Building energy consumption prediction method and device based on meta-strategy reinforcement learning
By using MAML meta-learning model to pre-train and fine-tune the DDPG model in building energy consumption prediction, the problem of slow convergence speed and insufficient generalization ability is solved, and more efficient energy consumption prediction is achieved.
Patent Information
- Application Number
- CN202411428878.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing building energy consumption prediction model has slow convergence speed and small training samples, resulting in limited generalization ability, and it is impossible to quickly adapt to the changes in the prediction scenario, affecting the accuracy.
Using a method based on meta-strategy reinforcement learning, the DDPG deep learning model is trained through the MAML meta-learning model, and the data of other buildings are pre-trained, and initialization parameters with strong generalization capabilities are obtained, and fine-tuned on the data of the target building. Update model parameters to improve prediction accuracy.
It significantly improves the convergence speed and prediction efficiency of the model, maintains high prediction accuracy with small sample size, and improves the accuracy of building energy consumption prediction.
Smart Images

Figure CN118966568A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of building energy consumption prediction, and specifically relates to a building energy consumption prediction method and device based on meta-strategy reinforcement learning. Background Art
[0002] Energy consumption prediction can play an important role in improving building energy efficiency, because efficient prediction of building energy consumption can help with energy consumption scheduling and management and building fault detection. In addition, it can also evaluate the operation strategies of different systems and evaluate the application value of the system by predicting the system energy consumption, so as to achieve the purpose of energy saving. With the development of artificial intelligence technology, machine learning methods have been introduced into this field. Compared with traditional building energy consumption prediction schemes, machine learning methods have the advantages of strong adaptability and simple modeling, and are favored by people.
[0003] The overall task of predicting building energy consumption can be regarded as a regression task. The required building energy consumption value is obtained by combining and regressing features of different dimensions. With the continuous development of neural network research, it has shown excellent performance in regression tasks in various fields. Neural networks achieve regression of nonlinear tasks through nonlinear activation functions of each layer. At the same time, reinforcement learning can realize the learning of serialized tasks, and the building energy consumption prediction task can be divided into serialized states. Applying the relevant emerging technologies of neural networks and reinforcement learning to building energy consumption prediction can better predict building energy consumption and thus achieve better energy management.
[0004] However, there are still some problems in the actual application of building energy consumption prediction using neural networks and reinforcement learning. Due to the inconsistent characteristics of building energy consumption in different specifications, different uses, and different environments, the parameters of the network model need to be adjusted accordingly when the prediction scenario changes, which will cause the problem of slow model convergence. In addition, there is a lack of data sets that can be used to train building energy consumption prediction models, the number of training samples is small, and the generalization ability of training tasks is limited. It is impossible to quickly adapt to changes in prediction scenarios, which ultimately seriously affects the accuracy of building energy consumption prediction. Summary of the invention
[0005] In view of the problems of slow model convergence speed and limited generalization ability caused by small number of training samples in current building energy consumption prediction schemes, in order to improve the energy consumption prediction accuracy of the model, the present invention provides a building energy consumption prediction method and device based on meta-strategy reinforcement learning, and the specific scheme is as follows.
[0006] The building energy consumption prediction method of the present invention comprises: Acquire characteristic data of the target building, the characteristic data including but not limited to: basic building data, environmental data of the area where the building is located, and time data; Based on the feature data, the energy consumption of the target building is predicted using the trained DDPG deep learning model; The training process of the DDPG deep learning model includes: Step 1: Collect historical characteristic data of the building complex and construct a training dataset; Step 2: preprocessing the training data set, and grouping the preprocessed training data set, wherein the preprocessed data of the target building is used as a meta-test set, and the preprocessed data of other buildings is used as a meta-training set; Step 3: Enter the MAML pre-training stage, use the meta-training set to train the DDPG deep learning model, and obtain generalized model parameters after training as initialization parameters of the model in the fine-tuning stage; Step 4: Enter the MAML fine-tuning stage and use the meta-test set to train the DDPG deep learning model. At this time, the initialization parameters in the DDPG deep learning model are the parameters obtained by pre-training in step 3. When the prediction error reaches the required threshold, the final DDPG deep learning model for building energy consumption prediction is obtained.
[0007] Optionally, the training process of the DDPG deep learning model further includes: Based on the real values of building energy consumption and corresponding feature data accumulated over time, the experience replay pool of the DDPG deep learning model in the fine-tuning phase is updated, and the model is trained again to update the parameters and complete the iteration of the model.
[0008] Optionally, the environmental data of the area where the building is located includes but is not limited to: air temperature, wind direction, wind speed, dew temperature, relative humidity, sea level pressure, and cloud coverage.
[0009] Optionally, the time data includes, but is not limited to: the month to which the date belongs, the day of the week to which the date belongs, and the day of the year to which the date belongs.
[0010] Optionally, the MAML pre-training stage includes: Step 31: Initialize the model parameters of the neural network ; Step 32: extracting data from the meta-training set to form a number of initial tasks, each task being divided into a support set and a query set; Step 33: Set the model parameters Assign to current task The unique network of ; Step 34: Using the Current Task The support set is task-based The learning rate ,right Perform an optimization and update , proceed to the next task; Step 35: After a batch of tasks is completed, based on the optimized , using the query set to calculate the loss , and calculate the loss right The gradient of , update the model parameters , get the new parameters ; Step 36: Repeat steps 31 to 35.
[0011] Optionally, the MAML fine-tuning stage includes: initializing the DDPG deep learning model using the model parameters obtained in the MAML pre-training stage, extracting a task from the meta-test set, using the support set in the task for training, using the test set in the task for prediction accuracy testing, and then adjusting the parameters.
[0012] The present invention provides a building energy consumption prediction system based on deep learning, comprising: A data acquisition module is configured to collect characteristic data of a target building, wherein the characteristic data includes but is not limited to: basic building data, environmental data of the area where the building is located, and time data; An energy consumption prediction module is configured to predict the energy consumption of a target building based on the feature data using a trained DDPG deep learning model; The DDPG deep learning model is trained using the training process described in any of the above items.
[0013] The present invention provides a building energy consumption prediction device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to implement the building energy consumption prediction method as described in any one of the above items when executing the computer program.
[0014] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the building energy consumption prediction method as described in any one of the above items is implemented.
[0015] The beneficial effects of the present invention are: The present invention introduces the MAML meta-learning model to train the DDPG deep learning model for predicting building energy consumption. To address the problem of small training data samples for the building energy consumption prediction model, the present invention groups the training data and uses the training data of buildings other than the target building for MAML pre-training, so that the model learns "prior knowledge" (initialization parameters). After initializing the DDPG deep learning model using the initialization parameters, the training data of the target building is used for training in the MAML fine-tuning stage. Compared with existing schemes, the training scheme of the present invention can show better training effects with a smaller sample size.
[0016] Experimental results show that compared with existing prediction schemes, the building energy consumption prediction method based on MAML+DDPG in the present invention can significantly improve the convergence speed of the model and improve the prediction efficiency while maintaining a high prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 It is a structural diagram of the DDPG network in the first embodiment of the present invention.
[0019] Figure 2 is a DDPG network update flow chart in Embodiment 1 of the present invention; Figure 3 It is a training flow chart of the building energy consumption prediction model based on MAML+DDPG of the present invention; Figure 4 It is a graph of the convergence of various models when MAE is used as the loss function. DETAILED DESCRIPTION
[0020] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0021] Embodiment 1: This embodiment provides a building energy consumption prediction method based on MAML+DDPG, including: (1) Obtain various characteristic data of the building group as the training data set for the prediction model; (2) Preprocess the training data set so that the data can meet the training requirements. During the processing, the data needs to be grouped, with the preprocessed data of the target building as the meta-test set and the preprocessed data of other buildings as the meta-training set; (3) Enter the MAML pre-training stage and use the meta-training set to train the model. The model used in the training process is the DDPG deep learning model. After training, the generalized model parameters are obtained, which are used as the initialization parameters of the model in the fine-tuning stage; (4) Enter the MAML fine-tuning phase and use the meta-test set to train the model. The initialization parameters of the network in DDPG are the parameters obtained by pre-training in (3). When the prediction error reaches the required threshold, the final training model is obtained; (5) Use the test set to evaluate the prediction accuracy of the model, calculate the prediction error and the convergence speed of the model, and compare it with other training methods.
[0022] The feature data comes from multiple different buildings, only one of which is the target building for which energy consumption prediction is required, and the other buildings are only used for meta-learning pre-training. These feature variables include but are not limited to factors such as temperature, wind direction, wind speed, dew temperature, relative humidity, sea level pressure, cloud coverage, historical power consumption, and time.
[0023] The operations of preprocessing the data include: correcting erroneous data; filling in missing historical information, supplementing it according to the average value of the previous and next data; picking up feature information; grouping data sets, etc. When picking up information, several items with strong correlation are selected as features. This embodiment selects seven features: temperature, wind direction, wind speed, dew temperature, sea level pressure, cloud coverage, and time. In addition, considering the implicit working days, statutory holidays and other information in time, it is split into three items: month, week, and day, that is, the month to which the day belongs, the day of the week, and the day of the year; therefore, the feature data of this embodiment is a 9-dimensional feature space. For different target buildings, the number and type of feature data selected will be different.
[0024] The MAML meta-learning model is a method that allows the model to learn "prior knowledge" (initialization parameters) through (many) N-way, K-shot task training; using this "prior knowledge", the model can be built faster on new N-way, K-shot tasks. The method is divided into two stages: pre-training stage and fine-tuning stage, corresponding to steps (3) and (4) respectively.
[0025] Pre-training is divided into two cycles: internal and external. Figure 3 The corresponding flow chart is drawn in the figure, and the general steps are as follows: (1) Initialize the model parameters of the neural network ; (2) According to the N-way, K-short experimental settings, data is extracted from the meta-training set to form several initial tasks. Each task is divided into a support set and a query set. (3) Network parameters Assign to current task The unique network of ; (4) Use the current task The support set is task-based The learning rate ,right Perform an optimization and update , proceed to the next task; (5) After a batch of tasks is completed, based on the optimized , use the query set to calculate the loss loss—— , and calculate right The gradient of , update the model parameters , get the new parameters ; (6) Repeat steps (2) to (5).
[0026] In the fine-tuning phase, the model parameters obtained by pre-training are used Initialize the model parameters, extract a task from the meta-test set each time during training, use the support set in the task for training, use the test set for accuracy testing, and adjust the parameters. Compared with pre-training, the fine-tuning phase only requires one gradient update per training, while the former requires two gradient updates.
[0027] The DDPG model (Deep Deterministic Policy Gradient Algorithm) is a reinforcement learning different-strategy algorithm for learning continuous actions. The specific network structure is shown in the attached Figure 1 shown.
[0028] The input of this model is the support set of the tasks in the meta-test set, which is also the environment of the reinforcement learning. Defined as the energy consumption of the building, the intuitive explanation is the prediction of the current state by reinforcement learning, and the prediction serves as the action of reinforcement learning. The current state S is the predicted building energy consumption at the current moment, and the next state is the predicted data at the next moment, and feedback is given according to the inverse of the absolute value error between the predicted value and the true value, as shown in formula (1).
[0029] (1) The Actor network makes the most appropriate prediction based on the current state, so the number of neurons in the input layer is 9, the number of neurons in the two hidden layers is set to 128, and the number of neurons in the output layer is 1. It is a prediction of building energy consumption, and the target-Actor has the same network structure.
[0030] The Critic network is equivalent to a state-action value function, whose input is the integration of state and action, that is, the integration of the input data and output data of the Actor network, with a total of 10 dimensions. Therefore, the input layer neurons of the network are 10, and the hidden layer is the same as the Actor network. The output layer is the evaluation of the actions taken by the Actor network. Similarly, the target-Critic network and the Critic network have the same structure.
[0031] The purpose of the two target networks is to help reinforcement learning converge more stably in neural network applications. The error between the target network and the actual network is used as a basis to help the network optimize. The error is shown in formula (2). For feedback, is the evaluation output of the target-Critic network for the next state and action, It is the evaluation output of the current state and action.
[0032] (2) After obtaining the error, gradient descent back propagation can be applied to update the error.
[0033] At the same time, since the Critic network is used to evaluate actions, the output of the Critic network corresponds to the loss of the Actor network. Since the network update is to improve the Critic network evaluation, we hope that its output is as large as possible. Therefore, we can perform gradient back propagation update along the output direction of the Critic network. The loss of the Actor network is shown in formula (3).
[0034] (3) After updating the above two networks, it is also necessary to soft-update the target network after a certain step length. The update method is shown in formula (4). is the weight of the target-Actor network, is the weight of the Actor network, is the weight of the target-Critic network, is the weight of the Critic network, is the soft update rate.
[0035]
[0036] (4) The real values of building energy consumption and the corresponding building characteristic data accumulated over time need to be preprocessed, and the experience replay pool of the DDPG model in the fine-tuning phase needs to be updated, and the model needs to be trained again to update the parameters and complete the model iteration.
[0037] Embodiment 2: This embodiment provides a building energy consumption prediction method based on MAML+DDPG. The data set used includes basic data of multiple buildings in 2016 (such as construction year, floor area, etc.), weather data (temperature, wind speed, etc.) and energy consumption meter reading data in each time period. The specific implementation steps are as follows: S1, obtain building group data, including the data of the target building and other buildings.
[0038] S2, perform data preprocessing. Table 1 shows the feature data extracted after preprocessing. The first 9 are used as input, and the tenth item of historical energy consumption is used as a reference benchmark for reinforcement learning. The preprocessed data are grouped, and the data of other buildings are used as a meta-training set, and the feature data of the target building is used as a meta-test set.
[0039] Table 1 Feature data after preprocessing
[0040] S3, enter the MAML pre-training stage, use the meta-training set to train the DDPG model, and after multiple epochs, obtain model parameters with strong generalization ability.
[0041] S4, enter the MAML fine-tuning stage, use the meta-test set to train the DDPG model, the network initialization parameters are the model parameters generated in the pre-training stage, at this time, the support set in each task of the meta-test set is used for model training, the query set is used for prediction, and the model network parameters are corrected.
[0042] S5, use the test set to evaluate the prediction accuracy of the model, calculate the prediction error and the convergence speed of the model, and compare with other training methods; use the three indicators of absolute value error (MAE), root mean square error (RMSE), and determination coefficient (R2) to compare the results. The experimental group is the model of the present invention (MAML+DDPG model), and the control group includes random forest model, DDPG model and back propagation neural network (BP).
[0043] The comparison results are shown in Table 2 and Figure 1 shown.
[0044] Table 2 Comparison of the prediction performance of each model under different parameters
[0045] It can be seen from Table 2 that the model proposed in this invention is better than the random forest model in terms of effect, and is similar to the DDPG model in performance, but slightly worse than the BP neural network. Figure 1 It can be seen that the model proposed in the present invention is significantly better than the other models in terms of convergence speed, and gradually converges after 50 epochs. Therefore, the building energy consumption prediction method based on DDPG+MAML in the present invention can effectively improve the convergence speed of the model while ensuring high prediction accuracy, thereby improving the prediction efficiency.
[0046] Embodiment three: This embodiment provides a building energy consumption prediction system, including: The data acquisition module is configured to collect characteristic data of the target building, the characteristic data including but not limited to: basic building data, environmental data of the area where the building is located, and time data; The energy consumption prediction module is configured to predict the energy consumption of the target building based on the feature data using the trained DDPG deep learning model; The DDPG deep learning model is trained using the training process described in Example 1.
[0047] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.
[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for predicting building energy consumption, It is characterized in that The method comprises: Acquire characteristic data of the target building, the characteristic data including but not limited to: basic building data, environmental data of the area where the building is located, and time data; Based on the feature data, the energy consumption of the target building is predicted using the trained DDPG deep learning model; The training process of the DDPG deep learning model includes: Step 1: Collect historical characteristic data of the building complex and construct a training dataset; Step 2: preprocessing the training data set, and grouping the preprocessed training data set, wherein the preprocessed data of the target building is used as a meta-test set, and the preprocessed data of other buildings is used as a meta-training set; Step 3: Enter the MAML pre-training stage, use the meta-training set to train the DDPG deep learning model, and obtain generalized model parameters after training as initialization parameters of the model in the fine-tuning stage; Step 4: Enter the MAML fine-tuning stage and use the meta-test set to train the DDPG deep learning model. At this time, the initialization parameters in the DDPG deep learning model are the parameters obtained by pre-training in step 3. When the prediction error reaches the required threshold, the final DDPG deep learning model for building energy consumption prediction is obtained.
2. The building energy consumption prediction method according to claim 1, It is characterized in that The training process of the DDPG deep learning model also includes: Based on the real values of building energy consumption and corresponding feature data accumulated over time, the experience replay pool of the DDPG deep learning model in the fine-tuning phase is updated, and the model is trained again to update the parameters and complete the iteration of the model.
3. The building energy consumption prediction method according to claim 1, It is characterized in that The environmental data of the area where the building is located includes but is not limited to: air temperature, wind direction, wind speed, dew temperature, relative humidity, sea level pressure, and cloud coverage.
4. The building energy consumption prediction method according to claim 1, It is characterized in that The time data includes but is not limited to: the month to which the date belongs, the day of the week to which the date belongs, and the day of the year to which the date belongs.
5. The building energy consumption prediction method according to claim 1, It is characterized in that The MAML pre-training phase includes: Step 31: Initialize the model parameters of the neural network ; Step 32: extracting data from the meta-training set to form a number of initial tasks, each task being divided into a support set and a query set; Step 33: Set the model parameters Assign to current task The unique network of ; Step 34: Using the Current Task The support set is task-based The learning rate ,right Perform an optimization and update , proceed to the next task; Step 35: After a batch of tasks is completed, based on the optimized , using the query set to calculate the loss , and calculate the loss right The gradient of , update the model parameters , get the new parameters ; Step 36: Repeat steps 31 to 35.
6. The building energy consumption prediction method according to claim 1, It is characterized in that The MAML fine-tuning phase includes: initializing the DDPG deep learning model using the model parameters obtained in the MAML pre-training phase, extracting a task from the meta-test set, training using the support set in the task, testing the prediction accuracy using the test set in the task, and then adjusting the parameters.
7. A building energy consumption prediction system, It is characterized in that The system comprises: A data acquisition module is configured to collect characteristic data of a target building, wherein the characteristic data includes but is not limited to: basic building data, environmental data of the area where the building is located, and time data; An energy consumption prediction module is configured to predict the energy consumption of a target building based on the feature data using a trained DDPG deep learning model; The DDPG deep learning model is trained using the training process described in any one of claims 1-6.
8. A building energy consumption prediction device, It is characterized in that It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the building energy consumption prediction method as described in any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the building energy consumption prediction method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Intelligent vehicle intersection decision-making method and device based on meta reinforcement learning, and medium
CN114537401A
Cited By
Soft soil pile foundation bearing capacity attenuation early warning method based on meta-learning strategy
CN119293617A
Meta-learning strategy based method for early warning of attenuation of bearing capacity of soft soil pile foundation
US12468955B1
