Building air conditioning optimization control method and system considering regional user individual comfort

Through a meta-learning-based method, learning and training a differentiated comfort reward function model, combining the energy consumption reward model for HVAC system control, the problem that the HVAC system is difficult to meet individual differentiated needs is solved, and efficient energy management and comfort is achieved Improved degree.

CN118794109BActive Publication Date: 2024-11-29SHANDONG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411266211.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-29
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

The existing HVAC system control method is difficult to effectively consider the personalized needs of users in different regions, resulting in low user comfort. In addition, traditional methods require high accuracy of the system model and complex calculations, making it difficult to achieve ideal control effects in practical applications.

Method used

Using the optimization control method based on meta-learning, we introduce the personalized comfort reward function model of regional users. By obtaining the comfort feedback data and historical operation data of users in different areas of the building, we describe the HVAC system as Markov decision-making process, learning and Training a differentiated comfort reward function model, combining the energy consumption reward model for policy network and value network training, and realizing personalized control decisions.

Benefits of technology

On the premise of ensuring the personalized comfort of regional users, we can achieve efficient energy management and energy conservation and emission reduction goals, reduce operation and maintenance costs, quickly adapt to the comfort needs of new regional users, and improve the convergence speed and user comfort of control strategies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118794109B_ABST
    Figure CN118794109B_ABST
Patent Text Reader

Abstract

The present invention proposes a building air conditioning optimization control method and system that considers the individual comfort of regional users, and relates to the technical field of building energy conservation and intelligent control. It includes describing the HVAC system control problem as a Markov decision process, defining the state, action and reward of each intelligent agent; using a meta-learning method to learn and obtain the initial model of the user comfort reward function; using the comfort feedback data of users in different regions to perform differentiated training on the initial model of the user comfort reward function, and obtain a differentiated comfort reward function model for each intelligent agent; based on the energy consumption reward model and the differentiated comfort reward function model of each intelligent agent, obtain the reward function of each intelligent agent, train the strategy network and value network of each intelligent agent, and use the trained strategy network to make control decisions for the building HVAC system. The present invention enables the HVAC system to achieve energy conservation and emission reduction on the premise of ensuring the individual comfort of regional users.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of building energy conservation and intelligent control, and in particular to a building air conditioning optimization control method and system that takes into account the individual comfort of regional users. Background Art

[0002] Building energy consumption accounts for a large proportion of the total national energy consumption, among which heating, ventilation and air conditioning systems (HVAC) are the main energy consuming equipment. With the acceleration of urbanization and the improvement of people's living standards, the comfort and energy-saving needs of the internal environment of buildings are constantly increasing. Due to differences between individuals, users in different regions may have different thermal preferences, which will affect the actual comfort level of users. In addition, due to the large climate differences between winter and summer, the seasonal energy demand peak is obvious, which further aggravates the energy consumption problem. In order to meet these challenges, improving the energy efficiency of HVAC systems and optimizing their control strategies have become urgent issues to be solved.

[0003] Existing HVAC system control methods mainly rely on traditional rule control methods. These methods have several problems in practical applications:

[0004] First, traditional control methods generally use standardized comfort evaluation indicators (such as the PMV model and ASHRAE Standard 55), which makes it difficult to effectively consider the personalized needs of users in different areas, resulting in low actual user comfort.

[0005] Secondly, traditional model-based control methods have high requirements on the accuracy of the system model and are computationally complex. The building HVAC system is a complex dynamic system with nonlinearity, multivariables, and strong coupling. Traditional methods often find it difficult to achieve ideal control effects in practical applications. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a building air conditioning optimization control method and system taking into account the individual comfort of regional users. The method adopts an optimization control method based on meta-learning, introduces a regional user personalized comfort reward function model, and ensures the personalized comfort requirements of users in different regions. The HVAC system can achieve efficient energy conservation and emission reduction goals while ensuring the individual comfort of regional users, while reducing operation and maintenance costs, which is convenient for practical application and promotion in various building environments.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] A first aspect of the present invention provides a building air conditioning optimization control method that takes into account the individual comfort of regional users.

[0009] The building air conditioning optimization control method considering the individual comfort of regional users includes the following steps:

[0010] Obtain comfort feedback data from users in different areas of the building, as well as historical operation data of the building's air conditioning system, and perform data preprocessing;

[0011] Describe the HVAC system control problem as a Markov decision process, define each agent and the state, action, and reward of each agent;

[0012] Using the meta-learning method, based on the comfort feedback data of users in multiple regions, the initial parameters of the comfort reward function of multiple regions are learned to obtain the initial model of the user comfort reward function;

[0013] The comfort feedback data of users in different regions are used to perform differentiated training and parameter update on the initial model of the user comfort reward function, and a differentiated comfort reward function model for each agent is obtained;

[0014] Based on the energy consumption reward model and the differentiated comfort reward function model of each agent, the reward function of each agent is obtained, the strategy network and value network of each agent are trained, and the trained strategy network is used to make control decisions for the building HVAC system.

[0015] A second aspect of the present invention provides a building air conditioning optimization control system that takes into account the individual comfort of regional users.

[0016] The building air conditioning optimization control system that takes into account the individual comfort of regional users includes:

[0017] The data acquisition module is configured to: acquire the comfort feedback data of users in different areas of the building and the historical operation data of the building air conditioning system, and perform data preprocessing;

[0018] The Markov process building module is configured to: formulate the HVAC system control problem as a Markov decision process, define each agent and the state, action, and reward of each agent;

[0019] The meta-learning training module is configured to: adopt a meta-learning method to learn initial parameters of the comfort reward function of multiple regions based on the comfort feedback data of users in multiple regions, and obtain an initial model of the user comfort reward function;

[0020] The meta-learning test module is configured to: use the comfort feedback data of users in different regions to perform differentiated training and parameter update on the initial model of the user comfort reward function, and obtain a differentiated comfort reward function model for each agent;

[0021] The optimization control strategy training module is configured as follows: combining the energy consumption reward model and the differentiated comfort reward function model of each agent to obtain the reward function of each agent, training the strategy network and value network of each agent, and using the trained strategy network to make control decisions for the building HVAC system.

[0022] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the building air conditioning optimization control method considering the individual comfort of regional users as described in the first aspect of the present invention.

[0023] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the building air conditioning optimization control method taking into account the individual comfort of regional users as described in the first aspect of the present invention are implemented.

[0024] One or more of the above technical solutions have the following beneficial effects:

[0025] The present invention provides a building air conditioning optimization control method and system that takes into account the individual comfort of regional users. The method adopts a meta-learning-based optimization control method and introduces a regional user individual comfort reward function model to ensure the individual comfort requirements of users in different regions. The HVAC system can achieve efficient energy management and energy conservation and emission reduction goals while ensuring the individual comfort of regional users, while reducing operation and maintenance costs, which is convenient for practical application and promotion in various building environments.

[0026] The present invention has the advantage of high efficiency, and can quickly adapt to the comfort requirements of users in new areas by using a meta-learning-based method, thereby improving the convergence speed of the control strategy.

[0027] The present invention has the advantage of personalization and can realize personalized comfort reward function model learning according to feedback from users in different regions, thereby improving user comfort.

[0028] The present invention has the advantage of energy saving, and the optimized control strategy can reduce energy consumption to the greatest extent while ensuring comfort.

[0029] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0032] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0033] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0034] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0035] Embodiment 1

[0036] In order to solve the problems mentioned in the above background technology, this embodiment discloses a building air conditioning optimization control method that takes into account the individual comfort of regional users, and adopts an optimization control method based on meta-learning to ensure the personalized comfort requirements of users in different regions. The specific scheme includes:

[0037] The meta-learning algorithm is used for training to obtain the initial model of the user comfort reward function, which is used as the initial model for learning the user comfort reward function in different areas.

[0038] In view of the differentiated comfort requirements of users in different regions, the comfort feedback data of users in different regions are collected respectively, and then the initial comfort reward function model is trained and updated with different parameters to obtain a comfort reward function model corresponding to each intelligent agent.

[0039] Based on the reward function that considers energy consumption and the differential comfort of each area, the HVAC optimization control neural network training is carried out, and the trained strategy network is used to make control decisions.

[0040] The present invention introduces a regional user personalized comfort reward function model, so that the HVAC system can achieve efficient energy conservation and emission reduction goals while ensuring the personalized comfort of regional users, while reducing operation and maintenance costs, which is convenient for practical application and promotion in various building environments.

[0041] like Figure 1 As shown, the building air conditioning optimization control method considering the individual comfort of regional users may include the following steps:

[0042] Obtain comfort feedback data from users in different areas of the building, as well as historical operation data of the building's air conditioning system, and perform data preprocessing;

[0043] Describe the HVAC system control problem as a Markov decision process, define each agent and the state, action, and reward of each agent;

[0044] Using the meta-learning method, based on the comfort feedback data of users in multiple regions, the initial parameters of the comfort reward function of multiple regions are learned to obtain the initial model of the user comfort reward function;

[0045] The comfort feedback data of users in different regions are used to perform differentiated training and parameter update on the initial model of the user comfort reward function, and a differentiated comfort reward function model for each agent is obtained;

[0046] Based on the energy consumption reward model and the differentiated comfort reward function model of each agent, the reward function of each agent is obtained, the strategy network and value network of each agent are trained, and the trained strategy network is used to make control decisions for the building HVAC system.

[0047] Furthermore, the detailed technical solution of this embodiment is as follows:

[0048] 1. Data collection and preprocessing

[0049] Collect user comfort feedback data in different areas and historical operation data of the building air conditioning system, including indoor and outdoor temperature, humidity, concentration, energy consumption, etc. The collected data is preprocessed through data cleaning, normalization and other methods to ensure the quality of the data used for training.

[0050] The processed data will be used as a data set in the meta-learning process to learn a comfort reward function model that meets the personalized comfort needs of users in different regions. The meta-learning model learns the comfort needs of users in different regions through these data and builds a personalized comfort model.

[0051] Further, including:

[0052] 1.1 Collection and preprocessing of comfort feedback data

[0053] Many factors affect the user's comfort level, such as air temperature, relative humidity, concentration, metabolic rate, clothing insulation, etc. However, in reality, due to user privacy issues, factors such as metabolic rate and clothing are difficult to monitor, and these factors do not change much. Therefore, these factors are fixed values ​​and no longer fed back. In order to adapt the controller to the comfort preferences of users in different regions, different user comfort levels in different regions will be used as part of the reward item when designing the optimization control method.

[0054] We use Indicates different areas of the system Different comfort rewards at time t, which are related to the user's feedback on the indoor environment. In order to collect feedback from occupants, we can provide them with a feedback device with five options such as "very uncomfortable", "uncomfortable", "medium", "comfortable" and "very comfortable".

[0055] After receiving different feedback from users in different areas, the five feedback options are quantified. The five options collected are quantized into -2, -1, 0, +1 and +2, which can be regarded as different areas in different indoor environments. Different real rewards at time t ,Right now:

[0056]

[0057] in, Indicates different areas of the system Different comfort rewards at time t; is an unknown function, Indicates area Indoor temperature, Indoor concentration, represents the indoor humidity. The present invention will use the meta-learning method to use Parameterized neural network to approximate.

[0058] 1.2 Collection and preprocessing of historical operating data of buildings

[0059] Collect building environmental data and HVAC system operation data. Environmental data includes: indoor temperature, humidity, concentration, outdoor temperature, humidity, wind speed, solar radiation, etc.; HVAC system data includes: HVAC system operating status, operating parameters (such as fan speed, chilled water temperature, heating water temperature, etc.), energy consumption, etc.

[0060] The collected data is preprocessed to ensure data quality and training effect. First, the data is cleaned to remove missing values ​​and outliers to ensure the integrity and accuracy of the data; then the data is standardized to make it on the same scale, using a standard normal distribution with a mean of 0 and a standard deviation of 1.

[0061] 2. Constructing a Markov process

[0062] This example considers a building air conditioning system with N zones. The controllable part of the HVAC system consists of an air handling unit (AHU) for the entire building and a variable air volume (VAV) box for each zone. Therefore, N+1 agents are considered in the MDP. We assume that the HVAC system only needs to consider its own strategy to solve the MDP and make sequential decisions, which only requires sample sequences of states, actions, and rewards.

[0063] The HVAC system control problem is described as a Markov decision process (MDP), which defines indoor temperature, indoor humidity, indoor The state (state) consists of concentration, the action (action) consists of regional mass flow, fresh air volume and chilled water flow, and the reward (reward) consists of energy consumption and comfort.

[0064] 2.1 State

[0065] The HVAC system has N zones, and the state of each zone i (1≤i≤N) at time t is defined. for:

[0066]

[0067] in, Indicates the outdoor temperature, represents the indoor temperature of area i, represents the indoor temperature of the adjacent area of ​​area i, Indicates the electricity price, represents the number of residents in area i, Indicates area i indoor concentration, Indicates the indoor humidity in area i; is the state of the i-th regional agent at time t, (1≤i≤N).

[0068] In addition to the N regions, define AHU as the N+1th agent, whose state at time t It is expressed as:

[0069]

[0070] in, is the state of the N+1th agent AHU at time t.

[0071] 2.2 Action

[0072] Define the action of each regional agent i (1≤i≤N) at time t as:

[0073]

[0074] in, is the action of agent i (1≤i≤N) at time t, Represents the regional mass flow of agent i (1≤i≤N).

[0075] Define the action of the N+1th agent at time t as:

[0076]

[0077] in, is the action of the N+1th agent at time t, Indicates the fresh air volume, Indicates the chilled water flow rate.

[0078] 2.3 Reward

[0079] The control goal of the present invention is to minimize the total energy consumption and maximize the comfort. For N regional agents, the reward can be divided into two parts:

[0080] ;

[0081] in, represents the reward of regional agent i (1≤i≤N) at time t; represents the comfort penalty of regional agent i (1≤i≤N), represents the energy consumption penalty of regional agent i (1≤i≤N), is a positive weight coefficient; is the state of regional agent i (1≤i≤N) at time t, is the action of regional agent i (1≤i≤N) at time t.

[0082] The comfort reward function is trained by the meta-learning method, and each agent has a different comfort reward model. The specific method will be explained in the next section.

[0083] A key goal of the AHU agent is to minimize the energy consumption of the entire system. At the same time, the AHU needs to consider the air quality standards of the entire building to ensure the environmental comfort of the entire building. Therefore, for the N+1th agent AHU, the reward function is expressed as:

[0084]

[0085] in, represents the reward of the N+1th agent AHU at time t; is a positive weight coefficient; represents the energy consumption penalty of the N+1th agent AHU; is the state of the N+1th agent AHU at time t; is the action of the N+1th agent AHU at time t.

[0086] 3. Meta-learning training task - initial model learning of comfort reward function

[0087] The meta-learning training task is to find an initial parameter that can quickly adapt to a new task by training on multiple tasks. Then, in the meta-learning test task phase, this initial parameter is used to enable the model to quickly adapt to a new task through a small amount of data training and a small number of gradient updates. (Advance)

[0088] The specific steps of the meta-learning training task include: task definition, parameter initialization for N different regions, task sampling, in-task training and meta-parameter update. Finally, the loss functions of users in N regions are averaged and the initial parameters are updated.

[0089] For N different regions, each task is defined as learning the comfort reward function of users in each region. First, the sampled data is divided into the Support set and the Query set; each task is trained using the data in the Support set, and the parameters of each task are updated by gradient descent; the trained parameters are tested in the Query set, and then the loss function of each task is calculated; the loss functions of these tasks are averaged, and the initial parameters of the meta-learning training task are updated by gradient descent.

[0090] The specific process of the meta-learning algorithm is as follows:

[0091] The task definition defines the comfort reward function of users in each region as a subtask. By sampling data from users in different regions, the meta-learning model learns how to quickly adapt to the needs of users in different regions.

[0092] Model initialization initializes the weight parameters θ of the meta-learning network;

[0093] Set the learning rates α and β, and the data batch size B.

[0094] Task sampling randomly samples B regional users from the dataset F to form B sub-training tasks;

[0095] For each subtask, we sample K sets of indoor environment status and user comfort feedback data pairs. (1≤k≤K) as the Support set, sample K′ groups of indoor environment status and user comfort feedback data pairs (1≤k′≤K′) as the query set, where and The indoor temperature, humidity, The indoor environment state composed of concentration, k and k′ are both counting parameters; and Indicates the comfort feedback data of indoor users on the indoor environment status.

[0096] In-task training First, train these B subtasks in the Support set to train the model parameters for each subtask.

[0097] Calculate the loss function:

[0098]

[0099] in, Indicates the actual comfort feedback data of indoor users on the indoor environment status obtained by sampling; Represented by the weight parameter The model represents the same input data The predicted output of Indicates that the Support set has K groups of data, and k represents the kth (1≤k≤K) group of data; represents the kth (1≤k≤K) group of indoor environment data in the Support set, and b represents the bth training subtask (1≤b≤B).

[0100] Update the parameters using gradient descent:

[0101]

[0102] in, is the weight parameter of the updated meta-learning network.

[0103] Then, use the parameters trained in the previous step to test in the query set.

[0104] Calculate the loss function for each task:

[0105]

[0106] in, is the k′th (1≤k′≤K′) group of indoor environment data in the query set; represents the model weight parameter obtained when testing with the Query set after training with the Support set; K′ represents the total number of K′ groups of data in the Query set. Represents the k′(1≤k′≤K′)th group of data.

[0107] (5) Meta Update

[0108] The loss functions of all users are averaged and the initial parameters are updated using gradient descent:

[0109]

[0110] Where b represents the b-th training subtask (1≤b≤B).

[0111] Meta-learning provides real-time personalized comfort feedback by training a comfort model that can quickly adapt to the personalized needs of users in different regions. During the meta-learning process, the model learns the comfort preferences of users in each region by sampling data from multiple regional user tasks. This process enables the model to quickly adjust parameters through a small amount of regional user feedback data, so that it can quickly adapt to new regional user tasks. As part of the reward function during the agent training process, the model helps optimize the control strategy of the HVAC system to achieve personalized comfort adjustment and efficient energy consumption management. This method effectively solves the problem that traditional comfort evaluation models cannot meet the differentiated needs of individuals, and improves the overall user satisfaction and the response speed of the system.

[0112] By adopting a meta-learning approach, through training on multiple tasks, an initial parameter that can quickly adapt to new tasks is learned, so that it can quickly adapt to the needs of new tasks or new users through a small amount of training data and a few gradient updates. When facing new users or new environments, there is no need to retrain the entire model. Only a small amount of data is needed to adjust the model to adapt to new users, improving the system response speed.

[0113] 4. Meta-learning test task: learning the reward function model for differentiated comfort of users in different regions

[0114] In order to obtain the comfort reward function model for users in each area, based on the initial model of the comfort reward function, different user comfort feedback data in each area is used for model training. For each task, the loss function is calculated, and the parameters are updated using the gradient descent method, and finally the comfort reward function model for the area is obtained. .

[0115] Specifically:

[0116] For different regional intelligent agent systems, their differentiated comfort feedback data are used to train the comfort reward function model separately.

[0117] For each comfort reward function task:

[0118] (1) Calculate the loss function. Use the feedback data of the area to calculate the loss function under the current parameters.

[0119] (2) Calculate parameter updates. Use the gradient descent method to update the current parameters.

[0120] The comfort reward function model of each agent can quickly adapt to the unique comfort requirements of the agent through a small amount of training data and a few gradient updates, and obtain the corresponding comfort reward function model of the agent. .

[0121] 5. Training on optimization control strategy of building HVAC system

[0122] Reward energy consumption Differentiated comfort rewards for users in different regions obtained from meta-learning test tasks Together, they serve as the reward function of each intelligent agent i (1≤i≤N+1) to update and train the strategy network and the value network, and ultimately obtain the optimal control strategy to meet the personalized comfort needs of regional users while reducing energy consumption.

[0123] The present invention adopts a centralized training-distributed decision-making training method. Each agent obtains the execution action in the current state according to its own strategy, and stores the obtained experience in its own experience buffer pool after interacting with the environment. After all agents interact with the environment, each agent randomly extracts experience from the experience pool to train its own neural network.

[0124] Based on the reward function composed of energy consumption reward and differentiated comfort reward, the HVAC system control neural network training is carried out, where each agent corresponds to a set of strategy networks and value networks, and the trained strategy network is used to make differentiated optimization control decisions for each agent. The specific process of training is:

[0125] Initialize each agent’s policy network and target strategy network , value network and target value network ;

[0126] Initialize the experience replay pool D of each agent;

[0127] Initialize all network parameters randomly.

[0128] At each time t, each agent i receives the state ;

[0129] Each agent i Using its strategy network Select Action ,in is the noise used for exploration.

[0130] Execute actions The agent performs actions in the environment , the environment returns to the new state and the reward for each agent ,in represents the set of actions of all agents at time t.

[0131] Store experience and experience Stored in the experience replay pool D, where Represents the set of rewards for all agents at time t.

[0132] Experience replay randomly draws a batch of experience samples from the experience replay pool D . The superscript j represents the jth empirical sample drawn. It indicates the next state that the current state of the j-th experience sample transfers to after action a.

[0133] Update the value network for each agent , calculate the target value :

[0134] ;

[0135] Among them, the superscript j represents the jth experience sample extracted; represents the next state of the jth sample of agent i; represents the target policy network of agent i; represents the target value network of agent i; Indicates that each agent i is based on the next state Target Policy Network The generated action, is the discount factor.

[0136] Minimize the loss function and update the value network parameters:

[0137] ;

[0138] in, represents the loss function of agent i, Represents the weight parameter of the value network update of agent i; represents the target value, represents the action of the jth experience sample of agent i, represents the value network of agent i; Represents the expected value operator, which is used to calculate the average loss over multiple samples.

[0139] Update the policy network For each agent i, update the policy network parameters through policy gradient:

[0140]

[0141] in, Represents the weight parameter of the policy network update; represents the objective function of the policy network.

[0142] Update the target network For each agent i, soft update the target network parameters:

[0143] ;

[0144] ;

[0145] in, is the soft update coefficient; Represents the network weight parameters; Represents the weight parameter of the target policy network update; Represents the weight parameter of the value network update; Represents the weight parameter of the target value network update.

[0146] Through the above method, this embodiment can achieve efficient, personalized and energy-saving control of the HVAC system while ensuring the personalized comfort of regional users, and is suitable for HVAC system optimization in various building environments.

[0147] Embodiment 2

[0148] This embodiment discloses a building air conditioning optimization control system that takes into account the individual comfort of regional users.

[0149] The building air conditioning optimization control system that takes into account the individual comfort of regional users includes:

[0150] The data acquisition module is configured to: acquire the comfort feedback data of users in different areas of the building and the historical operation data of the building air conditioning system, and perform data preprocessing;

[0151] The Markov process building module is configured to: formulate the HVAC system control problem as a Markov decision process, define each agent and the state, action, and reward of each agent;

[0152] The meta-learning training module is configured to: adopt a meta-learning method to learn initial parameters of the comfort reward function of multiple regions based on the comfort feedback data of users in multiple regions, and obtain an initial model of the user comfort reward function;

[0153] The meta-learning test module is configured to: use the comfort feedback data of users in different regions to perform differentiated training and parameter update on the initial model of the user comfort reward function, and obtain a differentiated comfort reward function model for each agent;

[0154] The optimization control strategy training module is configured as follows: combining the energy consumption reward model and the differentiated comfort reward function model of each agent to obtain the reward function of each agent, training the strategy network and value network of each agent, and using the trained strategy network to make control decisions for the building HVAC system.

[0155] Embodiment 3

[0156] The purpose of this embodiment is to provide a computer-readable storage medium.

[0157] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the building air conditioning optimization control method taking into account the individual comfort of regional users as described in Example 1 of the present disclosure.

[0158] Embodiment 4

[0159] The purpose of this embodiment is to provide an electronic device.

[0160] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the building air conditioning optimization control method considering the individual comfort of regional users as described in Example 1 of the present disclosure are implemented.

[0161] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0162] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0163] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A building air conditioning optimization control method considering the individual comfort of regional users, characterized in that: The following steps are involved: Obtain comfort feedback data from users in different areas of the building, as well as historical operation data of the building's air conditioning system, and perform data preprocessing; Describe the HVAC system control problem as a Markov decision process, define each agent and the state, action, and reward of each agent; Using the meta-learning method, based on the comfort feedback data of users in multiple regions, the initial parameters of the comfort reward function of multiple regions are learned to obtain the initial model of the user comfort reward function; The comfort feedback data of users in different regions are used to perform differentiated training and parameter update on the initial model of the user comfort reward function, and a differentiated comfort reward function model for each agent is obtained; Based on the energy consumption reward model and the differentiated comfort reward function model of each agent, the reward function of each agent is obtained, the strategy network and value network of each agent are trained, and the trained strategy network is used to make control decisions for the building HVAC system. Using the meta-learning method, the initial model of the user comfort reward function is obtained, which is: Define the comfort reward function of users in each area as a subtask; For each subtask, obtain multiple sets of indoor environment status and comfort feedback data from regional users and divide them into Support set and Query set. Use the data in the Support set to train each subtask separately, and update the parameters of each subtask through gradient descent; Use the trained parameters to test in the query set, and then calculate the loss function for each subtask; Average the loss functions of multiple subtasks and use gradient descent to update the initial parameters of the meta-learning training task; Calculate the loss function: ; in, Indicates the actual comfort feedback data of indoor users on the indoor environment status obtained by sampling; Represented by the weight parameter The model represents the same input data The predicted output of Supportset has K groups of data, k represents the kth group, 1≤k≤K groups of data; represents the kth group of indoor environment data in the Support set, b represents the bth training subtask, 1≤b≤B; Update the parameters using gradient descent: ; in, is the weight parameter of the updated meta-learning network; Calculate the loss function for each subtask: ; in, is the k′th group of indoor environment data in the query set; represents the model weight parameter obtained when testing with the Query set after training with the Support set; K′ represents the total number of K′ groups of data in the Query set. represents the k′th group of data, 1≤k′≤K′; The loss functions of all users are averaged and the initial parameters are updated using gradient descent: ; Where b represents the b-th training subtask, 1≤b≤B.

2. The building air conditioning optimization control method considering the individual comfort of regional users according to claim 1, characterized in that: The comfort feedback data is the user's different feelings about the indoor environment, and the indoor environment includes air temperature, relative humidity and The historical operation data of the building air conditioning system includes building environment data and HVAC system operation data, and the building environment data includes indoor temperature, indoor humidity, indoor concentration, outdoor temperature, outdoor humidity, wind speed and solar radiation; the HVAC system operation data includes fan speed, chilled water temperature and heating water temperature.

3. The building air conditioning optimization control method considering the individual comfort of regional users as claimed in claim 1, characterized in that: Consider a building with N areas, define N areas as N area agents, and define the air handling unit AHU used for the entire building as the N+1th agent AHU; Definition includes indoor temperature, indoor humidity and indoor The state of concentration; Define actions including zone mass flow, fresh air flow, and chilled water flow; Define a reward that includes both energy consumption and comfort.

4. The building air conditioning optimization control method considering the individual comfort of regional users as claimed in claim 3, characterized in that: The states defined for each agent include: For N regional agents, the state of each region i at time t for: ; Among them, 1≤i≤N; Indicates the outdoor temperature, represents the indoor temperature of area i, represents the indoor temperature of the adjacent area of ​​area i, Indicates the electricity price, represents the number of residents in area i, Indicates area i indoor concentration, Indicates the indoor humidity in area i; The state of the N+1th agent AHU at time t It is expressed as: ; Define the action of each regional agent i, 1≤i≤N at time t : ; in, represents the regional mass flow of agent i (1≤i≤N); Define the action of the N+1th agent AHU at time t : ; in, Indicates the fresh air volume, Indicates the chilled water flow rate; For N regional agents, the reward function for: ; Among them, 1≤i≤N; represents the comfort penalty of regional agent i, represents the energy consumption penalty of regional agent i, is a positive weight coefficient; is the state of regional agent i at time t, is the action of regional agent i at time t; For the N+1th agent AHU, the reward function for: in, represents the energy consumption penalty of the N+1th agent AHU; is a positive weight coefficient; is the state of the N+1th agent AHU at time t; is the action of the N+1th agent AHU at time t.

5. The building air conditioning optimization control method considering the individual comfort of regional users according to claim 1, characterized in that: Train each agent’s strategy network and value network, including: Each agent i interacts with the environment at time t to obtain the state , and uses its policy network to select and execute actions , get the new state returned by the environment and the reward for each agent ; Experience Store them in the experience replay pool, and randomly extract a batch of experience samples from the experience replay pool; Calculate the target value for each agent i, minimize the loss function, and update the value network parameters; For each agent i, the policy network parameters are updated through policy gradient.

6. The building air conditioning optimization control method considering the individual comfort of regional users as claimed in claim 5, characterized in that: For each agent i, soft update the target network parameters: ; ; in, is the soft update coefficient; Represents the network weight parameters; is the policy network of the ith agent, is the target policy network of the ith agent, is the value network of the ith agent, is the target value network of the i-th agent; Represents the weight parameter updated by the policy network; Represents the weight parameter of the target policy network update; Represents the weight parameter of the value network update; Represents the weight parameter of the target value network update.

7. The building air conditioning optimization control system considering the individual comfort of regional users is characterized by: include: The data acquisition module is configured to: acquire the comfort feedback data of users in different areas of the building and the historical operation data of the building air conditioning system, and perform data preprocessing; The Markov process building module is configured to: formulate the HVAC system control problem as a Markov decision process, define each agent and the state, action, and reward of each agent; The meta-learning training module is configured to: adopt a meta-learning method to learn initial parameters of the comfort reward function of multiple regions based on the comfort feedback data of users in multiple regions, and obtain an initial model of the user comfort reward function; The meta-learning test module is configured to: use the comfort feedback data of users in different regions to perform differentiated training and parameter update on the initial model of the user comfort reward function, and obtain a differentiated comfort reward function model for each agent; The optimization control strategy training module is configured to: combine the energy consumption reward model and the differentiated comfort reward function model of each agent to obtain the reward function of each agent, train the strategy network and value network of each agent, and use the trained strategy network to make control decisions for the building HVAC system; Using the meta-learning method, the initial model of the user comfort reward function is obtained, which is: Define the comfort reward function of users in each area as a subtask; For each subtask, obtain multiple sets of indoor environment status and comfort feedback data from regional users and divide them into Support set and Query set. Use the data in the Support set to train each subtask separately, and update the parameters of each subtask through gradient descent; Use the trained parameters to test in the query set, and then calculate the loss function for each subtask; Average the loss functions of multiple subtasks and use gradient descent to update the initial parameters of the meta-learning training task; Calculate the loss function: ; in, Indicates the actual comfort feedback data of indoor users on the indoor environment status obtained by sampling; Represented by the weight parameter The model represents the same input data The predicted output of Supportset has K groups of data, k represents the kth group, 1≤k≤K groups of data; represents the kth group of indoor environment data in the Support set, b represents the bth training subtask, 1≤b≤B; Update the parameters using gradient descent: ; in, is the weight parameter of the updated meta-learning network; Calculate the loss function for each subtask: ; in, is the k′th group of indoor environment data in the query set; represents the model weight parameter obtained when testing with the Query set after training with the Support set; K′ represents the total number of K′ groups of data in the Query set. represents the k′th group of data, 1≤k′≤K′; The loss functions of all users are averaged and the initial parameters are updated using gradient descent: ; Where b represents the b-th training subtask, 1≤b≤B.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the building air conditioning optimization control method considering the individual comfort of regional users as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the building air conditioning optimization control method considering the individual comfort of regional users as described in any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Subway platform HVAC control method based on multi-agent deep reinforcement learning

    CN115289619A

  • For hiearchical decomposition deep reinforcement learning for an artificial intelligence model

    WO2018236674A1