Central air conditioner control method, system, equipment, medium and program product

Abstract

The invention discloses a central air conditioner control method, system and device, a medium and a program product, and relates to the technical field of data processing and control systems.The method comprises the steps that a central air conditioner system control model is built; the method comprises the following steps: constructing an actor network, and embedding an optimization control model consisting of a global critic network and a local critic network of a GAT (Graphics Attention Transform); and training the optimization control model, and controlling the refrigeration temperature set value and the heating temperature set value of each area according to the current state observation variable of each area by using the trained actor network. A critic network is designed into a global critic network and a local critic network, the problem that regional thermal comfort awards and central air conditioning system total energy consumption awards conflict with each other in the training process is relieved, a graph attention network is fused into the global critic network, coupling information in the system is fully extracted, and the optimization control performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and control systems, and in particular to a central air-conditioning control method, system, equipment, medium and program product. Background Art

[0002] Existing central air conditioning control methods are mainly divided into three categories: (1) Rule-based controls (RBC) method: This method relies on fixed rules and lacks flexibility in nature. Therefore, it is difficult to effectively balance energy efficiency and thermal comfort in central air-conditioning systems in complex commercial buildings.

[0003] (2) Model Predictive Control (MPC) method, which optimizes energy consumption and thermal comfort through dynamic models, but it is highly dependent on the accuracy of the dynamic models. However, due to the uncertainty of building thermal parameters and the complexity of the operation mechanism of the central air-conditioning system, it is challenging to develop an accurate dynamic model for the central air-conditioning system of commercial buildings.

[0004] (3) Deep reinforcement learning (DRL) method, which learns the optimal control strategy through direct interaction with the environment and does not require complex dynamic models, making it suitable for HVAC (Heating Ventilation and Air Conditioning) systems.

[0005] Deep reinforcement learning is divided into single-agent deep reinforcement learning and multi-agent deep reinforcement learning (Multi-Agent Deep Reinforcement Learning, MADRL). Due to the interaction between various areas in the central air-conditioning system of commercial buildings, single-agent reinforcement learning cannot effectively achieve collaborative optimization among various agents.

[0006] Although MADRL can effectively coordinate the collaborative optimization between different regions, the central air-conditioning system of commercial buildings has significant energy consumption coupling and thermal coupling characteristics, which brings certain challenges to MADRL. First, the optimization goal of the central air-conditioning system of commercial buildings is the total energy consumption of the central air-conditioning system and the thermal comfort of each region. Therefore, the reward function of MADRL is usually a combination of regional thermal comfort rewards and total energy consumption rewards of the central air-conditioning system. However, this reward combination will conflict with each other during the training process, resulting in unstable learning strategies. In addition, MADRL mainly focuses on the strategy coordination between agents, but does not effectively model the relationship between system energy consumption coupling and thermal coupling, resulting in a decrease in control performance. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a central air-conditioning control method, system, device, medium and program product, and designs the critic network in multi-agent reinforcement learning into a global critic network and a local critic network, which effectively alleviates the conflict between the regional thermal comfort reward and the total energy consumption reward of the central air-conditioning system during the training process and eliminates unstable strategies; at the same time, the Graph Attention Network (GAT) is integrated into the global critic network to fully extract the coupling information within the system, thereby improving the optimization control performance.

[0008] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a central air conditioning control method, comprising: Construct a central air-conditioning system control model for N areas, including state observation variables defined by the indoor and outdoor temperature and humidity of each area, actions defined by the cooling and heating temperature set points of each area, and rewards defined by the total energy consumption of the central air-conditioning system and the regional thermal comfort; An optimization control model consisting of an actor network, a global critic network embedded in GAT, and a local critic network is constructed; the actor network takes the state observation variables of a single region as input and outputs the control action for the central air-conditioning system; the global critic network embedded in GAT takes the state observation variables and actions of all regions as input and outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system; the local critic network takes the state observation variables and actions of a single region as input and outputs the local state-action value corresponding to the regional thermal comfort reward; The optimization control model is trained, and the trained actor network is used to control the cooling temperature set value and heating temperature set value of each area of ​​the central air-conditioning system according to the current state observation variables of each area.

[0009] As an optional implementation, the state observation variables include outdoor air temperature, outdoor air relative humidity, number of occupants in each area, indoor air temperature, indoor air relative humidity, heating temperature setting value, cooling temperature setting value and total energy consumption of the central air conditioning system; The action includes incremental adjustment of the heating temperature setting value and the cooling temperature setting value, setting the range of the incremental adjustment, and the value range of the heating temperature setting value and the cooling temperature setting value, and the heating temperature setting value is lower than the cooling temperature setting value; Rewards include global rewards and local rewards ; ; ; ; in, and It is an adjustable weight; is the total energy consumption of the central air conditioning system; is the thermal comfort bonus of area i at time t, is the occupancy status of area i at time t, where unoccupied is 0 and occupied is 1; is the thermal comfort threshold; is the predicted dissatisfaction percentage of region i at time t.

[0010] As an optional implementation, the global critic network embedded with GAT includes a GAT layer and three fully connected layers after the GAT layer. ;in, is the GAT layer, It is a three-layer fully connected layer; is the state observation variable of all regions; For all areas of action; are the parameters of the global critic network embedded in GAT.

[0011] As an optional implementation, the processing of the global critic network embedded in GAT includes: Each region corresponds to an agent. In the GAT layer, each agent is a node in the graph. The feature vector of each node is composed of the state observation variable-action pair of the corresponding agent. The input of the GAT layer is the feature vector of all nodes. The attention coefficient of each node is calculated. : ; Use the attention coefficient to aggregate the neighboring nodes of node i to update the feature vector of node i; ; in, is a learnable weight matrix, is a learnable attention weight vector, || is a vector concatenation operation, represents transposition, LeakyReLU(.) is the activation function, is the set of neighbor nodes of node i; is the feature vector of node i, node j, and node k; σ(.) is the activation function, is the updated feature vector of node i; It is the state dimension after node aggregation; The updated feature vectors of all nodes are concatenated to obtain a global feature vector, which is passed through three fully connected layers to generate a global state-action value.

[0012] As an optional implementation, during training, the loss function for updating the parameters of the global critic network embedded in GAT is for: ; Among them, the target value for: ; Where γ is the discount factor, is the global critic network embedded in GAT for region i The target network, The parameters are , The update is updated using the soft update mechanism. The parameters are , is the target network of the actor network in region i, The parameters are , min is the minimum value operation; is the state observation variable of all regions; For all areas of action; A global reward for all regions; is the global reward of region i; is the global state observation variable at the next moment of sampling; D is the experience replay buffer pool; is the state observation variable of the sampling area i at the next moment; E is the expected operation function; is the action computed by the target actor network for region i.

[0013] As an alternative implementation, the loss function for updating the local critic network parameters is for: ; Among them, the target value for: ; in, For the local critic network The target network, The parameters are , The update is updated using the soft update mechanism. The parameters are ; Local rewards for all regions; is the state observation variable of region i; is the action of region i; is the local reward of region i.

[0014] As an alternative implementation, through the policy gradient function Update the parameters of the actor network; ; The first part on the right side of the equation is the global policy gradient calculated by the global critic network embedded in GAT, and the second part is the local policy gradient calculated by the local critic network; is the state observation variable of all regions; is the action of all areas; D is the experience replay buffer pool; For actor networks Parameters, The update is updated using the soft update mechanism; represents the gradient; is the state observation variable of region i; is the action of region i; is the first global critic network embedded in GAT, N is the total number of regions; is a local critic network.

[0015] In a second aspect, the present invention provides a central air conditioning control system, comprising: A control model building module is configured to build a central air-conditioning system control model including N areas, including state observation variables defined by indoor and outdoor temperature and humidity of each area, actions defined by cooling temperature set points and heating temperature set points of each area, and rewards defined by total energy consumption of the central air-conditioning system and regional thermal comfort; The optimization control model construction module is configured to construct an optimization control model composed of an actor network, a global critic network embedded in GAT, and a local critic network; wherein the actor network takes the state observation variables of a single region as input and outputs the control action for the central air-conditioning system, the global critic network embedded in GAT takes the state observation variables and actions of all regions as input and outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system, and the local critic network takes the state observation variables and actions of a single region as input and outputs the local state-action value corresponding to the regional thermal comfort reward; The control module is configured to train the optimization control model, and control the cooling temperature setting value and the heating temperature setting value of each area of ​​the central air-conditioning system according to the current state observation variables of each area with the trained actor network.

[0016] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0018] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a central air conditioning control method, specifically a multi-agent deep reinforcement learning central air conditioning control method based on decomposition critic network and graph attention network, which is used to achieve efficient energy saving and thermal comfort control of central air conditioning systems in commercial buildings. It has the following advantages: (1) Efficiency: By designing the critic network in multi-agent reinforcement learning as a global critic network and a local critic network, the conflict between the regional thermal comfort reward and the total energy consumption reward of the central air-conditioning system during the training process can be effectively alleviated, eliminating unstable strategies. At the same time, by integrating the graph attention network into the global critic network, the coupling information within the system can be fully extracted, thereby improving the optimization control performance.

[0020] (2) Energy saving: It can achieve energy-saving control of central air-conditioning systems while ensuring the thermal comfort of users of central air-conditioning systems in commercial buildings.

[0021] (3) Real-time: The trained actor network only needs to be deployed in the central air-conditioning system of a commercial building. Since the forward propagation speed of the actor network is extremely fast, real-time optimization control of the central air-conditioning system can be achieved.

[0022] The present invention proposes a central air-conditioning control method, system, device, medium and program product. First, the optimization control problem of the central air-conditioning system of a commercial building is modeled as a Markov game, and the corresponding state, action and reward are constructed. Secondly, the actor and critic network architecture of multi-agent reinforcement learning is designed, wherein the actor network takes the state observation of a single agent as input and outputs the control action for the central air-conditioning system. In order to solve the conflict problem between the regional thermal comfort reward and the total energy consumption reward of the central air-conditioning system during the training process, the critic network is designed as a global critic network and a local critic network. The input of the global critic network is the state observation and action of all agents, and the output is the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system. The local critic network takes the state observation variables and actions of a single agent as input, and the output is the local state-action value corresponding to the regional thermal comfort reward. At the same time, in order to fully model and learn the energy consumption coupling and thermal coupling relationship in the central air-conditioning system of a commercial building, GAT is embedded in the global critic network to improve the optimization control performance.

[0023] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0025] Figure 1 This is a flow chart of the central air conditioning control method provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0028] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "comprise" and any variation are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0030] Example 1 This embodiment proposes a central air conditioning control method, specifically a multi-agent deep reinforcement learning central air conditioning control method based on a decomposed critic network and a graph attention network. Figure 1 As shown, including: Construct a central air-conditioning system control model for N areas, including state observation variables defined by the indoor and outdoor temperature and humidity of each area, actions defined by the cooling and heating temperature set points of each area, and rewards defined by the total energy consumption of the central air-conditioning system and the regional thermal comfort; An optimization control model consisting of an actor network, a global critic network embedded in GAT, and a local critic network is constructed; the actor network takes the state observation variables of a single region as input and outputs the control action for the central air-conditioning system; the global critic network embedded in GAT takes the state observation variables and actions of all regions as input and outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system; the local critic network takes the state observation variables and actions of a single region as input and outputs the local state-action value corresponding to the regional thermal comfort reward; The optimization control model is trained, and the trained actor network is used to control the cooling temperature set value and heating temperature set value of each area of ​​the central air-conditioning system according to the current state observation variables of each area.

[0031] In this embodiment, the method is aimed at a typical central air-conditioning system of a commercial building with N areas. The optimization control problem of the central air-conditioning system of a commercial building with N areas is modeled as a Markov Game (MG) containing N agents. At the same time, the state observation variables are defined by the indoor and outdoor temperature and humidity of each area (agent), the actions are defined by the cooling temperature set value and the heating temperature set value of each area (agent), and the rewards are defined by the total energy consumption of the central air-conditioning system and the regional thermal comfort.

[0032] In this embodiment, the process of constructing MG is specifically as follows: The optimization control problem of a central air-conditioning system in a commercial building with N zones is modeled as a MG consisting of N agents, where each zone i corresponds to an agent i.

[0033] MG consists of the following components: a set of global states , the set of actions for each agent , and the set of local state observations of each agent Each agent i uses a random strategy To select actions, the strategy is based on the state transition function Generate the next global state, each agent i also has a reward .

[0034] In this embodiment, the reward is divided into two parts: global and local. Determined by the global state and actions of all agents, local rewards It only depends on the local state observation and action of agent i. Since MADRL does not require information of the state transition function, this embodiment mainly focuses on designing the state, action and reward of MG.

[0035] MG's status, actions and rewards are as follows: (1) Status: For region i ( ), whose corresponding local observation of agent i at time t Defined as: (1); Where N is the total number of regions; is the index for the day of the week, is the index of the hour of the day, Index of the time of day (its value depends on the selected interval, for example, if the interval is 15 minutes, then ); is the outdoor air temperature; is the relative humidity of outdoor air; is the number of people occupying area i; is the indoor air temperature; is the relative humidity of indoor air; is the heating temperature setting value; is the cooling temperature setting value; is the total energy consumption of the central air-conditioning system.

[0036] Finally, the local state observations of all agents are expressed as , this embodiment selects As global state .

[0037] (2) Action: Agent i at time Action Defined as: (2); in, For incremental adjustments to the heating temperature setpoint, For incremental adjustment of the cooling temperature set point, the incremental adjustment range is limited to Inside.

[0038] In addition, the heating temperature setting value and cooling temperature setpoint Restricted to range, and the heating temperature setting value is always lower than the cooling temperature setting value. Finally, the actions of all agents are expressed as .

[0039] (3) Rewards: The role of rewards is to provide immediate feedback for the actions of the agent. For the optimization control problem of the central air-conditioning system in a commercial building, the reward function must balance two key indicators: energy consumption and thermal comfort. As the energy consumption index, the predicted percentage of dissatisfied (PPD) is used as the thermal comfort evaluation index.

[0040] In order to eliminate unit differences, the total energy consumption and PPD of the central air-conditioning system are normalized. At the moment Rewards Defined as: (3); in, is the normalized PPD; and are adjustable weights used to balance energy consumption and thermal comfort indicators according to the optimization goal; Indicates the occupancy status of area i at time t, where unoccupied is 0 and occupied is 1.

[0041] To ensure that the PPD value remains within the thermal comfort threshold range of the ASHRAE standard (ASHRAE standard is the abbreviation of the American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., translated as the American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.), the normalized thermal comfort bonus Redefine as: (4); in, is the thermal comfort threshold; is the PPD of region i at time t.

[0042] During MADRL training, the PPD reward focuses on the thermal comfort of a single area, while the total energy consumption reward of the central air conditioning system reflects the collective behavior of all areas. This entanglement between local rewards and global rewards may lead to unstable strategies. To alleviate this problem, this embodiment will reward Decomposed into global rewards and local rewards ; (5); (6); Finally, the global rewards and local rewards of all agents are expressed as and .

[0043] In this embodiment, an actor and critic network architecture of multi-agent reinforcement learning is constructed. The actor network takes the state observation of the regional agent as input and outputs the optimal control action for the central air-conditioning system. At the same time, in order to solve the entanglement problem between the regional thermal comfort reward and the total energy consumption reward of the central air-conditioning system during the training process, the critic network is designed as a global critic network and a local critic network. The global critic network takes the state observation and action of all regional agents as input and outputs the global state-action value corresponding to the total energy consumption of the central air-conditioning system. The local critic network takes the state observation and action of each regional agent as input and outputs the local state-action value related to the evaluation of regional thermal comfort. In addition, in order to fully model and learn the energy consumption coupling and thermal coupling in the central air-conditioning system of commercial buildings, GAT is embedded in the global critic network to improve the optimization control performance.

[0044] Specific: Each agent has two GAT-based global critic networks, a local critic network, an actor network and its corresponding target network.

[0045] Specifically, for agent i, the actor network is represented as , the parameters are , input local state observation , output action , output action As input to the local critic network and the global critic network, perform the output action The final state observation is used as the input of the local critic network and the global critic network.

[0046] The local critic network of agent i is represented as , the parameters are , with the local state observation of agent i and actions As input, the local state-action value corresponding to the local reward (regional thermal comfort reward) is calculated; among them, the local state observation input to the local critic network is sampled from the experience replay pool, and a set of data in the experience replay pool is composed of the state observations obtained after executing the action output by the actor network.

[0047] Two GAT-based global critic networks for agent i and The parameters are and , and then take the minimum value of the output of these two networks during training, as shown in formula (15); these two networks observe the state of all agents and actions As input, the global state-action value corresponding to the global reward (the total energy consumption reward of the central air-conditioning system) is calculated.

[0048] Specifically, in order to fully model and learn the energy coupling and thermal coupling relationships within the central air-conditioning system of commercial buildings, the GAT-based global critic network consists of one GAT layer and three fully connected layers.

[0049] For the intelligent agent , its global critic network is as follows (7): (7); in, is the GAT layer, represents the three fully connected layers applied after the GAT layer; A global critic network embedded in GAT Parameters.

[0050] In the GAT layer, each agent is regarded as a node in the graph, and its feature vector consists of the state-action pair of the corresponding agent. The input of the GAT layer includes the feature vectors of all nodes, expressed as formula (8): (8); in, is the feature vector of node i, is the dimension of the feature vector, and dim(.) is the operation to obtain the dimension of the vector.

[0051] GAT uses the attention mechanism to assign different importance to the neighboring nodes of each node, thereby modeling and learning the interaction between agents. Calculation is as follows (9): (9); in, is a learnable weight matrix used to transform node feature vectors, is a learnable attention weight vector, || represents a vector concatenation operation, represents transposition, LeakyReLU(.) is the activation function, Represents the set of neighbor nodes of node i; is the feature vector of node i, node j, and node k; It is the state dimension after node aggregation.

[0052] Afterwards, the attention coefficient is used to aggregate the neighbor node information of node i to update the feature vector of node i, which is calculated as formula (10): (10); Among them, σ(.) is the activation function, j represents the jth neighbor node, is the feature vector of node j, is the updated feature vector of node i.

[0053] After updating the feature vectors of all nodes, concatenate them into a global feature vector H: (11).

[0054] Finally, the global feature vector is passed through three fully connected layers to generate the final global state-action value: (12).

[0055] It can be understood that the actor network takes the observation variables of a single agent as input and outputs the optimal control actions for the central air-conditioning system, and the local critic network takes the observation variables and actions of a single agent as input and outputs the local state-action values corresponding to the regional thermal comfort reward. The processing procedures of the two networks belong to the conventional processing procedures of the actor network and the critic network, which will not be elaborated here.

[0056] In this embodiment, the agent interacts with the simulation environment of the commercial building central air-conditioning system a large number of times, collects training data and trains the actor network, the global critic network and the local critic network.

[0057] At time during the training process, agent i obtains the local state observation and generates an action according to its actor network as follows: (13); where is the Gaussian noise at time

[0058] Subsequently, agent i receives the global reward and the local reward and obtains the local state observation at the next time. When all N agents complete this cycle, the multi-agent system stores the experience tuple in the experience replay buffer D.

[0059] When the number of experiences in the experience replay buffer D reaches the threshold, a mini-batch is randomly sampled from D to train the actor network and the critic network; where m is the index subscript of the experience data; K is the number of experiences taken out of the experience replay pool each time.

[0060] Specifically, the parameters of the global critic network are updated using the loss function shown in Equation (14) : (14).

[0061] where the target value is defined as: (15); where γ is the discount factor, is the target network of the global critic network of the embedded GAT of agent i and the parameters of are , The parameter of , is the target network of the actor network of agent i, The parameter of ; represents sampling from the experience replay buffer D; is the state observation variable of all regions; is the action of all regions; is the global reward of all regions obtained by sampling; is the global reward of agent i; is the global state observation variable obtained by sampling The corresponding global state observation variable at the next moment; D is the experience replay buffer; is the state observation variable of agent i at the next moment obtained by sampling; min is the minimum operation, applying this operation to the target networks of the two global critic networks can effectively alleviate the problem of overestimation of Q values; E is the expectation operation function; is the action calculated by the target actor network of agent i.

[0062] By implementing a delayed policy update mechanism, the dependence of the actor network on the critic network is significantly reduced. For this purpose, this embodiment selects to delay the update of the local critic network and the actor network every T steps.

[0063] Similarly, based on the loss function shown in Equation (16) Update the parameters of the local critic network of agent i: (16).

[0064] Among them, the target value is defined as: (17); Among them, is the target network of the local critic network , The parameter of , The parameter of ; is the local reward of all regions; is the state observation variable of region i; is the action of region i; is the local reward of region i.

[0065] Through the policy gradient function shown in Equation (18) Update the parameters of the actor network of agent i.

[0066] (18); Wherein, is the state observation variable of all regions; are the actions of all regions; D is the experience replay buffer pool; is the actor network parameters; represents the gradient; is the state observation variable of region i; is the action of region i; is the global critic network of the first embedded GAT, and N is the total number of regions; is the local critic network.

[0067] Equation (18) contains two parts: the first part is the global policy gradient calculated by the global critic network, and the second part is the local policy gradient calculated by the local critic network. Through this decomposed policy gradient, the problem of policy instability caused by the entanglement of global and local reward signals can be effectively alleviated.

[0068] In addition, the respective target networks of agent i are updated through the following soft update mechanism to obtain the parameters of the updated respective target networks , , : (19); (20); (21); Wherein, is the soft update coefficient, which is used to adjust the update rate of the target network.

[0069] In this embodiment, after training is completed, the parameters of the actor network model are fixed, and the actor network is deployed in the central air-conditioning system of commercial buildings, and real-time optimal control can be achieved.

[0070] In this embodiment, first, the optimization control problem of the central air-conditioning system in commercial buildings is modeled as a Markov game, and the corresponding states, actions, and rewards are constructed. Secondly, the actor and critic network architectures of multi-agent reinforcement learning are designed. The actor network takes the state observation of a single agent as input and outputs the control actions for the central air-conditioning system. To solve the conflict problem between the regional thermal comfort reward and the total energy consumption reward of the central air-conditioning system during the training process, the critic network is designed as a global critic network and a local critic network. The input of the global critic network is the state observations and actions of all agents, and it outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system. The local critic network takes the state observation variables and actions of a single agent as input and outputs the local state-action value corresponding to the regional thermal comfort reward. At the same time, in order to fully model and learn the energy consumption coupling and thermal coupling relationships in the central air-conditioning system of commercial buildings, a GAT is embedded in the global critic network to improve the optimization control performance. Finally, through a large number of interactions with the simulation environment of the central air-conditioning system in commercial buildings, the actor network, the global and local critic networks, and their target networks are trained, and the trained actor network is deployed to achieve the real-time optimization control of the central air-conditioning system in commercial buildings.

[0071] Embodiment 2 This embodiment provides a central air-conditioning control system, including: A control model construction module, configured to construct a control model for the central air-conditioning system including N regions, including state observation variables defined by the indoor and outdoor temperature and humidity of each region, actions defined by the cooling temperature set value and heating temperature set value of each region, and rewards defined by the total energy consumption of the central air-conditioning system and regional thermal comfort; An optimization control model construction module, configured to construct an optimization control model composed of an actor network, a global critic network embedded with a GAT, and a local critic network; wherein, the actor network takes the state observation variables of a single region as input and outputs the control actions for the central air-conditioning system. The global critic network embedded with a GAT takes the state observation variables and actions of all regions as input and outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system. The local critic network takes the state observation variables and actions of a single region as input and outputs the local state-action value corresponding to the regional thermal comfort reward; A control module, configured to train the optimization control model, and use the trained actor network to control the cooling temperature set value and heating temperature set value of each region of the central air-conditioning system according to the current state observation variables of each region.

[0072] It should be noted here that the above modules correspond to the steps described in Embodiment 1. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0073] In more embodiments, there is also provided: An electronic device, including a memory and a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the method described in Embodiment 1 is completed. For the sake of brevity, it will not be elaborated here.

[0074] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0075] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0076] A computer-readable storage medium for storing computer instructions, which when executed by the processor, complete the method described in Embodiment 1.

[0077] The method in Embodiment 1 can be directly embodied as being executed and completed by a hardware processor, or by a combination of hardware and software modules in the processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0078] A computer program product, including a computer program, which when executed by the processor, implements the method described in Embodiment 1.

[0079] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which are executed in a device on a target real or virtual processor to perform the processes / methods described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided as needed among the program modules. The machine-executable instructions for the program modules can be executed within local or distributed devices. In a distributed device, the program modules can be located in local and remote storage media.

[0080] The computer program code for implementing the method of the present invention can be written in one or more programming languages. This computer program code can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program code is executed by the computer or other programmable data processing device, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the computer, partially on the computer, as a stand-alone software package, partially on the computer and partially on a remote computer, or entirely on a remote computer or server.

[0081] In the context of the present invention, the computer program code or related data can be carried by any suitable carrier so that a device, apparatus, or processor can perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals can include electrical, optical, radio, sound, or other forms of propagated signals, such as carrier waves, infrared signals, etc.

[0082] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with this embodiment can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0083] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that based on the technical solution of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. A central air conditioning control method, characterized in that: include: Construct a central air-conditioning system control model including N areas, including state observation variables defined by the indoor and outdoor temperature and humidity of each area, actions defined by the cooling temperature setpoint and heating temperature setpoint of each area, and rewards defined by the total energy consumption of the central air-conditioning system and the regional thermal comfort; An optimization control model consisting of an actor network, a global critic network embedded in GAT, and a local critic network is constructed; the actor network takes the state observation variables of a single region as input and outputs the control action for the central air-conditioning system; the global critic network embedded in GAT takes the state observation variables and actions of all regions as input and outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system; the local critic network takes the state observation variables and actions of a single region as input and outputs the local state-action value corresponding to the regional thermal comfort reward; The optimization control model is trained, and the trained actor network is used to control the cooling temperature set value and heating temperature set value of each area of ​​the central air-conditioning system according to the current state observation variables of each area.

2. A central air conditioning control method as claimed in claim 1, characterized in that: The state observation variables include outdoor air temperature, outdoor air relative humidity, number of occupants in each area, indoor air temperature, indoor air relative humidity, heating temperature setpoint, cooling temperature setpoint, and total energy consumption of the central air conditioning system; The action includes incremental adjustment of the heating temperature setting value and the cooling temperature setting value, setting the range of the incremental adjustment, and the value range of the heating temperature setting value and the cooling temperature setting value, and the heating temperature setting value is lower than the cooling temperature setting value; Rewards include global rewards and local rewards ; ； ； ； in, and It is an adjustable weight; is the total energy consumption of the central air conditioning system; is the thermal comfort bonus of area i at time t, is the occupancy status of area i at time t, where unoccupied is 0 and occupied is 1; is the thermal comfort threshold; is the predicted dissatisfaction percentage of region i at time t.

3. A central air conditioning control method as claimed in claim 1, characterized in that: The global critic network embedded with GAT consists of a GAT layer and three fully connected layers after the GAT layer. ;in, is the GAT layer, It is a three-layer fully connected layer; is the state observation variable of all regions; For all areas of action; A global critic network embedded in GAT Parameters.

4. A central air conditioning control method as claimed in claim 3, characterized in that: The processing of the global critic network embedded in GAT includes: Each region corresponds to an agent. In the GAT layer, each agent is a node in the graph. The feature vector of each node is composed of the state observation variable-action pair of the corresponding agent. The input of the GAT layer is the feature vector of all nodes. The attention coefficient of each node is calculated. : ； Use the attention coefficient to aggregate the neighboring nodes of node i to update the feature vector of node i; ； in, is a learnable weight matrix, is a learnable attention weight vector, || is a vector concatenation operation, represents transposition, LeakyReLU(.) is the activation function, is the set of neighbor nodes of node i; is the feature vector of node i, node j, and node k; σ(.) is the activation function, is the updated feature vector of node i; It is the state dimension after node aggregation; The updated feature vectors of all nodes are concatenated to obtain a global feature vector, which is passed through three fully connected layers to generate a global state-action value.

5. A central air conditioning control method as claimed in claim 1, characterized in that: During training, the loss function for updating the parameters of the global critic network embedded in GAT is for: ； Among them, the target value for: ； Where γ is the discount factor, is the global critic network embedded in GAT for region i The target network, The parameters are , The update is updated using the soft update mechanism. The parameters are , is the target network of the actor network in region i, The parameters are , min is the minimum value operation; is the state observation variable of all regions; For all areas of action; A global reward for all regions; is the global reward of region i; is the global state observation variable at the next moment of sampling; D is the experience replay buffer pool; is the state observation variable of the sampling area i at the next moment; E is the expected operation function; is the action calculated by the target actor network in region i; Loss function for updating local critic network parameters for: ； Among them, the target value for: ； in, For the local critic network The target network, The parameters are , The update is updated using the soft update mechanism. The parameters are ; Local rewards for all regions; is the state observation variable of region i; is the action of region i; is the local reward of region i.

6. A central air conditioning control method as claimed in claim 1, characterized in that: Through the policy gradient function Update the parameters of the actor network; ； The first part on the right side of the equation is the global policy gradient calculated by the global critic network embedded in GAT, and the second part is the local policy gradient calculated by the local critic network; is the state observation variable of all regions; is the action of all areas; D is the experience replay buffer pool; For actor networks Parameters, The update is updated using the soft update mechanism; represents the gradient; is the state observation variable of region i; is the action of region i; is the first global critic network embedded in GAT, N is the total number of regions; is a local critic network.

7. A central air conditioning control system, characterized in that: include: A control model building module is configured to build a central air-conditioning system control model including N areas, including state observation variables defined by indoor and outdoor temperature and humidity of each area, actions defined by cooling temperature set points and heating temperature set points of each area, and rewards defined by total energy consumption of the central air-conditioning system and regional thermal comfort; The optimization control model construction module is configured to construct an optimization control model composed of an actor network, a global critic network embedded in GAT, and a local critic network; wherein the actor network takes the state observation variables of a single region as input and outputs the control action for the central air-conditioning system, the global critic network embedded in GAT takes the state observation variables and actions of all regions as input and outputs the global state-action value corresponding to the total energy consumption reward of the central air-conditioning system, and the local critic network takes the state observation variables and actions of a single region as input and outputs the local state-action value corresponding to the regional thermal comfort reward; The control module is configured to train the optimization control model, and control the cooling temperature setting value and the heating temperature setting value of each area of ​​the central air-conditioning system according to the current state observation variables of each area with the trained actor network.

8. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images