Building group-virtual power plant collaborative regulation method considering user thermal comfort elasticity

Abstract

This invention discloses a building cluster-virtual power plant collaborative control method considering the elasticity of user thermal comfort, belonging to the field of smart grid and distributed energy management technology. The method includes: collecting frequency regulation / peak shaving commands from the power grid dispatch center and operating status parameters of the building cluster; constructing a thermodynamic-electrical power coupling model of the building cluster to generate an initial control strategy; using reinforcement learning to collaboratively optimize the initial control strategy with the goal of minimizing power grid command tracking error and the user thermal comfort elasticity range as a constraint, generating a global optimization strategy; validating the global optimization strategy according to the thermal comfort elasticity range; and converting the validated global optimization strategy into control commands executable by building equipment. This invention, through a centralized training and distributed execution mechanism, ensures collaborative optimization that tracks power grid commands and respects overall comfort, effectively improving the capacity and efficiency of the building cluster in power grid control while prioritizing user experience.

Description

Technical Field

[0001] This invention relates to the field of smart grid and distributed energy management technology, and in particular to a building cluster-virtual power plant collaborative control method that takes into account the flexibility of user thermal comfort. Background Technology

[0002] With the increasing proportion of renewable energy, especially the volatility of wind and solar power, the frequency and voltage fluctuations of the power grid are becoming increasingly severe, leading to a growing demand for grid regulation resources. Against this backdrop, building air conditioning systems are widely used as a flexible load regulation resource; however, traditional building air conditioning load regulation methods still face several challenges. Existing technologies typically employ traditional start-stop control or temperature step adjustment methods. While these methods are simple and easy to implement, they cannot achieve precise temperature control and are prone to causing significant temperature fluctuations during load regulation, affecting user thermal comfort.

[0003] Furthermore, existing methods typically involve independent control of air conditioning in individual buildings, lacking a coordinated control mechanism for the entire building complex. The participation efficiency of a single building in grid frequency regulation and peak shaving is limited, especially in scenarios involving coordinated control of multiple buildings. This results in the inability to fully utilize the thermal inertia and temperature control resources between buildings, leading to low system control efficiency and an inability to effectively balance the conflict between grid demand and user comfort. Simultaneously, existing technologies exhibit low control precision and slow response speed, making it difficult to meet the grid's frequency regulation and peak shaving requirements.

[0004] Therefore, traditional air conditioning load regulation methods have significant limitations in terms of resource utilization, user experience, and system efficiency, making it difficult to meet the higher regulation requirements of future power grids. Existing technologies have failed to fully consider how to improve the collaborative efficiency of building clusters participating in power grid regulation while ensuring user comfort, particularly in terms of refined regulation and rapid response. A more efficient, flexible, and precise building cluster regulation method is urgently needed. Summary of the Invention

[0005] Purpose of the invention: In view of the above problems, the purpose of this invention is to provide a building cluster-virtual power plant collaborative control method that takes into account the flexibility of user thermal comfort.

[0006] Technical solution: The building cluster-virtual power plant collaborative control method of the present invention, which considers the flexibility of user thermal comfort, includes the following steps:

[0007] Step 1: Collect frequency regulation / peak regulation commands from the power grid dispatch center and the operating status parameters of the building cluster in real time, and transmit them to the virtual power plant aggregation layer through communication and data interfaces;

[0008] Step 2: In the virtual power plant aggregation layer, construct a thermodynamic-electric power coupling model of the building cluster based on the received data, and generate an initial control strategy;

[0009] Step 3: With the goal of minimizing the power grid command tracking error and the user thermal comfort elasticity range as the constraint, reinforcement learning is used to collaboratively optimize the initial control strategy and generate a global optimization strategy.

[0010] Step 4: Validate the global optimization strategy based on the thermal comfort elasticity range. If the building temperature change caused by the strategy is within the preset elasticity range, proceed to Step 5; otherwise, return to Step 3 to re-optimize.

[0011] Step 5: Convert the verified global optimization strategy into control commands that can be executed by building equipment to drive load equipment, including the air conditioning system, to perform power regulation.

[0012] Step 6: Collect building status data and power grid response data in real time after regulation, calculate command tracking accuracy and comfort satisfaction rate, and detect system operation status. If the regulation effect does not meet expectations or an abnormal state occurs, immediately trigger the feedback mechanism and return the evaluation results to Step 3 for strategy iteration and optimization until the regulation target is met.

[0013] Further, step 2 includes:

[0014] A dynamic coupling relationship between indoor temperature changes and the power consumption of the air conditioning system is established, denoted as a thermodynamic-electrical power coupling model, to describe the influence of building thermal inertia on power regulation response. The mathematical expression is:

[0015] ,

[0016] In the formula, The predicted trajectory representing the total power of the cluster. express Time of the first Real-time power consumption of the building's air conditioning This indicates the number of buildings participating in the virtual power plant's aggregated control. express Time of the first Real-time cooling / heating capacity of the building's air conditioning system. express Time of the first The real-time performance coefficient of the air conditioner;

[0017] Generate an initial control strategy that meets the preliminary constraints of grid demand, denoted as... ,in, Represented as the first Indoor temperature trajectory of a building This represents the temperature adjustment amount set for the building's air conditioning system.

[0018] Furthermore, step 3 includes:

[0019] Step 301: Treat each building as an independent intelligent agent, denoted as... Introducing a graph attention mechanism into a centralized Critic value network;

[0020] Step 302: The frequency regulation / peak shaving demand targets issued by the power grid dispatch center, the real-time operating status of the building complex, and the user comfort flexibility range are spatiotemporally aligned to form a global status view. , is represented as:

[0021] ,

[0022] in, express The real-time frequency deviation at any given moment is the specific value of the frequency modulation signal; express The excitation signal at any given time is the specific value of the peak-shaving signal. Indicates including the first Standardized status data including indoor temperature of a building, user-set temperature, thermal comfort range, and real-time power of air conditioning.

[0023] Step 303, define the action space, denoted as ,

[0024] in, This indicates the total power reference target, including the total power reference target for frequency-modulated signals and the total power reference target for peak-modulated signals; Represents the cooperative control strategy function;

[0025] Step 304, define the adaptive reward function, the expression of which is:

[0026] ,

[0027] In the formula, For grid-side instruction tracking rewards, This is a penalty for indoor temperature deviating from the user's thermal comfort range. This is a penalty term for power regulation smoothness. , , These are weighting coefficients that are dynamically adjusted according to the power grid's operating status.

[0028] Step 305: Introduce an adversary modeling mechanism to update the Actor policy network of the building agent;

[0029] Step 306: Optimize the centralized Critic value network by minimizing temporal difference learning, and use the optimized centralized Critic value network to evaluate the long-term value of joint actions and obtain the policy functions of each building agent.

[0030] Step 307: With the goal of minimizing the power grid command tracking error, construct the objective function, expressed as:

[0031] ,

[0032] In the formula, This represents the actual aggregate power of the building complex. This is represented as a power grid frequency regulation / peak shaving reference command. This refers to the indoor temperature of the building. This indicates the temperature setting for the building. This represents the change in building air conditioning control parameters. , , Represented as weighting coefficients;

[0033] The constraints for the objective function are constructed as follows:

[0034] ,

[0035] In the formula, , These represent the upper and lower limits of the acceptable temperature for building users, respectively, with representing the highest acceptable temperature for building users. , These represent the upper and lower limits of the operating power of the building's air conditioning system. This represents the maximum temperature adjustment range allowed by the user.

[0036] The objective function value is calculated based on the policy function to obtain the optimal policy function for each building agent.

[0037] Furthermore, in step 301, a graph attention mechanism is introduced into the centralized Critic value network, including:

[0038] Treat each building as an individual intelligent agent node, and model the building cluster as a graph structure. ,in The set of building agents and the edge set represent the set of building agents. Indicates the relationships between buildings;

[0039] At time t, construct the node feature vector of the building. The node feature vector includes one or more of the following: indoor temperature, user-set temperature, thermal comfort elasticity range, and real-time air conditioning power.

[0040] For any node and its neighboring nodes Calculate attention score , is represented as:

[0041] ,

[0042] In the formula, Indicates neighboring buildings For the target building The original importance score; , This represents the current status information of the building. It is a trainable linear transformation matrix used to map the original features to a feature space that is more suitable for comparison and aggregation; This is a trainable attention parameter vector; For activation functions;

[0043] The degree of influence on other nodes in the neighborhood is calculated using attention scoring, and is expressed as follows:

[0044] ,

[0045] In the formula, Attention weights represent the weights used when updating building nodes. When representing neighboring building nodes For nodes The relative degree of influence; Indicates with buildings A collection of buildings that are related;

[0046] Using attention weights The fused representation is obtained by weighted summation of the features of neighboring nodes. Furthermore, the fusion representation is used as part of the input features of the centralized Critic value network to output the value Q of the joint state-action pair.

[0047] Further, step 306 includes:

[0048] When the policy network of the building agents is updated, the temperature regulation actions of other building agents are estimated through the adversary modeling network based on the real-time collected building group state S(t). and will the action of this intelligent agent Combined with the estimated action to form a joint action vector Input the value network Critic to compute policy gradients and update the Actor network parameters for each building;

[0049] The formula for calculating the policy gradient is as follows:

[0050] ,

[0051] In the formula, Represents the Actor network parameters The direction of the fine-tuning This represents the state of samples taken from the experience playback pool D. With joint actions Seeking expectations, Indicates the adjustment amount. Indicates the first Temperature control strategies for buildings Indicates the first The current status of the building.

[0052] Furthermore, step 306 also includes:

[0053] The Critic loss function is constructed with the goal of minimizing temporal difference learning, and its expression is:

[0054] ,

[0055] in, This represents the expectation of the state-action-reward-next state samples obtained from the experience replay pool. For the first A Critic network in parameters Below the joint state-action pair Predict the Q value; The target Q value is defined as:

[0056] ,

[0057] in, As a discount factor, Represented as the target network's next state-action pair Value prediction; For timely rewards, the expression is:

[0058] ,

[0059] in, This represents the grid-side performance bonus, used to track the aggregated power of the building complex in response to grid commands. This indicates the degree to which the building's indoor temperature deviates from the user's set comfort range. This is expressed as the real-time total power output relative to the controlled building air conditioning load. Indicates the first The building sets the temperature. This is expressed as the user-allowed temperature deviation value; Represented as power grid side weighting coefficient and comfort weighting coefficient;

[0060] A priority experience replay pool D is constructed, and experience samples are stored in the priority experience replay pool. Priorities are assigned based on the temporal difference error samples of the value network, and experience samples are sampled according to the sampling probability corresponding to the priority to update the value network parameters. The samples include:

[0061] ,

[0062] in, This indicates a combined state of the building complex. , This represents the reward for the combined actions output by the intelligent agents in each building.

[0063] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0064] 1. This invention constructs a virtual battery aggregation model for the frequency regulation and peak shaving needs of the power grid. By establishing the coupling relationship between the building thermodynamic equation and the power model, the power regulation potential brought about by the building's thermal inertia is accurately quantified. The MADDPG algorithm is used to dynamically decompose the power grid's frequency regulation signal and peak shaving command into temperature fine-tuning strategies for each building. Experimental results show that this model reduces the frequency regulation response time of the building cluster to the second level, improves the peak load regulation capability, and effectively solves the problems of slow response speed and limited regulation capacity of traditional control methods.

[0065] 2. This invention innovatively designs a multi-objective optimization function that balances the accuracy of power grid frequency regulation and peak shaving with user comfort. Through the collaborative optimization of the reward function, it ensures that the system can quickly track the power grid frequency regulation and peak shaving commands and strictly constrain temperature changes within the user-preset elastic range. This design enables the system to achieve rapid power response through temperature fine-tuning in frequency regulation scenarios and to smooth the load curve through pre-cooling / preheating strategies in peak shaving scenarios, thus achieving a perfect balance between dual power grid service objectives and user experience.

[0066] 3. This invention employs a centralized training-distributed execution architecture and a closed-loop security verification mechanism to work together, significantly improving the reliability of frequency regulation and peak shaving services. Through pre-verification of the thermal comfort elastic range determination module and in-process control of command smoothing, it ensures both the rapid response characteristics required for frequency regulation and keeps the rate of temperature change within a range imperceptible to the user. Combined with real-time feedback optimization, the system ensures that the indoor temperature is always maintained within the comfortable range while continuously providing frequency regulation and peak shaving services, effectively solving the safety and stability problems when flexible loads participate in grid regulation. Attached Figure Description

[0067] Figure 1 This is a flowchart of the present invention;

[0068] Figure 2 This is a flowchart of the present invention;

[0069] Figure 3 A performance comparison chart of the traditional control system and the present invention in tracking power grid demand;

[0070] Figure 4 This is a comparison chart showing the performance improvement of the system of the present invention;

[0071] Figure 5 This is a comparison chart of the key performance indicators of the traditional control system and the present invention. Detailed Implementation

[0072] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the embodiments of the present invention, and not all structures.

[0073] In the following description, specific details such as target system architecture and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0074] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0075] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0076] Furthermore, in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0077] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include the target features, structures, or characteristics described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0078] Combination Figure 1 and Figure 2 As shown in this embodiment, the building cluster-virtual power plant collaborative control method considering user thermal comfort flexibility includes the following steps:

[0079] Step 1: Collect frequency regulation / peak regulation commands from the power grid dispatch center and the operating status parameters of the building cluster in real time, and transmit them to the virtual power plant aggregation layer through communication and data interfaces.

[0080] In one example, the power grid dispatch center performs frequency regulation and peak shaving, where the frequency regulation signal is related to the frequency deviation, as shown below:

[0081] ,

[0082] In the formula, Indicating frequency deviation is the core content of frequency modulation signals sent down from the power grid level; This represents the frequency actually measured instantaneously by the power grid. Indicates the rated frequency of the power grid;

[0083] when The time indicates that there is an overcapacity in power generation and the frequency is too high. The power grid needs to increase its load or reduce power generation. The virtual power plant should instruct the building complex to appropriately increase its power consumption. This indicates insufficient power generation, low frequency, and the need for the power grid to reduce load or increase power generation.

[0084] By utilizing the instantaneous power response characteristics of air conditioning loads to changes in set temperature, the building complex is transformed into a giant virtual battery, providing frequency regulation to the power grid by increasing or decreasing the load.

[0085] The peak-shaving signal is related to time, predicted load, and price, and its formula is as follows:

[0086] ,

[0087] In the formula, Indicates time The excitation signal is the specific value of the peak-shaving signal. Indicates the price coefficient. Indicates time Forecasted load, This indicates the baseline load level; any load exceeding this level requires peak load reduction.

[0088] By utilizing the building's thermal energy storage characteristics, peak grid loads are shifted to off-peak periods through pre-cooling or pre-heating, smoothing the daily load curve. All controls are limited within the user-defined thermal comfort range.

[0089] Simultaneously, operating parameters such as indoor temperature, set temperature, comfort zone boundary value, air conditioning operating power and energy efficiency ratio of the building complex are collected to form a complete data package characterizing the building's thermal dynamics and regulation potential.

[0090] The frequency regulation demand of the power grid is dynamically matched with the real-time adjustable capacity of the building complex to generate standardized data units, which are heterogeneous data.

[0091] Heterogeneous data is validated, filtered, and unified to form a high-quality standardized data pool. Key parameters such as temperature, temperature range, and power are extracted to construct a core state vector, denoted as:

[0092] ,

[0093] in, Indicates the current indoor temperature; This indicates the user-defined base temperature; This indicates the set range of thermal comfort flexibility. This represents the current instantaneous power of the air conditioning system.

[0094] The frequency regulation and peak shaving signals issued by the power grid dispatch center are converted into specific total power targets that can be executed by the building cluster:

[0095] The conversion formula for frequency modulation signals is:

[0096] ,

[0097] In the formula, This represents the total power that needs to be tracked in the building cluster; It is represented as the proportional gain coefficient.

[0098] The formula for peak-shaving signal conversion is as follows:

[0099]

[0100] In the formula, This represents the total power that the building cluster should achieve. This is represented as the baseline load. Represented as Forecasted load at any given time.

[0101] The downlink commands and uplink data are spatiotemporally aligned and correlated to form a global state view. , is represented as:

[0102]

[0103] In the formula, Representing the Standardized status data for each building.

[0104] Step 2: In the virtual power plant aggregation layer, construct a thermodynamic-electric power coupling model of the building cluster based on the received data, and generate an initial control strategy.

[0105] Further, step 2 includes:

[0106] The building cluster aggregation model, i.e., the thermodynamic model of a single building, is established as follows:

[0107] ,

[0108] In the formula, Indicates equivalent heat capacity, Indicates time Indoor temperature, Indicates time The outdoor temperature; Represented as equivalent thermal resistance, Indicates time The cooling / heating power generated by the air conditioning system This represents indoor thermal disturbance, including heat generated by people, equipment, and solar radiation. This thermodynamic model shows that due to thermal inertia, adjusting the air conditioning power will not cause an instantaneous change in indoor temperature, but rather a slow adjustment within a comfortable range.

[0109] The electrical model of an air conditioning system, that is, influencing the temperature by controlling the setpoint. Ultimately, the power changes linearly, expressed as:

[0110] ,

[0111] In the formula, Represented as time Air conditioner power consumption Expressed as energy efficiency ratio;

[0112] A dynamic coupling relationship between indoor temperature changes and the power consumption of the air conditioning system is established, denoted as a thermodynamic-electrical power coupling model, to describe the influence of building thermal inertia on power regulation response. The mathematical expression is:

[0113] ,

[0114] In the formula, The predicted trajectory representing the total power of the cluster. express Time of the first Real-time power consumption of the building's air conditioning This indicates the number of buildings participating in the virtual power plant's aggregated control. express Time of the first Real-time cooling / heating capacity of the building's air conditioning system. express Time of the first The real-time performance coefficient of the air conditioner;

[0115] When new parameters for each building actor network are obtained, the required power is calculated based on the new set temperature. Simultaneously, this aggregation formula is used to sum the results, yielding a predicted trajectory of the cluster's total power over a future period. ;

[0116] Through the global state view and total power reference target Based on this, a coordinated control strategy is calculated, and an initial control strategy that meets the preliminary constraints of grid demand is generated, denoted as... ,in, Represented as the first Indoor temperature trajectory of a building This represents the temperature adjustment amount set for the building's air conditioning system.

[0117] Step 3: With the goal of minimizing the power grid command tracking error and the user thermal comfort elasticity range as the constraint, reinforcement learning is used to perform collaborative optimization of the initial control strategy (denoted as the MADDPG algorithm) to generate a global optimization strategy.

[0118] Through a centralized training and decentralized execution mechanism, the temperature regulation strategies of each building are iteratively optimized, ultimately generating a global optimization strategy that meets the constraints of power grid command tracking accuracy and user thermal comfort.

[0119] Furthermore, step 3 includes:

[0120] Step 301: Treat each building as an independent intelligent agent, denoted as... Introducing a graph attention mechanism into a centralized Critic value network;

[0121] Step 302: The frequency regulation / peak shaving demand targets issued by the power grid dispatch center, the real-time operating status of the building complex, and the user comfort flexibility range are spatiotemporally aligned to form a global status view. , is represented as:

[0122] ,

[0123] in, express The real-time frequency deviation at any given moment is the specific value of the frequency modulation signal; express The excitation signal at any given time is the specific value of the peak-shaving signal. Indicates including the first Standardized status data including indoor temperature of a building, user-set temperature, thermal comfort range, and real-time power of air conditioning.

[0124] Step 303, define the action space, denoted as ,

[0125] in, This indicates the total power reference target, including the total power reference target for frequency-modulated signals and the total power reference target for peak-modulated signals; Represents the cooperative control strategy function;

[0126] Regarding the cooperative control strategy function It is used to establish the mapping relationship between the status of the building group / the observed control actions of each building. At time... The control actions of the building intelligent agent are represented as follows:

[0127] ,

[0128] in, For local state observation of building intelligent agents, For the corresponding Actor network parameters, The change range of building air conditioning control quantity; the control actions output by all building intelligent agents together constitute a joint action, which is used to achieve coordinated tracking of the total power reference target;

[0129] Step 304, define the adaptive reward function, the expression of which is:

[0130] ,

[0131] In the formula, For grid-side instruction tracking rewards, This is a penalty for indoor temperature deviating from the user's thermal comfort range. This is a penalty term for power regulation smoothness. , , These are weighting coefficients that are dynamically adjusted according to the power grid's operating status.

[0132] Step 305: Introduce an adversary modeling mechanism to update the Actor policy network of the building agent;

[0133] Step 306: Optimize the centralized Critic value network by minimizing temporal difference learning, and use the optimized centralized Critic value network to evaluate the long-term value of joint actions and obtain the policy functions of each building agent.

[0134] Step 307: Construct an objective function with the goal of minimizing grid command tracking error. This objective function is used to unify grid command tracking and user comfort / equipment availability under the same optimization criterion when building clusters participate in virtual power plant control, thus penalizing the aggregated power of the building clusters. With grid reference power The deviation is designed to ensure the building complex responds as accurately as possible to frequency modulation / peak shaving commands; and to penalize the indoor temperature of each building. Deviation from user-set temperature Used to constrain the impact on thermal comfort; penalizing changes in control quantity. The range of the setpoint / power is adjusted to avoid frequent and large fluctuations, thereby improving execution smoothness and device friendliness; the expression for this objective function is:

[0135] ,

[0136] In the formula, This represents the actual aggregate power of the building complex. This is represented as a power grid frequency regulation / peak shaving reference command. This refers to the indoor temperature of the building. This indicates the temperature setting for the building. This represents the change in building air conditioning control parameters. , , Represented as weighting coefficients;

[0137] The constraints for the objective function are constructed as follows:

[0138] ,

[0139] In the formula, , These represent the upper and lower limits of the acceptable temperature for building users, respectively, with representing the highest acceptable temperature for building users. , These represent the upper and lower limits of the operating power of the building's air conditioning system. This represents the maximum temperature adjustment range allowed by the user.

[0140] The objective function value is calculated based on the policy function to obtain the optimal policy function for each building agent.

[0141] Furthermore, in step 301, a graph attention mechanism is introduced into the centralized Critic value network, including:

[0142] Treat each building as an individual intelligent agent node, and model the building cluster as a graph structure. ,in The set of building agents and the edge set represent the set of building agents. Indicates the relationships between buildings;

[0143] At any moment Construct the node feature vectors of the building. The node feature vector includes one or more of the following: indoor temperature, user-set temperature, thermal comfort elasticity range, and real-time air conditioning power.

[0144] For any node and its neighboring nodes Calculate attention score , is represented as:

[0145] ,

[0146] In the formula, Indicates neighboring buildings For the target building The original importance score; For activation functions; This is a trainable attention parameter vector; It is a trainable linear transformation matrix used to map the original features to a feature space that is more suitable for comparison and aggregation; , This represents the current status information of the building.

[0147] The degree of influence on other nodes in the neighborhood is calculated using attention scoring, and is expressed as follows:

[0148] ,

[0149] In the formula, Attention weights represent the weights used when updating building nodes. When representing neighboring building nodes For nodes The relative influence of the attention weight indicates that the corresponding neighboring buildings contribute more to the subsequent feature aggregation and value assessment. Indicates with buildings A collection of buildings that are related;

[0150] Using attention weights The fused representation is obtained by weighted summation of the features of neighboring nodes. Furthermore, the fusion representation is used as part of the input features of the centralized Critic value network to output the value Q of the joint state-action pair.

[0151] Further, step 306 includes:

[0152] When the policy network of the building agents is updated, the temperature regulation actions of other building agents are estimated through the adversary modeling network based on the real-time collected building group state S(t). and will the action of this intelligent agent Combined with the estimated action to form a joint action vector Input the value network Critic to compute policy gradients and update the Actor network parameters for each building;

[0153] The adversary modeling network is not a fixed network structure directly given in the existing scenario, but a network structure built on the MADDPG framework and combined with the characteristics of the building cluster-virtual power plant collaborative control task. The Actor network is used to output the temperature regulation actions of each building agent, the Critic network is used to evaluate the long-term value of the joint state-action pair, and the adversary modeling network is used to estimate the actions of other building agents, thereby realizing multi-building collaborative optimization control;

[0154] The formula for calculating the policy gradient is as follows:

[0155] ,

[0156] In the formula, Represents the Actor network parameters The direction of the fine-tuning This represents the state of samples taken from the experience playback pool D. With joint actions Seeking expectations, Indicates the adjustment amount. Indicates the first Temperature control strategies for buildings Indicates the first The current status of the building.

[0157] Based on the Critic network, the local decision-making policy of each building, i.e., its Actor network, is optimized through policy gradient calculation. This allows each building's policy update to take into account the behavior of other buildings, enabling all buildings to work collaboratively.

[0158] Furthermore, step 306 also includes:

[0159] The Critic loss function is constructed with the goal of minimizing temporal difference learning, and the network parameters are... By minimizing temporal difference learning, the centralized Critic network is corrected and optimized. This enables it to accurately predict the long-term value of any joint state-action pair (S,A), providing reliable and efficient gradient directions for subsequent Actor network policy updates, ensuring that the collaborative control strategy of the entire building cluster can converge stably and efficiently to the optimal solution; the expression for the Critic loss function is:

[0160] ,

[0161] in, This represents the expectation of the state-action-reward-next state samples obtained from the experience replay pool. For the first A Critic network in parameters Below the joint state-action pair Predict the Q value; The target Q value is defined as:

[0162] ,

[0163] in, As a discount factor, Represented as the target network's next state-action pair Value prediction; For timely rewards, the expression is:

[0164]

[0165] in, This represents the grid-side performance bonus, used to track the aggregated power of the building complex in response to grid commands. This indicates the degree to which the building's indoor temperature deviates from the user's set comfort range. This is expressed as the real-time total power output relative to the controlled building air conditioning load. Indicates the first The building sets the temperature. This is expressed as the user-allowed temperature deviation value; Represented as power grid side weighting coefficient and comfort weighting coefficient;

[0166] A priority experience replay pool D is constructed, and experience samples are stored in the priority experience replay pool. Priorities are assigned based on the temporal difference error samples of the value network, and experience samples are sampled according to the sampling probability corresponding to the priority to update the value network parameters. The samples include:

[0167] ,

[0168] in, This indicates a combined state of the building complex. , This represents the reward for the combined actions output by the intelligent agents in each building.

[0169] Step 4: Validate the global optimization strategy based on the thermal comfort elasticity range. If the building temperature change caused by the strategy is within the preset elasticity range, proceed to Step 5; otherwise, return to Step 3 to re-optimize.

[0170] The user thermal comfort elasticity range is a temperature range pre-quantified based on the subjective acceptance boundaries of different user groups for hot and cold sensations. Its lower limit and upper limit correspond to different comfort thresholds.

[0171] Step 5: The verified global optimization strategy is converted into control commands that can be executed by building equipment to drive the load equipment, including the air conditioning system, to perform power regulation.

[0172] After verification, a smoothing process is adopted, which decomposes a large temperature adjustment into multiple small, gradual fine-tuning commands. This allows for a smooth transition to the target setpoint within a timescale imperceptible to the user. This process protects the equipment's lifespan and also achieves a smooth transition of power changes.

[0173] Control commands are completed and collaboratively distributed to target building equipment. For scenarios with high timeliness requirements, such as frequency regulation, precise time synchronization management ensures that most devices within the cluster can respond to commands within a similar time window, forming a rapid and consistent aggregated power effect to meet the grid's second-level or minute-level regulation needs. After the commands are issued, the execution status of the devices is continuously monitored and fed back to the monitoring device in real time, thereby providing closed-loop verification of the strategy execution effect for upstream optimization algorithms and supporting the system's continuous adaptive optimization.

[0174] Step 6: Collect building status data and power grid response data in real time after regulation, calculate command tracking accuracy and comfort satisfaction rate, and detect system operation status. If the regulation effect does not meet expectations or an abnormal state occurs, immediately trigger the feedback mechanism and return the evaluation results to Step 3 for strategy iteration and optimization until the regulation target is met.

[0175] The command tracking accuracy is the error value between the power grid dispatch command and the actual response power; the comfort satisfaction rate refers to the proportion of the time that the indoor temperature is within the preset comfort range to the total control time.

[0176] By deploying a sensor network across various buildings, the total aggregated power of the building cluster is collected in real time to assess the responsiveness to grid frequency regulation or peak shaving demands. Simultaneously, key performance indicators are collected. Indoor temperature, air conditioning system operating status, energy consumption, and other parameters in each building are continuously monitored to assess the impact of control measures on the building's internal environment and the equipment itself, ensuring the system operates within a safe and comfortable range.

[0177] Using the collected data, the effectiveness of this control action is precisely quantified and evaluated: the deviation between the actual aggregated power and the target value is calculated to determine whether the accuracy requirements of the power grid service are met. The indoor temperature of all users is monitored to ensure it remains strictly within the preset thermal comfort range, guaranteeing the seamless nature of the control measures.

[0178] Figure 3 The results of the grid demand tracking performance comparison show that, compared with the traditional control system, the building cluster-virtual power plant collaborative control method proposed in this invention, which considers the elasticity of user thermal comfort, has superior demand tracking performance. The traditional control system, which ignores the elasticity of user comfort and lacks collaborative optimization, has a root mean square error (RMSE) of 105.2 kW for demand tracking, while the RMSE of the proposed method is 85.8 kW, a reduction of approximately 18.4%. This demonstrates that the present invention can effectively aggregate building air conditioning loads to form a unified and adjustable flexible whole, resulting in smoother aggregated power output and more precise response to dynamic grid demand commands, thereby improving the availability and quality of the building cluster as a grid control resource. Simultaneously, the invention fully considers the elasticity of user thermal comfort during the control process, ensuring that indoor temperature is always maintained within the user's acceptable comfort range.

[0179] Figure 4 The performance improvement comparison chart quantifies the technical advantages of this invention from three core dimensions: In terms of tracking accuracy, this invention reduces grid command tracking error by 18.0% through the MADDPG algorithm for collaborative optimization control of building clusters, significantly improving the accuracy of power regulation; in terms of power stability, the smooth command processing mechanism based on the thermal comfort elastic range reduces output power fluctuation by 18.5%, effectively avoiding power surges caused by traditional regulation; and in terms of response speed, the power-temperature decoupling characteristic achieved by utilizing building thermal inertia shortens system response delay by 16.8%, achieving second-level regulation capability. This invention, through a technical path combining comfort constraint embedding with multi-agent collaboration, can simultaneously ensure user experience and grid regulation efficiency, resolving the core contradiction in the large-scale application of flexible loads.

[0180] The traditional method and the method of this invention are compared in terms of response delay, power fluctuation amplitude, temperature fluctuation amplitude, and average tracking error. The comparison results are as follows: Figure 5As shown in Table 1, the results demonstrate that the method of this invention outperforms the traditional method in all indicators: response delay is reduced from 12.5 s to 10.8 s, an improvement of 13.6%; power fluctuation amplitude is reduced from ±18.3 kW to ±15.1 kW, an improvement of 17.5%; temperature fluctuation amplitude is reduced from ±1.8℃ to ±1.5℃, an improvement of 16.7%; and average tracking error is reduced from 9.7 to 8.3, an improvement of 14.4%. This indicates that the present invention can effectively reduce power and temperature fluctuations while improving the grid demand response speed and tracking accuracy, thus balancing regulation stability and user thermal comfort. In summary, the building cluster-virtual power plant collaborative regulation method considering user thermal comfort elasticity proposed in this invention exhibits superior performance in terms of response speed, output stability, user comfort, and demand tracking accuracy, verifying the effectiveness and advancement of the method.

[0181] Table 1. Comparison of comprehensive performance indicators between traditional methods and the method of this invention .

Claims

1. A building cluster-virtual power plant collaborative control method considering the elasticity of user thermal comfort, characterized in that, Includes the following steps: Step 1: Collect frequency regulation / peak regulation commands from the power grid dispatch center and the operating status parameters of the building cluster in real time, and transmit them to the virtual power plant aggregation layer through communication and data interfaces; Step 2: In the virtual power plant aggregation layer, construct a thermodynamic-electric power coupling model of the building cluster based on the received data, and generate an initial control strategy; Step 3: With the goal of minimizing the power grid command tracking error and the user thermal comfort elasticity range as the constraint, reinforcement learning is used to collaboratively optimize the initial control strategy and generate a global optimization strategy. Step 4: Validate the global optimization strategy based on the thermal comfort elasticity range. If the building temperature change caused by the strategy is within the preset elasticity range, proceed to Step 5; otherwise, return to Step 3 to re-optimize. Step 5: Convert the verified global optimization strategy into control commands that can be executed by building equipment to drive load equipment, including the air conditioning system, to perform power regulation. Step 6: Collect building status data and power grid response data in real time after regulation, calculate command tracking accuracy and comfort satisfaction rate, and detect system operation status. If the regulation effect does not meet expectations or an abnormal state occurs, immediately trigger the feedback mechanism and return the evaluation results to Step 3 for strategy iteration and optimization until the regulation target is met.

2. The building cluster-virtual power plant collaborative control method considering user thermal comfort flexibility according to claim 1, characterized in that, Step 2 includes: A dynamic coupling relationship between indoor temperature changes and the power consumption of the air conditioning system is established, denoted as a thermodynamic-electrical power coupling model, to describe the influence of building thermal inertia on power regulation response. The mathematical expression is: ， In the formula, The predicted trajectory representing the total power of the cluster. express Time of the first Real-time power consumption of the building's air conditioning This indicates the number of buildings participating in the virtual power plant's aggregated control. express Time of the first Real-time cooling / heating capacity of the building's air conditioning system. express Time of the first The real-time performance coefficient of the air conditioner; Generate an initial control strategy that meets the preliminary constraints of grid demand, denoted as... ,in, Represented as the first Indoor temperature trajectory of a building This indicates the temperature adjustment amount set for the building's air conditioning system.

3. The building cluster-virtual power plant collaborative control method considering user thermal comfort flexibility according to claim 2, characterized in that, Step 3 includes: Step 301: Treat each building as an independent intelligent agent, denoted as... Introducing a graph attention mechanism into a centralized Critic value network; Step 302: The frequency regulation / peak shaving demand targets issued by the power grid dispatch center, the real-time operating status of the building complex, and the user comfort flexibility range are spatiotemporally aligned to form a global status view. , represented as: ， in, express The real-time frequency deviation at any given moment is the specific value of the frequency modulation signal; express The excitation signal at any given time is the specific value of the peak-shaving signal. Indicates including the first Standardized status data including indoor temperature of the building, user-set temperature, thermal comfort range, and real-time power of the air conditioner; Step 303, define the action space, denoted as , in, This indicates the total power reference target, including the total power reference target for frequency-modulated signals and the total power reference target for peak-modulated signals; Represents the cooperative control strategy function; Step 304, define the adaptive reward function, the expression of which is: ， In the formula, For grid-side instruction tracking rewards, This is a penalty for indoor temperature deviating from the user's thermal comfort range. This is a penalty term for power regulation smoothness. , , These are weighting coefficients that are dynamically adjusted according to the power grid's operating status. Step 305: Introduce an adversary modeling mechanism to update the Actor policy network of the building agent; Step 306: Optimize the centralized Critic value network by minimizing temporal difference learning, and use the optimized centralized Critic value network to evaluate the long-term value of joint actions and obtain the policy functions of each building agent. Step 307: With the goal of minimizing the power grid command tracking error, construct the objective function, the expression of which is: ， In the formula, This represents the actual aggregate power of the building complex. This is represented as a power grid frequency regulation / peak shaving reference command. This refers to the indoor temperature of the building. This indicates the temperature setting for the building. This represents the change in building air conditioning control parameters. , , Represented as weighting coefficients; The constraints for the objective function are constructed as follows: ， In the formula, , These represent the upper and lower limits of the acceptable temperature for building users, respectively, with denoted as the highest acceptable temperature for building users. , These represent the upper and lower limits of the operating power of the building's air conditioning system. This represents the maximum temperature adjustment range allowed by the user. The objective function value is calculated based on the policy function to obtain the optimal policy function for each building agent.

4. The building cluster-virtual power plant collaborative control method considering user thermal comfort flexibility according to claim 3, characterized in that, In step 301, a graph attention mechanism is introduced into the centralized Critic value network, including: Treat each building as an individual intelligent agent node, and model the building cluster as a graph structure. ,in The set of building agents and the edge set represent the set of building agents. Indicates the relationships between buildings; At time t, construct the node feature vector of the building. The node feature vector includes one or more of the following: indoor temperature, user-set temperature, thermal comfort elasticity range, and real-time air conditioning power. For any node and its neighboring nodes Calculate attention score , represented as: ， In the formula, Indicates neighboring buildings For the target building The original importance score; , This represents the current status information of the building. It is a trainable linear transformation matrix used to map the original features to a feature space that is more suitable for comparison and aggregation; This is a trainable attention parameter vector; For activation functions; The degree of influence on other nodes in the neighborhood is calculated using attention scoring, and is expressed as follows: ， In the formula, Attention weights represent the weights used when updating building nodes. When representing neighboring building nodes For nodes The relative degree of influence; Indicates with buildings A collection of buildings that are related; Using attention weights The fused representation is obtained by weighted summation of the features of neighboring nodes. Furthermore, the fusion representation is used as part of the input features of the centralized Critic value network to output the value Q of the joint state-action pair.

5. The building cluster-virtual power plant collaborative control method considering user thermal comfort flexibility according to claim 4, characterized in that, Step 306 includes: When the policy network of the building agents is updated, the temperature regulation actions of other building agents are estimated through the adversary modeling network based on the real-time collected building group state S(t). and will the action of this intelligent agent Combined with the estimated action to form a joint action vector Input the value network Critic to compute policy gradients and update the Actor network parameters for each building; The formula for calculating the policy gradient is as follows: ， In the formula, Represents the Actor network parameters The direction of the fine-tuning This represents the state of samples taken from the experience playback pool D. With joint actions Seeking expectations, Indicates the adjustment amount. Indicates the first Temperature control strategies for buildings Indicates the first The current status of the building.

6. The building cluster-virtual power plant collaborative control method considering user thermal comfort flexibility according to claim 5, characterized in that, Step 306 also includes: The Critic loss function is constructed with the goal of minimizing temporal difference learning, and its expression is: ， in, This represents the expectation of the state-action-reward-next state samples obtained from the experience replay pool. For the first A Critic network in parameters Below the joint state-action pair Predict the Q value; The target Q value is defined as: ， in, As a discount factor, Represented as the target network's next state-action pair Value prediction; For timely rewards, the expression is: ， in, This represents the grid-side performance bonus, used to track the aggregated power of the building complex in response to grid commands. This indicates the degree to which the building's indoor temperature deviates from the user's set comfort range. This represents the real-time total power output relative to the controlled building air conditioning load. Indicates the first The building sets the temperature. This is represented as the user-allowed temperature deviation value; Represented as power grid side weighting coefficient and comfort weighting coefficient; A priority experience replay pool D is constructed, and experience samples are stored in the priority experience replay pool. Priorities are assigned based on the temporal difference error samples of the value network, and experience samples are sampled according to the sampling probability corresponding to the priority to update the value network parameters. The samples include: ， in, This indicates a combined state of the building complex. , This represents the reward for the combined actions output by the intelligent agents in each building.

Images