Online collaborative operation method for grid-interacting residential buildings based on learning and optimization

Abstract

The application discloses a grid-interacting residential building online collaborative operation method based on learning and optimization, belongs to the technical field of cross between grid-interacting residential building collaborative operation and artificial intelligence, and comprises the following steps: establishing a single residential building operation cost minimization problem during non-grid service; modeling the single residential building operation cost minimization problem into a Markov decision process; training a residential building energy management intelligent agent by adopting a cross normalization algorithm to obtain an energy management strategy; considering total power limitation set by a power distribution system operator, establishing a double-layer collaborative operation cost minimization problem of a residential building group; based on the energy management strategy obtained by each residential building and an optimization auxiliary bisection search, solving the above collaborative operation cost minimization problem and executing decision-making; and the power distribution system operator calculates economic compensation according to energy management decision-making of each residential building before and after collaboration and issues the economic compensation. The application can effectively save energy cost and has stronger grid service capability.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of grid-interactive residential building collaborative operation and artificial intelligence, specifically a learning and optimization-based online collaborative operation method for grid-interactive residential buildings. Background Technology

[0002] The current power grid faces three pressures: continuously growing electricity demand leading to a widening peak-valley gap, the increasing proportion of renewable energy generation causing increased power output volatility, and rigid constraints on transmission and distribution capacity limiting system regulation margin. As a major energy consumer, accounting for 40%-70% of total electricity consumption, the building sector's load regulation potential is strategically valuable in alleviating grid pressure. Grid-interactive residential buildings, by coordinating adjustable resources such as photovoltaic power generation, energy storage systems, and HVAC systems, can provide peak power reduction services to the grid while ensuring user thermal comfort. However, the regulation capacity of a single grid-interactive residential building is typically less than 10 kilowatts, and the fragmented nature of its dispersed resources prevents it from independently meeting the grid's 100-kilowatt-level regulation needs. Furthermore, the lack of multi-building collaboration mechanisms to aggregate dispersed regulation capabilities into system-level service capabilities, the inability of existing technologies to accurately quantify the contribution of individuals to grid services, and the difficulty in establishing a fair economic compensation mechanism to incentivize continuous user participation hinder the large-scale application of building clusters participating in grid interaction.

[0003] Existing operational methods for grid-interactive residential buildings can be broadly categorized into two types: model-based and learning-based methods. The former relies on precise parameters such as equivalent thermal resistance and heat capacity, which are difficult to obtain in practice and result in large model errors. Furthermore, the model is designed only for a single building, failing to achieve coordinated scheduling across multiple buildings and neglecting grid service demands. The latter, while avoiding explicit modeling, requires continuous uploading of real-time data such as photovoltaic power generation, indoor temperature, and load demand to a central server during training, posing a significant risk of user privacy breaches. Simultaneously, these methods focus on the economic optimization of a single building, failing to establish cross-building power coordination mechanisms or quantify the power adjustments made by individual buildings to meet grid constraints. This leads to a complete lack of economic compensation mechanisms, hindering the motivation for continued user participation. Neither of these methods considers the collaborative operation of multiple grid-interactive residential buildings to provide grid services and jointly optimize their respective energy costs and user comfort under privacy protection. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a learning and optimization-based online collaborative operation method for grid-interactive residential buildings, which can minimize the operating costs of grid-interactive residential building complexes while meeting the needs of users for thermal comfort and grid services.

[0005] To achieve the above objectives, the present invention employs the following technical solution: The present invention provides a learning and optimization-based online collaborative operation method for power grid-interactive residential buildings, comprising the following steps:

[0006] Step 1: Minimize the operating cost of a single grid-interactive residential building during non-grid service periods, while ensuring that the indoor temperature is within a comfortable range;

[0007] Step 2: Transform the problem of minimizing the operating cost of a single grid-interactive residential building into a Markov decision process, and design its environmental state, actions, and reward functions;

[0008] Step 3: Apply cross-normalization Algorithm training of a power grid-interactive residential building energy management agent to obtain energy management strategies;

[0009] Step 4: Considering the total power limitations of the power distribution system operator, establish a system to minimize the cost of two-layer collaborative operation of the power grid-interactive residential building complex;

[0010] Step 5: Based on the energy management strategy obtained for each grid-interactive residential building, solve the above-mentioned problem of minimizing collaborative operation costs using an optimization-assisted binary search and execute the decision;

[0011] Step 6: The power distribution system operator calculates economic compensation based on the energy management decisions of each grid-interactive residential building before and after collaboration, and distributes the compensation to each residential building.

[0012] Furthermore, the expression for minimizing the operating cost of a single grid-interactive residential building in step 1 is as follows:

[0013] ,

[0014] ,

[0015] ,

[0016] ,

[0017] ,

[0018] ,

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] in: Represents the mathematical expectation. Denotes the upper limit of the sequence. Indicates time slot, Indicates the total number of time slots. This represents the load aggregator index. This represents an index of residential buildings with grid interaction. express Energy trading costs per time slot express Depreciation cost of time-slot energy storage systems; express The price at which electricity is purchased from the public grid during time slots; express The price of electricity sold to the public grid during time slots; express Electricity trading volume between time slots and the public grid, This indicates that electricity is purchased from the power grid. This indicates the sale of electricity to the power grid; express Duration of the time slot; This represents the depreciation cost factor for energy storage systems; express The charging and discharging power of the time-slot energy storage system Indicates charging. Indicates discharge; express Indoor temperature in the time slot; This indicates the lower limit of a comfortable indoor temperature. This indicates the upper limit of the comfortable indoor temperature. express Outdoor temperature in the time slot; express The energy level of time-slot energy storage systems This indicates the minimum energy level of the energy storage system. Indicates the maximum energy level of the energy storage system. This indicates the maximum charging power of the energy storage system. This indicates the maximum discharge power of the energy storage system. This represents the charging efficiency coefficient of the energy storage system. This represents the discharge efficiency coefficient of the energy storage system; express Input power of time-slot HVAC system This indicates the maximum input power of the HVAC system; The conversion function representing the conversion of solar irradiance into heat increment; express Random thermal disturbances in time slots; express Photovoltaic power generation per time slot; express The total power demand of time-slot non-transferable loads, the decision variables of this optimization problem are The charging and discharging power of time-slot energy storage system Input power of HVAC system .

[0024] Furthermore, in step 2, the state, action, and reward functions of the Markov decision process are related to the load aggregator. Interconnected power grid residential buildings The design is as follows:

[0025] ,

[0026] ,

[0027] ,

[0028] in: express The time slot is the relative time slot of the day; express The state of the residential building environment with grid interaction in this time slot; express The actions of the smart entity in the power grid-interactive residential building during the time slot; express The rewards obtained by the smart agent of the power grid interactive residential building in the time slot Indicates the scaling factor. , This indicates a deviation from the thermal comfort temperature.

[0029] Furthermore, the cross-normalization in step 3 The algorithm is characterized by: [the following is a description of the algorithm's behavior in relation to the load aggregator] Interconnected power grid residential buildings Its intelligent agents include an actor network. First Commentator Network and the Second Critics Network ;

[0030] The actor network Input is the load aggregator Interconnected power grid residential buildings The state is represented by a Gaussian action distribution, consisting of a mean vector and a standard deviation vector; the actor network... The actor network includes an input layer, multiple hidden layers, and an output layer. The number of neurons in the input layer and the load aggregation quotient Interconnected power grid residential buildings The number of state components is equal, and the number of neurons in the output layer is twice the number of action components of the corresponding agent;

[0031] First Commentator Network Input is the load aggregator Interconnected power grid residential buildings The state and its corresponding agent's actions are defined, and the output is the state-action value; the first commentator network It consists of one input layer, multiple hidden layers, and one output layer. The number of neurons in the input layer is equal to the sum of the number of environmental state components and the number of agent action components. The number of neurons in the output layer is 1. (Second commentator network) Structure, inputs, and outputs of the first commentator network same.

[0032] Furthermore, in step 3, the load aggregator... Interconnected power grid residential buildings The training process for its intelligent agent is as follows:

[0033] Step 3-1: Initialize and load aggregator Interconnected power grid residential buildings Actor Network Corresponding to Intelligent Agent and the First Critics Network Second Commentator Network The weight parameters are used to initialize the experience replay buffer. ;

[0034] Step 3-2: Acquisition of corresponding intelligent agents and load aggregator Interconnected power grid residential buildings exist Environmental conditions of time slots ;

[0035] Step 3-3: The corresponding agent will Input to actor network Get action ;

[0036] Steps 3-4: The corresponding agent interacts with the load aggregator. Interconnected power grid residential buildings Execution of actions Receive a reward and load aggregator Interconnected power grid residential buildings exist The state of the time slot ;

[0037] Steps 3-5: Transfer the empirical tuples Stored in the experience playback buffer In the middle, and ordered ;

[0038] Steps 3-6: From the experience replay buffer We randomly sample small batches of empirical tuples and use cross-normalization. Algorithm updates the actor network corresponding to the agent and the First Critics Network Second Commentator Network ;

[0039] Steps 3-2 to 3-6 are repeated until the preset number of training steps is reached.

[0040] Furthermore, the cross-normalization in step 3 The algorithm's loss function expression in relation to the load aggregator Interconnected power grid residential buildings as follows:

[0041] ,

[0042] ,

[0043] ,

[0044] in: Represents the network loss function for actors. It is the sample size sampled from the experience replay buffer in small batches. Indicates the index in the sampled sample. Represents the entropy coefficient. It is the first in the sample One state, Indicates the first each state Input to actor network The obtained motion estimate; , This indicates a network index of critics. Indicates the first A commentator's network loss function, Indicates the first A commentator's network loss function, It is the first in the sample One reward, It is the first sample. The state after the transition, Represents a discount factor. Indicates the first The state after the transition Input to actor network The obtained motion estimate, This indicates that gradient calculation has been stopped. Represents the entropy coefficient loss function. Represents the target entropy. For the first commentator network, For the second network of critics.

[0045] Furthermore, the cross-normalization in step 3 When calculating the commentator network loss, the algorithm computes the state-action value and the next-state-action value in parallel within the same mini-batch. The calculation expression is compared with the load aggregator. Interconnected power grid residential buildings The performance is as follows:

[0046] ,

[0047] in: This indicates that the data is concatenated along the batch dimension and then normalized in the batch. Indicates the first A commentator's network loss function, It is the first in the sample One state, It is the first in the sample One action, It is the first sample. The state after the transition, Indicates the first The state after the transition Input to actor network The obtained motion estimate.

[0048] Furthermore, in step 4, the lower-level optimization problem of minimizing the cost of the two-layer collaborative operation of the power grid-interactive residential building complex is P1, and the expression for the upper-level optimization problem is as follows:

[0049] ,

[0050] ,

[0051] ,

[0052] ,

[0053] ,

[0054] ,

[0055] in: This indicates the unit compensation price for power distribution system operators; This represents the mapping from peak power quotas to the total power demand of all grid-connected residential buildings; This represents the sum of peak power allowances for all grid-connected residential buildings. This represents the sum of peak power allowances for all grid-connected residential buildings. The maximum value, Indicates with load aggregator Interconnected power grid residential buildings Peak power quota; Indicates the number of load aggregators. Indicates load aggregator The number of residential buildings with grid interconnection capabilities; Let P1 be the decision variable for the optimization problem. express The optimal value, Let P2 be the decision variable for optimization problem P2. express The optimal value, Indicates the scaling factor; This represents the optimization problem P2 involving the load aggregator. Interconnected power grid residential buildings Electricity trading volume with the power grid express Take the optimal value Time and load aggregator Interconnected power grid residential buildings Electricity trading volume with the power grid This indicates the peak limit set by the power distribution system operator; It represents the second norm of a vector.

[0056] Furthermore, the optimization problem solved by the auxiliary binary search in step 5 is related to the load aggregator. Interconnected power grid residential buildings The Chinese character is represented as:

[0057] ,

[0058] ,

[0059] ,

[0060] ,

[0061] in: This represents the decision variable for the charging and discharging power of the energy storage system in optimization problem P3. This represents the HVAC input power decision variable in optimization problem P3; This represents the variable representing the electricity transaction volume with the power grid in optimization problem P3; In the optimization problem P1 The optimal charge and discharge power of the time-slot energy storage system In the optimization problem P1 Optimal input power for time-slotted HVAC systems; This indicates the maximum allowable power reduction value for HVAC systems.

[0062] Furthermore, the collaborative operation process in step 5 is as follows: The time slots are represented as follows:

[0063] (1) Initialize peak power quota ;

[0064] (2) For each load aggregator Interconnected power grid residential buildings Its agent observes the current state and infers the initial optimal decision. and will correspond to the initial power requirements Send to the relevant load aggregator ;

[0065] (3) Calculate the initial total power requirement If it is less than the power limit set by the power distribution system operator Then the optimal decision is executed directly. ;

[0066] (4) If the initial total power requirement Power exceeding the power limit set by the power distribution system operator Then, perform a binary search to adjust the peak power quota; for the first... Peak power quota in the next binary search iteration Each with load aggregator Interconnected power grid residential buildings Solve the single-slot optimization problem P3 to obtain the adjusted decision. and the corresponding power requirements Send to the relevant load aggregator Calculate the total power demand If it is less than the power limit set by the power distribution system operator. Then the adjusted decision will be implemented. If it is still greater than the power limit set by the power distribution system operator ,but Repeat the above search steps until the total power requirement is met. Meet the peak power limits set by the power distribution system operator Or the maximum number of searches has been reached.

[0067] (5) Each power grid-interactive residential building implements the final decision to obtain the power adjustment amount, and the power distribution system operator calculates and distributes the economic compensation fee based on the power adjustment amount.

[0068] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0069] The present invention provides a learning and optimization-based online collaborative operation method for power grid-interactive residential buildings, which utilizes cross-normalization. A two-tier decision-making framework, combining algorithms and an optimization-assisted binary search algorithm, achieves coordinated optimization of building energy costs and grid service performance while maintaining high user comfort. This scheme, through a two-tiered collaborative optimization mechanism, aggregates the distributed grid-interactive residential building regulation capacity from the individual kilowatt level to the system-level hundreds of kilowatt level. Local cross-normalization is employed. The distributed decision-making correction architecture, which combines intelligent agent decision-making and collaborative layer optimization-assisted binary search, effectively protects sensitive user information. Each grid-interactive residential building only needs to upload its power demand to the load aggregator, without sharing private data such as photovoltaic output and indoor temperature. By quantifying individual contributions through an economic compensation mechanism, the architecture incentivizes continued participation, achieving the dual goals of electricity cost savings and user participation incentives. This provides key technical support for the large-scale application of grid-interactive residential building clusters. Attached Figure Description

[0070] Figure 1Flowchart of the online collaborative operation method for power grid-interactive residential buildings based on learning and optimization provided by the present invention;

[0071] Figure 2 This is a comparison chart of the total power demand of the method of this invention and other methods for all grid-interactive residential buildings during grid service periods;

[0072] Figure 3 This diagram illustrates the process of adjusting the total power demand of all grid-interactive residential buildings during a specific time slot of the power grid service period using the method of this invention. Detailed Implementation

[0073] To clearly illustrate the technical solution and innovative advantages of the present invention, specific embodiments of the present invention will now be described in conjunction with the accompanying drawings. It should be specifically noted that these embodiments are for illustrative purposes only and do not constitute a limitation on the claims of the present invention.

[0074] This solution proposes an online collaborative operation method for grid-interactive residential buildings based on learning and optimization. Each grid-interactive residential building consists of distributed photovoltaic power generation devices, a battery energy storage system, rigid loads, and a building energy management system. Specifically, the battery energy storage system dynamically stores surplus photovoltaic energy and provides energy feedback during power shortages to optimize household energy costs. The load units include two types: rigid loads and controllable loads. Rigid loads refer to essential appliances that meet the user's living needs (such as televisions, microwave ovens, etc.), while controllable loads specifically refer to HVAC systems with power regulation capabilities. The smart home energy management system performs hourly dynamic optimization decisions on the charging and discharging power of the energy storage system and the input power of the HVAC system based on real-time acquired operating parameters (including but not limited to photovoltaic power generation, dynamic electricity price signals, ambient temperature, and rigid load requirements), thereby achieving collaborative operation of multiple grid-interactive residential buildings and maximizing economic benefits.

[0075] like Figure 1 As shown, the online collaborative operation method for power grid-interactive residential buildings based on learning and optimization includes the following steps:

[0076] Step 1: Minimize the operating cost of a single grid-interactive residential building during non-grid service periods, while ensuring that the indoor temperature is within a comfortable range;

[0077] Step 2: Transform the problem of minimizing the operating cost of a single grid-interactive residential building into a Markov decision process, and design its environmental state, actions, and reward functions;

[0078] Step 3: Apply cross-normalization Algorithm training of a power grid-interactive residential building energy management agent to obtain energy management strategies;

[0079] Step 4: Considering the total power limitations of the power distribution system operator, establish a system to minimize the cost of two-layer collaborative operation of the power grid-interactive residential building complex;

[0080] Step 5: Based on the energy management strategy obtained for each grid-interactive residential building, solve the above-mentioned problem of minimizing collaborative operation costs using an optimization-assisted binary search and execute the decision;

[0081] Step 6: The power distribution system operator calculates economic compensation based on the energy management decisions of each grid-interactive residential building before and after collaboration, and distributes it to each residential building.

[0082] Furthermore, the expression for minimizing the operating cost of a single grid-interactive residential building in step 1 is as follows:

[0083] ,

[0084] ,

[0085] ,

[0086] ,

[0087] ,

[0088] ,

[0089] ,

[0090] ,

[0091] ,

[0092] ,

[0093] in: Represents the mathematical expectation. Denotes the upper limit of the sequence. Indicates time slot, Indicates the total number of time slots. This represents the load aggregator index. This represents an index of residential buildings with grid interaction. express Energy trading costs per time slot express Depreciation cost of time-slot energy storage systems; express The price at which electricity is purchased from the public grid during time slots; express The price of electricity sold to the public grid during time slots; express Electricity trading volume between time slots and the public grid, This indicates that electricity is purchased from the power grid. This indicates the sale of electricity to the power grid; express Duration of the time slot; This represents the depreciation cost factor for energy storage systems; express The charging and discharging power of the time-slot energy storage system Indicates charging. Indicates discharge; express Indoor temperature in the time slot; This indicates the lower limit of a comfortable indoor temperature. This indicates the upper limit of the comfortable indoor temperature. express Outdoor temperature in the time slot; express The energy level of time-slot energy storage systems This indicates the minimum energy level of the energy storage system. Indicates the maximum energy level of the energy storage system. This indicates the maximum charging power of the energy storage system. This indicates the maximum discharge power of the energy storage system. This represents the charging efficiency coefficient of the energy storage system. This represents the discharge efficiency coefficient of the energy storage system; express Input power of time-slot HVAC system This indicates the maximum input power of the HVAC system; The conversion function representing the conversion of solar irradiance into heat increment; express Random thermal disturbances in time slots; express Photovoltaic power generation per time slot; express The total power demand of time-slot non-transferable loads. The decision variables for this optimization problem are: The charging and discharging power of time-slot energy storage system Input power of HVAC system .

[0094] Furthermore, in step 2, the state, action, and reward functions of the Markov decision process are related to the load aggregator. Interconnected power grid residential buildings The design is as follows:

[0095] ,

[0096] ,

[0097] ,

[0098] in: express The time slot is the relative time slot of the day; express The state of the residential building environment with grid interaction in this time slot; express The actions of the smart entity in the power grid-interactive residential building during the time slot; express The rewards obtained by the smart agent of the power grid interactive residential building in the time slot Indicates the scaling factor. , This indicates a deviation from the thermal comfort temperature.

[0099] Furthermore, the cross-normalization in step 3 The algorithm is characterized by: [the following is a description of the algorithm's behavior in relation to the load aggregator] Interconnected power grid residential buildings Its intelligent agents include an actor network. and the First Critics Network Second Commentator Network ;

[0100] The actor network Input is the load aggregator Interconnected power grid residential buildings The state is represented by a Gaussian action distribution, consisting of a mean vector and a standard deviation vector; the actor network... The actor network includes an input layer, multiple hidden layers, and an output layer. The number of neurons in the input layer and the load aggregation quotient Interconnected power grid residential buildings The number of state components is equal, and the number of neurons in the output layer is twice the number of action components of the corresponding agent;

[0101] The first critic network Input is the load aggregator Interconnected power grid residential buildings The state and its corresponding agent's actions are defined, and the output is the state-action value; the first commentator network The network consists of one input layer, multiple hidden layers, and one output layer. The number of neurons in the input layer is equal to the sum of the number of environmental state components and the number of agent action components. The output layer has one neuron. This is the second critic network. The structure, inputs, and outputs are consistent with the first commentator network. same.

[0102] Furthermore, in step 3, the load aggregator... Interconnected power grid residential buildings The training process for its intelligent agent is as follows:

[0103] Step 3-1: Initialize and load aggregator Interconnected power grid residential buildings Actor Network Corresponding to Intelligent Agent and the First Critics Network Second Commentator Network The weight parameters are used to initialize the experience replay buffer. ;

[0104] Step 3-2: Acquisition of corresponding intelligent agents and load aggregator Interconnected power grid residential buildings exist Environmental conditions of time slots ;

[0105] Step 3-3: The corresponding agent will Input to actor network Get action ;

[0106] Steps 3-4: The corresponding agent interacts with the load aggregator. Interconnected power grid residential buildings Execution of actions Receive a reward and load aggregator Interconnected power grid residential buildings exist The state of the time slot ;

[0107] Steps 3-5: Transfer the empirical tuples Stored in the experience playback buffer In the middle, and ordered ;

[0108] Steps 3-6: From the experience replay buffer We randomly sample small batches of empirical tuples and use cross-normalization. Algorithm updates the actor network corresponding to the agent and the First Critics Network Second Commentator Network ;

[0109] Steps 3-2 to 3-6 are repeated until the preset number of training steps is reached.

[0110] Furthermore, the cross-normalization in step 3 The algorithm's loss function expression in relation to the load aggregator Interconnected power grid residential buildings The following is a summary:

[0111] ,

[0112] ,

[0113] ,

[0114] in: Represents the network loss function for actors. It is the sample size sampled from the experience replay buffer in small batches. Indicates the index in the sampled sample. Represents the entropy coefficient. It is the first in the sample One state, Indicates the first each state Input to actor network The obtained motion estimate; , This indicates a network index of critics. Indicates the first A commentator's network loss function, Indicates the first A commentator's network loss function, It is the first in the sample One reward, It is the first sample. The state after the transition, Represents a discount factor. Indicates the first The state after the transition Input to actor network The obtained motion estimate, This indicates that gradient calculation has been stopped. Represents the entropy coefficient loss function. Represents the target entropy. For the first commentator network, For the second network of critics.

[0115] Furthermore, the cross-normalization in step 3 When calculating the commentator network loss, the algorithm computes the state-action value and the next-state-action value in parallel within the same mini-batch. The calculation expression is compared with the load aggregator. Interconnected power grid residential buildings The performance is as follows:

[0116] ,

[0117] in: This indicates that the data is concatenated along the batch dimension and then normalized in the batch. Indicates the first A commentator's network loss function, It is the first in the sample One state, It is the first in the sample One action, It is the first sample. The state after the transition, Indicates the first The state after the transition Input to actor network The obtained motion estimate.

[0118] Furthermore, in step 4, the lower-level optimization problem of minimizing the cost of the two-layer collaborative operation of the power grid-interactive residential building complex is P1, and the expression for the upper-level optimization problem is as follows:

[0119] ,

[0120] ,

[0121] ,

[0122] ,

[0123] ,

[0124] ,

[0125] in: This indicates the unit compensation price for power distribution system operators; This represents the mapping from peak power quotas to the total power demand of all grid-connected residential buildings; This represents the sum of peak power allowances for all grid-connected residential buildings. This represents the sum of peak power allowances for all grid-connected residential buildings. The maximum value, Indicates with load aggregator Interconnected power grid residential buildings Peak power quota; Indicates the number of load aggregators. Indicates load aggregator The number of residential buildings with grid interconnection capabilities; Let P1 be the decision variable for the optimization problem. express The optimal value, Let P2 be the decision variable for optimization problem P2. express The optimal value, Indicates the scaling factor; This represents the optimization problem P2 involving the load aggregator. Interconnected power grid residential buildings Electricity trading volume with the power grid express Take the optimal value Time and load aggregator Interconnected power grid residential buildings Electricity trading volume with the power grid This indicates the peak limit set by the power distribution system operator; It represents the second norm of a vector.

[0126] Furthermore, the optimization problem solved by the auxiliary binary search in step 5 is related to the load aggregator. Interconnected power grid residential buildings The Chinese character is represented as:

[0127] ,

[0128] ,

[0129] ,

[0130] ,

[0131] in: This represents the decision variable for the charging and discharging power of the energy storage system in optimization problem P3. This represents the HVAC input power decision variable in optimization problem P3; This represents the variable representing the electricity transaction volume with the power grid in optimization problem P3; In the optimization problem P1 The optimal charge and discharge power of the time-slot energy storage system In the optimization problem P1 Optimal input power for time-slotted HVAC systems; This indicates the maximum allowable power reduction value for HVAC systems.

[0132] Furthermore, the collaborative operation process in step 5 is as follows: The time slots are represented as follows:

[0133] (1) Initialize peak power quota ;

[0134] (2) For each load aggregator Interconnected power grid residential buildings Its agent observes the current state and infers the initial optimal decision. and will correspond to the initial power requirements Send to the relevant load aggregator ;

[0135] (3) Calculate the initial total power requirement If it is less than the power limit set by the power distribution system operator Then the optimal decision is executed directly. ;

[0136] (4) If the initial total power requirement Power exceeding the power limit set by the power distribution system operator Then, perform a binary search to adjust the peak power quota; for the first... Peak power quota in the next binary search iteration Each with load aggregator Interconnected power grid residential buildings Solve the single-slot optimization problem P3 to obtain the adjusted decision. and the corresponding power requirements Send to the relevant load aggregator Calculate the total power demand If it is less than the power limit set by the power distribution system operator. Then the adjusted decision will be implemented. If it is still greater than the power limit set by the power distribution system operator ,but Repeat the above search steps until the total power requirement is met. Meet the peak power limits set by the power distribution system operator Or the maximum number of searches has been reached.

[0137] (5) Each power grid-interactive residential building implements the final decision to obtain the power adjustment amount, and the power distribution system operator calculates and distributes the economic compensation fee based on the power adjustment amount.

[0138] To demonstrate the performance of the method of the present invention, three comparative schemes are introduced:

[0139] Comparison Scheme 1: Rule-based energy management strategy, where each grid-interactive residential building adopts a fixed threshold control strategy and operates independently. This scheme first determines the input power of the HVAC system. When the indoor temperature is within the set upper and lower limits of the comfort temperature, the input power of the HVAC system is given according to the same value of its previous control time slot. When the indoor temperature exceeds the upper limit of the indoor temperature, the input power of the HVAC system is the rated power. When the indoor temperature is below the lower limit of the indoor temperature, the input power of the HVAC system is 0. Then, the charging and discharging power of the energy storage system is determined, that is, the difference between the photovoltaic power generation power of the current control time slot and the sum of the input power of the HVAC system and the rigid load power is used to obtain the charging and discharging power of the energy storage system.

[0140] Comparison Scheme 2: A collaborative operation method based on the Twin Delayed Deep Deterministic policy gradient algorithm (TD3) and rule-assisted binary search. In this scheme, each grid-interactive residential building uses the classic Twin Delayed Deep Deterministic policy gradient algorithm to train an energy management agent to make initial decisions. When the agent's initial decisions cannot meet the grid service requirements, rule-assisted binary search is used to iteratively adjust the peak power quota. When the net power quota is non-negative, if the total power demand is not exceeded, the original decision is maintained; otherwise, the input power of the HVAC system and the charging and discharging power of the energy storage system are scaled proportionally. When the net power quota is negative, the energy storage system is set to operate only in the discharge state, and the input power of the HVAC system and the charging and discharging power of the energy storage system are given according to the designed adjustment rules. For details of the decision adjustment rules, please refer to the following literature: L.Yu, Z.Chen, D.Yue, Y.Ye, G.Strbac and Y.Wang, "Coordinated Operation Optimization of Grid-Interactive Residential Buildings Based on Neural Network-Assisted Hierarchical Model Predictive Control", IEEE Transactions on Automation Science and Engineering, vol.22, pp.13441-13457, 2025.

[0141] Comparison Scheme 3: A collaborative operation method based on the twin-delay deep deterministic strategy gradient algorithm and optimization-assisted binary search. In this scheme, each grid-interactive residential building uses the classic twin-delay deep deterministic strategy algorithm to train an energy management agent to make initial decisions. When the agent's initial decisions cannot meet the grid service requirements, the optimization-assisted binary search method of this invention is used to iteratively adjust the peak power quota to regulate the total power demand of all grid-interactive residential buildings.

[0142] For comparison, the following four performance metrics are defined: monthly total energy cost, building average hourly temperature deviation, average peak violation percentage, and power adjustment. Among them, the monthly total energy cost and building average hourly temperature deviation metrics are related to the individual interests of grid-connected residential buildings, while the average peak violation percentage metric is used to measure the group coordination performance of grid-connected residential buildings.

[0143] Table 1 summarizes the performance indicators of the method of the present invention and the comparative scheme. Figure 2This is a graph showing the variation of total power demand along the operating slots during the grid service period for the method of this invention and the comparative scheme. Figure 3 This is a diagram illustrating the iterative adjustment process of total power demand when various grid-interactive residential buildings coordinate to respond to grid services in the method of this invention.

[0144] Table 1 - Summary of Performance of the Method of the Invention and Comparative Schemes

[0145]

[0146] As shown in Table 1, compared to all comparative schemes, the method of the present invention can reduce the total monthly energy cost by 11.96%-25.99% (assuming an economic compensation of US$0.1 per kilowatt-hour per household), effectively helping users save energy costs and demonstrating good economic benefits. The average hourly temperature deviation of the building using the method of the present invention is better than that of comparative schemes 1 and 2, at only 0.0053 degrees Celsius per hour. Although comparative scheme 3 is slightly better than the present invention in this indicator, the average hourly temperature deviation of the building in the present invention, comparative schemes 2 and 3 are all controlled below 0.01 degrees Celsius per hour, and all three can meet the user's thermal comfort requirements. The average peak violation percentage of the method of the present invention and comparative scheme 3, which also uses the optimized cooperative binary search method of the present invention, is 0.00%, a reduction of 100.00% compared to other schemes. This indicates that the optimized cooperative binary search method of the present invention has stronger peak power regulation capability and grid service response capability compared to the rule-based cooperative binary search method in comparative scheme 2. Comparison Scheme 3 and Comparison Scheme 2 use the same energy management strategy in a single grid-interactive residential building, but the power adjustment of Comparison Scheme 3 is higher than that of Comparison Scheme 2, which also proves this point.

[0147] like Figure 2 As shown, both the method of the present invention and the comparative scheme 3 can limit the total power demand of each grid interactive building to the total power limit specified by the distribution system operator during the grid service period. However, the comparative scheme 1 does not perform collaborative operation, and the comparative scheme 2 uses rule-assisted binary search for collaborative operation, neither of which can fully meet the grid service requirements. This further proves that the optimized assisted binary search in the method of the present invention has superior collaborative performance.

[0148] like Figure 3 As shown, the method of the present invention can ensure a monotonic relationship between the total power demand and the total power quota of each grid-interactive residential building, thereby limiting the total power demand to the total power limit specified by the distribution system operator through binary search of the total power quota.

[0149] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A learning-and-optimization-based online collaborative operation method for power grid-interactive residential buildings, characterized in that: Includes the following steps: Step 1: Minimize the operating cost of a single grid-interactive residential building during non-grid service periods, while ensuring that the indoor temperature is within a comfortable range; The expression for minimizing the operating cost of a single grid-interactive residential building in step 1 is as follows: Optimization problem P1 , ， ， ， ， ， ， ， ， ， in: Represents the mathematical expectation. Denotes the upper limit of the sequence. Indicates time slot, Indicates the total number of time slots. This represents the load aggregator index. This represents an index of residential buildings with grid interaction. express Energy trading costs per time slot express Depreciation cost of time-slot energy storage systems; express The price at which electricity is purchased from the public grid during time slots; express The price of electricity sold to the public grid during time slots; express Electricity trading volume between time slots and the public grid, This indicates that electricity is purchased from the power grid. This indicates the sale of electricity to the power grid; express Duration of the time slot; This represents the depreciation cost factor for energy storage systems. express The charging and discharging power of the time-slot energy storage system Indicates charging. Indicates discharge; express Indoor temperature in the time slot; This indicates the lower limit of a comfortable indoor temperature. This indicates the upper limit of a comfortable indoor temperature. express The outdoor temperature of the time slot; express The energy level of time-slot energy storage systems This indicates the minimum energy level of the energy storage system. Indicates the maximum energy level of the energy storage system. This indicates the maximum charging power of the energy storage system. This indicates the maximum discharge power of the energy storage system. This represents the charging efficiency coefficient of the energy storage system. This represents the discharge efficiency coefficient of the energy storage system; express Input power of time-slot HVAC system This indicates the maximum input power of the HVAC system; The conversion function representing the conversion of solar irradiance into heat increment; express Random thermal disturbances in time slots; express Photovoltaic power generation per time slot; express The total power demand of time-slot non-transferable loads, the decision variables of this optimization problem are The charging and discharging power of time-slot energy storage system Input power of HVAC system ; Step 2: Transform the problem of minimizing the operating cost of a single grid-interactive residential building into a Markov decision process, and design its environmental state, actions, and reward functions; Step 3: Apply cross-normalization Algorithm training of a power grid-interactive residential building energy management agent to obtain energy management strategies; Step 4: Considering the total power limitations of the power distribution system operator, establish a system to minimize the cost of two-layer collaborative operation of the power grid-interactive residential building complex; Step 5: Based on the energy management strategy obtained for each grid-interactive residential building, solve the above-mentioned problem of minimizing collaborative operation costs using an optimization-assisted binary search and execute the decision; Step 6: The power distribution system operator calculates economic compensation based on the energy management decisions of each grid-interactive residential building before and after collaboration, and distributes the compensation to each residential building.

2. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 1, characterized in that, In step 2, the state, action, and reward function of the Markov decision process are related to the load aggregator. Interconnected power grid residential buildings The design is as follows: ， ， ， in: express The time slot is the relative time slot of the day; express The state of the residential building environment with grid interaction in this time slot. express Electricity price per time slot; express The actions of the smart agent in the power grid-interactive residential building during the time slot. express The rewards obtained by the smart agent of the power grid interactive residential building in the time slot Indicates the scaling factor. , This indicates a deviation from the thermal comfort temperature. express Photovoltaic power generation per time slot; express Total power demand of time-slot non-transferable loads express The energy level of time-slot energy storage systems express Outdoor temperature in the time slot express Indoor temperature in time slots, express The charging and discharging power of the time-slot energy storage system express Input power of time-slot HVAC system express Energy trading costs per time slot express Depreciation cost of time-slot energy storage systems.

3. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 1, characterized in that, Cross-normalization in step 3 The algorithm is characterized by: [the following is a description of the algorithm's behavior in relation to the load aggregator] Interconnected power grid residential buildings Its intelligent agents include an actor network. First Commentator Network and the Second Critics Network ; Actor Network Input is the load aggregator Interconnected power grid residential buildings The state is represented by a Gaussian action distribution, consisting of a mean vector and a standard deviation vector; actor network. It includes an input layer, multiple hidden layers, and an output layer; actor network Number of neurons in the input layer and load aggregation quotient Interconnected power grid residential buildings The number of state components is equal, and the number of neurons in the output layer is twice the number of action components of the corresponding agent; First Commentator Network Input is the load aggregator Interconnected power grid residential buildings The state and its corresponding agent's actions are defined, and the output is the state-action value; First Commentator Network It consists of one input layer, multiple hidden layers, and one output layer. The number of neurons in the input layer is equal to the sum of the number of environmental state components and the number of agent action components. The number of neurons in the output layer is 1. (Second commentator network) Structure, inputs, and outputs of the first commentator network same.

4. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 1, characterized in that, In step 3, for the load aggregator Interconnected power grid residential buildings The training process for its intelligent agent is as follows: Step 3-1: Initialize and load aggregator Interconnected power grid residential buildings Actor Network Corresponding to Intelligent Agent First Commentator Network and the Second Critics Network The weight parameters are used to initialize the experience replay buffer. ; Step 3-2: Acquisition of corresponding intelligent agents and load aggregator Interconnected power grid residential buildings exist Environmental conditions of time slots ; Step 3-3: The corresponding agent records the environmental state Input to actor network Get action ; Steps 3-4: The corresponding agent interacts with the load aggregator. Interconnected power grid residential buildings Perform actions in the environment Receive a reward and load aggregator Interconnected power grid residential buildings exist The state of the time slot ; Steps 3-5: Transfer the empirical tuples Stored in the experience playback buffer In the middle, and ordered ; Steps 3-6: From the experience replay buffer We randomly sample small batches of empirical tuples and use cross-normalization. Algorithm updates the actor network corresponding to the agent First Commentator Network and the Second Critics Network ; Steps 3-2 to 3-6 are repeated until the preset number of training steps is reached.

5. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 1, characterized in that, Cross-normalization in step 3 The algorithm's loss function expression in relation to the load aggregator Interconnected power grid residential buildings The following is a summary: ， ， ， in: Represents the network loss function for actors. It is the sample size sampled from the experience replay buffer in small batches. Indicates the index in the sampled sample. Represents the entropy coefficient. It is the first in the sample One state, It is the first in the sample One action, Indicates the first each state Input to actor network The obtained motion estimate; , This indicates a network index of critics. Indicates the first A commentator's network loss function, Indicates the first A commentator's network loss function, It is the first in the sample One reward, It is the first sample. The state after the transition, Represents a discount factor. Indicates the first The state after the transition Input to actor network The obtained motion estimate, This indicates that gradient calculation has been stopped. Represents the entropy coefficient loss function. Represents the target entropy. For the first commentator network, For the second network of critics.

6. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 1, characterized in that, Cross-normalization in step 3 When calculating the commentator network loss, the algorithm computes the state-action value and the next-state-action value in parallel within the same mini-batch. The calculation expression is compared with the load aggregator. Interconnected power grid residential buildings The performance is as follows: ， in: This indicates that the data is concatenated along the batch dimension and then normalized in the batch. Indicates the first A commentator's network loss function, It is the first in the sample One state, It is the first in the sample One action, It is the first sample. The state after the transition, Indicates the first The state after the transition Input to actor network The obtained motion estimate.

7. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 1, characterized in that, In step 4, the lower-level optimization problem of minimizing the cost of two-layer collaborative operation of the power grid-interactive residential building complex is P1, and the expression for the upper-level optimization problem is as follows: Optimization problem P2 , ， ， ， ， ， in: This indicates the unit compensation price for the power distribution system operator; This represents the mapping from peak power quotas to the total power demand of all grid-connected residential buildings; This represents the sum of peak power allowances for all grid-connected residential buildings. This represents the sum of peak power allowances for all grid-connected residential buildings. The maximum value, Indicates with load aggregator Interconnected power grid residential buildings Peak power quota; Indicates the number of load aggregators. Indicates load aggregator The number of residential buildings with grid interconnection capabilities; Let P1 be the decision variable for the optimization problem. express The optimal value, Let P2 be the decision variable for optimization problem P2. express The optimal value, Indicates the scaling factor; This represents the optimization problem P2 involving the load aggregator. Interconnected power grid residential buildings Electricity trading volume with the power grid express Take the optimal value Time and load aggregator Interconnected power grid residential buildings Electricity trading volume with the power grid This indicates the peak limit set by the power distribution system operator; It represents the second norm of a vector.

8. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 7, characterized in that, In step 5, the optimization problem solved by the auxiliary binary search is related to the load aggregator. Interconnected power grid residential buildings The Chinese character is represented as: Optimization problem P3 , ， ， ， in: This represents the decision variable for the charging and discharging power of the energy storage system in optimization problem P3. This represents the HVAC input power decision variable in optimization problem P3. Indicates with load aggregator Interconnected power grid residential buildings Peak power quota; This represents the variable representing the electricity transaction volume with the power grid in optimization problem P3; In the optimization problem P1 The optimal charge and discharge power of the time-slot energy storage system In the optimization problem P1 Optimal input power for time-slotted HVAC systems; This indicates the maximum allowable power reduction value for HVAC systems.

9. The online collaborative operation method for power grid-interactive residential buildings based on learning and optimization according to claim 8, characterized in that, The collaborative operation process in step 5 is in The time slots are represented as follows: (1) Initialize peak power quota ; (2) For load aggregator Interconnected power grid residential buildings Its agent observes the current state and infers the initial optimal decision. and will correspond to the initial power requirements Send to the relevant load aggregator ; (3) Calculate the initial total power requirement If it is less than the power limit set by the power distribution system operator Then the optimal decision is executed directly. ; (4) If the initial total power requirement Power exceeding the power limit set by the power distribution system operator Then, a binary search is performed to adjust the peak power quota; for the first... Peak power quota in the next binary search iteration Each with load aggregator Interconnected power grid residential buildings Solve the single-slot optimization problem P3 to obtain the adjusted decision. and the corresponding power requirements Send to the relevant load aggregator Calculate the total power demand ; If it is less than the power limit set by the power distribution system operator Then the adjusted decision will be implemented. If it is still greater than the power limit set by the power distribution system operator ,but Repeat the above search steps until the total power requirement is met. Meet the peak power limits set by the power distribution system operator Or the maximum number of searches has been reached; (5) Each power grid-interactive residential building implements the final decision to obtain the power adjustment amount, and the power distribution system operator calculates and distributes the economic compensation fee based on the power adjustment amount.

Images