Electricity-carbon combined demand response scheduling decision method, device and equipment

Abstract

The present application discloses a demand response scheduling decision method, device and equipment based on electricity-carbon combination, which relates to the field of distributed energy response scheduling. The method includes: obtaining user-side information data; the information data includes the electricity-carbon joint node marginal electricity price signal corresponding to the operator on the distribution network side; the information data is determined based on the node marginal electricity price model of the electricity-carbon combination on the distribution network side; the node marginal electricity price model is determined by the operator on the distribution network side using the distribution network branch flow algorithm; determining the user-side demand response model; the user-side demand response model is based on the information data and is constructed with the goal of minimizing the power coefficient; using Markov decision making to mathematically transform the user-side demand response model, and using deep reinforcement learning methods to iteratively optimize and solve, to obtain a scheduling decision plan to coordinate the user-side load on the user side with distributed energy generation. This application aims to achieve coordinated adjustment and scheduling of distributed energy and user load.

Description

Technical Field

[0001] The present application relates to the field of distributed energy response scheduling, and in particular to a demand response scheduling decision method, device and equipment based on electricity-carbon combination. Background Art

[0002] With the widespread access of distributed energy in the power system, the flexibility and resilience of the system have been significantly enhanced, while bringing considerable environmental benefits. However, existing research still has shortcomings in the coordinated optimization of demand response and distributed energy, especially in the design of mechanisms to effectively motivate users to actively participate in demand response. At present, most methods focus on traditional power flow models and economic dispatch strategies when dispatching and optimizing power systems, and fail to fully consider the behavioral incentives of users in a diversified power market environment.

[0003] Some existing methods attempt to respond to demand through simple electricity price signals, but in the complex and ever-changing electricity market, the user response effect is often not ideal. Too single incentive mechanisms and scheduling strategies fail to fully mobilize the enthusiasm of users, and may lead to unbalanced resource allocation in practical applications. Due to the large differences in users' geographical locations, electricity demand, and connected distributed energy, the existing mechanisms are still limited in fairness and response effects. In addition, although some studies have attempted to optimize scheduling strategies on the user side, they are mostly based on idealized assumptions and lack in-depth modeling of user demand response behavior and consideration of actual constraints. Strategies and mechanisms that effectively incentivize users to participate in demand response are relatively scarce, and the potential of distributed energy and user load adjustment cannot be fully tapped. Summary of the invention

[0004] The purpose of this application is to provide a demand response scheduling decision method, device and equipment based on electricity-carbon combination, which can realize the coordinated adjustment and scheduling of distributed energy and user load.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In the first aspect, the present application provides a demand response scheduling decision method based on electricity-carbon combination, which is applied to a distributed framework system of a distribution network; the distributed framework system of the distribution network includes: a connected communication layer and a power layer; the communication layer includes a demand response aggregator for connecting a user end and a distribution network; the power layer is used to distribute and manage the power flow of the distribution network; the demand response scheduling decision method based on electricity-carbon combination includes:

[0007] Acquire user-side information data; the information data includes: the marginal electricity price signal of the electricity-carbon joint node corresponding to the operator at the distribution network end; the information data is determined based on the node marginal electricity price model of the electricity-carbon joint at the distribution network side; wherein the node marginal electricity price model is determined by the operator at the distribution network end using the distribution network branch flow algorithm; the node marginal electricity price model includes: objective function and constraint conditions;

[0008] Determining a user-side demand response model; the user-side demand response model is constructed based on the information data with the goal of minimizing the power coefficient;

[0009] The user-side demand response model is mathematically transformed using Markov decision making, and an iterative optimization solution is performed using a deep reinforcement learning method to obtain a scheduling decision solution; the scheduling decision solution is used to coordinate the user-side load at the user end with distributed energy generation.

[0010] Optionally, the objective function is expressed as:

[0011] ;

[0012] in, is the objective function; for distributed generation costs; Carbon emission intensity for distributed generation; is the distributed generation power; is the cost of photovoltaic power generation; is the photovoltaic power generation power; The cost of energy storage decay; Generates power for the energy storage system; The electricity price of the upper power grid; is the carbon emission intensity of the parent power grid; is the active power of the upper grid.

[0013] Optionally, the constraints include: power balance constraint, reactive power balance constraint, voltage difference constraint, power flow limitation of transmission line, voltage magnitude constraint, line current limitation, distributed power generation power limitation, capacity limitation of distributed generator, range limitation of reactive power of distributed generator, range limitation of active power of energy storage system, power range of renewable energy and power factor limitation of renewable energy.

[0014] Optionally, the objective function corresponding to the user-side demand response model is:

[0015] ;

[0016] ;

[0017] ;

[0018] in, is the objective function corresponding to the user-side demand response model; is the total number of time steps; is the total number of nodes participating in the response; is the time step number; The node number that participates in the response; It is the marginal electricity price signal of the electricity-carbon joint node; is the active load of the node after demand response; Nodes participating in the response Maximum load capacity; is the active load before demand response; The maximum response amount.

[0019] Optionally, the user-side demand response model is mathematically transformed by using Markov decision making, and a deep reinforcement learning method is used for iterative optimization and solving to obtain a scheduling decision solution, which specifically includes:

[0020] Based on the user-side demand response model, Markov decision is used to convert response to decision in discrete time steps to obtain a conversion result; the conversion result includes: an intelligent agent, a global state, a local observation, an action set, a reward function, and a state transfer function; the intelligent agent represents the user-side demand response participant at the user end; the global state is the user-side demand response state at the user end; the local observation includes the active load of the node after the demand response and the marginal price signal of the electric-carbon joint node; the action set is a set of the power layer to allocate and manage the power flow of the distribution network;

[0021] A deep reinforcement learning method is adopted, and an entropy regularization strategy based on the entropy term is used to maximize the cumulative expected reward of the agent. The conversion results are iteratively optimized and solved to obtain a scheduling decision plan.

[0022] Optionally, the process of iterative optimization and solving using a deep reinforcement learning method specifically includes:

[0023] Acquire initial parameter data; the initial parameter data includes: weight data, local observations, action sets and reward functions; the local observations, action sets and reward functions are stored in a playback buffer;

[0024] For each time step, each agent selects action data from the action set based on the current corresponding local observation and determines the current cumulative expected reward based on the reward function;

[0025] Based on the current cumulative expected reward, the action data and the current corresponding local observation are used as local experience and stored in the playback buffer;

[0026] Sample batch data from the replay buffer, based on the formulas corresponding to the policy network loss function and the value network loss function:

[0027] ;

[0028] ;

[0029] Update weight data to obtain updated weights;

[0030] in, is the policy network loss function; Mathematical symbols for expectation; is the probability distribution of the strategy network of agent i; is the observation value of the i-th agent at the t-th time step; for the playback buffer; The soft Q value estimated by the value network represents the Q value at a given state and action; is the action value of the ith agent at the tth time step; is the entropy coefficient; is the state value at the tth time step; is the action before the tth time step; is the reward value of the i-th agent at the t-th time step; is the state value of the t+1th time step; The Q value of the target Q value network is usually updated softly (soft target network) to avoid overestimation; is the target value; is the value network loss function; is the probability distribution;

[0031] Based on the updated weights and local experience, the global state and local observations are updated according to all selected action data to obtain a scheduling decision plan.

[0032] In a second aspect, the present application provides a demand response scheduling decision-making device based on electricity-carbon combination, comprising:

[0033] A data acquisition module, used to acquire user-side information data; the information data includes: the marginal electricity price signal of the electricity-carbon joint node corresponding to the operator at the distribution network end; the information data is determined based on the node marginal electricity price model of the electricity-carbon joint at the distribution network side; wherein the node marginal electricity price model is determined by the operator at the distribution network end using the distribution network branch flow algorithm; the node marginal electricity price model includes: objective function and constraint conditions;

[0034] A model determination module, used to determine a user-side demand response model; the user-side demand response model is constructed based on the information data with the goal of minimizing the power coefficient;

[0035] The scheme determination module is used to use Markov decision to mathematically transform the user-side demand response model, and use deep reinforcement learning method to iteratively optimize and solve to obtain a scheduling decision scheme; the scheduling decision scheme is used to coordinate the user-side load at the user end with distributed energy generation.

[0036] In the third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned demand response scheduling decision method based on electricity-carbon combination.

[0037] According to the specific embodiments provided in this application, this application has the following technical effects:

[0038] The present application provides a method, device and equipment for demand response scheduling decision-making based on electricity-carbon combination. By obtaining the electricity-carbon joint node marginal electricity price signal corresponding to the operator at the distribution network end, the user-side demand response model is determined; the user-side demand response model is mathematically transformed by using Markov decision making, and the deep reinforcement learning method is used for iterative optimization and solution to obtain a scheduling decision plan. Since the electricity-carbon joint node marginal electricity price signal is obtained based on the node marginal electricity price model determined by the distribution network operator using the distribution network branch flow algorithm, it fully considers the behavioral incentives of users in a multi-electric power market environment. In addition, the user-side demand response model is constructed with the goal of minimizing the power coefficient, which can mobilize the user-side user response enthusiasm. In addition, by using the deep reinforcement learning method for iterative optimization and solution, the electricity demand, that is, the distributed energy that meets and is connected to the user side, can be coordinated, and there will be no problem of large differences. It can give full play to the potential of distributed energy and user load adjustment, thereby realizing the coordinated adjustment and scheduling of distributed energy and user load. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flow chart of the demand response scheduling decision method based on electricity-carbon combination provided in this application;

[0041] Figure 2 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] This application aims to fill the gaps in existing solutions by designing an innovative demand response mechanism, providing precise user-side incentive strategies, and encouraging users to actively adjust their electricity consumption behavior based on electricity price signals to achieve more efficient resource use and response flexibility. This mechanism aims to strengthen the interaction between users and the power grid, improve the economy and sustainability of system operation, and promote the wider and more effective application of distributed energy.

[0044] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] In an exemplary embodiment, Figure 1 As shown, a demand response scheduling decision method based on electricity-carbon combination is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to a server as an example for explanation, and includes the following steps.

[0046] The present application provides a demand response scheduling decision method based on electricity-carbon combination, which is applied to a distribution network distributed framework system. The distribution network distributed framework system includes: a connected communication layer and a power layer; the communication layer includes a demand response aggregator for connecting the user end and the distribution network; the power layer is used to distribute and manage the power flow of the distribution network.

[0047] like Figure 1 As shown, the demand response scheduling decision method based on electricity-carbon combination provided in this application includes:

[0048] Step 100: Obtain user-side information data. The information data includes: the node marginal price signal of the electricity-carbon combination corresponding to the operator at the distribution network end; the information data is determined based on the node marginal price model of the electricity-carbon combination at the distribution network side; wherein the node marginal price model is determined by the operator at the distribution network end using the distribution network branch flow algorithm; the node marginal price model includes: objective function and constraints.

[0049] Step 200: Determine a user-side demand response model. The user-side demand response model is constructed based on information data with the goal of minimizing the power coefficient.

[0050] Step 300: Use Markov decision making to mathematically transform the user-side demand response model, and use deep reinforcement learning methods to iteratively optimize and solve, and obtain a scheduling decision plan. The scheduling decision plan is used to coordinate the user-side load at the user end with distributed energy generation.

[0051] In one embodiment, the objective function is expressed as:

[0052] .

[0053] in, is the objective function; for distributed generation costs; Carbon emission intensity for distributed generation; is the distributed generation power; is the cost of photovoltaic power generation; is the photovoltaic power generation power; The cost of energy storage decay; Generates power for the energy storage system; The electricity price of the upper power grid; is the carbon emission intensity of the parent power grid; is the active power of the upper grid.

[0054] The constraints include: power balance constraint, reactive power balance constraint, voltage difference constraint, power flow limitation of transmission line, voltage magnitude constraint, line current limitation, distributed generation power limitation, distributed generator capacity limitation, distributed generator reactive power range limitation, energy storage system active power range limitation, renewable energy power range and renewable energy power factor limitation.

[0055] The objective function corresponding to the user-side demand response model is:

[0056] .

[0057] .

[0058] .

[0059] in, is the objective function corresponding to the user-side demand response model; is the total number of time steps; is the total number of nodes participating in the response; is the time step number; The node number that participates in the response; It is the marginal electricity price signal of the electricity-carbon joint node; is the active load of the node after demand response; Nodes participating in the response Maximum load capacity; is the active load before demand response; The maximum response amount.

[0060] In one embodiment, a user-side demand response model is mathematically transformed using Markov decision making, and a deep reinforcement learning method is used for iterative optimization and solution to obtain a scheduling decision solution, which specifically includes:

[0061] Based on the user-side demand response model, Markov decision making is used to convert response to decision in discrete time steps to obtain the conversion result; the conversion result includes: agent, global state, local observation, action set, reward function and state transfer function; the agent represents the user-side demand response participant at the user end; the global state is the user-side demand response state at the user end; the local observation includes the active load of the node after demand response and the marginal price signal of the electricity-carbon joint node; the action set is the set of power layer allocation and management of the power flow of the distribution network.

[0062] The deep reinforcement learning method is adopted, and the entropy regularization strategy based on the entropy term aims to maximize the cumulative expected reward of the agent. The conversion results are iteratively optimized and solved to obtain the scheduling decision plan.

[0063] The process of iterative optimization and solution using deep reinforcement learning methods includes:

[0064] Initial parameter data is obtained; the initial parameter data includes: weight data, local observations, action sets and reward functions; the local observations, action sets and reward functions are stored in the playback buffer.

[0065] For each time step, each agent selects action data from the action set based on the current corresponding local observation and determines the current cumulative expected reward based on the reward function.

[0066] Based on the current cumulative expected reward, the action data and the current corresponding local observation are used as local experience and stored in the playback buffer.

[0067] Sample batch data from the replay buffer based on the formulas corresponding to the policy network loss function and the value network loss function:

[0068] .

[0069] .

[0070] Update weight data to obtain updated weights;

[0071] in, is the policy network loss function; Mathematical symbols for expectation; is the probability distribution of the strategy network of agent i; is the observation value of the i-th agent at the t-th time step; for the playback buffer; The soft Q value estimated by the value network represents the Q value at a given state and action; is the action value of the ith agent at the tth time step; is the entropy coefficient; is the state value at the tth time step; is the action before the tth time step; is the reward value of the i-th agent at the t-th time step; is the state value of the t+1th time step; The Q value of the target Q value network is usually updated softly (soft target network) to avoid overestimation; is the target value; is the value network loss function; is a probability distribution.

[0072] Based on the updated weights and local experience, the global state and local observations are updated according to all selected action data to obtain a scheduling decision plan.

[0073] This application aims to achieve effective coordination of user-side demand response in the distribution network, based on the distributed framework system of the distribution network. The architecture of the system is divided into two parts: the communication layer and the power layer. The communication layer is operated by the demand response aggregator (Aggregator). As a bridge between the user side and the distribution network, the aggregator can collect, transmit and exchange limited information to help users on the user side make scientific and reasonable decisions in a complex electricity market environment. The power layer involves the actual distribution and management of power flows, and reasonably configures distributed energy resources (DERs), including energy demand (ED) response units, distributed generation (DG), and renewable energy-based generators, namely renewable energy systems (RES), to ensure the diversity and sustainability of the energy structure.

[0074] Users receive the C-LMP signal of the electricity-carbon joint node marginal price calculated in real time by the Distribution System Operator (DSO), where LMP refers to the Locational Marginal Price node marginal price, to optimize their electricity consumption mode, achieve flexible response and maximize energy efficiency under low-carbon conditions. DSO plays a key role in this mechanism, responsible for managing controllable distributed energy and performing optimal power flow (OPF) calculations to ensure the stability and security of distribution network operation.

[0075] Different from the traditional centralized control method, this application assumes that all users at the user end have the ability to make independent decisions and have independent electricity demand and preferences. Users operate under a distributed framework, and personal information is not stored in the central control system, thus ensuring privacy and data security. The introduction of the aggregator aims to bridge the problem of information asymmetry between users, provide them with necessary information and reasonable incentive mechanisms, promote collaboration and resource sharing among users, so as to optimize the overall demand response behavior and improve the flexibility of the system and user participation.

[0076] In practical applications, the operating steps of the method mentioned in this application can be as follows.

[0077] S1: Establish a user-side demand response model.

[0078] The goal of the user-side demand response model is to minimize electricity costs and optimize response behavior, that is, it is built with the goal of minimizing the power coefficient.

[0079] The objective function corresponding to the user-side demand response model can be expressed as:

[0080] (1)

[0081] in, is the objective function corresponding to the user-side demand response model; is the total number of time steps; is the total number of nodes participating in the response; is the time step number; The node number that participates in the response; It is the marginal electricity price signal of the electricity-carbon joint node; is the active load of the node after demand response.

[0082] Responsive afterload constraints:

[0083] (2)

[0084] in, Nodes participating in the response The maximum load capacity.

[0085] (3)

[0086] in, is the active load before demand response; The maximum response amount.

[0087] S2: The user-side demand response model is formulated as a distributed partially observable Markov decision process.

[0088] The significance of transforming the demand response problem on the user side into a distributed partially observable Markov decision process is to structure and formalize the complex power demand response coordination problem, making it suitable for solving by reinforcement learning algorithms. The role of this transformation is mainly reflected in three aspects: first, it provides a clear mathematical modeling for the problem, making elements such as state, action, and reward clear, which is convenient for algorithm processing; second, by introducing partially observable characteristics, it can effectively deal with the uncertainty in the power system and the randomness of user behavior, thereby enhancing the adaptability of the model to the real environment; finally, the distributed design supports users to make independent decisions with only local information, thereby realizing decentralized control, ensuring data privacy and improving the flexibility of the system. In short, this modeling step not only provides a basis for the application of multi-agent reinforcement learning, but also enables users to perform real-time response optimization after receiving the marginal electricity price signal of the electricity-carbon joint node, which promotes the improvement of energy efficiency and system stability.

[0089] Specifically, under discrete time steps, the distributed partially observable Markov decision process is defined as, which includes I agents (representing the user-side demand response participants), a set of global states , a set of local observations , a set of actions , a set of reward functions And the state transfer function . is the discount factor. It is the global state.

[0090] Let the time interval between two consecutive time steps be Hour.

[0091] At each time step , each agent According to its local observation, that is, the observation value of the i-th agent at the t-th time step , and based on the control strategy Select an action , which is the action value of the ith agent at the tth time step. The environment is based on the state transition function Transfer to the next state and get the state value of the t+1th time step Each agent Get rewards And get new local observations This process continues until each agent The trajectory of observations, actions, and rewards : , which is mapped to .

[0092] in, For the The local observation of an agent at the first time step; For the The action data of an agent at the first time step; For the The reward of an agent at the first time step; For the The local observation of an agent at the second time step; For the The agent in The reward for each time step.

[0093] Each agent Aims to maximize its cumulative discounted reward, that is, the cumulative expected reward ,in is the discount factor, Hours is the daily time frame. is the attenuation coefficient.

[0094] Observation: At time step , each agent The local observation of is: The observation consists of two parts: exogenous states that are not affected by the action, including the node location of users with demand response capabilities , the active power demand of the node and reactive power demand and the active power of renewable energy at the node , as the endogenous state of the action feedback signal, is modeled as the node load after demand response.

[0095] Action: At time step , each agent Control its actions: .

[0096] in, is the node active power demand response change; For intelligent agents Action collection.

[0097] State transition: State transition is determined by the following formula: .

[0098] State transition is affected by the current state of the environment , local observations of all agents and the action before the tth time step And the random parameters of the environment affected by the combination of.

[0099] in, Active power price for the main grid; Reactive power price for the main grid; is the active load of the node after demand response; Reactive power demand.

[0100] Determined by market conditions, , Affected by energy use behavior, the proportion of renewable energy generation The distribution of these factors is very difficult to model explicitly. In the field of machine learning, reinforcement learning solves this problem through a data-driven approach that does not rely on an accurate model of the underlying uncertainty, but learns dynamic characteristics directly from the data source.

[0101] Reward: At time step At the end, each agent Get its rewards . First, users connected to the grid who can provide demand response aim to maximize revenue by providing active and reactive power services. This can be calculated based on the size of the demand response volume (i.e., action) and the marginal electricity price of the node to which it is connected to the electricity-carbon union. Secondly, all users need to ensure that their total load can meet the daily production needs of the enterprise. Since the distributed partially observable Markov decision process is a dynamic decision process, it requires sufficient total load physical constraints as part of the environment, which makes it impossible for the demand response regulation agent to directly obtain it. Therefore, a reward shaping mechanism is needed to punish violations of this constraint, and it is assumed that the mechanism is effective in solving this challenge.

[0102] When users choose to participate in grid demand response, Node marginal electricity price based on electricity-carbon combination Calculated charging (discharging) costs (benefits), is the active load before demand response. The second is the penalty for not meeting the sufficient load at the end of each day, where is the penalty factor corresponding to the demand response item, , is the active load before demand response.

[0103] .

[0104] S3: Node marginal electricity price C-LMP model of electricity-carbon integration on the distribution network side.

[0105] The C-LMP model of the electricity-carbon joint distribution network plays a key bridging role, aiming to integrate electricity costs and carbon emission costs into an actionable real-time price signal to guide users and distributed energy participants to optimize their behavior. It serves as the input of the user demand response model to help users adjust their electricity strategies based on the signal. At the same time, it is incorporated into the environmental observations in the Markov Decision Processes (MDP) modeling to support users in optimizing decisions at each time step.

[0106] According to the current regulations of the carbon trading market, the current carbon trading market mainly focuses on direct carbon emissions in scope 1. In this context, through the advanced flue gas continuous emission monitoring system (CEMS), it is assumed that the grid operator can monitor the data of all direct carbon emission sources in the jurisdiction in real time. Since power generation companies have participated in the national carbon market, the carbon emission cost has been embedded in their power generation costs, so the electricity price paid by users can be regarded as covering their carbon emission responsibilities. In the same power supply area, users are regarded as sharing a unified carbon emission responsibility. The marginal node carbon electricity price (C-LMP) is calculated through the real-time power generation side carbon price to share the environmental responsibility in the region more fairly and avoid the power generation companies and high energy-consuming enterprises bearing too high a carbon burden alone. This method introduces carbon price signals, which helps to motivate power generation companies to carry out low-carbon transformation and avoid the complexity of the green electricity market auction process, thereby rationalizing the coupling relationship between the carbon market and the electricity market.

[0107] More specifically, a distribution network branch flow algorithm operated by a DSO is introduced. For each time step t, the DSO will solve the following optimization problem according to the existing situation, calculate the marginal electricity price of the electricity-carbon combination, and then adjust the internal power generation equipment to maximize its own objective function. The objective function is a cost minimization problem, including the power cost of distributed generation (DG), photovoltaics (PV), energy storage systems (ESS), and power purchases from the upper-level power grid.

[0108] (4)

[0109] The constraints are as follows:

[0110] (5)

[0111] (6)

[0112] (7)

[0113] (8)

[0114] (9)

[0115] (10)

[0116] (11)

[0117] (12)

[0118] (13)

[0119] (14)

[0120] (15)

[0121] (16)

[0122] Among them, formula (5) is the node The power balance constraint ensures that the total power generation is equal to the total load demand minus the transmission power loss, where is the active power of the upper grid, is the distributed generation active power, is the active power of renewable energy, is the node active load after demand response, It is the active network loss of the line.

[0123] Assemble for the upper grid; For generator collection; Gather for renewable energy generation; Index the generator; is a set of demand response nodes; Indexing for demand response; is the node number of any connected node; is a network branch collection; A collection of network nodes.

[0124] The above formula (6) is the node The reactive power balance constraint ensures that the total reactive power generation is equal to the total reactive power demand and the transmission reactive power loss, where and is the square value of the voltage of any adjacent node, and are the resistance and reactance on the corresponding connection line, is the square of the current of any adjacent node. is the reactive power of the upper grid. is the reactive power of distributed generation, is the reactive power of renewable energy, is the node reactive load after demand response, It is the active network loss of the line.

[0125] The above formula (7) is the connection The node voltage difference constraint represents the relationship between the voltage difference and the transmission power.

[0126] Formula (8) above is the power flow limit of the transmission line, ensuring that the power flow does not exceed the line capacity.

[0127] Formula (9) above is the node voltage size constraint, which ensures that the node voltage is within the safe range, where and are the squares of the maximum and minimum node voltages, respectively.

[0128] Formula (10) above is the line current limit, which ensures that the line current does not exceed the maximum capacity, where It is the maximum value of the square of the current magnitude of the adjacent nodes.

[0129] The formula (11) above is the limit of distributed generation power, ensuring that the power is within the allowable range. and are the minimum and maximum values ​​of distributed generation power respectively.

[0130] Formula (12) above is the capacity limit of the distributed generator, ensuring that the apparent power does not exceed the maximum capacity The formula (13) above is the range limit of the reactive power of distributed generators, ensuring that the reactive power is within the allowable range, where is the power angle of the distributed generator.

[0131] Formula (14) above is the range limit of the active power of the energy storage system, ensuring that the energy storage system operates within the allowable range. and They are the minimum and maximum values ​​of energy storage charging and discharging respectively. is the energy storage index number; For energy storage collection.

[0132] Formula (15) above is the power range of renewable energy, ensuring that its output power is within the allowable range, where is the maximum power of renewable energy.

[0133] Formula (16) above is the power factor limit of renewable energy, which limits the reactive power output, where is the power angle of the renewable energy generator.

[0134] S4: Use the MASAC algorithm to solve the above distributed partially observable Markov decision process.

[0135] Multi-agent Soft Actor-Critic MASAC is a deep reinforcement learning method that extends the single-agent soft actor-critic SAC algorithm and is suitable for multi-agent environments. In MASAC, each agent is trained through an independent policy network (Actor) and value network (Critic) to optimize its strategy and maximize its benefits in a dynamic environment. The goal of the algorithm is to achieve more stable and robust policy learning in multi-agent systems by maximizing the cumulative expected reward of each agent while introducing an entropy term to encourage policy exploration.

[0136] Policy update: Each agent Strategy According to its local observation Select Action , the goal of policy updating is to maximize the following objective function.

[0137] (17)

[0138] in, is the entropy coefficient, which is used to balance exploration and utilization, is the playback buffer.

[0139] Value function update: Each agent Critics Network The update is performed using the temporal difference (TD) method. The update objective is to minimize the following mean square error.

[0140] (18)

[0141] in, is the target value.

[0142] (19)

[0143] Entropy Regularization: Entropy Term The introduction of ensures that the strategy will not converge to a deterministic strategy too early, thereby increasing the possibility of exploration. Can be fixed or dynamically updated via an automatic tuning strategy to control the level of exploration.

[0144] The MASAC algorithm is used to solve the above distributed partially observable Markov decision process as follows.

[0145] against The steps of the MASAC algorithm for agents are:

[0146] Parameter initialization: Initialization parameters The policy and critic networks for each agent and copy them to their respective target network weights .

[0147] Experience Collection: Initialize a shared replay buffer for all agents , the agent interacts with the environment, collects observations, actions and rewards, and stores them in the replay buffer.

[0148] Main loop:

[0149] At each time step , each agent Choose actions and interact with the environment based on the current observations: For each episode (i.e., each day), arrive Start loop:

[0150] Random process initialization: Initialize a random process for action exploration ,in is the variance.

[0151] State and observation initialization: record the experience of each agent and update the global and local state .

[0152] Time step loop: For each time step (i.e., every hour) arrive Start loop.

[0153] Action selection: For each agent According to current observations Select Action .

[0154] Execute actions: All the actions of the agents Applied to distribution network.

[0155] Power flow calculation and node marginal price acquisition: The distribution system operator (DSO) solves the branch power flow algorithm (4) and obtains . is the LMP of each node.

[0156] Reward and Observation Records: For each agent, observe the current reward and the next observation , then the local experience Passed to the demand response aggregator.

[0157] Experience storage: The demand response aggregator connects the C-LMP and local experience together and stores it in the playback buffer middle.

[0158] State and Observation Updates: Update Global State and local observation .

[0159] Mini-batch sampling: from the playback buffer Sampling a mini-batch of data . Where j is the sequence number and J is the total number of sampled data. and All are data from sampled mini-batch data.

[0160] Critics Network Update: Updates to online critics network.

[0161] Network Weight Update: Updates online network weights.

[0162] End the round cycle.

[0163] By using reinforcement learning for solving, this application realizes the intelligent regulation and benefit maximization of user-side response behavior. Specifically, the reinforcement learning algorithm enables users (or agents) to learn and optimize their electricity consumption strategies through dynamic interaction with the environment. After receiving real-time market signals such as the joint node marginal price of electricity and carbon (C-LMP), users respond intelligently through the trained model and flexibly adjust the electricity consumption mode to adapt to changes in prices and carbon emissions. The reward function guides users to maximize benefits when optimizing behaviors, such as increasing electricity consumption during low-price or low-carbon periods and reducing electricity consumption during high prices, thereby effectively reducing costs. The adaptive characteristics of reinforcement learning enable users to adjust decisions in real time in the face of market fluctuations and maximize long-term benefits. At the same time, multi-agent reinforcement learning (MARL) further supports users to respond collaboratively in a distributed framework, improving the overall efficiency and stability of the system. Therefore, this solution method not only realizes intelligent user response, but also promotes the efficient and low-carbon operation of the distribution network.

[0164] In summary, this application aims to optimize the electric-carbon joint node marginal electricity price strategy in the active distribution network. By constructing a distributed partially observable Markov decision process and applying the multi-agent soft actor-critic (MASAC) algorithm, the invention realizes the intelligent regulation and benefit maximization of the user-side response behavior. The implementation of the calculation process enables the demand response aggregator to accurately adjust in real time based on the electric-carbon joint regional marginal price mechanism, which promotes the autonomous collaboration and flexible response of the user side. The mechanism effectively coordinates the user-side load and distributed energy generation, ensures the stability and low-carbon development of the distribution network, and provides users with economic incentives for active participation. This invention has shown significant advantages in achieving fair sharing of environmental responsibilities, reducing overall carbon emissions, and improving energy utilization efficiency, and has promoted the important role of the distribution network in the green energy transition. The mechanism and algorithm proposed in this application are expected to be applied to a wider range of power system scenarios, further promote the maturity and promotion of demand response technology, and provide new solutions for realizing an intelligent and low-carbon energy system.

[0165] Based on the same inventive concept, the embodiment of the present application also provides a demand response scheduling decision device based on electricity-carbon integration for implementing the above-mentioned demand response scheduling decision method based on electricity-carbon integration. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the demand response scheduling decision device based on electricity-carbon integration provided below can refer to the above limitations on the demand response scheduling decision method based on electricity-carbon integration, and will not be repeated here.

[0166] In an exemplary embodiment, a demand response scheduling decision device based on electricity-carbon combination is provided, comprising:

[0167] The data acquisition module is used to obtain user-side information data. The information data includes: the node marginal price signal of the electricity-carbon joint corresponding to the operator on the distribution network side; the information data is determined based on the node marginal price model of the electricity-carbon joint on the distribution network side; wherein, the node marginal price model is determined by the operator on the distribution network side using the distribution network branch flow algorithm; the node marginal price model includes: objective function and constraints.

[0168] The model determination module is used to determine the user-side demand response model. The user-side demand response model is constructed based on the information data with the goal of minimizing the power coefficient.

[0169] The scheme determination module is used to mathematically transform the user-side demand response model using Markov decision making, and iteratively optimize and solve using deep reinforcement learning methods to obtain a scheduling decision scheme. The scheduling decision scheme is used to coordinate the user-side load at the user end with distributed energy generation.

[0170] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store user-side demand response scheduling decision data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a demand response scheduling decision method based on electricity-carbon combination is implemented.

[0171] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0172] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the corresponding device owner. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0173] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0174] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A demand response scheduling decision method based on electricity-carbon combination, characterized in that: The electricity-carbon combined demand response scheduling decision method is applied to the distributed framework system of the distribution network; The distribution network distributed framework system includes: a connected communication layer and a power layer; the communication layer includes a demand response aggregator for connecting the user end and the distribution network; the power layer is used to distribute and manage the power flow of the distribution network; the demand response scheduling decision method based on electricity-carbon combination includes: Acquire user-side information data; the information data includes: the marginal electricity price signal of the electricity-carbon joint node corresponding to the operator at the distribution network end; the information data is determined based on the node marginal electricity price model of the electricity-carbon joint at the distribution network side; wherein the node marginal electricity price model is determined by the operator at the distribution network end using the distribution network branch flow algorithm; the node marginal electricity price model includes: objective function and constraint conditions; The expression of the objective function is: Where F is the objective function; for distributed generation costs; Carbon emission intensity for distributed generation; is the distributed generation power; is the cost of photovoltaic power generation; is the photovoltaic power generation power; The cost of energy storage decay; Generates power for the energy storage system; The electricity price of the upper power grid; is the carbon emission intensity of the parent power grid; is the active power of the upper power grid; Determining a user-side demand response model; the user-side demand response model is constructed based on the information data with the goal of minimizing the power coefficient; The user-side demand response model is mathematically transformed using Markov decision making, and a deep reinforcement learning method is used for iterative optimization and solution to obtain a scheduling decision solution, which specifically includes: Based on the user-side demand response model, Markov decision is used to convert response to decision in discrete time steps to obtain a conversion result; the conversion result includes: an intelligent agent, a global state, a local observation, an action set, a reward function, and a state transfer function; the intelligent agent represents the user-side demand response participant at the user end; the global state is the user-side demand response state at the user end; the local observation includes the active load of the node after the demand response and the marginal price signal of the electric-carbon joint node; the action set is a set of the power layer to allocate and manage the power flow of the distribution network; A deep reinforcement learning method is adopted, and an entropy regularization strategy based on the entropy term is used to iteratively optimize and solve the conversion results with the goal of maximizing the cumulative expected reward of the intelligent agent to obtain a scheduling decision plan; the scheduling decision plan is used to coordinate the user-side load at the user end with the distributed energy generation.

2. The demand response scheduling decision method based on electricity-carbon combination according to claim 1 is characterized in that: The constraints include: power balance constraint, reactive power balance constraint, voltage difference constraint, power flow limitation of transmission line, voltage magnitude constraint, line current limitation, distributed power generation limitation, capacity limitation of distributed generator, range limitation of reactive power of distributed generator, range limitation of active power of energy storage system, power range of renewable energy and power factor limitation of renewable energy.

3. The demand response scheduling decision method based on electricity-carbon combination according to claim 1 is characterized in that: The objective function corresponding to the user-side demand response model is: Where D is the objective function corresponding to the user-side demand response model; T is the total number of time steps; B is the total number of nodes participating in the response; t is the time step number; b is the node number participating in the response; It is the marginal electricity price signal of the electricity-carbon joint node; is the active load of the node after demand response; is the maximum load of node b participating in the response; P b，t is the active load before demand response; ΔP b，max is the maximum response amount.

4. The demand response scheduling decision method based on electricity-carbon combination according to claim 1 is characterized in that: The process of iterative optimization and solution using deep reinforcement learning methods includes: Acquire initial parameter data; the initial parameter data includes: weight data, local observations, action sets and reward functions; the local observations, action sets and reward functions are stored in a playback buffer; For each time step, each agent selects action data from the action set based on the current corresponding local observation and determines the current cumulative expected reward based on the reward function; Based on the current cumulative expected reward, the action data and the current corresponding local observation are used as local experience and stored in the playback buffer; Sample batch data from the replay buffer, based on the formulas corresponding to the policy network loss function and the value network loss function: Update weight data to obtain updated weights; Among them, J(π i ) is the policy network loss function; Mathematical symbol for expectation; π i (a i，t |o i，t ) is the probability distribution of the strategy network of agent i; i，t is the observation value of the i-th agent at the t-th time step; is the playback buffer; Q i (o i，t , a i，t ) is the soft Q value estimated by the value network, which represents the Q value at a given state and action; a i，t is the action value of the ith agent at the tth time step; α is the entropy coefficient; s t is the state value at the tth time step; a 1：I，t is the action before the tth time step; r i，t is the reward value of the ith agent at the tth time step; s t+1 is the state value of the t+1th time step; Q i (s t , a 1:I，t ) is the Q value of the target Q value network; y i，t is the target value; L(Q i ) is the value network loss function; π i is the probability distribution; Based on the update weights and local experience, the global state and local observations are updated according to all selected action data to obtain a scheduling decision plan.

5. A demand response scheduling decision-making device based on electricity-carbon combination, characterized in that: The demand response scheduling decision device based on electricity-carbon combination is implemented by the demand response scheduling decision method based on electricity-carbon combination according to any one of claims 1 to 4; the demand response scheduling decision device based on electricity-carbon combination includes: A data acquisition module, used to acquire user-side information data; the information data includes: the marginal electricity price signal of the electricity-carbon joint node corresponding to the operator at the distribution network end; the information data is determined based on the node marginal electricity price model of the electricity-carbon joint at the distribution network side; wherein the node marginal electricity price model is determined by the operator at the distribution network end using the distribution network branch flow algorithm; the node marginal electricity price model includes: objective function and constraint conditions; A model determination module, used to determine a user-side demand response model; the user-side demand response model is constructed based on the information data with the goal of minimizing the power coefficient; The scheme determination module is used to use Markov decision to mathematically transform the user-side demand response model, and use deep reinforcement learning method to iteratively optimize and solve to obtain a scheduling decision scheme; the scheduling decision scheme is used to coordinate the user-side load at the user end with distributed energy generation.

6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the demand response scheduling decision method based on electricity-carbon combination as described in any one of claims 1 to 4.

Images