Micro-grid dispatching method and system fusing user side and electric vehicle pricing mechanism

Abstract

The application discloses a micro-grid dispatching method and system fusing a user side and an electric vehicle pricing mechanism, determines a user side time-of-use electricity price and an electric vehicle dynamic electricity price, establishes a collaborative dispatching model, takes a weighted sum of step rewards of each dispatching moment, final rewards of a dispatching period, and expert intervention rewards of pricing guidance as a total reward function, establishes an evaluation index system of the total reward function, calculates the step rewards of each dispatching moment and the final rewards of the dispatching period based on the evaluation index system, establishes a supply-demand balance index based on the pricing guidance, determines whether expert intervention is needed according to a system energy storage state of charge and the supply-demand balance index, respectively sets the expert intervention rewards when the expert intervention is needed and the expert intervention rewards when the expert intervention is not needed, takes maximization of expected cumulative total rewards in the dispatching period as an optimization target, solves the collaborative dispatching model, obtains collaborative dispatching optimal instructions, and realizes collaborative optimization of system side energy dispatching and demand side pricing incentives.

Description

Technical Field

[0001] This invention relates to the field of power system operation and control technology, and in particular to a microgrid reinforcement learning optimization scheduling method and system that integrates user-side and electric vehicle pricing mechanisms. Background Technology

[0002] As the core carrier for distributed energy consumption, microgrids require energy management that balances the intermittency of renewable energy sources such as wind and solar power, the elasticity of user-side loads, and the mobile energy storage characteristics of electric vehicles. This is a crucial link in achieving efficient energy utilization and flexible grid regulation. With the large-scale popularization of electric vehicles and the development of user-side demand response technologies, the scheduling potential of demand-side resources is gradually becoming apparent. However, existing microgrid optimization scheduling methods still have significant limitations.

[0003] Current microgrid energy management models based on deep reinforcement learning mostly only consider the energy balance and economic cost on the system side, failing to deeply couple user-side demand response and the pricing incentive mechanism for electric vehicle charging and discharging with the reinforcement learning framework, resulting in low demand-side resource utilization efficiency. Furthermore, existing technologies do not incorporate electric vehicle cluster charging and discharging as independent scheduling actions into the reinforcement learning action space, treating it only as a passive load, thus failing to fully leverage the mobile energy storage and peak-shaving capabilities of electric vehicles. In addition, the reward function design of traditional reinforcement learning lacks consideration of demand-side indicators, resulting in sparse reward signals and requiring improvements in algorithm convergence speed and scheduling robustness. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a microgrid reinforcement learning scheduling method and system that integrates user-side and electric vehicle pricing mechanisms. It incorporates pricing incentives and demand-side scheduling actions into a reinforcement learning framework, achieving synergistic optimization of system-side energy scheduling and demand-side pricing incentives, thereby improving the economic operability, renewable energy absorption rate, and supply-demand balance of the microgrid system.

[0005] The present invention adopts the following technical solution.

[0006] This invention proposes a microgrid reinforcement learning scheduling method that integrates user-side and electric vehicle pricing mechanisms, comprising: Acquire operational status data of the microgrid system; determine user-side time-of-use electricity pricing and dynamic electricity pricing for electric vehicles; Establish a collaborative scheduling model, using the weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing as the total reward function of the collaborative scheduling model; Establish an evaluation index system for the total reward function; based on the evaluation index system, calculate the step reward at each scheduling moment and the final reward for the scheduling cycle; establish a supply and demand balance index based on pricing guidance; determine whether expert intervention is needed based on the system's energy storage state of charge and supply and demand balance index; set expert intervention rewards when expert intervention is needed and expert intervention rewards when no expert intervention is needed. With the goal of maximizing the expected cumulative total reward within the scheduling cycle, the cooperative scheduling model is iteratively solved to obtain the optimal cooperative scheduling command for the microgrid.

[0007] Operational status data includes: wind power output, photovoltaic power output, critical load power, responsive flexible load baseline power, state of charge of high power density energy storage, state of charge of high capacity energy storage, and average state of charge of electric vehicle clusters.

[0008] A target function for user-side time-of-use electricity price is established using user-side response quantification indicators and electricity price deviation penalty terms. The user-side time-of-use electricity price is determined with the goal of maximizing the target function. Based on the user-side time-of-use electricity price and the total load-side electricity demand under the incentive of user-side time-of-use electricity price, the dynamic electricity price for electric vehicles is determined under the real-time supply and demand balance constraint of the system.

[0009] The optimization objective is to maximize the objective function, as shown in the following equation:

[0010] In the formula, For scheduling time The objective function of user-side time-of-use electricity pricing, For scheduling time responsive flexible load power, To provide a responsive reference power for flexible loads, For user-side response metrics, This is the penalty coefficient for electricity price deviation. For scheduling time User-side time-of-use electricity pricing This is the benchmark electricity price for users.

[0011] The responsive flexible load power under the user-side time-of-use pricing incentive is shown in the following formula:

[0012] In the formula, For scheduling time responsive flexible load power under time-of-use pricing incentives on the user side This refers to the elasticity coefficient of user-side demand response. >0; The total load-side electricity demand under the user-side time-of-use pricing incentive is shown in the following formula:

[0013] In the formula, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives For scheduling time The critical load power.

[0014] Dynamic electricity price for electric vehicles As shown in the following formula:

[0015]

[0016] In the formula, For scheduling time Dynamic electricity pricing for electric vehicles For scheduling time The system's remaining power, This is the dynamic electricity price adjustment coefficient. For scheduling time Wind power output, For scheduling time Photovoltaic power output.

[0017] A state space is established using the state of charge of wind power output, photovoltaic power output, high power density energy storage, high capacity energy storage, average state of charge of electric vehicle clusters, total load-side power demand under user-side time-of-use pricing incentives, user-side time-of-use pricing, and dynamic pricing of electric vehicles. The action space is established using the charging and discharging power of high power density energy storage, the charging and discharging power of high capacity energy storage, the interaction power between microgrids and the main grid, and the charging and discharging power of electric vehicle clusters. The total reward function is shown below:

[0018] In the formula, For scheduling time Step-by-step rewards As the final reward, For scheduling time Pricing-guided expert intervention rewards. , , All are weighting coefficients, satisfying + + =1.

[0019] The evaluation index system for the total reward function includes: system self-balancing rate, system comprehensive operating cost index, system reliability rate, user-side responsiveness, and electric vehicle charging and discharging satisfaction. At each scheduling time Calculate step rewards As shown in the following formula:

[0020] In the formula, The penalty coefficient is... For scheduling time The Middle The system self-balancing rate at each time step For scheduling time The Middle System reliability rate at each time step For scheduling time The Middle User-side responsiveness at each time step For scheduling time The Middle Satisfaction with electric vehicle charging and discharging at a specific time step. For scheduling time The Middle The system's overall operating cost index at each time step.

[0021] Scheduling time System self-balancing rate involving electric vehicles As shown in the following formula:

[0022] In the formula, For scheduling time Main network interaction power, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives , Scheduling time The charging and discharging power of electric vehicle clusters.

[0023] Scheduling time System comprehensive operating cost index As shown in the following formula:

[0024] In the formula, For scheduling time The total net cost of the system. For scheduling time The benchmark cost without energy storage or pricing.

[0025] Scheduling time User-side and electric vehicle system reliability As shown in the following formula:

[0026] In the formula, For scheduling time Wind power output, For scheduling time Photovoltaic power output, For scheduling time High power density energy storage charging and discharging power, For scheduling time High-capacity energy storage charging and discharging power, For scheduling time The interaction power between the microgrid and the main grid, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives , Scheduling time The charging and discharging power of electric vehicle clusters.

[0027] Scheduling time User-side responsiveness As shown in the following formula:

[0028] In the formula, For scheduling time responsive flexible load power under time-of-use pricing incentives on the user side This is the reference power for responsive flexible loads.

[0029] Scheduling time Electric vehicle charging and discharging satisfaction As shown in the following formula:

[0030]

[0031]

[0032] In the formula, These are the weighting coefficients. [0,1], For scheduling time Price satisfaction with electric vehicles For scheduling time Travel satisfaction with electric vehicles; For scheduling time The cost of charging and discharging electric vehicle clusters , These represent the lower and upper limits of the charging and discharging cost for electric vehicle clusters, respectively. For scheduling time electric vehicles The state of charge, For electric vehicles, the reference state of charge , These are the lower and upper limits of the state of charge (SOC) of electric vehicles, respectively. This refers to the number of electric vehicles.

[0033] Throughout the scheduling cycle Calculate the final reward at the end. As shown in the following formula:

[0034] In the formula, , , , , Scheduling time The rewards include system self-balancing rate rewards, economic cost rewards, system reliability rate rewards, user-side responsiveness rewards, and electric vehicle charging and discharging satisfaction rewards.

[0035] Scheduling time The system self-balancing rate reward is shown in the following formula:

[0036] In the formula, As the reward coefficient, For scheduling time The energy storage discharge power, with the superscript E indicating high power density energy storage or high capacity energy storage. For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives Scheduling time The discharge power of the electric vehicle cluster; Scheduling time The economic cost reward is shown in the following formula:

[0037] Scheduling time The reliability bonus is shown in the following formula:

[0038] Scheduling time The user-side responsiveness reward is shown in the following formula:

[0039] Scheduling time The electric vehicle charging and discharging satisfaction reward is shown in the following formula: .

[0040] The supply and demand balance indicator based on pricing guidance is shown in the following formula:

[0041] In the formula, For scheduling time Supply and demand balance indicators For scheduling time The system's remaining power, , Scheduling time The charging and discharging power of electric vehicle clusters For scheduling time Total electricity demand on the load side under the incentive of time-of-use pricing on the user side; When scheduling time System energy storage state of charge Greater than 85% and A value greater than 0.1 indicates a power surplus in the system, requiring expert intervention to force energy storage charging and generate expert-intervention energy storage power charging commands. The superscript E indicates high power density energy storage or high capacity energy storage. When scheduling time System energy storage state of charge Less than 25% and A value less than -0.1 indicates a power deficit in the system, requiring expert intervention to force energy storage discharge and generate an expert-mandated energy storage power discharge command. ; When scheduling time System energy storage state of charge Not more than 85% and not less than 25%, or Not greater than 0.1 and When the value is not less than -0.1, no expert intervention is required; the system's energy storage intelligent agent autonomously makes decisions and generates energy storage power commands. ; Scheduling time Expert intervention reward when expert intervention is required Intervention reward coefficient Expert intervention reward when no intervention is needed Intervention penalty coefficient .

[0042] With the goal of maximizing the expected cumulative total reward within the scheduling cycle, the DDQN algorithm and PG algorithm are used to iteratively solve the cooperative scheduling model to obtain the optimal cooperative scheduling command for the microgrid. The expected total cumulative reward is maximized as shown in the following formula:

[0043] In the formula, To optimize the objective function, Let be the policy function, representing the time at which the scheduling occurs. state space Lower execution action space The probability, For policy function The expected function under, The scheduling period is [number].

[0044] In another aspect, this invention proposes a microgrid reinforcement learning scheduling system that integrates user-side and electric vehicle pricing mechanisms, comprising: The pricing module is used to acquire operating status data of the microgrid system and determine the user-side time-of-use electricity price and the dynamic electricity price for electric vehicles. The modeling module is used to establish a collaborative scheduling model. The total reward function of the collaborative scheduling model is the weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing. An evaluation index system for the total reward function is established. Based on the evaluation index system, the step rewards at each scheduling moment and the final reward of the scheduling cycle are calculated. A supply-demand balance index based on pricing guidance is established. Based on the system's energy storage state of charge and the supply-demand balance index, it is determined whether expert intervention is needed. Expert intervention rewards are set separately for when expert intervention is needed and when it is not. The scheduling module is used to iteratively solve the collaborative scheduling model with the optimization objective of maximizing the expected cumulative total reward within the scheduling cycle, and obtain the optimal collaborative scheduling instruction for the microgrid.

[0045] The present invention is also a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to perform operations according to the instructions to execute the steps of the method.

[0046] The present invention is also a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0047] Compared with the prior art, the beneficial effects of the present invention include at least the following: 1. This invention incorporates the charging and discharging power of electric vehicle clusters into the reinforcement learning action space, and combines dynamic electricity price incentives with scheduling command control to give full play to the mobile energy storage and peak-shaving role of electric vehicles. This solves the problem of existing technologies treating electric vehicles as passive loads and having low utilization efficiency, and significantly improves the scheduling value of demand-side resources.

[0048] 2. The two-stage pricing mechanism constructed in this invention sets time-of-use and dynamic electricity prices for flexible loads on the user side and electric vehicles respectively, realizing differentiated pricing incentives on the demand side, effectively capturing the price sensitivity characteristics of different demand-side resources, and improving user responsiveness and electric vehicle charging and discharging satisfaction.

[0049] 3. This invention integrates the two-stage pricing results into the state space of the Markov decision process and incorporates demand-side pricing indicators into the multi-stage reward function, thereby achieving deep coupling between the pricing mechanism and the deep reinforcement learning framework. This makes the agent's decisions more closely aligned with the actual response characteristics of the demand side and avoids the problem of incomplete decision-making basis in traditional models.

[0050] 4. The pricing-guided expert intervention strategy designed in this invention, based on the supply and demand balance state constraining the agent's action space, effectively reduces the randomness of reinforcement learning training, avoids the agent from falling into suboptimal scheduling strategies, and improves the convergence speed and scheduling robustness of the algorithm.

[0051] 5. This invention supports the implementation of two algorithms, DDQN and PG, and is suitable for discrete and continuous action space requirements. Among them, the PG algorithm performs better in continuous scheduling decision-making. Its convergence speed, scheduling economy and real-time performance can meet the real-time scheduling requirements of actual microgrid operation, and can achieve multi-objective optimization of minimizing system operating costs and maximizing renewable energy consumption rate. Attached Figure Description

[0052] Figure 1 The flowchart illustrates a microgrid reinforcement learning optimization scheduling method that integrates user-side and electric vehicle pricing mechanisms, as provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0054] In Embodiment 1, this invention provides a reinforcement learning-based optimization scheduling method for microgrids that integrates user-side and electric vehicle pricing mechanisms. The microgrid system includes wind and solar power generation equipment, high power density energy storage (HPE), high capacity energy storage (HCE), critical loads (CL), responsive flexible loads (FL), electric vehicle clusters (EVs), a main grid interface, and a pricing and scheduling center. The method performs continuous rolling optimization with a preset scheduling cycle, such as... Figure 1 As shown, the method includes: Step 1: Obtain the operating status data of the microgrid system.

[0055] Specifically, at the scheduling time Acquire operational status data of the microgrid system, including but not limited to: wind power output. Photovoltaic power output Critical load power Response to flexible load reference power HPE state of charge HCE state of charge Average state of charge of electric vehicle clusters .

[0056] Step 2: Establish the objective function of user-side time-of-use electricity price using user-side response quantification indicators and electricity price deviation penalty terms. Determine the user-side time-of-use electricity price by maximizing the objective function. Based on the user-side time-of-use electricity price and the total load-side electricity demand under the incentive of user-side time-of-use electricity price, determine the dynamic electricity price of electric vehicles under the real-time supply and demand balance constraint of the system.

[0057] In this embodiment, the user-side time-of-use electricity price is determined based on a two-stage pricing mechanism. and dynamic electricity pricing for electric vehicles ; Specifically, step 2 includes: Step 2.1: Establish the objective function of the user-side time-of-use electricity price using the user-side response metric and the electricity price deviation penalty term. Maximize the objective function to determine the user-side time-of-use electricity price that achieves a balance between maximizing user-side responsiveness and microgrid pricing revenue. As shown in the following formula:

[0058] In the formula, For scheduling time The objective function of user-side time-of-use electricity pricing, For scheduling time responsive flexible load power, To provide a responsive reference power for flexible loads, For user-side response metrics, This is the penalty coefficient for electricity price deviation. For scheduling time User-side time-of-use electricity pricing The benchmark electricity price for users; By maximizing the quantifiable metrics of user-side response, the demand response potential of flexible loads is fully released, thereby maximizing user-side responsiveness. A price deviation penalty term is used to constrain time-of-use pricing within a reasonable range, preventing revenue losses or loss of user responsiveness, and ensuring balanced microgrid pricing revenue. By maximizing the calculation results of the objective function, the optimal time-of-use price that balances user responsiveness and microgrid operating revenue is found.

[0059] [ , ], , These represent the lower and upper limits of the time-of-use electricity price for users, respectively.

[0060] Step 2.2: Determine the total load-side power demand under the user-side time-of-use pricing incentive based on the critical load power and the responsive flexible load power under the user-side time-of-use pricing incentive. Among them, the response characteristics of responsive flexible loads under time-of-use pricing incentives are as follows:

[0061] In the formula, For scheduling time responsive flexible load power under time-of-use pricing incentives on the user side This refers to the elasticity coefficient of user-side demand response. >0; The total load-side electricity demand under the user-side time-of-use pricing incentive is the sum of the critical load power and the responsive flexible load power under the user-side time-of-use pricing incentive, as shown in the following formula:

[0062] In the formula, For scheduling time Total electricity demand on the load side under the incentive of time-of-use pricing on the user side.

[0063] Step 2.3: Based on the user-side time-of-use electricity price and the total load-side electricity demand under the incentive of the user-side time-of-use electricity price, determine the dynamic electricity price for electric vehicles under the real-time supply and demand balance constraint of the system. As shown in the following formula:

[0064]

[0065] In the formula, For scheduling time Dynamic electricity pricing for electric vehicles For scheduling time The system's remaining power, This is the dynamic electricity price adjustment coefficient. For scheduling time Wind power output, For scheduling time Photovoltaic power output; >0 indicates a system power surplus, reducing... Encourage EV charging; <0 indicates a system power deficit, which needs to be increased. Excite the EV to discharge.

[0066] Step 3: Establish a collaborative scheduling model. The state space of the collaborative scheduling model is the operating status data, the user-side time-of-use electricity price, and the dynamic electricity price of electric vehicles. The action space of the collaborative scheduling model is the collaborative scheduling command of the microgrid. The total reward function of the collaborative scheduling model is the weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing.

[0067] Specifically, the collected operational status data will be compared with the two-stage pricing results. , The inputs are fed into a pre-trained reinforcement learning decision model to obtain coordinated scheduling instructions. The reinforcement learning decision model supports two algorithms: Dual Deep Q-Network (DDQN) and Policy Gradient (PG). It employs a multi-stage reward function (P-MSRM) with a fusion pricing mechanism. The training objective is to maximize the cumulative reward to minimize the overall operating cost of the microgrid. The coordinated scheduling instructions include: HPE charging and discharging power... HCE charging and discharging power Interaction power between microgrid and main grid Electric vehicle cluster charging and discharging power .

[0068] This invention incorporates the charging and discharging power of electric vehicle clusters into the reinforcement learning action space, and combines dynamic electricity price incentives with scheduling command control to fully leverage the mobile energy storage and peak-shaving role of electric vehicles. This solves the problem of existing technologies treating electric vehicles as passive loads and having low utilization efficiency, and significantly improves the scheduling value of demand-side resources.

[0069] Specifically, step 3 includes: Step 3.1: Construct a reinforcement learning decision model based on Markov Decision Process (MDP) as a collaborative scheduling model, including: state space, action space and reward function; The reinforcement learning decision model is denoted as The state space Integrating the results of the two-stage pricing process, room for maneuver Incorporating the charging and discharging power of the electric vehicle cluster, reward function The multi-stage total reward function (P-MSRM) of the integrated pricing mechanism is used, where P represents the probability distribution of the system transitioning to the next running state after the agent performs a scheduling action in the current system state. The model solves for the optimal scheduling strategy through DDQN and PG algorithms. The state space is established using partial operational status data of the microgrid system, total load-side electricity demand under user-side time-of-use pricing incentives, user-side time-of-use pricing, and dynamic pricing for electric vehicles; scheduling time... The state space of reinforcement learning decision-making models It is a high-dimensional continuous space, including system-side operating state, demand-side pricing state, and electric vehicle cluster state, with a total of 8 dimensions, defined as: =[ , , , , , , , ] ; The action space is established using coordinated scheduling instructions; scheduling time Reinforce the action space of learning decision-making models The scheduling decision variables for the intelligent agent include the charging and discharging power of the electric vehicle cluster, comprising four dimensions, defined as follows: =[ , , , ] ; Each action variable satisfies its own physical constraints, and the action space is a constrained continuous space. The PG algorithm directly processes continuous actions, while DDQN processes them through action discretization.

[0070] Step 3.2: The weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing is used as the total reward function for integrating the three-stage rewards, as shown in the following formula:

[0071] In the formula, For scheduling time Step-by-step rewards As the final reward, For scheduling time Pricing-guided expert intervention rewards. , , All are weighting coefficients, satisfying + + =1, prioritizing the final global performance.

[0072] The multi-stage reward function (P-MSRM) of the fusion pricing mechanism proposed in this invention includes: step reward Final Rewards Incentives for expert intervention in pricing guidance The total reward is the weighted sum of the rewards at each stage. The core design idea is to incorporate user-side responsiveness, electric vehicle charging and discharging satisfaction, and the system economic cost under pricing incentives into the reward evaluation. At the same time, expert intervention guided by pricing is used to prevent the agent from falling into suboptimal strategies.

[0073] From the perspective of integrating reinforcement learning theory and microgrid optimal scheduling theory, the three-dimensional multi-stage reward function constructed in this invention, consisting of step reward, final reward, and pricing-guided expert intervention reward, breaks through the core theoretical limitations of existing technologies. Traditional reward functions used in existing microgrid reinforcement learning scheduling schemes generally suffer from three major theoretical defects: First, the reward signal is sparse, relying solely on single-step immediate rewards or global rewards at the end of the cycle, leading to excessively large variance in the policy gradient solution of the Markov Decision Process (MDP), making it impossible to theoretically guarantee algorithm convergence; second, it is prone to short-sighted decision-making bias, failing to consider both single-step process constraints and full-cycle global optimization, making policy solutions prone to getting trapped in local optima and unable to achieve Pareto optimality in multi-objective optimization; third, the purely data-driven design is disconnected from prior knowledge in the microgrid scheduling field, failing to incorporate demand-side pricing rules and grid operation constraints into the reward mechanism, resulting in a large ineffective exploration space for the agent, easily leading to suboptimal strategies that violate physical rules and supply-demand balance logic, and failing to achieve a theoretical closed loop between pricing incentives and scheduling decisions.

[0074] The three-dimensional reward function of this invention achieves three core improvements at the theoretical level: First, the step reward decomposes the global multi-objective optimization objective into single-step immediate penalty constraints, providing dense reward feedback for each step of the MDP's state transition. This fundamentally solves the problem of sparse reward signals, reduces the variance of policy gradient calculation, and provides a theoretical guarantee for the algorithm's convergence. Second, the final reward anchors the full-cycle optimization direction through a stepped global incentive, correcting the short-sighted decision bias that is easily caused by single-step rewards. Theoretically, it achieves a Nash equilibrium between immediate process constraints and long-term global optimization, ensuring that the policy solution can converge to the scheduling policy with the best overall performance throughout the entire cycle. Third, the expert intervention reward guided by pricing transforms the supply and demand balance rules output by the two-stage pricing mechanism and the prior knowledge of microgrid operation into the reward constraints and action correction mechanism of MDP. Theoretically, this achieves a deep integration of domain priors and data-driven reinforcement learning framework, significantly reducing the ineffective exploration space of the agent and reducing the complexity of strategy solution. At the same time, it ensures the deep coupling between demand-side pricing incentives and system-side energy scheduling from a mechanism perspective, constructing a complete theoretical closed loop of "pricing-response-scheduling-optimization".

[0075] Step 4: Establish an evaluation index system for the total reward function; based on the evaluation index system, calculate the step reward at each scheduling time and the final reward for the scheduling cycle; establish a supply and demand balance index based on pricing guidance; determine whether expert intervention is needed based on the system's energy storage state of charge and the supply and demand balance index; set expert intervention rewards when expert intervention is needed and expert intervention rewards when no expert intervention is needed.

[0076] Specifically, step 4 includes: Step 4.1: Establish the evaluation index system consisting of five core evaluation indicators for the total reward function; The total reward function involves five core evaluation indicators, specifically defined as follows: 1) System self-balancing rate

[0077] The system self-balancing rate with electric vehicle participation reflects the ability of distributed energy sources, energy storage, and EVs to absorb local loads, as shown in the following formula:

[0078] In the formula, For scheduling time The self-balancing rate of the system involving electric vehicles For scheduling time Main network interaction power, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives , Scheduling time The charging and discharging power of electric vehicle clusters; [0,1], The closer the value is to 1, the stronger the system's self-balancing ability.

[0079] 2) System overall operating cost index

[0080] The normalized system integrated operating cost index reflects the economic operation level of the microgrid dispatching strategy and quantifies the degree to which the dispatching scheme achieves the core optimization objective of minimizing integrated operating costs, as shown in the following formula:

[0081] In the formula, For scheduling time The overall system operating cost index, For scheduling time The total net cost of the system. For scheduling time The benchmark cost without energy storage or pricing;

[0082] In the formula, For scheduling time The total net cost of the system. For scheduling time The power interaction cost between the microgrid and the main grid, For scheduling time The charging, discharging, operation, and maintenance costs of electric vehicle clusters For scheduling time The microgrid electricity sales revenue corresponding to the user-side time-of-use electricity price. For scheduling time The revenue from electricity sales corresponding to the discharge of electric vehicle clusters participating in system peak shaving.

[0083] 3) System reliability rate

[0084] The system reliability rate on the user side and EV side reflects the system's ability to meet the load and EV requirements, as shown in the following formula:

[0085] In the formula, For scheduling time User-side and EV system reliability ≥1 indicates sufficient power supply. <1 indicates insufficient power supply, triggering a reliability penalty.

[0086] 4) User-side responsiveness

[0087] User-side responsiveness reflects the degree of response of flexible loads on the user side under time-of-use pricing incentives, as shown in the following formula:

[0088] In the formula, For scheduling time User-side responsiveness For scheduling time responsive flexible load power under time-of-use pricing incentives on the user side Reference power for responsive flexible loads; [0,1], The closer to 1, the higher the responsiveness.

[0089] 5) Satisfaction with electric vehicle charging and discharging

[0090] Electric vehicle charging and discharging satisfaction reflects the overall satisfaction of electric vehicle owners under dynamic electricity pricing incentives, including price satisfaction and travel satisfaction, as shown in the following formula:

[0091] In the formula, For scheduling time Satisfaction with electric vehicle charging and discharging These are the weighting coefficients. [0,1], For scheduling time Price satisfaction with electric vehicles For scheduling time Travel satisfaction with electric vehicles;

[0092]

[0093] In the formula, For scheduling time The cost of charging and discharging electric vehicle clusters , These represent the lower and upper limits of the charging and discharging cost for electric vehicle clusters, respectively. For scheduling time electric vehicles The state of charge, To provide a reference state of charge for electric vehicles in order to ensure travel needs are met. , These are the lower and upper limits of the state of charge (SOC) of electric vehicles, respectively. This refers to the number of electric vehicles.

[0094] Step 4.2: Calculate the step reward at each scheduling time based on the evaluation index system; Step-by-step rewards For immediate penalty-based rewards, at each scheduling time The calculation, reflecting the comprehensive impact of the current action on the system's economic cost, self-balancing rate, reliability rate, and demand-side responsiveness, is designed as follows:

[0095] In the formula, The penalty coefficient is preferably set within the range of [1, 5]. For scheduling time The Middle The system self-balancing rate at each time step For scheduling time The Middle System reliability rate at each time step For scheduling time The Middle User-side responsiveness at each time step For scheduling time The Middle Satisfaction with electric vehicle charging and discharging at a specific time step. For scheduling time The Middle The overall system operating cost index for each time step; Step 4.3: Calculate the final reward for the scheduling cycle based on the evaluation index system; Final reward For the global reward, at the end of each training episode ( = The calculation reflects the overall performance throughout the entire scheduling cycle and is designed with a tiered reward system, providing additional rewards for metrics that reach performance thresholds. The formula is as follows:

[0096] In the formula, , , , , Scheduling time System self-balancing rate reward, economic cost reward, system reliability rate reward, user-side responsiveness reward, and electric vehicle charging and discharging satisfaction reward; The five sub-rewards correspond to five core evaluation indicators and are designed in a tiered manner: 1) System self-balancing rate reward:

[0097] In the formula, The reward coefficient is preferably set within the range of [1, 5], which matches the dimension of the penalty coefficient in the step reward. For scheduling time The energy storage and discharge power, where E represents HPE or HCE; 2) Economic cost incentive:

[0098] 3) Reliability Bonus:

[0099] 4) User-side responsiveness rewards:

[0100] 5) Electric vehicle charging and discharging satisfaction reward:

[0101] Step 4.4: Establish supply and demand balance indicators based on pricing guidance; determine whether expert intervention is needed based on the system's energy storage state of charge and supply and demand balance indicators; set expert intervention rewards when expert intervention is needed and expert intervention rewards when no expert intervention is needed.

[0102] Furthermore, if expert intervention is required, an energy storage power charging command or energy storage power discharging command under expert intervention will be generated; if no expert intervention is required, the energy storage power command will be generated by the intelligent agent.

[0103] Pricing-guided expert intervention rewards As a constraint-based reward, the action space of the agent is constrained based on the supply and demand balance result of the two-stage pricing: when the system state exceeds the optimal range guided by pricing, the action of energy storage / EV is forcibly adjusted and an intervention penalty is given; when the intervention is not triggered, an intervention reward is given to avoid the agent falling into a suboptimal strategy.

[0104] The supply and demand balance indicator based on pricing guidance is shown in the following formula:

[0105] In the formula, For scheduling time Supply and demand balance indicators For scheduling time The system's remaining power, , Scheduling time The charging and discharging power of electric vehicle clusters For scheduling time Total electricity demand on the load side under the incentive of time-of-use pricing on the user side; When scheduling time System energy storage state of charge Greater than 85% and A value greater than 0.1 indicates a power surplus in the system, requiring expert intervention. When scheduling time System energy storage state of charge Less than 25% and A value less than -0.1 indicates a power deficit in the system, requiring expert intervention. When scheduling time System energy storage state of charge Not more than 85% and not less than 25%, or Not greater than 0.1 and When the value is not less than -0.1, no expert intervention is required; Expert intervention rewards during expert intervention Intervention reward coefficient The preferred value range is [1,5]; expert intervention reward when no intervention is required. Intervention penalty coefficient The preferred value range is [ 5, [1] is used to impose negative penalties on decisions made by agents that cause the system to exceed its limits, thereby guiding agents to avoid operational risks.

[0106] Step 4.5, when scheduling time System energy storage state of charge Greater than 85% and When the value is greater than 0.1, expert intervention is required to force energy storage charging and generate an expert-intervention energy storage power charging command. When scheduling time System energy storage state of charge Less than 25% and When the value is less than -0.1, expert intervention is required to force the energy storage to discharge, generating an expert-intervention energy storage power discharge command. When scheduling time System energy storage state of charge Not more than 85% and not less than 25%, or Not greater than 0.1 and When the value is not less than -0.1, no expert intervention is required; the system's energy storage intelligent agent autonomously makes decisions and generates energy storage power commands. ; This invention uses the generation of instructions based on whether expert intervention is required to adjust the energy storage charging and discharging power as an action space constraint, which is responsible for immediate safety backup and can quickly smooth out the current power imbalance when the energy storage is close to being fully charged or depleted. Furthermore, the dynamic electricity price for electric vehicles is adjusted based on the power status of energy storage, with a reduction during periods of power surplus. Encourage EV charging and increase power when there is a power deficit. Incentivizing EV discharge. In this embodiment, the dynamic electricity price adjustment coefficient for electric vehicles ranges from [0.1, 10], with a base value of 1. It can be dynamically adjusted within this range based on the degree of supply-demand imbalance in the system. The expert intervention and electricity price linkage mechanism simultaneously achieve dual optimization of the reinforcement learning action space and state space: the forced constraint on the energy storage action space enables rapid mitigation of power imbalance at the current moment, ensuring the real-time operational safety of the system; the linkage adjustment of the dynamic electricity price for electric vehicles changes the core input variable of the reinforcement learning state space, guiding the spatiotemporal transfer of electric vehicle charging and discharging behavior through price incentives, achieving supply-demand optimization throughout the entire scheduling cycle. The combination of these two mechanisms forms a complete control system of "immediate safety net - long-term global optimization".

[0107] Step 5: With the goal of maximizing the expected cumulative total reward within the scheduling cycle, the DDQN algorithm and PG algorithm are used to iteratively solve the cooperative scheduling model to obtain the optimal cooperative scheduling instruction for the microgrid.

[0108] The ultimate optimization goal of MDP is to find the optimal strategy. This enables the agent to complete the entire scheduling cycle. Maximize the expected cumulative total reward within:

[0109] In the formula, To optimize the objective function, Let be the policy function, representing the time at which the scheduling occurs. state space Lower execution action space The probability, For policy function The expected function under, The scheduling period is [number].

[0110] Preferably, the reinforcement learning decision model employs two algorithms, DDQN-P-MSRM and PG-P-MSRM, to solve for the optimal policy in the MDP. DDQN handles the discrete action space, while PG handles the continuous action space, making it more suitable for the continuous scheduling decision in this paper. The core framework of the algorithm is as follows: a two-stage pricing module generates the demand-side state → the MDP state space is updated → the agent executes actions → P-MSRM calculates multi-stage rewards → the agent's network parameters are updated.

[0111] The DDQN-P-MSRM algorithm addresses the overestimation problem of DQN through a dual-Q network, separating action selection from action evaluation. Its network update formula is:

[0112] In the formula, For scheduling time The current Q network parameters, For scheduling time Target Q-network parameters, For learning rate, As a discount factor, For scheduling time The action, For scheduling time The gradient of the current Q-network parameters; Action discretization: dividing the continuous action space =[ , , , The action variable is discretized into several action intervals, and the discrete step length of each action variable is 1 / 10 of the maximum power of the device, so as to ensure the precision of the action and the efficiency of calculation.

[0113] The PG-P-MSRM algorithm is a policy gradient algorithm that directly applies the policy function. Parameterize the algorithm and maximize the expected cumulative reward through gradient ascent. Its core formula is: 1) Expected cumulative rewards

[0114]

[0115] 2) Policy gradient

[0116]

[0117]

[0118] In the formula, For scheduling time The cumulative discount reward is based on the current scheduling time. Starting from the beginning and ending at time T of the scheduling cycle, the future cumulative discount weighted value of the total reward output by the multi-stage reward function (P-MSRM) of the integrated pricing mechanism is used to quantify the long-term impact of the current scheduling action on the overall scheduling revenue.

[0119] 3) Parameter update

[0120] A Gaussian policy is used to model the continuous action space, and the policy function is:

[0121] In the formula, It is a multidimensional Gaussian probability density function. The average action output by the policy network represents continuous scheduling decisions. For the action variance, It is an identity matrix.

[0122] The policy network structure uses a 3-layer fully connected neural network with 100 neurons in each layer. The activation function is ReLU, and the output layer has linear activation (corresponding to the mean of consecutive actions).

[0123] Step 6, at the scheduling time Execute the optimal instructions for coordinated scheduling.

[0124] During scheduling Execute the optimal coordinated scheduling instruction to control the charging and discharging behavior of HPE and HCE, the main network interaction behavior, and the charging and discharging behavior of electric vehicle clusters, verify and satisfy the system power balance and various operational constraints; enter the next scheduling time t+1, repeat the above steps 1-6, until the preset scheduling cycle is completed.

[0125] Specifically, the execution of the optimal instruction in the coordinated scheduling must satisfy the operating models and constraints of each component, as follows: 1) Electric vehicle cluster charging and discharging model Electric vehicle clusters (EVs) serve as mobile energy storage, and their charging and discharging power is affected by dynamic electricity pricing. With the dual constraints of optimal coordinated scheduling instructions, the total charging and discharging power of the cluster is:

[0126]

[0127] In the formula, For scheduling time electric vehicles The charging and discharging power, with charging being positive and discharging being negative. For the number of electric vehicles, , These are the maximum charging and discharging power of the electric vehicle, respectively. The state of charge (SOC) constraint for EVs is:

[0128]

[0129] In the formula, For scheduling time electric vehicles The state of charge, , These are the lower and upper limits of the state of charge (SOC) for electric vehicles, respectively. This refers to the charging and discharging time. The charging and discharging efficiency of electric vehicles is divided into charging efficiency. and discharge efficiency , For the rated capacity of electric vehicles, The self-discharge rate of electric vehicles; The average state of charge of the electric vehicle cluster is .

[0130] 2) Energy storage equipment operation model The charging and discharging power of HPE and HCE and the SOC constraint are satisfied as follows:

[0131]

[0132]

[0133] In the formula, For scheduling time Energy storage charging and discharging power, This represents the upper limit of the energy storage charging and discharging power. For scheduling time The state of charge, , These are the lower and upper limits of the state of charge, respectively. For scheduling time The self-discharge rate of energy storage, For scheduling time The energy storage charging and discharging power command value, For scheduling time The charging and discharging efficiency of energy storage. This refers to the rated capacity of the energy storage.

[0134] 3) Mainnet Interaction Model The interaction power between the microgrid and the main grid satisfies:

[0135] In the formula, This represents the maximum interaction power between the microgrid and the main grid. Mainnet interaction cost for:

[0136] in, For scheduling time The interaction cost between the microgrid and the main grid. For scheduling time Real-time electricity price on the main network.

[0137] 4) Core constraints for system operation The execution of coordinated dispatch optimization instructions must strictly meet the power balance constraints and renewable energy absorption rate constraints of the microgrid, including: 4.1) Power balance constraints Considering the participation of users and electric vehicles, the real-time power balance of the system is as follows:

[0138] 4.2) Renewable energy consumption rate constraints

[0139] In the formula, For renewable energy consumption rate, This represents the lower limit of the renewable energy integration rate. For scheduling time The amount of wind / solar power that is curtailed.

[0140] If the above constraints are not met, the controller will dynamically adjust the charging and discharging power of the energy storage and electric vehicle cluster until the constraints are met.

[0141] In Embodiment 2, this invention provides a microgrid system that integrates user-side and electric vehicle pricing mechanisms, used to execute the microgrid reinforcement learning optimization scheduling method for integrating user-side and electric vehicle pricing mechanisms described in Embodiment 1, comprising: The pricing module is used to acquire operating status data of the microgrid system and determine the user-side time-of-use electricity price and the dynamic electricity price for electric vehicles. The modeling module is used to establish a collaborative scheduling model. The total reward function of the collaborative scheduling model is the weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing. An evaluation index system for the total reward function is established. Based on the evaluation index system, the step rewards at each scheduling moment and the final reward of the scheduling cycle are calculated. A supply-demand balance index based on pricing guidance is established. Based on the system's energy storage state of charge and the supply-demand balance index, it is determined whether expert intervention is needed. Expert intervention rewards are set separately for when expert intervention is needed and when it is not. The scheduling module is used to iteratively solve the collaborative scheduling model with the optimization objective of maximizing the expected cumulative total reward within the scheduling cycle, so as to obtain the optimal collaborative scheduling instruction for the microgrid.

[0142] Embodiment 3 of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the microgrid reinforcement learning optimization scheduling method according to Embodiment 1, which integrates user-side and electric vehicle pricing mechanisms.

[0143] Embodiment 4 of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the microgrid reinforcement learning optimization scheduling method according to Embodiment 1, which integrates user-side and electric vehicle pricing mechanisms.

[0144] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0145] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0146] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0147] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A microgrid dispatching method integrating user-side and electric vehicle pricing mechanisms, characterized in that, include: Acquire operational status data of the microgrid system; Determine the user-side time-of-use electricity price and the dynamic electricity price for electric vehicles; Establish a collaborative scheduling model, using the weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing as the total reward function of the collaborative scheduling model; Establish an evaluation index system for the total reward function; based on the evaluation index system, calculate the step reward at each scheduling moment and the final reward for the scheduling cycle; establish a supply and demand balance index based on pricing guidance; determine whether expert intervention is needed based on the system's energy storage state of charge and supply and demand balance index. Set separate expert intervention rewards for when expert intervention is required and for when expert intervention is not required; With the goal of maximizing the expected cumulative total reward within the scheduling cycle, the cooperative scheduling model is iteratively solved to obtain the optimal cooperative scheduling command for the microgrid.

2. The microgrid dispatching method integrating user-side and electric vehicle pricing mechanisms according to claim 1, characterized in that, Operational status data includes: wind power output, photovoltaic power output, critical load power, responsive flexible load baseline power, state of charge of high power density energy storage, state of charge of high capacity energy storage, and average state of charge of electric vehicle clusters. A target function for user-side time-of-use electricity price is established using user-side response quantification indicators and electricity price deviation penalty terms. The user-side time-of-use electricity price is determined with the goal of maximizing the target function. Based on the user-side time-of-use electricity price and the total load-side electricity demand under the incentive of user-side time-of-use electricity price, the dynamic electricity price for electric vehicles is determined under the real-time supply and demand balance constraint of the system. The objective function is maximized as the optimization goal, as shown in the following equation: In the formula, For scheduling time The objective function of user-side time-of-use electricity pricing, For scheduling time responsive flexible load power, To provide a responsive reference power for flexible loads, For user-side response metrics, This is the penalty coefficient for electricity price deviation. For scheduling time User-side time-of-use electricity pricing This is the benchmark electricity price for users.

3. The microgrid dispatching method integrating user-side and electric vehicle pricing mechanisms according to claim 2, characterized in that, The responsive flexible load power under the user-side time-of-use pricing incentive is shown in the following formula: In the formula, For scheduling time Responsive flexible load power under time-of-use pricing incentives on the user side This refers to the elasticity coefficient of user-side demand response. >0; The total load-side electricity demand under the user-side time-of-use pricing incentive is shown in the following formula: In the formula, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives For scheduling time Key load power; Dynamic electricity price for electric vehicles As shown in the following formula: In the formula, For scheduling time Dynamic electricity pricing for electric vehicles For scheduling time The system's remaining power, This is the dynamic electricity price adjustment coefficient. For scheduling time Wind power output, For scheduling time Photovoltaic power output.

4. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 2, characterized in that, A state space is established using the state of charge of wind power output, photovoltaic power output, high power density energy storage, high capacity energy storage, average state of charge of electric vehicle clusters, total load-side power demand under user-side time-of-use pricing incentives, user-side time-of-use pricing, and dynamic pricing of electric vehicles. The action space is established using the charging and discharging power of high power density energy storage, the charging and discharging power of high capacity energy storage, the interaction power between microgrids and the main grid, and the charging and discharging power of electric vehicle clusters.

5. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 1, characterized in that, The total reward function is shown below: In the formula, For scheduling time Step-by-step rewards, As the final reward, For scheduling time Pricing-guided expert intervention rewards. , , All are weighting coefficients, satisfying + + =1.

6. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 1, characterized in that, The evaluation index system for the total reward function includes: system self-balancing rate, system comprehensive operating cost index, system reliability rate, user-side responsiveness, and electric vehicle charging and discharging satisfaction. At each scheduling time Calculate step rewards As shown in the following formula: In the formula, The penalty coefficient is... For scheduling time The Middle The system self-balancing rate at each time step For scheduling time The Middle System reliability rate at each time step For scheduling time The Middle User-side responsiveness at each time step For scheduling time The Middle Satisfaction with electric vehicle charging and discharging at a specific time step. For scheduling time The Middle The system's overall operating cost index at each time step.

7. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 6, characterized in that, Scheduling time System self-balancing rate involving electric vehicles As shown in the following formula: In the formula, For scheduling time Main network interaction power, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives , Scheduling time The charging and discharging power of electric vehicle clusters.

8. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 6, characterized in that, Scheduling time System comprehensive operating cost index As shown in the following formula: In the formula, For scheduling time The total net cost of the system. For scheduling time The benchmark cost without energy storage or pricing.

9. The microgrid dispatching method integrating user-side and electric vehicle pricing mechanisms according to claim 6, characterized in that, Scheduling time User-side and electric vehicle system reliability As shown in the following formula: In the formula, For scheduling time Wind power output, For scheduling time Photovoltaic power output, For scheduling time High power density energy storage charging and discharging power, For scheduling time High-capacity energy storage charging and discharging power, For scheduling time The interaction power between the microgrid and the main grid, For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives , Scheduling time The charging and discharging power of electric vehicle clusters.

10. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 6, characterized in that, Scheduling time User-side responsiveness As shown in the following formula: In the formula, For scheduling time Responsive flexible load power under time-of-use pricing incentives on the user side This is the reference power for responsive flexible loads.

11. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 6, characterized in that, Scheduling time Electric vehicle charging and discharging satisfaction As shown in the following formula: In the formula, These are the weighting coefficients. [0,1], For scheduling time Price satisfaction with electric vehicles For scheduling time Travel satisfaction with electric vehicles; For scheduling time The cost of charging and discharging electric vehicle clusters , These represent the lower and upper limits of the charging and discharging cost for electric vehicle clusters, respectively. For scheduling time electric vehicles The state of charge, For electric vehicles, the reference state of charge , These are the lower and upper limits of the state of charge (SOC) of electric vehicles, respectively. This refers to the number of electric vehicles.

12. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 6, characterized in that, Throughout the scheduling cycle Calculate the final reward at the end. As shown in the following formula: In the formula, , , , , Scheduling time The rewards include system self-balancing rate rewards, economic cost rewards, system reliability rate rewards, user-side responsiveness rewards, and electric vehicle charging and discharging satisfaction rewards.

13. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 12, characterized in that, Scheduling time The system self-balancing rate reward is shown in the following formula: In the formula, As the reward coefficient, For scheduling time The energy storage discharge power, with the superscript E indicating high power density energy storage or high capacity energy storage. For scheduling time Total load-side electricity demand under user-side time-of-use pricing incentives Scheduling time The discharge power of the electric vehicle cluster; Scheduling time The economic cost reward is shown in the following formula: Scheduling time The reliability bonus is shown in the following formula: Scheduling time The user-side responsiveness reward is shown in the following formula: Scheduling time The electric vehicle charging and discharging satisfaction reward is shown in the following formula: 。 14. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 5, characterized in that, The supply and demand balance indicator based on pricing guidance is shown in the following formula: In the formula, For scheduling time Supply and demand balance indicators For scheduling time The system's remaining power, , Scheduling time The charging and discharging power of electric vehicle clusters For scheduling time Total electricity demand on the load side under the incentive of time-of-use pricing on the user side; When scheduling time System energy storage state of charge Greater than 85% and A value greater than 0.1 indicates a power surplus in the system, requiring expert intervention to force energy storage charging and generate expert-intervention energy storage power charging commands. The superscript E indicates high power density energy storage or high capacity energy storage. When scheduling time System energy storage state of charge Less than 25% and A value less than -0.1 indicates a power deficit in the system, requiring expert intervention to force energy storage discharge and generate an expert-intervention energy storage power discharge command. ; When scheduling time System energy storage state of charge Not more than 85% and not less than 25%, or Not greater than 0.1 and When the value is not less than -0.1, no expert intervention is required; the system's energy storage intelligent agent autonomously makes decisions and generates energy storage power commands. ; Scheduling time Expert intervention reward when expert intervention is required Intervention reward coefficient Expert intervention reward when no intervention is needed Intervention penalty coefficient .

15. The microgrid scheduling method integrating user-side and electric vehicle pricing mechanisms according to claim 1, characterized in that, With the goal of maximizing the expected cumulative total reward within the scheduling cycle, the DDQN algorithm and PG algorithm are used to iteratively solve the cooperative scheduling model to obtain the optimal cooperative scheduling command for the microgrid. The expected total cumulative reward is maximized as shown in the following formula: In the formula, To optimize the objective function, Let be the policy function, representing the time at which the scheduling occurs. state space Lower execution action space The probability, For policy function The expected function under, The scheduling period is [number].

16. A microgrid dispatching system integrating user-side and electric vehicle pricing mechanisms, used to implement the microgrid dispatching method integrating user-side and electric vehicle pricing mechanisms as described in any one of claims 1 to 15, characterized in that, include: The pricing module is used to acquire operating status data of the microgrid system; Determine the user-side time-of-use electricity price and the dynamic electricity price for electric vehicles; The modeling module is used to establish a collaborative scheduling model. The total reward function of the collaborative scheduling model is the weighted sum of the step rewards at each scheduling moment, the final reward of the scheduling cycle, and the expert intervention reward guided by pricing. An evaluation index system for the total reward function is established. Based on the evaluation index system, the step rewards at each scheduling moment and the final reward of the scheduling cycle are calculated. A supply and demand balance index based on pricing guidance is established. Based on the system's energy storage state of charge and the supply and demand balance index, it is determined whether expert intervention is needed. Set separate expert intervention rewards for when expert intervention is required and for when expert intervention is not required; The scheduling module is used to iteratively solve the collaborative scheduling model with the optimization objective of maximizing the expected cumulative total reward within the scheduling cycle, so as to obtain the optimal collaborative scheduling instruction for the microgrid.

17. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-15.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-15.

Images