Electric vehicle low-carbon demand response scheduling method considering vehicle network interaction

Abstract

The invention discloses an electric vehicle low-carbon demand response scheduling method considering vehicle-network interaction. The method comprises the following steps: calculating carbon emission factors of nodes of a power distribution network; classifying electric vehicle users, establishing a differentiated utility function based on a random utility theory, and calibrating the weight of the utility function; constructing a deep reinforcement learning scheduling model, including designing a corresponding state space, an action space and a reward function, constructing an Actor-Critic network, training the Actor-Critic network by using a PPO algorithm, and obtaining a trained Actor-Critic network; acquiring a carbon excitation coefficient by using the trained Actor-Critic network, determining an electricity price according to the carbon excitation coefficient, and guiding an electric vehicle user to respond; the electric vehicle user is guided to select low-carbon nodes and time periods for charging, so that the overall carbon emission of the system is reduced, and the operation efficiency of a power grid is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electric vehicle charging scheduling, and particularly relates to an electric vehicle low-carbon demand response scheduling method and system considering vehicle-to-grid interaction. BACKGROUND

[0002] With the rapid popularization of electric vehicles (EVs), their charging load has a significant impact on the operation of distribution networks. On the one hand, electric vehicles, as flexible loads, can participate in grid regulation through demand response mechanisms to achieve peak-valley regulation and renewable energy consumption; on the other hand, the charging process of electric vehicles produces carbon emissions, especially when the charging power comes from high-carbon generating units, which will exacerbate the carbon footprint of the power system. At present, existing electric vehicle demand response scheduling methods mainly focus on economic cost optimization or grid stability, such as time-shift charging strategies based on price signals, but ignore the spatial and temporal differences in carbon emissions, and cannot effectively guide users to choose low-carbon time periods and nodes for charging and discharging. Moreover, related research often ignores the vehicle-to-grid (V2G) capability of EVs, which not only is a transferable charging load, but also is a bi-directionally adjustable distributed energy storage. V2G will affect the distribution network power flow and node carbon emission factor. At the same time, user behavior is highly heterogeneous, and existing user response models mostly use a single utility function, which cannot accurately capture the differentiated response characteristics of price-sensitive, convenience-priority, and time-constrained users. In addition, traditional optimization methods are inefficient in handling high-dimensional uncertainty, and cannot adapt to real-time changes in the grid and real-time responses of users. Therefore, a low-carbon scheduling method that considers vehicle-to-grid interaction, carbon emission tracking, and multi-level user response is needed to achieve the coordinated optimization of carbon emission reduction, economic benefits, and grid safety. SUMMARY

[0003] The purpose of the present application is to provide an electric vehicle low-carbon demand response scheduling method and system considering vehicle-to-grid interaction, which calculates the carbon emission factor of each node in the distribution network in real time, models the multi-level response characteristics of users, and uses a deep reinforcement learning algorithm to develop differentiated electricity price strategies to guide electric vehicle users to choose low-carbon nodes and time periods for charging, thereby reducing the overall carbon emissions of the system, improving the efficiency of the grid operation, and ensuring the fairness and operability of user response behavior.

[0004] To solve the problems in the background art, the present application adopts the following technical solutions:

[0005] The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction comprises the following steps:

[0006] Collecting distribution network topology, load, distributed power and user charging history data, and calculating the carbon emission factor of each node in the distribution network;

[0007] A multi-level response characteristic model of electric vehicle users is constructed: electric vehicle users are classified, a differentiated utility function based on the random utility theory is established, and the weight of the utility function is calibrated. Based on the constructed utility function, a multinomial logistic model is used to calculate the probability of users selecting each charging station, so as to obtain the charging load of each node;

[0008] A deep reinforcement learning scheduling model is constructed, including designing the corresponding state space, action space and reward function, constructing an Actor-Critic network, training the Actor-Critic network by using the PPO algorithm, and obtaining the trained Actor-Critic network;

[0009] The trained Actor-Critic network is used to obtain the carbon incentive coefficient, and the electricity price is determined according to the carbon incentive coefficient to guide the response of electric vehicle users.

[0010] Further, the electric vehicle users are classified, a differentiated utility function based on the random utility theory is established, and the weight of the utility function is calibrated as follows:

[0011] The electric vehicle users are divided into price-sensitive users, convenience-priority users and time-constrained users;

[0012] A differentiated utility function based on the random utility theory is established, and the utility function of k-type user j selecting charging station i at time t is:

[0013] ;

[0014] wherein, is the electricity price level of charging station i at time t; is the distance from the current location of electric vehicle user j to charging station i; is the expected waiting time of charging station i at time t; is the service quality score of charging station i; is the ideal discharge amount of electric vehicle user j; is a random error term; the weight parameter 、 、 、 、 characterizes the sensitivity weight of different types of electric vehicle users to the factors of electricity price level, charging distance, waiting time, service quality and ideal discharge amount:

[0015] The maximum likelihood estimation method based on historical data is used to calibrate the weight parameter: it is assumed that the user selection follows the principle of utility maximization, and the likelihood function of the selection probability is constructed based on the information when the user makes the selection; the optimal sensitivity weight parameter value is solved by maximizing the likelihood function.

[0016] Further, the likelihood function of the selection probability is constructed based on the information when the user makes the selection; the method for solving the optimal sensitivity weight parameter value by maximizing the likelihood function is:

[0017] The form of the likelihood function is: wherein, is the number of k type electric vehicle users, is the number of observation time periods, is the actual selection of electric vehicle user j at time t,

[0018] In order to solve the optimal weight parameter, the log-likelihood function needs to be maximized, and the partial derivative of the weight parameter is zero to obtain the optimization condition: Since the equation set is nonlinear, Newton-Raphson method or quasi-Newton method and other numerical optimization algorithms can be used to solve it.

[0019] Further, based on the constructed utility function, a multinomial logistic model is used to calculate the probability of user selection of each charging station, so as to obtain the charging load of each node. The method is:

[0020] The probability of k type user j selecting charging station i at time t is: wherein, is the total number of selectable charging stations, is a temperature parameter, which controls the randomness of user selection;

[0021] Let the total number of k type users in the system be , and the average charging demand of each user be , then the expected charging load of charging station i at time t is : wherein, is the average probability of k type users selecting charging station i for charging: .

[0022] Further, the state space includes a node voltage state vector, a line load rate vector, a node carbon emission factor vector, a system average carbon emission factor, a carbon emission factor standard deviation, a classified user demand vector, a user V2G available capacity, a user battery state of charge distribution, a spatial demand distribution entropy, a user response sensitivity index, and a time characteristic;

[0023] The action space is defined as a two-way carbon incentive coefficient wherein, are the charging carbon incentive coefficient and the discharging carbon incentive coefficient, respectively;

[0024] The reward function is:

[0025] ​ ;

[0026] wherein, , , are the weight coefficients of economic benefit, environmental benefit and constraint penalty, respectively;

[0027] Economic benefit reward reflects the economic benefit of the current scheduling strategy, and the calculation formula is:

[0028] ;

[0029] wherein, the electricity sales revenue ; the generation cost , wherein is the discharging capacity of the electric vehicle user of the charging station i, is the marginal generation cost of the unit u; is the active power output of the unit u;

[0030] Environmental benefit reward quantifies the carbon emission reduction value achieved by the current scheduling strategy:

[0031] ;

[0032] wherein, is the system carbon emission of the baseline scenario, is the actual carbon emission under the current scheduling strategy, is the economic value of carbon emission; the actual carbon emission is calculated as: ; the baseline carbon emission is calculated based on the load distribution under the uniform electricity price: ; wherein, represents the baseline charging load of the charging station i;

[0033] Constraint penalty term is used to ensure that the scheduling strategy meets the requirements of safe operation of the power grid, including voltage constraint penalty, line capacity constraint penalty and electricity price rationality penalty.

[0034] Further, the Actor-Critic network is constructed, and the method for training the Actor-Critic network by using the PPO algorithm is:

[0035] The Actor-Critic network structure includes an Actor network and a Critic network, wherein the Actor network takes the state space vector as input, outputs the probability distribution parameters of the carbon incentive coefficient , and realizes the mapping from state to action; the Critic network takes the state space vector As input, output the value estimate in this state , provide a baseline for policy update

[0036] The overall loss function when training the Actor-Critic network with the PPO algorithm is:

[0037] ;

[0038] Wherein, ; c is the weight coefficient of the value function loss and the entropy reward is the probability ratio of the new and old policies is the advantage function estimate is the step size of the control policy update Indicates: if , output ; if , output ; if , output ; minimize the value function prediction error , wherein is the target value, and the calculation method is: ; the value estimation function is a value network with parameters, which can estimate the state , wherein is the time difference error is the GAE parameter, which balances the bias and variance is the value estimate of state S at time t.

[0039] Further, the method for determining the electricity price according to the carbon incentive coefficient and guiding the electric vehicle user to respond is:

[0040] The carbon incentive coefficient based on the output of the agent and the obtained node carbon emission factor , the real-time electricity price of each node is calculated according to the following formula:

[0041] , ;

[0042] Wherein, is the charging price of the charging station i is the discharging price of the charging station i are the charging and discharging baseline prices respectively , are the maximum charging and discharging price changes allowed respectively , are the maximum and minimum values of the node carbon emission factor in the system at the current time respectively

[0043] In addition, real-time electricity pricing needs to meet the following constraints:

[0044] (1) Non-negativity constraint of electricity price: To avoid negative electricity prices;

[0045] (2) Constraints on electricity price fluctuations: , To avoid user dissatisfaction caused by drastic fluctuations in electricity prices;

[0046] (3) Balance of income and expenditure constraint: To ensure that the electricity price covers the cost of electricity sales; among which, This represents the electricity sales cost of charging station i;

[0047] (4) SOC requirement constraints: ;in, , These represent the charging and discharging efficiencies of electric vehicle j, respectively. , , respectively, represent the charging and discharging power of electric vehicle j at time t;

[0048] SOC satisfies upper and lower bound constraints. ,in This represents the state of charge of electric vehicle j at time t. These represent the upper and lower limits of the state of charge, respectively.

[0049] Considering a vehicle-to-grid (V2G) interactive low-carbon demand response scheduling system for electric vehicles, including:

[0050] The carbon emission factor acquisition module is used to collect historical data on distribution network topology, load, distributed power sources, and user charging, and to calculate the carbon emission factor of each node in the distribution network.

[0051] The electric vehicle user multi-level response characteristic model construction module is used to construct an electric vehicle user multi-level response characteristic model: classify electric vehicle users, establish a differentiated utility function based on stochastic utility theory, and calibrate the weights of the utility function. Based on the constructed utility function, a multinomial logic model is used to calculate the probability of users choosing each charging station, thereby obtaining the charging load of each node.

[0052] The deep reinforcement learning scheduling model construction module is used to build a deep reinforcement learning scheduling model, including designing the corresponding state space, action space and reward function, constructing the Actor-Critic network, training the Actor-Critic network using the PPO algorithm, and obtaining the trained Actor-Critic network.

[0053] The electric vehicle user-guided response module is used to obtain the carbon incentive coefficient using a trained Actor-Critic network, determine the electricity price based on the carbon incentive coefficient, and guide electric vehicle users to respond accordingly.

[0054] Furthermore, the present invention adopts the following technical solution:

[0055] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction as described above.

[0056] Furthermore, the present invention adopts the following technical solution:

[0057] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction as described above.

[0058] The beneficial technical effects of this invention are as follows:

[0059] This invention innovatively combines real-time carbon emission factor calculation of power distribution networks with electric vehicle demand response scheduling, breaking through the limitations of traditional scheduling methods that only consider economic factors. Unlike the simplistic assumptions about user behavior in existing studies, this method constructs a differentiated utility function based on stochastic utility theory, classifying users into three categories: price-sensitive, convenience-prioritized, and time-constrained. It uses a Multinomial Logit model to accurately predict user selection probabilities, significantly improving the accuracy and practicality of user response prediction. It uses deep reinforcement learning to solve the problem, achieving end-to-end learning from environmental perception to decision output, overcoming the difficulty of solving traditional optimization methods under complex constraints. Attached Figure Description

[0060] Figure 1 This is a flowchart of the low-carbon demand response scheduling method for electric vehicles that considers vehicle-to-grid interaction, provided in Embodiment 1 of the present invention. Detailed Implementation

[0061] The method and apparatus for calculating dynamic carbon emission factors based on spatiotemporal graph convolutional networks provided by the present invention will be further described clearly and completely below with reference to the accompanying drawings:

[0062] Example 1

[0063] like Figure 1 The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction provided in this embodiment includes the following steps:

[0064] S1. Collect historical data on distribution network topology, load, distributed power sources and user charging, and calculate the carbon emission factor of each node in the distribution network.

[0065] S2. Construct a multi-level response characteristic model for electric vehicle users: classify electric vehicle users, establish a differentiated utility function based on stochastic utility theory, and calibrate the weights of the utility function. Based on the constructed utility function, use a multinomial logic model to calculate the probability of users choosing each charging station, thereby obtaining the charging load of each node.

[0066] S3. Construct a deep reinforcement learning scheduling model, including designing the corresponding state space, action space and reward function, constructing an Actor-Critic network, training the Actor-Critic network using the PPO algorithm, and obtaining the trained Actor-Critic network.

[0067] S4. Use the trained Actor-Critic network to obtain the carbon incentive coefficient, and determine the electricity price based on the carbon incentive coefficient to guide electric vehicle users to respond.

[0068] Specifically, the method for collecting historical data on distribution network topology, loads, distributed generation, and user charging in step S1, and calculating the carbon emission factor of each node in the distribution network, is as follows:

[0069] First, data such as grid topology information, node load data, distributed generation output, and upstream grid power supply information are collected in real time from the distribution network automation system to perform a complete power flow calculation on the distribution network, thereby obtaining the power flow distribution in the distribution network.

[0070] The unit of nodal carbon emission factor is kg CO2 / kWh, which represents the carbon emissions generated per unit of electricity consumption. Its calculation process is based on power flow distribution. Essentially, it traces the source of electricity consumption for each load and binds carbon emissions in the power system to power transmission to achieve carbon emission tracking and allocation. Ultimately, it binds system carbon emissions to power flow, assesses the carbon intensity of each distribution network node in real time, and provides a quantitative basis for differentiated electricity pricing and dispatch.

[0071] Because the operating status of the distribution network and the output of generator units change in real time, network operation data is re-collected and calculated every preset time interval (e.g., 15 minutes). The output of the carbon emission factor calculation is a real-time updated vector. Where n is the total number of distribution network nodes, Let d be the carbon emission factor at time t. The spatiotemporal differences in the carbon emission factor provide a quantitative basis for formulating differentiated electricity prices, guiding electric vehicle users to choose low-carbon charging nodes. It should be noted that the carbon emission factor can be calculated using methods such as power flow tracking, marginal generator sets, and machine learning. Any one of these methods can be used in this embodiment.

[0072] In this embodiment, carbon emission reduction benefits are calculated based on the charging and discharging behavior of electric vehicle users according to the node carbon emission factor, and their baseline charging and discharging data are obtained. Based on this, the calculation is performed on each electric vehicle user after demand response.

[0073] In step S2, when constructing the multi-level response characteristic model of electric vehicle users, electric vehicle users are first classified, and a differentiated utility function based on stochastic utility theory is established. Specifically:

[0074] Electric vehicle (EV) users exhibit significant heterogeneity in their charging behavior. Different user types show marked differences in their sensitivity to price signals, convenience, and time factors. Accurately modeling the multi-level response characteristics of users is crucial for achieving effective demand response scheduling. Based on historical charging behavior data of EV users, this invention categorizes EV users into three types:

[0075] Price-sensitive users: These users are sensitive to changes in electricity prices and are willing to adjust their charging time and location in order to obtain lower charging costs. They usually have more flexible travel arrangements.

[0076] Convenience-first users: These users prioritize the convenience of charging, choosing charging stations that are nearby or have good facilities, and are relatively less sensitive to price changes.

[0077] Time-constrained users: These users have a strong time urgency in their charging needs, such as commercial vehicles like taxis and ride-hailing vehicles, which have limited flexibility in charging time and location.

[0078] To quantify the charging preferences of different types of electric vehicle users, a differentiated utility function based on stochastic utility theory is established. For user j of type k choosing charging station i at time t, the utility function is:

[0079] ;

[0080] in, The electricity price level of charging station i at time t directly affects the user's economic costs; The distance from the current location of electric vehicle user j to charging station i reflects spatial convenience; Let be the expected waiting time of charging station i at time t, reflecting the time cost; The service quality score for charging station i includes the number of charging piles, charging power, and surrounding facilities; For electric vehicle users, the ideal discharge capacity; The random error term reflects the uncertainty of user preferences; weight parameters , , , , The sensitivity weights of different types of electric vehicle users to factors such as electricity price level, charging distance, waiting time, service quality, and ideal discharge capacity are represented as follows:

[0081] For price-sensitive users (defined as k=1): For convenience-first users (defined as k = 2): and For time-constrained users (defined as k = 3): .

[0082] Next, the weight parameters are calibrated using the maximum likelihood estimation method based on historical data: historical charging selection data of different types of electric vehicle users are collected, including information such as the selected charging station, the price level at that time, distance, and waiting time; assuming that the user's choice follows the principle of utility maximization, a likelihood function of the selection probability is constructed based on the information when the user makes the choice; the optimal sensitivity weight parameter value is solved by maximizing the likelihood function.

[0083] The likelihood function has the following form: ,in, The number of users of type k electric vehicles. The number of observation periods. For the actual choice made by electric vehicle user j at time t. To find the optimal weight parameters, we need to maximize the log-likelihood function and take its partial derivative with respect to the weight parameters, setting it to zero to obtain the optimization conditions. Since the system of equations is nonlinear, it can be solved using numerical optimization algorithms such as the Newton-Raphson method or the quasi-Newton method.

[0084] Next, based on the constructed utility function, a multinomial logic model is used to calculate the probability of a user choosing each charging station, thereby obtaining the charging load of each node. Specifically:

[0085] Based on the constructed utility function, a Multinomial Logit (MNL) model is used to calculate the probability of a user choosing each charging station. This model assumes that the user will choose the charging station with the highest utility, but due to the presence of a random error term, the selection behavior is probabilistic. The probability of a user j of type k choosing charging station i at time t is: ,in, This represents the total number of available charging stations. This is a temperature parameter that controls the degree of randomness in user selection.

[0086] Temperature parameters Physical meaning: When When users have complete determinism, they will always choose the charging station with the highest utility; when At that time, the user's choice is completely random, and each charging station has an equal probability of being selected, which is moderate. The value can reflect the balance between rationality and randomness in user choices; different types of users respond differently to price signals, requiring differentiated adjustment through temperature parameters. In other words, price-sensitive users respond most rationally and definitively, while time-constrained users make the most random choices.

[0087] Based on the selection probability of each user, the aggregate charging load distribution of each node in the system can be calculated: Let the total number of users of type k in the system be... The average charging demand of each user is Then the expected charging load of charging station i at time t for: ,in, The average probability of a user of type k choosing charging station i for charging: .

[0088] In summary, the user selection probability matrix can be obtained: This describes the selection preferences of different types of users for various charging stations; node load distribution vector: This reflects the spatial distribution of electric vehicle charging load; these outputs will serve as an important component of the deep reinforcement learning environment model, used to: construct the agent's state space to describe the current state of the user's response; establish a state transition model for the environment to predict the user's response to the scheduling strategy; and calculate the reward function to evaluate the effectiveness of the scheduling strategy.

[0089] Step S3 involves constructing a deep reinforcement learning scheduling model, including designing the corresponding state space, action space, and reward function; constructing an Actor-Critic network; and training the Actor-Critic network using the PPO algorithm to obtain the trained Actor-Critic network. The details are as follows:

[0090] This invention models the low-carbon demand response scheduling problem for electric vehicles as a Markov Decision Process (MDP). In this framework: the agent is defined as the distribution network operator's scheduling system, responsible for formulating and issuing differentiated electricity pricing strategies, capable of obtaining the overall network's operating status and carbon emission information, and rapidly making electricity price adjustment decisions within each scheduling cycle; the environment is defined as a composite system encompassing the distribution network's physical characteristics (power flow distribution, voltage state, line load, and other physical constraints) and the electric vehicle user group (electric vehicle users' charging needs, response behavior, and spatial distribution characteristics).

[0091] The state space designed in this invention includes the following four subspaces:

[0092] (a) Power grid physical state subspace

[0093] The physical state of the power grid describes the real-time operation of the distribution network and provides security constraint information for dispatching decisions. The physical state subspace of the power grid includes: node voltage state vectors: This reflects the voltage amplitude level at each node; line load rate vector: ,in This represents the load rate of the e-th line. This represents the active power of line e at time t. This indicates the upper limit of the active power capacity of line e. Total number of lines; Transformer load status: reflects the load level of each level of transformer, ensuring that it does not exceed the equipment capacity limit;

[0094] (b) Carbon emission state subspace

[0095] Carbon emission status is directly derived from the calculated nodal carbon emission factors and is the core basis for formulating low-carbon scheduling strategies. The carbon emission status subspace includes: nodal carbon emission factor vector: It reflects the real-time carbon emission intensity of each node; the statistical characteristics of carbon emission factors include the system average carbon emission factor. and standard deviation This reflects the degree of spatial differences in carbon emissions.

[0096] (c) User response state subspace

[0097] User response states, based on the constructed multi-level response characteristics of electric vehicle users, describe the charging needs and response characteristics of the electric vehicle user group. The user response state subspace includes: categorized user demand vectors. , , , These represent the total charging needs of price-sensitive, convenience-first, and time-constrained users at time t, respectively; and the available V2G capacity for users. User battery state of charge (SOC) distribution Spatial demand distribution entropy: This reflects the spatial concentration of charging demand; user response sensitivity indicators: A vector of user response coefficients to price signals, calculated based on historical response data;

[0098] (d) Temporal feature subspace

[0099] The temporal characteristics reflect the temporal regularity of electric vehicle charging demand and power grid operation. The temporal characteristic subspace includes: Time-moment characteristics: (This represents the normalized value of the number of hours corresponding to time t, which is divided by 24 to normalize to the [0,1] interval, and is used to reflect the time pattern within a day.) (This represents the normalized value of the week corresponding to time t, which is divided by 7 to normalize to the [0,1] interval, used to reflect the periodicity within a week), using normalized time encoding; Historical state memory: key state variables from the previous few moments, enhancing the Markov property of the state (the above two variables are historical state information. By introducing the above variables, the current state can more completely contain the information needed for decision-making, especially the temporal characteristics of user behavior).

[0100] The state variables of the above subspaces are concatenated to form a unified state vector: To ensure the stability of neural network training, the obtained state vector needs to be standardized.

[0101] The action space designed in this invention is as follows:

[0102] The agent's action is defined as the bidirectional carbon excitation coefficient. ,in, These are the charging carbon incentive coefficient and the discharging carbon incentive coefficient, respectively. This parameter directly controls the formulation of differentiated electricity prices for each node. The physical meaning of the carbon incentive coefficient is: the intensity of the system's incentive for low-carbon charging behavior. The larger the value, the more obvious the price advantage of the low-carbon node.

[0103] Carbon excitation coefficient based on agent output And the node carbon emission factor calculated above The real-time electricity price at each node is calculated using the following formula: , ;in, The charging price for charging station i. The discharge price for charging station i. These are the baseline prices for charging and discharging, respectively. , These are the maximum permissible changes in charging and discharging electricity prices, which are typically determined based on the electricity price before demand response. , These represent the maximum and minimum values ​​of the carbon emission factor at each node in the system at the current moment. This design ensures that the spatial difference in electricity price is positively correlated with the difference in carbon emission, while parameter constraints guarantee the rationality of electricity price and user acceptance.

[0104] To ensure the generated electricity pricing strategy is practically feasible, the following constraints need to be set:

[0105] (1) Non-negativity constraint of electricity price: To avoid negative electricity prices;

[0106] (2) Constraints on electricity price fluctuations: , To avoid user dissatisfaction caused by drastic fluctuations in electricity prices;

[0107] (3) Balance of income and expenditure constraint: To ensure that the electricity price covers the cost of electricity sales; among which, This represents the electricity sales cost of charging station i;

[0108] (4) SOC requirement constraints: ;in, , These represent the charging and discharging efficiencies of electric vehicle j, respectively. , , respectively, represent the charging and discharging power of electric vehicle j at time t;

[0109] SOC satisfies upper and lower bound constraints. ,in This represents the state of charge of electric vehicle j at time t. These represent the upper and lower limits of the state of charge, respectively;

[0110] When a constraint is violated, a projection algorithm is used to map the action to the feasible region, that is, to select the infeasible action at this time as the nearest feasible action.

[0111] The system's immediate reward function is designed as a weighted combination of three components:

[0112] ,in , , These are the weighting coefficients for economic benefits, environmental benefits, and constraints / penalties, respectively.

[0113] The economic benefit reward reflects the economic returns of the current scheduling strategy, and the calculation formula is as follows: ;

[0114] Among them, electricity sales revenue Electricity generation cost ,in The amount of electricity discharged by electric vehicle users at charging station i. The marginal cost of generating electricity for unit u; For the active power output of unit u;

[0115] Environmental benefit rewards quantify the carbon emission reduction value achieved by the current dispatch strategy, and can also be regarded as the revenue obtained by grid operators in the carbon market: ;in, This represents the system's carbon emissions under the baseline scenario (no differentiated electricity pricing). This represents the actual carbon emissions under the current scheduling strategy. The economic value of carbon emissions (carbon price); actual carbon emissions are calculated as follows: The baseline carbon emissions are based on load distribution under uniform electricity prices. Perform the calculation: ,in, This represents the baseline charging load of charging station i.

[0116] Constraint penalties are used to ensure that the dispatch strategy meets the requirements for safe grid operation, including:

[0117] Voltage constraint penalty:

[0118] ,in and These are the upper and lower limits of the node voltage allowed during system operation;

[0119] Line capacity constraint penalty:

[0120] ,in This represents the maximum allowable current carrying capacity of the e-th line;

[0121] Penalties for unreasonable electricity pricing:

[0122] ;

[0123] in, The maximum allowable electricity price;

[0124] The total constraint penalty is:

[0125] ;

[0126] in, , , These are the coefficients corresponding to voltage constraint penalties, line capacity constraint penalties, and electricity price reasonableness penalties, respectively.

[0127] Based on the state space, action space, and reward function designed in the previous step, this method uses an Actor-Critic architecture to implement policy learning for the agent and employs the Proximal Policy Optimization (PPO) algorithm to train and schedule the agent, achieving end-to-end learning from environmental perception to decision output.

[0128] Specifically, the Actor-Critic network structure is designed as follows:

[0129] The network structure consists of two main components: the Actor network (policy network): which uses state vectors... As input, the output carbon excitation coefficient The probability distribution parameters are used to map states to actions; Critic network (value network): uses state vectors... As input, output the value estimate for that state. This provides a baseline for policy updates.

[0130] The probability distribution parameters output by the Actor network and A normal distribution is defined. The agent samples the specific carbon excitation coefficients from this distribution: Sampled from the Actor network Actions representing an intelligent agent must satisfy predefined constraints, which are handled in the following ways: This ensures that the carbon excitation coefficient remains within the set upper and lower limits;

[0131] The Critic network is responsible for evaluating the value of a state, providing an accurate baseline estimate for the Actor network's policy updates. The value function learned by the Critic network is defined as the expected cumulative reward obtainable from the current state using the current policy: ,in, for Instant rewards for each moment This is a discount factor that reflects the importance attached to future rewards.

[0132] The core idea of ​​the PPO algorithm is to limit the difference between the old and new strategies. The objective function is designed as follows:

[0133] ;

[0134] in: The ratio of the probability of the new strategy to the probability of the old strategy is used to characterize the tendency of the new strategy to take the current action. This is the advantage function estimate, reflecting the merits of the current action relative to the average level. The step size for updating the control strategy. Indicates: If Output ;if Output ;if Output .

[0135] To reduce variance and improve estimation accuracy, the Generalized Advantage Estimation (GAE) method is used to design an advantage function to evaluate the relative value of a specific state-action pair: ,in For timing difference error, For GAE parameters, balance bias and variance (when When this happens, the model will only focus on the next step's gain, resulting in a smaller variance but a larger deviation from the true advantage; when... In this case, the model will focus on long-term returns, resulting in a smaller deviation from the actual returns. However, it will be more susceptible to short-term environmental randomness, leading to a larger variance. For estimating the value of state S;

[0136] The training objective of the Critic network is to minimize the prediction error of the value function. ,in The target value is calculated as follows: Value estimation function It is a parameter of A value network that can value states;

[0137] Based on the above, the overall loss function of the PPO algorithm is: , where c is the weighting coefficient of the value function loss and entropy reward.

[0138] Experience collection is fundamental to reinforcement learning training. This method collects training data through real-time interaction between the agent and the environment. The agent interacts with the environment according to the current policy, collecting a complete decision trajectory: Since PPO is an online policy algorithm, but in actual implementation there is the reuse of experience, importance sampling correction is required: calculate the probability ratio of the new and old policies, and if the ratio exceeds the preset threshold, reweight or remove the samples to ensure training stability.

[0139] It should be noted that the network parameter update uses the stochastic gradient descent method, and the specific steps are as follows:

[0140] Forward propagation: Batch data is input into the Actor and Critic networks to calculate policy probabilities and state values;

[0141] Loss calculation: Calculate the total loss based on the objective function;

[0142] Gradient calculation: The gradient of each network parameter is calculated using the backpropagation algorithm;

[0143] Parameter update: Update network parameters using the Adam optimizer;

[0144] Design multiple convergence criteria to ensure the effectiveness of the training process:

[0145] First, determine whether the reward function converges, i.e., the average reward change over multiple consecutive rounds is less than a preset threshold; second, determine the stability of the strategy, i.e., the change in strategy parameters is less than a threshold; finally, determine whether the value function converges, i.e., the state value prediction error stabilizes at a low level.

[0146] After training, the deep reinforcement learning algorithm outputs the following key results:

[0147] Trained Actor network parameters Used for online scheduling decisions;

[0148] Trained Critic network parameters Used for strategy performance evaluation.

[0149] In practical applications, Input the Actor network to obtain the distribution parameters of the carbon excitation coefficients: To ensure the stability of scheduling decisions, a deterministic strategy is usually adopted, directly using the distribution mean as the output action: At the same time, constraints are checked to ensure that the generated carbon incentive coefficient meets the requirements. Finally, the generated electricity price is determined based on the carbon incentive coefficient to guide electric vehicle users to respond.

[0150] Example 2

[0151] This embodiment provides a low-carbon demand response scheduling system for electric vehicles that considers vehicle-to-grid interaction, including:

[0152] The carbon emission factor acquisition module is used to collect historical data on distribution network topology, load, distributed power sources, and user charging, and to calculate the carbon emission factor of each node in the distribution network.

[0153] The electric vehicle user multi-level response characteristic model construction module is used to construct an electric vehicle user multi-level response characteristic model: classify electric vehicle users, establish a differentiated utility function based on stochastic utility theory, and calibrate the weights of the utility function. Based on the constructed utility function, a multinomial logic model is used to calculate the probability of users choosing each charging station, thereby obtaining the charging load of each node.

[0154] The deep reinforcement learning scheduling model construction module is used to build a deep reinforcement learning scheduling model, including designing the corresponding state space, action space and reward function, constructing the Actor-Critic network, training the Actor-Critic network using the PPO algorithm, and obtaining the trained Actor-Critic network.

[0155] The electric vehicle user-guided response module is used to obtain the carbon incentive coefficient using a trained Actor-Critic network, determine the electricity price based on the carbon incentive coefficient, and guide electric vehicle users to respond accordingly.

[0156] Furthermore, the present invention adopts the following technical solution:

[0157] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction as described above.

[0158] Furthermore, the present invention adopts the following technical solution:

[0159] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction as described above.

[0160] From the above description of the embodiments, those skilled in the art will clearly understand that the facilities of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Embodiments of the present invention can be implemented using existing processors, or by dedicated processors used for this or other purposes for suitable systems, or by hardwired systems. Embodiments of the present invention also include non-transitory computer-readable storage media, comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon; such machine-readable media can be any available medium accessible by a general-purpose or special-purpose computer or other machine with a processor. For example, such machine-readable media can include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of machine-executable instructions or data structures and is accessible by a general-purpose or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or other communication connection (hardwired, wireless, or a combination of hardwired and wireless), that connection is also considered a machine-readable medium.

[0161] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A low-carbon demand response scheduling method for electric vehicles considering vehicle-to-grid interaction, characterized in that, The method includes the following steps: Collect historical data on distribution network topology, loads, distributed power sources, and user charging to calculate the carbon emission factor of each node in the distribution network. Constructing a multi-level response characteristic model for electric vehicle users: classifying electric vehicle users, establishing a differentiated utility function based on stochastic utility theory, and calibrating the weights of the utility function. Based on the constructed utility function, a multinomial logic model is used to calculate the probability of users choosing each charging station, thereby obtaining the charging load of each node. Construct a deep reinforcement learning scheduling model, including designing the corresponding state space, action space and reward function, constructing an Actor-Critic network, training the Actor-Critic network using the PPO algorithm, and obtaining the trained Actor-Critic network. The carbon incentive coefficient is obtained using a trained Actor-Critic network, and the electricity price is determined based on the carbon incentive coefficient to guide electric vehicle users to respond.

2. The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction according to claim 1, characterized in that, The method for classifying electric vehicle users, establishing differentiated utility functions based on stochastic utility theory, and labeling the weights of the utility functions is as follows: Electric vehicle users are categorized into price-sensitive users, convenience-first users, and time-constrained users. A differentiated utility function based on stochastic utility theory is established. For user j of type k choosing charging station i at time t, the utility function is: ； in, Let be the electricity price level of charging station i at time t; Let i be the distance from the current location of electric vehicle user j to charging station i. Let be the expected waiting time for charging station i at time t; Rate the service quality of charging station i; For electric vehicle users, the ideal discharge capacity; For random error terms; weight parameters , , , , The sensitivity weights of different types of electric vehicle users to factors such as electricity price level, charging distance, waiting time, service quality, and ideal discharge capacity are represented as follows: The maximum likelihood estimation method based on historical data is used to calibrate the weight parameters: assuming that user choices follow the principle of utility maximization, a likelihood function of the choice probability is constructed based on the information when the user makes the choice; the optimal sensitivity weight parameter value is solved by maximizing the likelihood function.

3. The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction according to claim 2, characterized in that, A likelihood function is constructed based on information about the user's choice. The optimal sensitivity weight parameter values ​​are then obtained by maximizing the likelihood function. The likelihood function has the following form: ,in, The number of users of type k electric vehicles. The number of observation periods. For the actual choice made by electric vehicle user j at time t. ; To find the optimal weight parameters, we need to maximize the log-likelihood function and take its partial derivative with respect to the weight parameters. Setting this derivative to zero yields the optimization conditions: Since the system of equations is nonlinear, it can be solved using numerical optimization algorithms such as the Newton-Raphson method or the quasi-Newton method.

4. The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction according to claim 2, characterized in that, Based on the constructed utility function, a multinomial logic model is used to calculate the probability of users choosing each charging station, thereby obtaining the charging load of each node. The probability that user j of type k chooses charging station i at time t is: ,in, This represents the total number of available charging stations. For temperature parameters, control the degree of randomness in user selection; Let the total number of users of type k in the system be . The average charging demand of each user is Then the expected charging load of charging station i at time t for: ,in, The average probability of a user of type k choosing charging station i for charging: .

5. The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction according to claim 1, characterized in that, The state space includes node voltage state vector, line load rate vector, node carbon emission factor vector, system average carbon emission factor, carbon emission factor standard deviation, classified user demand vector, user V2G available capacity, user battery state of charge distribution, spatial demand distribution entropy, user response sensitivity index, and time characteristics. The action space is defined as the bidirectional carbon excitation coefficient. ,in, These are the carbon excitation coefficient during charging and the carbon excitation coefficient during discharging, respectively. The reward function is: ； in, , , These are the weighting coefficients for economic benefits, environmental benefits, and constraints / penalties, respectively. Economic benefit reward The economic benefits of the current scheduling strategy are reflected in the following formula: ； Among them, electricity sales revenue Electricity generation cost ,in The amount of electricity discharged by electric vehicle users at charging station i. The marginal cost of generating electricity for unit u; For the active power output of unit u; Environmental benefit rewards Quantify the carbon reduction value achieved by the current scheduling strategy: ； in, This represents the system's carbon emissions under the baseline scenario. This represents the actual carbon emissions under the current scheduling strategy. The economic value of carbon emissions; actual carbon emissions are calculated as follows: The baseline carbon emissions are based on load distribution under uniform electricity prices. Perform the calculation: ,in, The baseline charging load represents charging station i; Constraints and penalties To ensure that dispatch strategies meet the requirements for safe operation of the power grid, penalties include voltage constraint penalties, line capacity constraint penalties, and electricity price reasonableness penalties.

6. The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction according to claim 1, characterized in that, The method for constructing an Actor-Critic network and training it using the PPO algorithm is as follows: The Actor-Critic network structure includes an Actor network and a Critic network, where the Actor network uses a state-space vector. As input, the output carbon excitation coefficient The probability distribution parameters are used to map states to actions; the Critic network uses state space vectors... As input, output the value estimate for that state. This provides a baseline for policy updates; The overall loss function when training an Actor-Critic network using the PPO algorithm is: ； in, c represents the weighting coefficients for the value function loss and entropy reward. This represents the probability ratio between the old and new strategies; For the estimation of the advantage function, To control the step size of the policy update; Indicates: If Output ;if Output ;if Output Minimize the prediction error of the value function ,in The target value is calculated as follows: Value estimation function It is a parameter of The value network can value the state; ,in For timing difference error, For GAE parameters, balance bias and variance, Let t be the value estimate of state S at time t.

7. The electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction according to claim 1, characterized in that, The method for incentivizing electric vehicle users to respond by determining electricity prices based on carbon incentive coefficients is as follows: Carbon excitation coefficient based on agent output and the obtained node carbon emission factors The real-time electricity price at each node is calculated using the following formula: ， ； in, The charging price for charging station i. The discharge price for charging station i. These are the baseline prices for charging and discharging, respectively. , These represent the maximum permissible changes in charging and discharging electricity prices, respectively. , These represent the maximum and minimum values ​​of the node carbon emission factor in the system at the current moment; In addition, real-time electricity pricing needs to meet the following constraints: (1) Non-negativity constraint of electricity price: To avoid negative electricity prices; (2) Constraints on electricity price fluctuations: , To avoid user dissatisfaction caused by drastic fluctuations in electricity prices; (3) Balance of income and expenditure constraint: To ensure that the electricity price covers the cost of electricity sales; among which, This represents the electricity sales cost of charging station i; (4) SOC requirement constraints: ;in, , These represent the charging and discharging efficiencies of electric vehicle j, respectively. , , respectively, represent the charging and discharging power of electric vehicle j at time t; SOC satisfies upper and lower bound constraints. ,in This represents the state of charge of electric vehicle j at time t. These represent the upper and lower limits of the state of charge, respectively.

8. A low-carbon demand response scheduling system for electric vehicles considering vehicle-to-grid interaction, characterized in that, include: The carbon emission factor acquisition module is used to collect historical data on distribution network topology, load, distributed power sources, and user charging, and to calculate the carbon emission factor of each node in the distribution network. The electric vehicle user multi-level response characteristic model construction module is used to construct an electric vehicle user multi-level response characteristic model: classify electric vehicle users, establish a differentiated utility function based on stochastic utility theory, and calibrate the weights of the utility function. Based on the constructed utility function, a multinomial logic model is used to calculate the probability of users choosing each charging station, thereby obtaining the charging load of each node. The deep reinforcement learning scheduling model construction module is used to build a deep reinforcement learning scheduling model, including designing the corresponding state space, action space and reward function, constructing the Actor-Critic network, training the Actor-Critic network using the PPO algorithm, and obtaining the trained Actor-Critic network. The electric vehicle user-guided response module is used to obtain the carbon incentive coefficient using a trained Actor-Critic network, determine the electricity price based on the carbon incentive coefficient, and guide electric vehicle users to respond accordingly.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the electric vehicle low-carbon demand response scheduling method considering vehicle-to-grid interaction as described in any one of claims 1 to 7.

Images