Charging station pricing and charging method based on multi-agent reinforcement learning robust reward function
Through the robust reward function of multi-agent reinforcement learning, the behavior and grid constraints of electric vehicles are processed, and the pricing and power declaration of charging stations are optimized, which solves the problems of uncertainty in electric vehicles and coupling relationship between distribution networks in the existing technology. Improves the revenue of the charging station and the stability of the power system.
Patent Information
- Application Number
- CN202410823734.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-25
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-06-25
AI Technical Summary
The existing technology is difficult to effectively deal with the uncertainty of electric vehicle behavior and the coupling relationship between the distribution network and the charging station, resulting in the inability to model charging demands, increasing the computational complexity, affecting the stability of the power system and the profit of the charging station. .
Using a robust reward function based on multi-agent reinforcement learning, by obtaining power grid models and historical data, establishing distribution network operation models and charging station operation models, calculating robust reward functions, optimizing charging station pricing and power declaration, and processing electric power Uncertainty in automotive behavior and grid constraints.
It improves the revenue of the charging station and the stability of the power system, balances the load of the distribution network, realizes the optimized pricing and charging decisions of the charging station in the dynamic market, and enhances the strategic accuracy under information privacy.
Smart Images

Figure CN118917901A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of electricity-transportation network collaborative optimization, and specifically relates to a charging station pricing and charging method based on multi-agent reinforcement learning robust reward function. Background Art
[0002] At present, there is a great investment in transportation electrification to promote energy transformation and decarbonization. This includes large-scale deployment of electric transportation networks and distribution networks. Close integration between cars and fast charging stations. In recent years, the scale of electric vehicles has grown rapidly. By 2030, the number of electric vehicles is expected to reach 1.81 billion, consuming 43,000GWh of electricity per year. The distribution network and cause local congestion, which may lead to large-scale power outages. The large amount of electricity consumed by random charging of electric vehicles has brought a huge burden to electric vehicle users, causing serious economic losses and negative social impacts.
[0003] However, there are two important gaps in the existing literature on charging station decision-making. First, most of the existing work encapsulates charging demand in the modeling of the transportation network. In fact, this is highly dependent on the complete and private information of the electric vehicle's origin-destination trip. However, the real environment is highly random and dynamic, and accurate demand modeling may not be realistic enough. Second, in some existing literature, electric vehicles are used as intelligent agents for decision-making. As the number of electric vehicles increases, the computational complexity of pricing optimization will increase significantly. Therefore, it is urgent to develop a charging station pricing and charging method based on a multi-agent reinforcement learning robust reward function to take into account the uncertainty of charging demand and the coupling relationship between the distribution network and electric vehicles. Summary of the invention
[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a charging station pricing and charging method based on a multi-agent reinforcement learning robust reward function in view of the deficiencies in the prior art. The present invention takes into account the efficient collaboration among the distribution network, electric vehicles, and charging stations. The robust reinforcement learning algorithm calculates the charging and pricing decisions of the charging station based on historical data to deal with the uncertainty of electric vehicle behavior. Grid-related constraints are introduced to coordinate the scheduling of power-charging stations. The present invention can not only promote the collaboration between the distribution network and electric vehicles under information barriers, but also help charging stations increase their own profits based on the dynamic market.
[0005] Technical solution: In order to solve the above technical problems, the present invention proposes a charging station pricing and charging method based on multi-agent reinforcement learning robust reward function, which includes the following steps:
[0006] Step 1: Obtain the network coefficient and operation coefficient of the power grid model, wherein the network coefficient includes the power grid topology, line resistance and impedance, and the operation coefficient includes the power generation coefficient of the generator set, the charging and discharging coefficient of the energy storage system, the photovoltaic inverter coefficient, and the charging station parameters;
[0007] Step 2: Obtain grid load demand, photovoltaic output, historical vehicle flow at charging stations, and historical charging demand scenario data;
[0008] Step 3: Based on the obtained distribution network parameters, with the grid operation constraints and unit operation constraints as constraints, and with the minimum distribution network operation cost as the objective function, a distribution network operation model based on the optimal power flow is established, and the node electricity price of the node to which the charging station belongs is obtained according to the model;
[0009] Step 4: The charging station is used as an intelligent agent. The node electricity price and historical data of the charging station obtained in step 3 are used as the input of the intelligent agent state. The vehicle charging demand update constraint, parking time update constraint, and charging power constraint are used as constraint conditions. The maximum comprehensive benefit of the charging station is used as the objective function. A charging station operation model is established. The declared power of the charging station and the distribution network is calculated based on the model.
[0010] Step 5. Based on the node electricity price obtained in step 3 and the declared power obtained in step 4, the robust reward function of the charging station is calculated using the multi-agent robust proximal strategy optimization algorithm. The process of constructing the robust reward function is the process of quantifying the difference between the reward in the worst case and the deterministic reward, and the difference is relaxed using the dual form of the Holder inequality and the Cauchy-Schwarz inequality.
[0011] Furthermore, in step 3, the distribution network operation model is:
[0012]
[0013] In the formula, F DN represents the operating cost of the distribution network, g represents the distributed power source, a g represents the second-order cost coefficient of distributed generation g, b g represents the first-order cost coefficient of distributed generation g, c g represents the constant cost coefficient of the distributed generation g, P g,t represents the output power of distributed generation g at time t, P sub,t represents the electric power purchased from the root node at time t, c sub,t represents the price of electricity purchased from the root power plant at time t, c DNe Indicates the charge and discharge cost coefficient of DNe energy storage, represents the discharge power of the energy storage numbered DNe at time t, represents the charging power of the DNe energy storage at time t, ccut represents the penalty coefficient for compliance reduction, j represents the load node, represents the load reduction amount of node j at time t.
[0014] Furthermore, in step 3, the distribution network operation constraints are:
[0015] P g,t -P g,t-1 ≤RU g (A-2)
[0016] P g,t-1 -P g,t ≤RD g (A-3)
[0017]
[0018] SOC DNe,min ≤SOC DNe,t ≤SOC DNe,max (A-5)
[0019]
[0020] P g,min ≤P g,t ≤P g,max (A-12)
[0021] Q g,min ≤Q g,t ≤Q g,max (A-13)
[0022]
[0023] P sub,min ≤P sub,t ≤P sub,max (A-15)
[0024]
[0025] In the formula, RU g and RD g They represent the upper and lower bounds of the power ramp of the distributed power supply g, SOC DNe,t ,SOC DNe,t+1 They represent the state of charge of the energy storage numbered DNe at time t and time t+1 respectively, Indicates the charging state of the energy storage numbered DNe at time t, represents the discharge state of the energy storage numbered DNe at time t, η ch and η dis Respectively represent the charging and discharging efficiency, Δt DNIndicates the time interval of the distribution network, SOC DNe,min ,SOC DNe,max They represent the minimum and maximum values of the energy storage charge state numbered DNe, P j,t represents the active load of node j at time t, represents the load reduction of node j at time t, f and k represent nodes in the distribution network, and N DN represents the set of distribution network nodes, F DN represents the set of lines flowing into node j in the distribution network, T DN represents the set of lines flowing out of node j in the distribution network, represents the active power load after node j is reduced at time t, Q j,t represents the reactive power load of node j at time t, P jf,t and P kj,t They represent the active power of line jf and line kj at time t, Q jf,t and Q kj,t They represent the reactive power of line jf and line kj at time t, R kj , X kj Respectively represent the resistance and reactance of line kj; I kj,t ,I jf,t They represent the currents of lines kj and jf at time t respectively; B DN represents the set of all lines in the distribution network, V f,t 、V j,t 、V k,t Respectively represent the voltage of nodes f, j, and k at time t, V k,min ,V k,max They represent the minimum and maximum limits of the node k voltage, I jf,max Indicates the maximum value of the line jf current, P g,min , P g,max They represent the minimum and maximum active output of distributed power source g, Q g represents the reactive power output of distributed generation g, Q g,min .and Q g,max They represent the maximum and minimum reactive output of distributed power source g, P sub,min and P sub,max They represent the minimum and maximum values of the power purchased by the root node, and They respectively represent the maximum power of discharging and charging of the energy storage device numbered DNe.
[0026] Furthermore, in step 4, the charging station operation model with the goal of maximizing revenue is:
[0027]
[0028] Charging demand update constraints:
[0029]
[0030] Remaining parking time update constraint:
[0031]
[0032] Charging power constraints:
[0033]
[0034] Collection definition:
[0035]
[0036] In the formula, C z,t represents the revenue of charging station z at time t, r z,t represents the service fee pricing of charging station z at time t, represents the time when electric car i arrives, represents the set of vehicles that arrive at charging station z at historical time t, D i (r z,t ) represents the charging demand of electric vehicle i under the service fee rz,t, pi represents the initial parking time of electric vehicle i, λ j,t represents the marginal electricity price of node j at time t, ω represents the penalty coefficient of electricity deviation, e z,t represents the total charging rate of charging station z at time t, represents the actual charging rate of charging station z at time t, They represent the remaining charging demand of electric vehicle i at time t and time t+1 respectively, They represent the remaining parking time of electric vehicle i at time t and time t+1, respectively, i,t represents the charging rate of electric vehicle i at time t, represents the maximum charging rate of electric vehicle i, represents the maximum total charging rate of charging station z, and They represent the set of electric vehicles in charging station z at time t and time t+1 respectively.
[0037] Furthermore, in step 5, the robust reward function is calculated as:
[0038]
[0039] In the formula, They represent the strategies of the charging station in uncertain and certain states respectively, A represents the action set of the charging station, S represents the state set of the charging station, Representation strategy and π CS The difference in reward function for the next charging station z, s t / s t+1 They represent the status of the charging station at time t and t+1 respectively. They represent the strategies π CS and The next charging station is in state s at time t t The probability of Represents strategy π CS and The next charging station is in state s at time t t The state value function, a t represents the action of the charging station at time t, r t represents the revenue of the charging station at time t, Respectively represent strategies The comprehensive income of the charging station is Representation strategy and π CS Next charging station status t The total variation distance.
[0040] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0041] Compared with the traditional charging station decision-making solution, the technical solution of the present invention handles the uncertainty of electric vehicle behavior by introducing a robust reward function. At the same time, in order to deal with the problem of information barriers, the intelligent agent makes decisions on its own behavior based on historical data. The results of the example test show that the method proposed by the present invention helps charging stations improve their own profits in a more complex real-world environment, improves the utilization rate of charging piles, balances the peak and valley values of the distribution network load, and improves the stability of the power system. The present invention takes into account the reality that there is an information barrier between the current transportation network and the charging station, and makes decisions through robust reinforcement learning based on historical data to ensure the precise strategy of the charging station under information privacy, and realizes the optimal pricing and charging decision of the charging station. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 is a flow chart of the method of the present invention;
[0043] Figure 2 is the utilization of charging stations under different service fees;
[0044] Figure 3 It is a comparison chart of the training results of the proposed algorithm and the existing algorithm. DETAILED DESCRIPTION
[0045] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0046] like Figure 1 As shown, the present invention proposes a charging station pricing and charging method based on a multi-agent reinforcement learning robust reward function, the method comprising the following steps:
[0047] Step 1: Obtain the network coefficient and operation coefficient of the power grid model, wherein the network coefficient includes the power grid topology, line resistance and impedance, and the operation coefficient includes the power generation coefficient of the generator set, the charging and discharging coefficient of the energy storage system, the photovoltaic inverter coefficient, and the charging station parameters;
[0048] Step 2: Obtain grid load demand, photovoltaic output, historical vehicle flow at charging stations, and historical charging demand scenario data;
[0049] Step 3: Based on the obtained distribution network parameters, with the grid operation constraints and unit operation constraints as constraints, and with the minimum distribution network operation cost as the objective function, a distribution network operation model based on the optimal power flow is established, and the node electricity price of the node to which the charging station belongs is obtained according to the model;
[0050] Step 4: The charging station is used as an intelligent agent. The node electricity price and historical data of the charging station obtained in step 3 are used as the input of the intelligent agent state. The vehicle charging demand update constraint, parking time update constraint, and charging power constraint are used as constraint conditions. The maximum comprehensive benefit of the charging station is used as the objective function. A charging station operation model is established. The declared power of the charging station and the distribution network is calculated based on the model.
[0051] Step 5. Based on the node electricity price obtained in step 3 and the declared power obtained in step 4, the robust reward function of the charging station is calculated using the multi-agent robust proximal strategy optimization algorithm. The process of constructing the robust reward function is the process of quantifying the difference between the reward in the worst case and the deterministic reward, and the difference is relaxed using the dual form of the Holder inequality and the Cauchy-Schwarz inequality.
[0052] Furthermore, in step 3, the distribution network operation model is:
[0053]
[0054] In the formula, F DN represents the operating cost of the distribution network, g represents the distributed power source, a g represents the second-order cost coefficient of distributed generation g, b g represents the first-order cost coefficient of distributed generation g, c grepresents the constant cost coefficient of the distributed generation g, P g,t represents the output power of distributed generation g at time t, P sub,t represents the electric power purchased from the root node at time t, c sub,t represents the price of electricity purchased from the root power plant at time t, c DNe Indicates the charge and discharge cost coefficient of DNe energy storage, represents the discharge power of the energy storage numbered DNe at time t, represents the charging power of the DNe energy storage at time t, c cut represents the penalty coefficient for compliance reduction, j represents the load node, represents the load reduction amount of node j at time t.
[0055] Furthermore, in step 3, the distribution network operation constraints are:
[0056] P g,t -P g,t-1 ≤RU g (A-2)
[0057] P g,t-1 -P g,t ≤RD g (A-3)
[0058]
[0059] SOC DNe,min ≤SOC DNe,t ≤SOC DNe,max (A-5)
[0060]
[0061] P g,min ≤P g,t ≤P g,max (A-12)
[0062] Q g,min ≤Q g,t ≤Q g,max (A-13)
[0063]
[0064] P sub,min ≤P sub,t ≤P sub,max (A-15)
[0065]
[0066] In the formula, RU g and RD gThey represent the upper and lower bounds of the power ramp of the distributed power supply g, SOC DNe,t ,SOC DNe,t+1 They represent the state of charge of the energy storage numbered DNe at time t and time t+1 respectively, Indicates the charging state of the energy storage numbered DNe at time t, represents the discharge state of the energy storage numbered DNe at time t, η ch and η dis Respectively represent the charge and discharge efficiency, Δt DN Indicates the time interval of the distribution network, SOC DNe,min ,SOC DNe,max They represent the minimum and maximum values of the energy storage charge state numbered DNe, P j,t represents the active load of node j at time t, represents the load reduction of node j at time t, f and k represent nodes in the distribution network, and N DN represents the set of distribution network nodes, F DN represents the set of lines flowing into node j in the distribution network, T DN represents the set of lines flowing out of node j in the distribution network, represents the active power load after node j is reduced at time t, Q j,t represents the reactive power load of node j at time t, P jf,t and P kj,t They represent the active power of line jf and line kj at time t, Q jf,t and Q kj,t They represent the reactive power of line jf and line kj at time t, R kj , X kj Respectively represent the resistance and reactance of line kj; I kj,t ,I jf,t They represent the currents of lines kj and jf at time t respectively; B DN represents the set of all lines in the distribution network, V f,t 、V j,t 、V k,t Respectively represent the voltage of nodes f, j, and k at time t, V k,min ,V k,max They represent the minimum and maximum limits of the node k voltage, I jf,max Indicates the maximum value of the line jf current, P g,min , P g,max They represent the minimum and maximum active output of distributed power source g, Q g represents the reactive power output of distributed generation g, Q g,min .and Q g,max They represent the maximum and minimum reactive output of distributed power source g, Psub,min and P sub,max They represent the minimum and maximum values of the power purchased by the root node, and They respectively represent the maximum power of discharging and charging of the energy storage device numbered DNe.
[0067] Furthermore, in step 4, the charging station operation model with the goal of maximizing revenue is:
[0068]
[0069] Charging demand update constraints:
[0070]
[0071] Remaining parking time update constraint:
[0072]
[0073] Charging power constraints:
[0074]
[0075] Collection definition:
[0076]
[0077] In the formula, C z,t represents the revenue of charging station z at time t, r z,t represents the service fee pricing of charging station z at time t, represents the time when electric car i arrives, represents the set of vehicles that arrive at charging station z at historical time t, D i (r z,t ) represents the charging demand of electric vehicle i under the service fee rz,t, pi represents the initial parking time of electric vehicle i, λ j,t represents the marginal electricity price of node j at time t, ω represents the penalty coefficient of electricity deviation, e z,t represents the total charging rate of charging station z at time t, represents the actual charging rate of charging station z at time t, They represent the remaining charging demand of electric vehicle i at time t and time t+1 respectively, They represent the remaining parking time of electric vehicle i at time t and time t+1, respectively, i,t represents the charging rate of electric vehicle i at time t, represents the maximum charging rate of electric vehicle i, represents the maximum total charging rate of charging station z, and They represent the set of electric vehicles in charging station z at time t and time t+1 respectively.
[0078] Furthermore, in step 5, the robust reward function is calculated as:
[0079]
[0080] In the formula, They represent the strategies of the charging station in uncertain and certain states respectively, A represents the action set of the charging station, S represents the state set of the charging station, Representation strategy and π CS The difference in reward function for the next charging station z, s t / s t+1 They represent the status of the charging station at time t and t+1 respectively. They represent the strategies π CS and The next charging station is in state s at time t t The probability of Represents strategy π CS and The next charging station is in state s at time t t The state value function, a t represents the action of the charging station at time t, r t represents the revenue of the charging station at time t, Respectively represent strategies The comprehensive income of the charging station is Representation strategy and π CS Next charging station status t The total variation distance.
[0081] Case Analysis
[0082] The following example illustrates the superiority of the charging station pricing and charging method based on multi-agent reinforcement learning robust reward function described in the present invention. Figure 2 The improved IEEE 33-node power system shown in FIG. 1 is a schematic diagram of an improved IEEE 33-node power system. In order to compare the superiority of the method proposed in the present invention, the charging station pricing and charging method based on multi-agent proximal strategy optimization and the charging station pricing and charging method based on robust multi-agent proximal strategy optimization proposed in the present invention are respectively used for power-electric vehicle collaborative scheduling. The present invention is implemented through the Pytorch platform and the Gurobi solver is used to solve the nonlinear programming problem.
[0083] Based on this example, the performance and scheduling results of the charging station pricing and charging method based on multi-agent proximal strategy optimization and the charging station pricing and charging method based on robust multi-agent proximal strategy optimization proposed in this invention are compared (the results are shown in Table 1), as well as the utilization of charging stations under different service fees (the results are shown in Table 2). Figure 2 ). This shows that compared with the model that ignores the uncertainty of electric vehicles, the method proposed in this invention can bring higher returns to charging stations in complex real-world scenarios (Table 1 and Figure 3 ), and charging stations are sensitive to environmental changes when formulating strategies, taking into account market supply and demand and the price sensitivity of electric vehicles.
[0084] Table 1 Comparison of prediction performance and scheduling results of different methods
[0085] Performance Indicators Multi-agent proximal policy optimization Robust multi-agent proximal policy optimization Iterations 60 121 Scheduling results / ¥ 1737.38 1139.95
[0086] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.
Claims
1. A charging station pricing and charging method based on multi-agent reinforcement learning robust reward function, characterized in that: The method comprises the following steps: Step 1: Obtain the network coefficient and operation coefficient of the power grid model, wherein the network coefficient includes the power grid topology, line resistance and impedance, and the operation coefficient includes the power generation coefficient of the generator set, the charging and discharging coefficient of the energy storage system, the photovoltaic inverter coefficient, and the charging station parameters; Step 2: Obtain grid load demand, photovoltaic output, historical vehicle flow at charging stations, and historical charging demand scenario data; Step 3: Based on the obtained distribution network parameters, with the grid operation constraints and unit operation constraints as constraints, and with the minimum distribution network operation cost as the objective function, a distribution network operation model based on the optimal power flow is established, and the node electricity price of the node to which the charging station belongs is obtained according to the model; Step 4: The charging station is used as an intelligent agent. The node electricity price and historical data of the charging station obtained in step 3 are used as the input of the intelligent agent state. The vehicle charging demand update constraint, parking time update constraint, and charging power constraint are used as constraint conditions. The maximum comprehensive benefit of the charging station is used as the objective function. A charging station operation model is established. The declared power of the charging station and the distribution network is calculated based on the model. Step 5. Based on the node electricity price obtained in step 3 and the declared power obtained in step 4, the robust reward function of the charging station is calculated using the multi-agent robust proximal strategy optimization algorithm. The process of constructing the robust reward function is the process of quantifying the difference between the reward in the worst case and the deterministic reward, and the difference is relaxed using the dual form of the Holder inequality and the Cauchy-Schwarz inequality.
2. A charging station pricing and charging method based on multi-agent reinforcement learning robust reward function according to claim 1, characterized in that: In step 3, the distribution network operation model is: In the formula, F DN represents the operating cost of the distribution network, g represents the distributed power source, a g represents the second-order cost coefficient of distributed generation g, b g represents the first-order cost coefficient of distributed generation g, c g represents the constant cost coefficient of the distributed generation g, P g,t represents the output power of distributed generation g at time t, P sub,t represents the electric power purchased from the root node at time t, c sub,t represents the price of electricity purchased from the root power plant at time t, c DNe Indicates the charge and discharge cost coefficient of DNe energy storage, represents the discharge power of the energy storage numbered DNe at time t, represents the charging power of the DNe energy storage at time t, c cut represents the penalty coefficient for compliance reduction, j represents the load node, represents the load reduction amount of node j at time t.
3. A charging station pricing and charging method based on multi-agent reinforcement learning robust reward function according to claim 2, characterized in that: In step 3, the distribution network operation constraints are: P g,t -P g,t-1 ≤RU g (A-2) P g,t-1 -P g,t ≤RD g (A-3) SOC DNe,min ≤SOC DNe,t ≤SOC DNe,max (A-5) P g,min ≤P g,t ≤P g,max (A-12) Q g,min ≤Q g,t ≤Q g,max (A-13) P sub,min ≤P sub,t ≤P sub,max (A-15) In the formula, RU g and RD g They represent the upper and lower bounds of the power ramp of the distributed power supply g, SOC DNe,t ,SOC DNe,t+1 They represent the state of charge of the energy storage numbered DNe at time t and time t+1 respectively, Indicates the charging state of the energy storage numbered DNe at time t, represents the discharge state of the energy storage numbered DNe at time t, η ch and η dis Respectively represent the charging and discharging efficiency, Δt DN Indicates the time interval of the distribution network, SOC DNe,min ,SOC DNe,max They represent the minimum and maximum values of the energy storage charge state numbered DNe, P j,t represents the active load of node j at time t, represents the load reduction of node j at time t, f and k represent nodes in the distribution network, and N DN represents the set of distribution network nodes, F DN represents the set of lines flowing into node j in the distribution network, T DN represents the set of lines flowing out of node j in the distribution network, represents the active power load after node j is reduced at time t, Q j,t represents the reactive power load of node j at time t, P jf,t and P kj,t They represent the active power of line jf and line kj at time t, Q jf,t and Q kj,t They represent the reactive power of line jf and line kj at time t, R kj , X kj Respectively represent the resistance and reactance of line kj; I kj,t ,I jf,t They represent the currents of lines kj and jf at time t respectively; B DN represents the set of all lines in the distribution network, V f,t 、V j,t 、V k,t Respectively represent the voltage of nodes f, j, and k at time t, V k,min ,V k,max They represent the minimum and maximum limits of the node k voltage, I jf,max Indicates the maximum value of the line jf current, P g,min , P g,max They represent the minimum and maximum active output of distributed power source g, Q g represents the reactive power output of distributed generation g, Q g,min .and Q g,max They represent the maximum and minimum reactive output of distributed power source g, P sub,min and P sub,max They represent the minimum and maximum values of the power purchased by the root node, and They represent the maximum power of DNe energy storage discharge and charge, respectively. j,t represents the marginal electricity price of node j at time t.
4. A charging station pricing and charging method based on multi-agent reinforcement learning robust reward function according to claim 3, characterized in that: In step 4, the charging station operation model with the goal of maximizing revenue is: Charging demand update constraints: Remaining parking time update constraint: Charging power constraints: Collection definition: In the formula, C z,t represents the revenue of charging station z at time t, r z,t represents the service fee pricing of charging station z at time t, represents the time when electric car i arrives, represents the set of vehicles that arrive at charging station z at historical time t, D i (r z,t ) represents the service fee r for electric vehicle i z,t The charging demand under i represents the initial parking time of electric vehicle i, λ j,t represents the marginal electricity price of node j at time t, ω represents the penalty coefficient of electricity deviation, e z,t represents the total charging rate of charging station z at time t, represents the actual charging rate of charging station z at time t, They represent the remaining charging demand of electric vehicle i at time t and time t+1 respectively, They represent the remaining parking time of electric vehicle i at time t and time t+1, respectively, i,t represents the charging rate of electric vehicle i at time t, represents the maximum charging rate of electric vehicle i, represents the maximum total charging rate of charging station z, and They represent the set of electric vehicles in charging station z at time t and time t+1 respectively.
5. A charging station pricing and charging method based on multi-agent reinforcement learning robust reward function according to claim 4, characterized in that: In step 5, the robust reward function is calculated as: In the formula, They represent the strategies of the charging station in uncertain and certain states respectively, A represents the action set of the charging station, S represents the state set of the charging station, Representation strategy and π CS The difference in reward function for the next charging station z, s t / s t+1 They represent the status of the charging station at time t and t+1 respectively. They represent the strategies π CS and The next charging station is in state s at time t t The probability of Represents strategy π CS and The next charging station is in state s at time t t The state value function, a t represents the action of the charging station at time t, r t represents the revenue of the charging station at time t, Respectively represent strategies The comprehensive income of the charging station is Representation strategy and π CS Next charging station status t The total variation distance.
Citation Information
Patent Citations
Time-phased joint pricing method and system for electric vehicle photovoltaic charging station network
CN113052402A
Power grid traffic system day-ahead distribution robust scheduling method based on potential game
CN114970191A
Vehicle-station-network interaction decision-making method based on multi-agent deep reinforcement learning
CN115879637A
Optical storage charging station operation optimization method and system based on near-end strategy optimization algorithm
CN115986834A
Self-supported temporary earth retaining structure and constructoin method of self-supported retaining wall using the same
KR102002059B1
Cited By
Electric vehicle charging method and device based on multi-agent deep reinforcement learning
CN119596705A
Method for jointly determining taxi scheduling strategy and charging station pricing strategy
CN121391344A
Method for jointly determining a taxi dispatching strategy and a charging station pricing strategy
CN121391344B
