Electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning

Abstract

The invention discloses an electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning, and the method comprises the steps: S1.1, constructing a double-layer user equalization (UE) demand modeling frame, constructing a dynamic origin-destination (OD) distribution model based on point-of-interest (POI) distribution and road network topology at an upper layer, and achieving traffic flow guided charging demand distribution at a lower layer through a probability model, according to the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning, the problems of demand modeling disjunction, insufficient space competition perception and low efficiency of multi-agent cooperation of a traditional method are effectively solved, the system income is improved, the congestion node proportion is reduced, the utilization rate variation coefficient optimization is reduced, and the method is suitable for large-scale popularization and application. And support can be provided for urban charging facility operation and power grid dispatching.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric vehicles, in particular to a dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning. BACKGROUND

[0002] With the explosive growth of electric vehicle (EV) ownership (as of 2024, some cities have reached tens of thousands of EVs), the operational efficiency and revenue balance of charging infrastructure have become key challenges. There are three major problems with existing charging station pricing strategies: 1. Demand modeling is disconnected from reality: Traditional static pricing relies on fixed demand assumptions and does not consider the spatiotemporal heterogeneity of road network traffic and user state of charge (SOC) distribution, resulting in low matching between pricing and actual charging demand; 2. Lack of spatial competition awareness: Single-agent pricing models ignore the spatial competition between charging stations, leading to "price wars" or "load concentration" among neighboring stations, reducing overall system efficiency; 3. Inefficient multi-agent collaboration: Existing multi-agent reinforcement learning (MARL) methods lack the use of road network topology, making it difficult to achieve global optimization under decentralized decision-making, and the calibration of hyperparameters relies on experience, resulting in poor model robustness.

[0003] Therefore, there is an urgent need for a dynamic pricing method that can accurately capture the spatiotemporal characteristics of charging demand, capture spatial competition among charging stations, and achieve efficient multi-agent collaboration, to address demand-pricing mismatch, spatial competition imbalance, and low system efficiency. SUMMARY

[0004] This section aims to outline some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] 1. Technical problems to be solved:

[0006] To solve the demand-pricing mismatch, spatial competition imbalance, and low system efficiency problems existing in the prior art, the present application is proposed.

[0007] Therefore, the purpose of the present application is to provide a dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning, which is suitable for large-scale electric vehicle charging demand and charging station competition environment at the city scale, and can provide technical support for charging facility revenue improvement, user charging cost optimization, and grid load balancing.

[0008] 2. Technical solutions:

[0009] To solve the above technical problems, according to one aspect of the present application, the present application provides the following technical solutions:

[0010] Comprising the following steps:

[0011] S1.1: Construct a double-layer user equilibrium (UE) demand modeling framework, the upper layer constructs a dynamic origin-destination (OD) assignment model based on point of interest (POI) distribution and road network topology, and the lower layer realizes charging demand assignment of traffic guidance through a probability model, and the objective function, constraint condition and demand mapping rule of the framework are defined;

[0012] S1.2: Design a graph attention enhanced multi-agent reinforcement learning (MARL) framework, integrate a graph attention network (GAT) and a multi-agent proximal policy optimization (MAPPO) algorithm, construct a "centralized training-decentralized execution" (CTDE) mechanism, and solve the problems of insufficient spatial competition perception and inefficient multi-agent cooperation in charging station (CS) pricing decision;

[0013] S1.3: Model optimization is carried out by using Karush-Kuhn-Tucker (KKT) condition and Bayesian optimization, the KKT system of the lower layer demand assignment problem is embedded into the upper layer OD flow optimization, is converted into a mathematical programming (MPEC) with equilibrium constraint, and the model super parameters (behavior utility coefficient, OD preference function weight) are calibrated through Bayesian optimization, so that the fitting degree of the model and the actual traffic data is improved;

[0014] S1.4: In the dynamic pricing decision closed loop, the interaction of "demand modeling-multi-agent pricing-environment feedback" is realized, the charging station agent outputs the pricing strategy based on local observation and neighborhood information aggregated by GAT, and updates the strategy combined with the charging demand and income data fed back by the UE demand model, so as to realize the system level income maximization and load balancing.

[0015] As a preferred scheme of the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning of the present application, the double-layer UE demand modeling framework in S1.1 comprises:

[0016] Upper layer OD assignment model construction;

[0017] Lower layer charging demand assignment model construction;

[0018] Demand mapping rule definition.

[0019] As a preferred scheme of the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning of the present application, the graph attention enhanced MARL framework in S1.2 comprises:

[0020] GAT integration;

[0021] MAPPO algorithm optimization;

[0022] Agent state and action definition;

[0023] Reward function design.

[0024] As a preferred scheme of the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning of the application, the model optimization in S1.3 includes:

[0025] KKT condition embedding;

[0026] Bayesian optimization calibration;

[0027] Relaxation variable introduction.

[0028] As a preferred scheme of the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning of the application, the dynamic pricing decision loop in S1.4 includes:

[0029] Environment initialization;

[0030] Multi-agent interaction;

[0031] Policy update;

[0032] Output result.

[0033] As a preferred scheme of the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning of the application, the objective function is defined as: minimizing the weighted sum of congestion delay, POI residual demand, and OD flow efficiency loss.

[0034] As a preferred scheme of the electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning of the application, the reward function design adopts a neighborhood index storage mechanism.

[0035] 3. Beneficial effects:

[0036] Compared with the prior art, the application has the beneficial effects that:

[0037] The electric vehicle charging station dynamic pricing method based on graph attention multi-agent reinforcement learning has the beneficial effects that:

[0038] 1. A "macroscopic OD allocation-microscopic demand allocation" double-layer model is constructed. The upper layer is based on POI distribution and road network topology, and aims to minimize congestion delay, POI residual demand, etc., to realize balanced allocation of OD flow. The lower layer combines the SOC distribution of EVs, converts road segment flow into charging demand, and realizes reasonable allocation of demand among charging stations through a distance and price weighted perception cost model, to provide real demand feedback for pricing decision.

[0039] 2. Based on MAPPO, integrate GAT to realize spatial competition perception. Each charging station is an agent, which aggregates the price and demand information of neighboring stations through GAT to generate an enhanced state representation; adopt the "centralized training-decentralized execution" mechanism, and in the training stage, stabilize the strategy update through the centralized critic network, and in the execution stage, only rely on local observation to output pricing, taking into account the collaboration and decentralized demand;

[0040] 3. Convert the double-layer demand model into a mathematical programming (MPEC) with equilibrium constraints through KKT conditions to realize the joint optimization of OD distribution and demand distribution; adopt Bayesian optimization to calibrate hyperparameters (behavior coefficient, OD preference weight), take the error between model predicted traffic and actual observed traffic as the target, and through Gaussian process and expected improvement criterion, improve the fitting degree of the model to the actual traffic data;

[0041] 4. Based on real EV trajectory data, build an experimental environment to realize the closed loop of "state collection-pricing output-demand feedback-strategy update". Through iterative training, the agent learns to adapt to the pricing strategy of spatial demand and spatial competition, and finally outputs a dynamic pricing scheme that takes into account the revenue and load balancing. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the present application will be described in detail below with reference to the drawings and detailed embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor. Among them:

[0043] Figure 1 It is a whole framework schematic diagram of the electric vehicle charging station dynamic pricing method based on the graph attention multi-agent reinforcement learning of the present application;

[0044] Figure 2 It is a double-layer UE demand modeling module flow chart of the electric vehicle charging station dynamic pricing method based on the graph attention multi-agent reinforcement learning of the present application;

[0045] Figure 3 It is a graph attention enhanced MARL module core architecture diagram of the electric vehicle charging station dynamic pricing method based on the graph attention multi-agent reinforcement learning of the present application;

[0046] Figure 4 It is a Bayesian optimization hyperparameter calibration trajectory diagram of the electric vehicle charging station dynamic pricing method based on the graph attention multi-agent reinforcement learning of the present application;

[0047] Figure 5A training performance comparison chart of different pricing methods of the electric vehicle charging station dynamic pricing method based on a graph attention multi-agent reinforcement learning of the application. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, characteristics and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0049] The application is described in detail in combination with the schematic diagram. In the detailed description of the embodiments of the application, the cross-sectional view of the device structure is locally enlarged without the general proportion for the convenience of description, and the schematic diagram is only an example, which should not limit the scope of protection of the application here. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in actual production.

[0050] The orientation or positional relationship indicated in the term is based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the application.

[0051] The connection mode in the term should be understood broadly, for example, "connection" can be fixed connection, detachable connection or integral connection; can be mechanical connection or electrical connection; can be direct connection or indirect connection through an intermediate medium; and can be internal communication of two elements. For those skilled in the art, the specific meaning of the above-mentioned term in the application can be understood according to the specific circumstances.

[0052] The embodiments of the application will be further described in detail below in combination with the drawings.

[0053] The application provides a whole structure schematic diagram of an embodiment of the electric vehicle charging station dynamic pricing method based on a graph attention multi-agent reinforcement learning, which comprises the following steps:

[0054] Please refer to Figures 1-5 The electric vehicle charging station dynamic pricing method based on a graph attention multi-agent reinforcement learning of the embodiment comprises the following steps:

[0055] S1.1: Construct a double-layer user equilibrium (UE) demand modeling framework, the upper layer constructs a dynamic origin-destination (OD) distribution model based on the point of interest (POI) distribution and the road network topology, and the lower layer realizes the charging demand distribution of traffic guidance through a probability model, and the objective function, the constraint condition and the demand mapping rule of the framework are clear;

[0056] S1.2: Design a graph attention enhanced multi-agent reinforcement learning (MARL) framework, integrate graph attention network (GAT) with multi-agent proximal policy optimization (MAPPO) algorithm, build a "centralized training-decentralized execution" (CTDE) mechanism, solve the problems of insufficient spatial competition perception and inefficient multi-agent cooperation in charging station (CS) pricing decision;

[0057] S1.3: Use Karush-Kuhn-Tucker (KKT) condition and Bayesian optimization for model optimization, embed the KKT system of lower-level demand allocation problem into upper-level OD flow optimization, convert it into mathematical programming with equilibrium constraints (MPEC), and calibrate model hyperparameters (behavior utility coefficient, OD preference function weight) through Bayesian optimization to improve the fitting degree of the model to actual traffic data;

[0058] S1.4: Implement the interaction of "demand modeling-multi-agent pricing-environment feedback" in the dynamic pricing decision loop, charging station agents output pricing strategies based on local observations and GAT aggregated neighborhood information, update strategies combined with charging demand and revenue data feedback from UE demand model to achieve system-level revenue maximization and load balancing.

[0059] It is worth noting that, specifically, the double-layer UE demand modeling framework in S1.1 includes:

[0060] Upper-level OD allocation model construction;

[0061] Lower-level charging demand allocation model construction;

[0062] Demand mapping rule definition.

[0063] Next, specifically, the graph attention enhanced MARL framework in S1.2 includes:

[0064] GAT integration;

[0065] MAPPO algorithm optimization;

[0066] Agent state and action definition;

[0067] Reward function design.

[0068] At the same time, specifically, the model optimization in S1.3 includes:

[0069] KKT condition embedding;

[0070] Bayesian optimization calibration;

[0071] Relaxation variable introduction.

[0072] Further, specifically, the dynamic pricing decision loop in S1.4 includes:

[0073] environment initialization;

[0074] multi-agent interaction;

[0075] policy update;

[0076] output result.

[0077] Meanwhile, specifically, the objective function is defined as: minimizing the weighted sum of congestion delay, POI residual demand, and OD flow efficiency loss.

[0078] Next, specifically, the reward function design adopts a neighborhood index storage mechanism.

[0079] Embodiment 1:

[0080] comprising the following steps:

[0081] S1.1: Construct a two-layer user equilibrium (UE) demand modeling framework, the upper layer constructs a dynamic origin-destination (OD) assignment model based on the distribution of points of interest (POI) and the topology of the road network, and the lower layer realizes the charging demand assignment of traffic guidance through a probability model, and the objective function, constraint condition and demand mapping rule of the framework are defined;

[0082] S1.2: Design a multi-agent reinforcement learning (MARL) framework enhanced by graph attention, integrate graph attention network (GAT) and multi-agent proximal policy optimization (MAPPO) algorithm, construct a "centralized training-decentralized execution" (CTDE) mechanism, solve the problems of insufficient spatial competition perception and inefficient multi-agent cooperation in charging station (CS) pricing decision;

[0083] S1.3: Use Karush-Kuhn-Tucker (KKT) condition and Bayesian optimization for model optimization, embed the KKT system of the lower layer demand assignment problem into the upper layer OD flow optimization, convert it into a mathematical programming with equilibrium constraints (MPEC), and calibrate the model hyperparameters (behavior utility coefficient, OD preference function weight) through Bayesian optimization, improve the fitting degree of the model to the actual traffic data;

[0084] S1.4: Realize the interaction of "demand modeling-multi-agent pricing-environment feedback" in the dynamic pricing decision loop, the charging station agent outputs the pricing strategy based on the local observation and the neighborhood information aggregated by GAT, updates the strategy combined with the charging demand and revenue data feedback by the UE demand model, realizes the system-level revenue maximization and load balancing.

[0085] It is worth noting that, specifically, the two-layer UE demand modeling framework in S1.1 includes:

[0086] The upper-layer OD distribution model is constructed, and the road network is represented as a graph containing nodes and road segments. The nodes cover POIs, charging stations, and traffic nodes, and the road segments are road segments connecting the nodes. The OD demand matrix is estimated based on the POI attributes and time activity patterns. The path set containing optional charging stop points is defined to minimize the congestion delay, POI remaining demand, OD flow efficiency loss, and price utility as the objective function, and to meet the OD demand balance, road segment capacity constraints, and other conditions.

[0087] The lower-layer charging demand distribution model is constructed based on the equilibrium road segment flow output by the upper layer. The basic charging request intensity is calculated based on the state of charge (SOC) distribution of electric vehicles (EVs) (only vehicles with SOC below a certain threshold are involved). The demand distribution is achieved by representing the continuous variable of the proportion of road segment allocation to the demand of each charging station, with the objective of minimizing the user-perceived cost (distance and price weighted) and the concentration of charging station load, and meeting the constraints of the sum of the proportion being 1 and the value range.

[0088] The demand mapping rule defines that when converting road segment flow into charging demand, distance decay coefficient and price sensitivity coefficient are introduced to ensure that demand allocation decreases with increasing spatial distance and rising charging price.

[0089] Next, specifically, the graph attention enhanced MARL framework in S1.2 includes:

[0090] GAT integration: Each charging station agent encodes local observations (historical demand, neighboring station price, traffic flow) into a feature vector, aggregates neighborhood node features to obtain enhanced representation by calculating attention weight and normalized weight, and improves spatial competition perception ability; The attention weight is obtained by processing the feature combination through the LeakyReLU activation function, and the normalized weight is realized by the exponential function and summation operation;

[0091] MAPPO algorithm optimization: The CTDE mechanism is adopted. In the training stage, the centralized critic network estimates the value function by inputting the neighborhood aggregated features, and calculates the generalized advantage estimation (GAE); The truncated agent target is used for policy update, which processes the importance sampling ratio through truncation to ensure the stability of the update;

[0092] Definition of agent state and action: The state space includes SOC distribution statistics, road segment flow intensity, neighboring station price average, and timestamp; The action space is a discrete price adjustment amount, which generates the final price based on the benchmark price and adjusts the price within a reasonable range;

[0093] The reward function is designed, and the basic reward is the product of the normalized price and the demand utilization rate (the normalized price is the ratio of the actual price to the maximum allowed price, and the demand utilization rate is the ratio of the service demand to the charging station capacity). An unserved demand penalty term is introduced (the ratio of unserved demand to capacity is attenuated by an exponential function).

[0094] Meanwhile, specifically, the model optimization in S1.3 includes:

[0095] KKT condition embedding, for the lower-level demand allocation problem, derive the KKT system containing the first-order optimality condition, dual feasibility, and complementary slackness, embed it as a constraint into the upper-level OD flow optimization, convert it into an MPEC model, and realize the joint optimization of OD allocation and demand allocation;

[0096] Bayesian optimization calibration, taking the squared error of the model predicted link flow and the actual observed flow as the target, using Gaussian process (GP) to build a proxy model, selecting hyperparameter candidates by expected improvement (EI) criterion, iteratively optimizing the behavior coefficient and OD preference function weight until convergence;

[0097] Relaxation variable introduction, adding a relaxation variable for unallocated demand at POI nodes in the OD allocation model, tolerating partial demand unmet, and improving the robustness of the model in supply-demand imbalance scenarios.

[0098] Further, specifically, the dynamic pricing decision-making closed loop in S1.4 includes:

[0099] Environment initialization, based on 3 months of trajectory data and charging records of 10,000 EVs, construct a 100m x 100m grid space index, extract the road network topology and POI distribution, and initialize 50 charging station agents;

[0100] Multi-agent interaction, at each time step (1 hour), the agent sequentially completes local state collection, aggregates neighborhood information through GAT, and outputs pricing actions. The UE demand model calculates demand allocation and revenue based on pricing actions, and then feeds back the reward to the agent;

[0101] Strategy update, the agent stores the experience (observation, action, reward, neighborhood index) into the buffer, and in the training phase, the centralized critic calculates the time difference (TD) target to minimize the mean square error loss and update the critic. The MAPPO is used to update the policy network;

[0102] Output results, after 4000 training cycles of iterative training, the dynamic pricing strategy of each charging station is output, achieving maximum system revenue, reducing the proportion of congestion nodes (≤10.2%), and optimizing the utilization rate coefficient of variation (≤0.18).

[0103] Meanwhile, specifically, the objective function is defined as minimizing the weighted sum of congestion delay, POI residual demand, and OD flow efficiency loss, where the congestion delay is calculated using the BPR congestion delay function (including free-flow time, the ratio of link flow to capacity, and a fixed coefficient), and the OD flow efficiency loss is calculated by the ratio of OD to total flow and OD preference function, which adopts a distance segmented exponential form, while introducing a numerical stability term to avoid calculation abnormalities.

[0104] Next, specifically, the reward function design adopts a neighborhood index storage mechanism, each experience record only contains the local observation of the agent, action, reward, and the index of the neighborhood node in the graph, and the neighborhood feature reconstruction enhanced representation is queried in real time during training to reduce storage overhead (reduce 30%-50% memory occupancy).

[0105] Embodiment 2:

[0106] The core includes a double-layer UE demand modeling module, a graph attention enhanced MARL module, a KKT and Bayesian optimization module, and a dynamic pricing decision closed loop module, which cooperatively realize the output of the charging station dynamic pricing strategy. The specific construction steps are as follows:

[0107] Step 1: Input data preparation, collect 3 months of trajectory data (including real-time position, SOC) of 10000 EVs in a city, charging records (charging position, duration, cost), road network data (extracted from OpenStreetMap), and POI distribution (including residential, commercial, hub type, etc.), to provide data support for subsequent modules;

[0108] Step 2: Environment construction, construct a 100m x 100m grid space index (map EV trajectory, POI and charging station location), and represent the road network as a graph containing nodes and links (nodes cover 843 elements, including POI, charging station, traffic hub), and use the breadth-first search algorithm to define the neighborhood competition range of each charging station;

[0109] Step 3: Double-layer UE demand modeling (attached Figure 2 ), build an "upper-layer OD allocation - lower-layer charging demand allocation" model, and output the calibrated demand allocation result;

[0110] Step 4: Graph attention enhanced MARL modeling (attached Figure 3 ), integrate GAT and MAPPO to build the core of agent decision-making, and design feature encoding, neighborhood aggregation, strategy training, and reward calculation mechanism;

[0111] Step 5: Model optimization (attached Figure 4 ), convert the double-layer model to MPEC through KKT condition, and use Bayesian optimization to calibrate hyperparameters to improve model fitting degree;

[0112] Step 6: Dynamic pricing decision-making closed loop and verification (Appendix Figure 5 ), realizing multi-agent interaction and policy updating, and outputting the final dynamic pricing strategy.

[0113] As shown in Figure 2 , the double-layer UE demand modeling module provides real demand feedback for pricing decisions, and the specific construction steps are as follows:

[0114] Step 1: Upper OD distribution model construction. Based on POI type and time activity pattern, generate OD demand matrix to reflect the travel demand intensity between different origins and destinations; minimize the weighted sum of congestion delay, POI residual demand, and OD flow efficiency loss as the target; POI residual demand is the outbound demand of POI that has not been allocated to the path, and OD flow efficiency loss is calculated by the ratio of OD to total flow and OD preference function; set the constraint condition to satisfy OD demand balance and road section capacity at the same time; finally, use the GUROBI solver to solve the model and output the equilibrium road section flow;

[0115] Step 2: Lower charging demand distribution model construction. Based on the equilibrium road section flow output by the upper layer and the SOC distribution of EVs, calculate the charging request intensity of each road section; only consider vehicles with SOC below a certain threshold, obtain the basic charging request intensity by multiplying the road flow and the probability of vehicles with SOC below the threshold; minimize user perceived cost and charging station load concentration as the target; user perceived cost is the weighted sum of distance and price, and load concentration is calculated by the square sum of demand allocation to different charging stations; set the total sum of demand allocation proportion of each road section to different charging stations to be 1, and the proportion value to be between 0 and 1; solve the model to get the demand allocation proportion of each road section to the charging station;

[0116] Step 3: Demand mapping verification: compare the demand allocation results output by the model with the actual charging records, adjust the distance decay coefficient and price sensitivity coefficient to ensure that the model prediction error is ≤5%, and output the calibrated demand model.

[0117] As shown in Figure 3 , the MARL module enhanced by graph attention provides the decision-making core for charging station agents, and the specific construction steps are as follows:

[0118] Step 1: GAT integration (spatial competition perception). Encode the local observation of the agent (service demand, overflow demand, and average price of neighboring stations) into a 64-dimensional feature vector, perform linear transformation and ReLU activation through a multi-layer perceptron (MLP), and extract key feature information; calculate the attention score between the agent and the neighboring agents based on the encoded feature vector, and then realize weight normalization through the exponential function and summation operation; aggregate the neighborhood features based on the normalized weights to generate an enhanced representation containing spatial competition information;

[0119] Step 2: MAPPO training (policy optimization). The enhanced representation of the neighborhood aggregation is input into a 2-layer MLP, which outputs a value function to evaluate the long-term return of the current state. The truncated actor target is used to calculate the action advantage through generalized advantage estimation (GAE). The neighborhood index mechanism is used to store only the local observations of the agent, actions, rewards, and indices of neighborhood nodes in the graph, and to query the neighborhood feature reconstruction enhanced representation in real time during training to reduce memory usage.

[0120] Step 3: Reward function design: The basic reward is the product of normalized price and demand utilization rate; the overflow demand penalty term is added to decay the ratio of unserved demand to capacity through an exponential function, avoiding demand loss due to excessive pricing of charging stations.

[0121] As shown in Figure 4 , the model optimization module improves model performance through mathematical transformation and hyperparameter calibration. The specific construction steps are as follows:

[0122] Step 1: KKT condition embedding (joint optimization of double-layer model). For the lower-layer charging demand allocation problem, the first-order optimality condition (marginal cost of demand allocation equals marginal revenue), dual feasibility (dual variables satisfy constraints), and complementary slackness condition (dual variables corresponding to ineffective constraints are 0) are derived to form a complete KKT system. Embed the KKT system as a constraint into the upper-layer OD allocation optimization model, convert it into a mathematical programming with equilibrium constraints (MPEC), and realize the joint optimization of OD allocation and demand allocation to ensure the consistency of the two-layer model.

[0123] Step 2: Bayesian optimization (hyperparameter calibration). Set the distance segment weight of behavior coefficient and OD preference function; use Gaussian process to construct a surrogate model, select square exponential kernel function to describe the non-linear relationship between hyperparameters and target loss; select hyperparameter candidates through expected improvement (EI) criterion to balance exploration (untried hyperparameter region) and utilization (known high-quality hyperparameter region); iterate for 30 times and converge to get the optimal hyperparameter combination, output the optimized model parameters.

[0124] As shown in Figure 5 , the dynamic pricing decision closed-loop module realizes policy training through "demand-pricing-feedback" interaction, and verifies the performance based on 3 months of real trajectory and charging data of 10,000 EVs in Hefei City, as follows:

[0125] Step 1: Initialize 50 charging station agents, each training cycle contains 24 time steps (1 hour / step), cumulative training 4000 cycles; in each time step, the agent completes local state collection (including service demand, adjacent station price, road section flow) in turn, aggregates adjacent station features to generate enhanced representation through GAT, and outputs discrete pricing actions based on benchmark price (adjustment range [-10, 10] yuan); the UE demand model calculates the demand allocation ratio of each road section to the charging station and the single station revenue based on the pricing action, and feeds back the reward containing the overflow demand penalty to the agent; the agent stores the "observation-action-reward-neighborhood index" experience into the buffer, and in the training stage, the centralized critic calculates the time difference (TD) target (reward + discount factor x next state value) to minimize the mean square error loss to update the critic network, and uses the truncated agent target of MAPPO and generalized advantage estimation (GAE) to update the policy network, ensuring that the policy adapts to the space-time demand fluctuations and the spatial competition relationship between charging stations;

[0126] Step 2: Compared with three mainstream multi-agent reinforcement learning algorithms MAPPO, QMIX and MADDPG, the final average reward of the method reaches 0.42 after 4000 cycles, which is 15% higher than the optimal baseline algorithm (MAPPO, final reward 0.36), reflecting better pricing strategy learning and convergence ability; during the continuous 3-day hour-level test period, the method realizes system revenue of 1650 yuan / hour (17.5% higher than the fixed pricing strategy), congestion node ratio (station utilization rate over 90% ratio) drops to 10.2% (45.5% lower than the fixed pricing), charging station utilization rate coefficient of variation is optimized to 0.18 (30.8% lower than the fixed pricing), while realizing single station revenue increase and regional charging load balance.

[0127] Although the present application has been described with reference to the embodiments above, various improvements can be made thereto and components thereof can be substituted with equivalents without departing from the scope of the present application. In particular, features in the embodiments disclosed by the present application can be combined with each other in any manner as long as there is no structural conflict, and the combinations are not exhaustively described in the specification only for the purpose of omitting pages and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning, characterized in that, Includes the following steps: S1.1: Construct a two-layer user balance (UE) demand modeling framework. The upper layer is based on the distribution of points of interest (POI) and road network topology to build a dynamic origin-destination (OD) allocation model. The lower layer realizes the charging demand allocation guided by traffic flow through a probability model. The objective function, constraints and demand mapping rules of the framework are defined. S1.2: Design a graph attention-enhanced multi-agent reinforcement learning (MARL) framework, integrating graph attention network (GAT) and multi-agent nearest neighbor policy optimization (MAPPO) algorithm, and construct a "centralized training-decentralized execution" (CTDE) mechanism to solve the problems of insufficient spatial competition perception and inefficient multi-agent collaboration in charging station (CS) pricing decisions; S1.3: The model is optimized using Karush-Kuhn-Tucker (KKT) conditions and Bayesian optimization. The KKT system of the lower-level demand allocation problem is embedded into the upper-level OD flow optimization, which is transformed into mathematical programming with equilibrium constraints (MPEC). The model hyperparameters (behavioral utility coefficients and OD preference function weights) are calibrated through Bayesian optimization to improve the fit between the model and actual traffic data. S1.4: In the dynamic pricing decision closed loop, the interaction of "demand modeling - multi-agent pricing - environmental feedback" is realized. The charging station agent outputs the pricing strategy based on the neighborhood information of local observation and GAT aggregation, and combines the charging demand and revenue data update strategy fed back by the UE demand model to achieve system-level revenue maximization and load balancing.

2. The dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning according to claim 1, characterized in that, The two-layer UE requirement modeling framework in S1.1 includes: Construction of upper-level OD allocation model; Construction of a lower-level charging demand allocation model; Demand mapping rule definition.

3. The dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning according to claim 2, characterized in that, The graph attention-enhanced MARL framework in S1.2 includes: GAT integration; MAPPO algorithm optimization; Definition of agent state and action; Reward function design.

4. The dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning according to claim 3, characterized in that, The model optimization in S1.3 includes: KKT conditional embedding; Bayesian optimization calibration; Introducing slack variables.

5. The dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning according to claim 4, characterized in that, The dynamic pricing decision closed loop in S1.4 includes: Environment initialization; Multi-agent interaction; Strategy update; Output results.

6. The dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning according to claim 5, characterized in that, The objective function is defined as: minimizing the weighted sum of congestion delay, POI remaining demand, and OD flow efficiency loss.

7. The dynamic pricing method for electric vehicle charging stations based on graph attention multi-agent reinforcement learning according to claim 6, characterized in that, The reward function is designed using a neighborhood index storage mechanism.

Images