A charging station real-time pricing system based on multi-agent cooperation

Abstract

The application discloses a kind of real-time pricing system of charging station based on multi-agent cooperation, it is characterized in that, including charging network layer including environmental state information acquisition unit and transmission state unit;Environmental state information acquisition unit is used to model as directed weighted graph structure according to the association of charging station, road network node and demand node;Transmission state unit reports the charging station state information to corresponding agent by V2X or cellular network;Multi-agent cooperation layer, multi-agent cooperation layer contains individual agent and Mixer network structure based on super network, Mixer network structure based on super network fuses local Q value into global Q value for collaborative training;Electricity price adjustment layer, the Q value corresponding to different electricity price adjustment action is calculated in current state, action selection is carried out using -greedy strategy, and each agent updates the electricity price of this charging station;Vehicle decision layer makes selection decision according to each site electricity price and utility function, adjusts the load distribution of charging network layer.

Description

Technical Field

[0001] This invention belongs to the field of pricing technology for electric vehicle charging infrastructure, specifically, it relates to a real-time pricing system for charging stations based on multi-agent collaboration. Background Technology

[0002] With the global energy structure transformation and the advancement of carbon neutrality goals, electric vehicles, as the core path for low-carbon development in the transportation sector, are rapidly gaining popularity; simultaneously, the total amount of charging infrastructure has also grown rapidly. While urban electric vehicle charging networks have reached a considerable scale, they still face significant challenges in terms of balanced spatial distribution, uniform time allocation, and optimal facility utilization. Existing charging pricing mechanisms are insufficient to effectively guide the rational distribution of vehicles and achieve efficient allocation of charging infrastructure resources.

[0003] Traditional charging station pricing typically employs a fixed electricity price strategy, meaning charging service fees are charged at a fixed rate and do not adjust based on changes in time, location, or supply and demand. Fixed electricity prices fail to reflect the true supply and demand of charging services, leading to excessive concentration of charging resources during peak demand periods and idle facilities during off-peak periods. Furthermore, fixed electricity prices lack incentive mechanisms for car owners, failing to regulate the spatial and temporal distribution of vehicles through price signals, making it difficult to optimize the utilization efficiency of charging facilities. To address the shortcomings of fixed electricity prices, the industry has proposed a series of rule-based dynamic pricing methods, primarily including time-of-use pricing and simple response pricing. Time-of-use pricing sets different price standards for different time periods based on the peak and off-peak characteristics of the power grid load, guiding car owners to avoid charging during peak hours; response pricing, on the other hand, adjusts the price proportionally based on the real-time occupancy rate or queue length of charging stations.

[0004] The drawback of existing technologies is that the existing fixed electricity price and simple rule pricing methods cannot dynamically adjust the electricity price according to the real-time load status of charging stations, queue length and vehicle distribution. This makes it difficult to effectively guide the rational distribution of vehicles in spatial and temporal dimensions, resulting in overcrowding at popular stations and idle resources at other stations. Summary of the Invention

[0005] To address the problem that existing charging station pricing methods cannot dynamically adjust electricity prices based on the real-time load status, queue length, and vehicle distribution of charging stations, making it difficult to effectively guide the rational distribution of vehicles in spatial and temporal dimensions, resulting in overcrowding at popular stations and idle resources at other stations, this invention provides a real-time pricing system for charging stations based on multi-agent collaboration.

[0006] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows: A real-time pricing system for charging stations based on multi-agent collaboration includes a charging network layer, a multi-agent collaboration layer, an electricity price adjustment layer, and a vehicle decision-making layer. The charging network layer includes an environmental status information acquisition unit and a transmission status unit; The environmental status information acquisition unit is used to model the relationship between charging stations, road network nodes and demand nodes into a directed weighted graph structure. The transmission status unit reports the charging station status information (load rate, queue length, electricity price) to the corresponding intelligent agent via V2X or cellular network; The multi-agent collaboration layer defines each charging station as an agent. The multi-agent collaboration layer includes... Each agent and a Mixer network structure based on a super network are used. Each agent calculates its local Q-value based on its local Q-network. The Mixer network structure based on the super network merges the local Q-values ​​into a global Q-value for collaborative training. In the electricity price adjustment layer, each agent inputs its observed state into its local Q-network to calculate the Q-value corresponding to different electricity price adjustment actions under the current state. The -greedy strategy selects actions, and each agent updates the electricity price at the charging station based on the selected actions; The vehicle decision-making layer makes selection decisions based on the electricity price and utility function of each station, and adjusts the load distribution of the charging network layer.

[0007] Furthermore, a directed weighted graph is adopted. This refers to the city's charging network, in which Represents a set of nodes. This represents the set of edges, used to effectively describe the geographical distribution of charging stations and the traffic relationships of vehicles in the road network; There are three types of nodes: charging station nodes are used to represent facilities that provide charging services. Each charging station node contains core attributes such as location coordinates, maximum charging power, current load status and real-time electricity price; road nodes are used to represent topological key points such as intersections and bifurcation points in the road network, and are used to connect each charging station node to form a complete road network structure; demand nodes are used to represent the origin and destination of vehicles, reflecting the spatial distribution of vehicles and travel demand. The attributes of an edge describe the travel relationship between adjacent nodes, mainly including: road segment distance, the physical distance between adjacent nodes; estimated travel time, the travel time of a vehicle on the road segment; and road type, used to distinguish between different levels of roads such as arterial roads, secondary arterial roads, and local roads, with different speed and safety restrictions for different road types. The specific attributes of each charging station node are designed as follows: station_id is the unique identifier of the charging station, used to distinguish different agents in a multi-agent system; position is the geographical coordinate of the charging station, which can be represented by latitude and longitude coordinates; max_power is the maximum charging power of the charging station, reflecting the power supply capacity constraint of the station; current_load is the current load rate of the charging station, with a value between 0 and 1, representing the ratio of the current actual output power to the maximum power; queue_length is the number of vehicles currently waiting to be charged; base_price is the base electricity price of the charging station, the benchmark value for pricing adjustments.

[0008] Furthermore, each charging station is defined as an intelligent agent, denoted as... ,in To determine the number of charging stations in the charging network, a centralized training-distributed execution framework is adopted. Each agent can access global information during the training phase and makes decisions based solely on local observations during the execution phase. State-space perception: the observed state of each agent includes the following elements: current electricity price. The charging service price currently in effect at this station; the current load rate. This represents the ratio of the charging station's current output power to its maximum power; the current number of vehicles in the queue. This reflects the number of vehicles waiting to charge at this station; the load status of adjacent stations, including the load rate and queue length of neighboring stations, is used to realize information transmission and coordination between stations; time period characteristics, including the current time and whether it is the weekend, are used to capture the periodic patterns in the time dimension; historical load characteristics reflect the recent load change trend, which is used to predict short-term demand changes. Action space selection: the actionable actions of each agent are defined as the electricity price adjustment range. The range of values ​​is ,in This represents the maximum magnitude of a single adjustment. The reward function is calculated using a comprehensive multi-objective reward function, which includes the following four components: The mathematical expression for the comprehensive reward function is: The meanings and calculation methods for each item are as follows: Load balancing reward: Incentivizes balanced load rates across sites, calculated as the negative of the global load rate variance. , In the formula, This represents the variance function, used to measure the load rate of each charging station. The degree of dispersion between them; For charging stations At any moment Load rate; Revenue Incentive: This incentivizes charging stations to increase operational revenue, calculated as the product of the current electricity price and the amount of electricity charged. , In the formula, For charging stations At any moment Electricity price; For charging stations At any moment The amount of charge; Queueing penalties: To curb excessively long queues, the severity of the penalty is proportional to the number of vehicles in the queue. , In the formula, Queuing penalty coefficient ( ), used to adjust the intensity of queuing penalties; For charging stations At any moment The number of vehicles queuing; Power constraint penalty: To prevent charging stations from overloading, a penalty is imposed when the power demanded by a station exceeds its maximum capacity. , In the formula, Power constraint penalty coefficient ( ), For charging stations At any moment The required power, For charging stations Maximum charging power capacity, This means that a positive penalty is only incurred when the required power exceeds the maximum capacity. The four reward items mentioned above are combined using a weighted summation method to form the total reward of this agent, with the weight parameters... Adjustments will be made based on actual operational needs.

[0009] Furthermore, the QMIX value decomposition algorithm architecture includes a hybrid network structure, monotonicity constraints of the hybrid network, a Mixer network structure based on hypernetworks, and a centralized learning-distributed execution framework. The hybrid network structure merges the local Q-values ​​of each agent into a global Q-value. The mathematical expression for the hybrid network is: The global Q-value (joint action value function) represents the value of a given global state. Under these circumstances, all intelligent agents take joint actions. The expected cumulative reward that can be obtained at that time; Joint action-observation history, which is the combination of actions and observation sequences of all agents over a number of past time steps; For intelligent agents The action-observation history records the intelligent agent. The trajectory of observed inputs and actions taken in the past; A joint action is the set of actions taken by all agents at the current moment. For intelligent agents The action taken by the agent, namely the amount of electricity price adjustment chosen by the agent; For intelligent agents The local Q-value is based solely on the agent's own action-observation history. and actions Calculated; It provides global status information, including the overall load, queuing, and pricing status of all charging stations; Nonlinear transformation function implemented for hybrid networks; in, The local Q-values ​​of each agent are... This is global status information. Nonlinear transformations implemented for hybrid networks; Monotonicity constraints for hybrid networks: During the execution phase, greedily selecting the action with the maximum local Q value is equivalent to selecting the joint action with the maximum global Q value. The Mixer network structure based on hypernetworks, with global state The input is a neural network, and its output is used to generate the weight parameters of each layer of the hybrid network; The centralized learning-distributed execution framework adopts the standard CTDE paradigm. During the training phase, all agents share global state information and collaboratively train the hybrid network parameters. Each agent calculates its local Q-value based on local observations, and the hybrid network combines these local Q-values ​​into a global Q-value, updating the network parameters through temporal difference learning.

[0010] During the execution phase, each agent independently selects actions based on local observations.

[0011] Furthermore, during the training phase of the centralized learning-distributed execution framework, agents exchange state information via V2X or cellular networks to support collaborative learning among multiple agents. Communication network: Each charging station's intelligent agent establishes a two-way communication link through V2X communication or cellular network; The information exchanged between adjacent stations includes the following status information: load rate, the current power utilization level of each station; queue length, the number of vehicles waiting to charge at each station; recent electricity price history, records of electricity price adjustments over the past period, used to analyze the impact of prices on demand; and expected arrival information, the number of vehicles expected to arrive in the short term based on historical data and real-time traffic conditions. The communication frequency and information exchange cycle can be configured according to actual needs.

[0012] Furthermore, the specific procedures for adjusting electricity prices include: Status acquisition: Each charging station agent collects its current status information at each decision point, including the current electricity price, current load rate, and current number of vehicles in the queue; at the same time, the agent receives status information updates from neighboring stations to construct a complete local observation vector. Q-value calculation: Each agent inputs its observation state into its local Q network to calculate the Q-value corresponding to different electricity price adjustment actions under the current state; Action selection, using -Greedy strategy for action selection: based on probability. Randomly select actions to explore new pricing strategies, using probability. Choose the optimal action that maximizes the Q value; Electricity price update: Based on the selected action, each agent updates the local electricity price. The electricity price update formula is as follows: ; In the formula, Let be the truncation function, defined as: if Then return ,like Then return Otherwise return ,Right now ; For intelligent agents Selected electricity price adjustment range; For charging stations The minimum allowable electricity price is used to guarantee the basic revenue of charging stations; This is the maximum permissible electricity price for charging station i, used to protect consumer rights; Reward calculation and network update: After executing pricing decisions, each agent calculates immediate rewards based on new market feedback, stores the experience tuple consisting of the current state, action, reward and next state in the replay buffer, and periodically samples and updates the parameters of the Q network and hybrid network.

[0013] Furthermore, the closed-loop feedback of electricity price and choice, and the closed-loop interaction mechanism between electricity price adjustments and vehicle selection behavior, guide the rational spatial distribution of vehicles through electricity price signals to achieve the global optimization goal: The vehicle behavior model states that when a vehicle has a charging need, the onboard system receives real-time electricity price information from nearby charging stations and combines it with factors such as distance to each station and estimated queuing time to calculate and select the utility value of each station. The vehicle then selects the charging station with the highest utility value. The utility function used comprehensively considers charging costs, travel costs, and waiting costs, and its mathematical expression is: in, Indicates vehicle Select site The utility value, For the site Electricity price, For vehicles to station distance, For the site Queue length, These are the weighting coefficients for the corresponding cost items; A closed-loop feedback mechanism ensures that vehicle selection behavior directly affects the load status of each charging station. The synergistic effect arises from the mutual influence of electricity pricing decisions at each charging station, and vehicle selection behavior will have an impact that is transmitted throughout the entire network.

[0014] Furthermore, it also includes a cloud server, which includes a training module and a global state management system. The training module is used for training tasks of the hybrid network, and the global state management system is used for global state management of the Mixer network structure based on the hypernetwork.

[0015] Compared with the prior art, the present invention has the following advantages: A multi-agent collaborative pricing mechanism based on the QMIX value decomposition algorithm was proposed, which realizes true collaborative optimization among multiple charging stations.

[0016] A comprehensive multi-objective reward function was designed, which includes four indicators: load balancing, revenue, queuing, and power constraints. Pareto optimization of the multi-objectives is achieved through weighted combination.

[0017] A joint optimization framework of electricity price and queuing is proposed, which guides the rational distribution of vehicles in the spatial dimension through electricity price signals, thereby indirectly optimizing queuing time. Attached Figure Description

[0018] Figure 1This is an overall structural block diagram of a real-time pricing system for charging stations based on multi-agent collaboration, as described in an embodiment of the present invention. Detailed Implementation

[0019] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0020] like Figure 1 As shown, this embodiment provides a real-time pricing system for charging stations based on multi-agent collaboration, including a charging network layer, a multi-agent collaboration layer, an electricity price adjustment layer, and a vehicle decision layer; The charging network layer includes an environmental status information acquisition unit and a transmission status unit; The environmental status information acquisition unit is used to model the relationship between charging stations, road network nodes and demand nodes into a directed weighted graph structure; each charging station reports its status information (load rate, queue length, electricity price) to the corresponding intelligent agent; The transmission status unit reports the charging station status information (load rate, queue length, electricity price) to the corresponding intelligent agent via V2X or cellular network; The multi-agent collaboration layer defines each charging station as an agent. The multi-agent collaboration layer includes... Each agent and a Mixer network structure based on a super network are used. Each agent calculates its local Q-value based on its local Q-network. The Mixer network structure based on the super network merges the local Q-values ​​into a global Q-value for collaborative training. In the electricity price adjustment layer, each agent inputs its observed state into its local Q-network to calculate the Q-value corresponding to different electricity price adjustment actions under the current state. The -greedy strategy selects actions, and each agent updates the electricity price at the charging station based on the selected actions; The vehicle decision-making layer makes selection decisions based on the electricity price and utility function of each station, and adjusts the load distribution of the charging network layer.

[0021] The cloud server includes a training module and a global state management module. The training module is used for training tasks of hybrid networks, and the global state management module is used for global state management of the Mixer network structure based on hypernetworks.

[0022] Using directed weighted graphs This refers to the city's charging network, in which Represents a set of nodes. This represents the set of edges. This graph model can effectively describe the geographical distribution of charging stations and the traffic relationships of vehicles in the road network;

[0023] There are three types of nodes: charging station nodes (CS) represent facilities that provide charging services. Each charging station node contains core attributes such as location coordinates, maximum charging power, current load status, and real-time electricity price; road nodes (RN) represent topological key points such as intersections and bifurcation points in the road network, and are used to connect the charging station nodes to form a complete road network structure; demand nodes (DN) represent the origin and destination of vehicles, reflecting the spatial distribution of vehicles and travel demand. The attributes of an edge describe the travel relationship between adjacent nodes, mainly including: road segment distance, which is the physical distance between adjacent nodes; estimated travel time, which is the time it takes for a vehicle to travel on the road segment; and road type, which is used to distinguish different levels of roads such as arterial roads, secondary arterial roads, and local roads. Different road types have different travel speeds and safety restrictions. The specific attributes of each charging station node are designed as follows: station_id is the unique identifier of the charging station, used to distinguish different agents in a multi-agent system; position is the geographical coordinate of the charging station, which can be represented by latitude and longitude coordinates; max_power is the maximum charging power of the charging station, reflecting the power supply capacity constraint of the station; current_load is the current load rate of the charging station, with a value between 0 and 1, representing the ratio of the current actual output power to the maximum power; queue_length is the number of vehicles currently waiting to be charged; base_price is the base electricity price of the charging station, i.e., the benchmark value for pricing adjustments.

[0024] Each charging station is defined as an intelligent agent, denoted as... ,in The number of charging stations in the charging network; a centralized training-distributed execution framework is adopted, in which each agent can access global information during the training phase and makes decisions based only on local observations during the execution phase; State-space perception: the observed state of each agent includes the following elements: current electricity price. This refers to the charging service price currently in effect at this station; the current load rate. This represents the ratio of the charging station's current output power to its maximum power; the current number of vehicles in the queue. This reflects the number of vehicles waiting to charge at this station; the load status of adjacent stations, including the load rate and queue length of neighboring stations, is used to realize information transmission and coordination between stations; time period characteristics, including the current time and whether it is the weekend, are used to capture the periodic patterns in the time dimension; historical load characteristics reflect the recent load change trend, which is used to predict short-term demand changes. Action space selection: the actionable actions of each agent are defined as the electricity price adjustment range. The range of values ​​is ,in This represents the maximum adjustment range in a single instance. The specific meaning of the action is to increase or decrease the price by a certain amount based on the current electricity price. The adjusted electricity price will affect the vehicle's choice of charging station, thereby regulating the load distribution.

[0025] The reward function is calculated using a comprehensive multi-objective reward function, which comprises the following four components: The mathematical expression for the comprehensive reward function is: The meanings and calculation methods for each item are as follows: Load balancing reward: Incentivizes balanced load rates across sites, calculated as the negative of the global load rate variance. , In the formula, This represents the variance function, used to measure the load rate of each charging station. The degree of dispersion between them; For charging stations At any moment The load factor is defined as the ratio of the current output power to the maximum charging power. The larger the variance, the more uneven the load distribution among the stations; when a negative value is taken, the algorithm tends to minimize the load factor variance, thereby promoting a more balanced load distribution among the stations.

[0026] Revenue Incentive: This incentivizes charging stations to increase operational revenue, calculated as the product of the current electricity price and the amount of electricity charged. , In the formula, For charging stations At any moment Electricity price (yuan / kWh); For charging stations At any moment The charging volume (in kWh) refers to the actual charging output of this station during that period. This reward is calculated as the product of the electricity price and the charging volume, encouraging stations to improve overall operating revenue through reasonable pricing.

[0027] Queueing penalties: To curb excessively long queues, the severity of the penalty is proportional to the number of vehicles in the queue. , In the formula, Queuing penalty coefficient ( ), used to adjust the intensity of queuing penalties; For charging stations At any moment The number of vehicles in the queue (unit: vehicles). This penalty is proportional to the number of vehicles in the queue; the more vehicles in the queue, the heavier the penalty, thereby suppressing excessively long queues at charging stations.

[0028] Power constraint penalty: To prevent charging stations from overloading, a penalty is imposed when the power demanded by a station exceeds its maximum capacity. , In the formula, Power constraint penalty coefficient ( ), used to adjust the intensity of power over-limit penalty; For charging stations At any moment The required power (in kW) is the sum of the charging power requested by all connected vehicles at the current moment. For charging stations The maximum charging power capacity (in kW) is the upper limit of the power supply capacity of the site; This means that a positive penalty is only applied when the demand exceeds the maximum capacity. This penalty prevents pricing strategies from directing too many vehicles to stations that are about to be overloaded, protecting the user charging experience and the safety of facility operation.

[0029] The four reward items mentioned above are combined using a weighted summation method to form the total reward of this agent, with the weight parameters... It can be adjusted according to actual operational needs.

[0030] Power constraint penalty The introduction of this concept is an innovation in existing charging pricing research. When charging demand exceeds the power supply capacity of a charging station, the actual charging power received by vehicles is forced to decrease, significantly extending charging time and deteriorating user experience. Traditional revenue-oriented or load-balancing-oriented methods cannot detect this constraint, potentially leading to excessive vehicle congregation at the same station during specific time periods. This invention, by adding a power constraint penalty term to the reward function, enables the pricing strategy to proactively avoid guiding too many vehicles to stations that are about to be overloaded, thereby protecting user experience and the safety of charging infrastructure.

[0031] The QMIX value decomposition algorithm architecture includes a hybrid network structure, monotonicity constraints of the hybrid network, a Mixer network structure based on hypernetworks, and a centralized learning-distributed execution framework. The hybrid network structure merges the local Q-values ​​of each agent into a global Q-value. The mathematical expression for the hybrid network is: in, The global Q-value (joint action value function) represents the value of a given global state. Under these circumstances, all intelligent agents take joint actions. The expected cumulative reward that can be obtained at that time; Joint action-observation history, which is the combination of actions and observation sequences of all agents over a number of past time steps; For intelligent agents The action-observation history records the intelligent agent. The trajectory of observed inputs and actions taken in the past; A joint action is the set of actions taken by all agents at the current moment. For intelligent agents The action taken by the agent, namely the amount of electricity price adjustment chosen by the agent; For intelligent agents The local Q-value is based solely on the agent's own action-observation history. and actions Calculated; It provides global status information, including the overall load, queuing, and pricing status of all charging stations; Nonlinear transformation functions implemented for hybrid networks; Monotonicity constraints for hybrid networks: The constraint ensures that, during the execution phase, greedily selecting the action with the largest local Q value is equivalent to selecting the joint action with the largest global Q value, thereby achieving distributed execution; The Mixer network structure based on hypernetworks, with global state The input is a neural network, and its output is used to generate the weight parameters for each layer of the hybrid network. An absolute value activation function ensures that the weight parameters are non-negative, thus satisfying the monotonicity constraint. The biases of the hybrid network are then determined by another method using the global state. This is used to generate a linear network as input. This design allows the hybrid network to dynamically adjust the combined weights of the Q-values ​​of each agent based on the global state, enhancing the algorithm's adaptability to different scenarios.

[0032] The centralized learning-distributed execution framework adopts the standard CTDE paradigm. During the training phase, all agents share global state information and collaboratively train the hybrid network parameters. Each agent calculates its local Q-value based on local observations, and the hybrid network combines these local Q-values ​​into a global Q-value, updating the network parameters through temporal difference learning.

[0033] During the execution phase, each agent independently selects actions based on local observations, without the need for additional communication and coordination, thus ensuring real-time decision-making. This framework leverages global information from the training phase to enhance collaborative performance while guaranteeing distributed real-time decision-making capabilities during the execution phase.

[0034] Each charging station is modeled as an independent agent, and the QMIX algorithm is used to achieve collaborative learning. Compared with traditional independent optimization methods, QMIX ensures the consistency between the global Q-value and the individual Q-value through monotonic value function decomposition. That is, during the execution phase, greedily selecting the action with the maximum local Q-value is equivalent to selecting the joint action with the maximum global Q-value. This characteristic enables each agent to maintain global optimality while making independent decisions based on local observations, avoiding the collaborative dilemma of "local optimum ≠ global optimum".

[0035] In the training phase of a centralized learning-distributed execution framework, agents exchange state information via V2X or cellular networks to support collaborative learning among multiple agents. The communication network establishes bidirectional communication links between the intelligent entities at each charging station via V2X or cellular networks. V2X communication is suitable for direct communication scenarios between vehicles and infrastructure, featuring low latency; cellular networks (such as 4G / 5G) provide wider coverage and are suitable for information exchange between charging stations within a city.

[0036] The information exchange between adjacent stations involves periodically exchanging the following status information: load rate (current power utilization level of each station); queue length (number of vehicles waiting to charge at each station); recent electricity price history (records of electricity price adjustments over past periods, used to analyze the impact of prices on demand); and expected arrival information (predicted number of vehicles arriving in the short term based on historical data and real-time traffic conditions). This information exchange allows each agent to consider the state changes of surrounding stations when calculating its local Q-value, achieving true collaborative optimization.

[0037] The communication frequency and information exchange cycle can be configured according to actual needs, with a default value of once per minute. The communication frequency setting needs to strike a balance between information timeliness and communication resource consumption: a higher communication frequency provides more timely information updates, facilitating rapid response to changing needs, but increases the load on the communication system; a lower communication frequency reduces system overhead, but may lead to information delays, affecting collaborative effectiveness.

[0038] The specific procedures for adjusting electricity prices include: State acquisition involves each charging station's intelligent agent collecting its current state information at each decision-making point, including the current electricity price, current load rate, and current number of vehicles in the queue. Simultaneously, the agent receives state information updates from neighboring stations, constructing a complete local observation vector. The timeliness of state acquisition is crucial to the quality of pricing decisions; this invention employs a synchronization mechanism to ensure the temporal consistency of state information across all stations.

[0039] Q-value calculation involves each agent inputting its observed state into its local Q-network to calculate the Q-value corresponding to different electricity price adjustment actions in the current state. The Q-value reflects the expected long-term cumulative reward that can be obtained by taking a specific action and serves as the basis for action selection. The parameters of the Q-network are continuously optimized during the training phase through experience replay and gradient descent, and are frozen during the execution phase for inference computation.

[0040] Action selection, using -Greedy strategy for action selection: based on probability. Randomly select actions to explore new pricing strategies, using probability. Choose the optimal action that maximizes the Q value. The value gradually decays as training progresses, initially encouraging ample exploration to discover the optimal strategy, and later reducing exploration to utilize learned knowledge. The action space is continuous but limited by the maximum adjustment range, ensuring smooth and controllable changes in electricity prices.

[0041] Electricity price updates: Based on the selected action, each agent updates the local electricity price. The basic principle of the update is to add the action value to the current electricity price, while ensuring that the new electricity price is between a preset minimum and maximum value. The electricity price update formula is:

[0042] In the formula, Let be the truncation function, defined as: if Then return ,like Then return Otherwise return ,Right now ; For intelligent agents Selected electricity price adjustment range; For charging stations The minimum allowable electricity price (yuan / kWh) is used to ensure the basic revenue of charging stations; This represents the maximum permissible electricity price (RMB / kWh) for charging station i, used to protect consumer rights. The cutoff function ensures that the adjusted electricity price remains within a reasonable range, avoiding drastic price fluctuations.

[0043] In the reward calculation and network update process, after executing a pricing decision, each agent calculates an immediate reward based on the new market feedback. The reward value reflects the comprehensive contribution of the decision to multiple objectives such as load balancing, revenue, queuing control, and power constraints. The system stores an experience tuple consisting of the current state, action, reward, and next state in a replay buffer, and periodically samples and updates the parameters of the Q-network and hybrid network to achieve continuous learning and policy optimization.

[0044] The closed-loop feedback mechanism of electricity price and vehicle selection, which is a closed-loop interaction mechanism between electricity price adjustments and vehicle selection behavior, guides the rational spatial distribution of vehicles through electricity price signals to achieve the global optimization goal. Regarding vehicle behavior models, when a vehicle has a charging need, the onboard system receives real-time electricity price information from nearby charging stations and, combined with factors such as distance to each station and estimated waiting time, calculates and selects the utility value of each station. Vehicles tend to choose the charging station with the highest utility value, and the design of the utility function directly affects the guidance effect.

[0045] The utility function used comprehensively considers charging costs (positively correlated with electricity prices), travel costs (positively correlated with distance), and waiting costs (positively correlated with queue length), and its mathematical expression is: in, Indicates vehicle Select site The utility value, For the site Electricity price, For vehicles to station distance, For the site Queue length, These are the weighting coefficients for the corresponding cost items. After weighing the three costs, the vehicle selects the charging station with the highest utility value.

[0046] In terms of the closed-loop feedback mechanism, vehicle selection behavior directly affects the load status of each charging station. When the electricity price at a certain station is high, some vehicles will choose to go to other stations, causing the load at that station to decrease; conversely, when the electricity price at a certain station is low, it will attract more vehicles, causing the load to increase. Load changes then affect the reward calculation and pricing decisions for the next period, forming a dynamic feedback loop. This closed-loop mechanism enables the charging station pricing strategy to continuously adapt to market changes and achieve adaptive load balancing.

[0047] Regarding the generation of synergistic effects, since the electricity pricing decisions of each charging station influence each other, and vehicle selection behavior transmits this influence to the entire network, the pricing strategy optimized through multi-agent collaborative learning in this invention can generate a global synergistic effect that surpasses single-station optimization. The pricing strategies of each station are no longer isolated decisions, but rather form a collaborative pattern of mutual cooperation, jointly achieving the dual goals of load balancing and optimal revenue.

[0048] There is a dynamic coupling between electricity prices and queuing: electricity prices influence vehicle selection behavior, which in turn determines queue lengths at each station. Queue lengths then serve as feedback information, influencing electricity price decisions for the next time period. This invention designs this closed-loop feedback mechanism, enabling electricity price adjustments to indirectly regulate queue waiting times. Compared to traditional methods that directly control queues, this invention transforms the queue control problem into a pricing optimization problem, using electricity prices as an economic lever to guide vehicle behavior, resulting in better scalability and real-time performance.

[0049] The mechanism of synergistic effect: Since the electricity pricing decisions of each charging station influence each other, and vehicle selection behavior transmits this influence to the entire network, the pricing strategy generated by multi-agent collaborative learning in this invention can produce a global synergistic effect that surpasses single-station optimization. The pricing strategies of each station are no longer isolated decisions, but form a cooperative pattern. For example, when a charging station in a commercial area is about to be overloaded due to low electricity prices, its neighboring stations will automatically increase their electricity prices through collaborative learning, forming a reasonable gradient and guiding some vehicles to charge at nearby stations.

[0050] Compared with the prior art, the present invention has the following advantages: A multi-agent collaborative pricing mechanism based on the QMIX value decomposition algorithm was proposed, which realizes true collaborative optimization among multiple charging stations.

[0051] A comprehensive multi-objective reward function was designed, which includes four indicators: load balancing, revenue, queuing, and power constraints. Pareto optimization of the multi-objectives is achieved through weighted combination.

[0052] A joint optimization framework of electricity price and queuing is proposed, which guides the rational distribution of vehicles in the spatial dimension through electricity price signals, thereby indirectly optimizing queuing time.

[0053] The above provides a detailed description of a real-time pricing system for charging stations based on multi-agent collaboration. The specific embodiments described are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A real-time pricing system for charging stations based on multi-agent collaboration, characterized in that, It includes the charging network layer, the multi-agent collaboration layer, the electricity price adjustment layer, and the vehicle decision-making layer; The charging network layer includes an environmental status information acquisition unit and a transmission status unit; The environmental status information acquisition unit is used to model the relationship between charging stations, road network nodes and demand nodes into a directed weighted graph structure. The transmission status unit reports the charging station status information to the corresponding intelligent agent via V2X or cellular network; The multi-agent collaboration layer defines each charging station as an agent. The multi-agent collaboration layer includes... Each agent and a Mixer network structure based on a super network are used. Each agent calculates its local Q-value based on its local Q-network. The Mixer network structure based on the super network merges the local Q-values ​​into a global Q-value for collaborative training. In the electricity price adjustment layer, each agent inputs its observed state information into its local Q-network to calculate the Q-value corresponding to different electricity price adjustment actions in the current state. The -greedy strategy selects actions, and each agent updates the electricity price at the charging station based on the selected actions; The vehicle decision-making layer makes selection decisions based on the electricity price and utility function of each station, and adjusts the load distribution of the charging network layer.

2. The real-time pricing system for charging stations based on multi-agent collaboration according to claim 1, characterized in that, Using directed weighted graphs This refers to the city's charging network, in which Represents a set of nodes. This represents the set of edges, used to effectively describe the geographical distribution of charging stations and the traffic relationships of vehicles in the road network; There are three types of nodes: charging station nodes represent facilities that provide charging services. Each charging station node contains core attributes such as location coordinates, maximum charging power, current load status, and real-time electricity price; road nodes represent topological key points such as intersections and bifurcation points in the road network, which are used to connect the charging station nodes to form a complete road network structure; and demand nodes represent the origin and destination of vehicles, reflecting the spatial distribution of vehicles and travel demand. The attributes of an edge describe the travel relationship between adjacent nodes, mainly including: road segment distance, the physical distance between adjacent nodes; and estimated travel time, the time it takes for a vehicle to travel on the road segment. Road type is used to distinguish between different levels of roads, such as arterial roads, secondary arterial roads, and local roads. Different road types have different speed and safety restrictions. The specific attributes of each charging station node are designed as follows: station_id is the unique identifier of the charging station, used to distinguish different agents in a multi-agent system; position is the geographical coordinate of the charging station, which can be represented by latitude and longitude coordinates; max_power is the maximum charging power of the charging station, reflecting the power supply capacity constraint of the station; current_load is the current load rate of the charging station, with a value between 0 and 1, representing the ratio of the current actual output power to the maximum power; queue_length is the number of vehicles currently waiting to be charged; base_price is the base electricity price of the charging station, the benchmark value for pricing adjustments.

3. A real-time pricing system for charging stations based on multi-agent collaboration as described in claim 2, characterized in that, Each charging station is defined as an intelligent agent, denoted as... ,in To determine the number of charging stations in the charging network, a centralized training-distributed execution framework is adopted. Each agent can access global information during the training phase and makes decisions based solely on local observations during the execution phase. State-space perception: the observed state of each agent includes the following elements: current electricity price. The charging service price currently in effect at this station; the current load rate. This represents the ratio of the charging station's current output power to its maximum power; the current number of vehicles in the queue. This reflects the number of vehicles waiting to charge at this station; the load status of adjacent stations, including the load rate and queue length of neighboring stations, is used to realize information transmission and coordination between stations. Time-period characteristics, including current time and whether it is a weekend, are used to capture periodic patterns in the time dimension; historical... Load characteristics reflect recent load change trends and are used to predict short-term demand changes; Action space selection: the actionable actions of each agent are defined as the electricity price adjustment range. The range of values ​​is ,in This represents the maximum magnitude of a single adjustment. The reward function is calculated using a comprehensive multi-objective reward function, which includes the following four components: The mathematical expression for the comprehensive reward function is: ， The meanings and calculation methods for each item are as follows: Load balancing reward: Incentivizes balanced load rates across sites, calculated as the negative of the global load rate variance. ， In the formula, This represents the variance function, used to measure the load rate of each charging station. The degree of dispersion between them; For charging stations At any moment Load rate; Revenue Incentive: This incentivizes charging stations to increase operational revenue, calculated as the product of the current electricity price and the amount of electricity charged. ， In the formula, For charging stations At any moment Electricity price; For charging stations At any moment The amount of charge; Queueing penalties: To curb excessively long queues, the severity of the penalty is proportional to the number of vehicles in the queue. ， In the formula, Queuing penalty coefficient ( ), used to adjust the intensity of queuing penalties; For charging stations At any moment The number of vehicles queuing; Power constraint penalty: To prevent charging stations from overloading, a penalty is imposed when the power demanded by a station exceeds its maximum capacity. ， In the formula, Power constraint penalty coefficient ( ), For charging stations At any moment The required power, For charging stations Maximum charging power capacity, This means that a positive penalty is only incurred when the required power exceeds the maximum capacity. The four reward items mentioned above are combined using a weighted summation method to form the total reward of this agent, with the weight parameters... Adjustments will be made based on actual operational needs.

4. A real-time pricing system for charging stations based on multi-agent collaboration as described in claim 3, characterized in that, The QMIX value decomposition algorithm architecture includes a hybrid network structure, monotonicity constraints of the hybrid network, a Mixer network structure based on hypernetworks, and a centralized learning-distributed execution framework. The hybrid network structure merges the local Q-values ​​of each agent into a global Q-value. The mathematical expression for the hybrid network is: ， The global Q-value (joint action value function) represents the value of a given global state. Under these circumstances, all intelligent agents take joint actions. The expected cumulative reward that can be obtained at that time; Joint action-observation history, which is the combination of actions and observation sequences of all agents over a number of past time steps; For intelligent agents The action-observation history records the intelligent agent. The trajectory of observed inputs and actions taken in the past; A joint action is the set of actions taken by all agents at the current moment. For intelligent agents The action taken by the agent, namely the amount of electricity price adjustment chosen by the agent; For intelligent agents The local Q-value is based solely on the agent's own action-observation history. and actions Calculated; It provides global status information, including the overall load, queuing, and pricing status of all charging stations; Nonlinear transformation function implemented for hybrid networks; Monotonicity constraints for hybrid networks: ， During the execution phase, greedily selecting the action with the maximum local Q value is equivalent to selecting the joint action with the maximum global Q value. The Mixer network structure based on hypernetworks, with global state The input is a neural network, and its output is used to generate the weight parameters of each layer of the hybrid network; The centralized learning-distributed execution framework adopts the standard CTDE paradigm. During the training phase, all agents share global state information and collaboratively train the hybrid network parameters. Each agent calculates its local Q-value based on local observations, and the hybrid network combines these local Q-values ​​into a global Q-value, updating the network parameters through temporal difference learning. During the execution phase, each agent independently selects actions based on local observations.

5. A real-time pricing system for charging stations based on multi-agent collaboration as described in claim 4, characterized in that, In the training phase of a centralized learning-distributed execution framework, agents exchange state information via V2X or cellular networks to support collaborative learning among multiple agents. Communication network: Each charging station's intelligent entity establishes a two-way communication link through V2X communication or cellular network; Information exchange content: Neighboring sites periodically exchange the following status information: load rate, the current power utilization level of each site; Queue length, the number of vehicles waiting to charge at each station; recent electricity price history, records of electricity price adjustments over several periods in the past, used to analyze the impact of prices on demand. And expected arrival information, which is the number of vehicles expected to arrive in the short term based on historical data and real-time traffic conditions; The communication frequency and information exchange cycle can be configured according to actual needs.

6. A real-time pricing system for charging stations based on multi-agent collaboration as described in claim 5, characterized in that, The specific process for adjusting electricity prices includes: Status acquisition: Each charging station agent collects its current status information at each decision point, including the current electricity price, current load rate, and current number of vehicles in the queue; at the same time, the agent receives status information updates from neighboring stations to construct a complete local observation vector. Q-value calculation: Each agent inputs its observation state into its local Q network to calculate the Q-value corresponding to different electricity price adjustment actions under the current state; Action selection, using -Greedy strategy for action selection: based on probability. Randomly select actions to explore new pricing strategies, using probability. Choose the optimal action that maximizes the Q value; Electricity price update: Based on the selected action, each agent updates the local electricity price. The electricity price update formula is as follows: ； In the formula, Let be the truncation function, defined as: if Then return ,like Then return Otherwise return ,Right now ; For intelligent agents Selected electricity price adjustment range; For charging stations The minimum allowable electricity price is used to guarantee the basic revenue of charging stations; This is the maximum permissible electricity price for charging station i, used to protect consumer rights; Reward calculation and network update: After executing pricing decisions, each agent calculates immediate rewards based on new market feedback, stores the experience tuple consisting of the current state, action, reward and next state in the replay buffer, and periodically samples and updates the parameters of the Q network and hybrid network.

7. A real-time pricing system for charging stations based on multi-agent collaboration as described in claim 6, characterized in that, The closed-loop feedback mechanism of electricity price and vehicle selection, which is a closed-loop interaction mechanism between electricity price adjustments and vehicle selection behavior, guides the rational spatial distribution of vehicles through electricity price signals to achieve the global optimization goal. The vehicle behavior model states that when a vehicle has a charging need, the onboard system receives real-time electricity price information from nearby charging stations and combines it with factors such as distance to each station and estimated queuing time to calculate and select the utility value of each station. The vehicle then selects the charging station with the highest utility value. The utility function used comprehensively considers charging costs, travel costs, and waiting costs, and its mathematical expression is: ， in, Indicates vehicle Select site The utility value, For the site Electricity price, For vehicles Arrive at the station distance, For the site Queue length, These are the weighting coefficients for the corresponding cost items; A closed-loop feedback mechanism ensures that vehicle selection behavior directly affects the load status of each charging station. The synergistic effect arises from the mutual influence of electricity pricing decisions at each charging station, and vehicle selection behavior will have an impact that is transmitted throughout the entire network.

8. A real-time pricing system for charging stations based on multi-agent collaboration as described in claim 4, characterized in that, It also includes a cloud server, which includes a training module and a global state management system. The training module is used for training tasks of hybrid networks, and the global state management system is used for global state management of the Mixer network structure based on hypernetworks.

Images