Hybrid electric vehicle charging scheduling system and method based on multi-agent near-end strategy optimization

Abstract

The invention relates to a hybrid electric vehicle charging scheduling system and method based on multi-agent near-end strategy optimization, and belongs to the field of intelligent traffic systems and electric vehicle charging scheduling. The method comprises the following steps: firstly, constructing a hybrid charging framework integrating a fixed charging station and a mobile energy distributor; secondly, establishing a refined traffic and energy consumption model considering vehicle following behaviors, road congestion and urban bus type MED operation conditions; then, a multi-agent near-end strategy optimization algorithm is adopted, a central controller serves as a global coordinator to be responsible for task scheduling and load balancing, and multi-agent combined optimization of electric vehicle charging mode selection and a driving path is achieved; and finally, converting an original complex mixed integer nonlinear optimization problem into an intelligent decision problem which can be solved online through a multi-agent reinforcement learning framework which is trained in a centralized manner and executed in a decentralized manner. According to the invention, the charging time, the driving energy consumption, the driving distance and the charging cost of the electric vehicle in the hybrid charging system can be effectively reduced.

Description

Technical Field

[0001] This invention relates to a hybrid electric vehicle charging scheduling system and method based on Multi-Agent Proximity Policy Optimization (MAPPO), belonging to the fields of intelligent transportation systems and electric vehicle charging scheduling. Background Technology

[0002] With the continuous growth in the number of electric vehicles (EVs), the charging scheduling problem has become an important research direction in intelligent transportation systems. Because EVs are heavily reliant on charging infrastructure during operation, their charging behavior not only affects the vehicle's own travel efficiency but also significantly impacts urban traffic flow and the utilization of charging resources. Existing EV charging scheduling schemes primarily rely on fixed charging stations. However, due to the fixed locations and limited service capacity of these stations, long waiting times and congestion are common during peak travel periods, while other areas may experience low charging resource utilization, making it difficult to achieve balanced charging resource utilization. To improve charging flexibility, some studies have introduced mobile charging, where mobile devices or vehicles with power supply capabilities provide charging services for EVs. However, in real urban traffic environments, the operational status of mobile charging resources is closely related to traffic flow changes, and their service capacity is affected by various factors such as travel routes, operational tasks, and energy constraints, making scheduling challenging and hindering efficient collaboration with fixed charging stations. Furthermore, urban road traffic exhibits significant dynamism and complexity, with vehicle travel time and energy consumption significantly influenced by traffic congestion levels and vehicle interaction behaviors. Existing charging scheduling methods fail to adequately consider dynamic traffic factors, making it difficult to accurately characterize vehicle driving and energy consumption characteristics under complex traffic conditions, thus affecting the applicability of scheduling strategies in real-world scenarios. Furthermore, electric vehicle charging scheduling involves collaborative decision-making among multiple vehicles and charging resources. Different vehicles exhibit competition and coupling relationships in charging selection and route planning, making it a complex dynamic optimization problem. Traditional scheduling methods struggle to achieve continuous optimization of overall performance when dealing with large vehicle fleets or frequently changing environments. Therefore, there is an urgent need for an electric vehicle charging scheduling method capable of collaboratively scheduling fixed charging stations and mobile charging resources in complex urban traffic environments, while also considering traffic operation characteristics and overall system costs, to improve the operational efficiency and stability of charging systems in large-scale application scenarios. Summary of the Invention

[0003] The technical problem solved by this invention is to address the problems of congestion at fixed charging stations, low utilization of mobile charging resources, and complex urban traffic dynamics in existing electric vehicle charging scheduling. This invention provides a hybrid charging scheduling system and method for electric vehicles, which adopts a multi-agent near-end strategy optimization (MAPPO) algorithm to effectively reduce the additional charging costs of electric vehicles.

[0004] The technical solution adopted in this invention is as follows: In order to achieve the above-mentioned objectives, this invention proposes a hybrid electric vehicle charging scheduling system and method based on multi-agent near-end strategy optimization. By performing unified modeling and collaborative scheduling of electric vehicles, fixed charging stations and mobile energy distributors, the joint optimization of electric vehicle charging mode selection and driving path planning is realized.

[0005] A hybrid electric vehicle charging scheduling system based on Multi-Agent Proximity Policy Optimization (MAPPO) includes: electric vehicles, fixed charging stations (FCS), mobile energy distributors (MEDs), roadside units, aggregators, and a central controller. Each electric vehicle is equipped with a wireless energy receiver that matches the wireless energy transmitter on the mobile energy distributor. When its own battery level is detected to be below a preset threshold, the electric vehicle sends a charging request to the roadside unit and receives charging scheduling strategies from the roadside unit. The electric vehicle acts as a mobile communication and execution node, moving according to a car-following model. The generated charging request includes current battery level, target battery level, current location, and destination information. Fixed charging stations are infrastructures with power supply capabilities deployed in fixed urban locations. They provide plug-in charging services to electric vehicles and report the location of the charging station, the usage status of charging piles, charging power, charging price, and the load status of the charging station to the aggregator. A bus-type mobile energy distributor operates along a preset bus route. While performing its predetermined operational tasks, it also acts as a mobile charging resource, providing dynamic charging services to electric vehicles via wireless energy transmission during its journey. The mobile energy distributor (MED) in this invention refers to a bus mobile energy distributor, which is an electric bus operating along a fixed bus route. The bus mobile energy distributor reports its operating status information to roadside units. The operating status information includes at least the current location, operating route information, and remaining available discharge energy. The roadside units are set up along the road and deployed along the roadside communication infrastructure to collect electric vehicle charging requests and their own status, as well as the operating status information of the mobile energy distributor. This information is then relayed to the central controller, and the charging scheduling decisions sent by the central controller are forwarded to the electric vehicles and the mobile energy distributor. The aggregator is set up near the fixed charging station to collect the operating status information sent by the fixed charging station and forward it to the central controller. At the same time, the charging scheduling decisions generated by the central controller are forwarded to the fixed charging station. The central controller, as a global coordinator, collects electric vehicle charging demands sent by the roadside units, the operating status of the mobile energy distributor, and information about fixed charging stations sent by the aggregator. Based on a multi-agent reinforcement learning algorithm, it makes decisions on the electric vehicle charging scheduling problem, generates the optimal hybrid charging scheduling strategy, and distributes the scheduling strategy to the roadside units and the aggregator for execution.

[0006] A method for electric vehicle charging scheduling in a hybrid charging system based on multi-agent near-end strategy optimization, comprising the following steps: Step 1: Electric vehicles that need charging send a charging request to the roadside unit. The charging request includes the current battery level, target battery level, current location and destination information. Step 2: Fixed charging stations send their own operating status information to the aggregator, and bus mobile energy distributors send their operating status information to the roadside units; Step 3: The roadside unit sends the collected vehicle information and mobile energy distributor information to the central controller, and the aggregator sends the collected fixed charging station information to the central controller. Step 4: The central controller constructs the system state based on the collected information and establishes an optimization model with the goal of minimizing the comprehensive cost, including the additional time cost of electric vehicle charging, the additional energy consumption cost of driving, the additional driving distance cost, and the charging cost. Step 5: The central controller uses a multi-agent near-end strategy optimization algorithm to generate a hybrid charging scheduling strategy, determine whether each electric vehicle can choose to charge at a fixed charging station or wirelessly dynamically charge with the bus mobile energy distributor, and plan the corresponding driving path. Step 6: The central controller will generate the optimal hybrid charging scheduling strategy and distribute it to electric vehicles, mobile energy distributors and fixed charging stations through roadside units and aggregators, respectively. Step 7: The electric vehicle completes the charging process according to the scheduling strategy, and the scheduling ends.

[0007] In the above method, mutual exclusion constraints are set for each electric vehicle during the scheduling process, so that each electric vehicle is only assigned to one fixed charging station or one mobile energy distributor. Combined with the energy constraints of the vehicle and the energy constraints of the bus mobile energy distributor, the feasibility of the scheduling result is guaranteed.

[0008] The multi-agent proximal policy optimization algorithm adopts a multi-agent reinforcement learning framework with centralized value function training and distributed policy execution.

[0009] The beneficial effects of this invention are as follows: First, it constructs a hybrid charging architecture integrating fixed charging stations and mobile energy distributors to address the spatial and temporal imbalance of urban electric vehicle charging demand and the shortage of charging facilities. Second, it establishes a refined traffic and energy consumption model that considers vehicle following behavior, road congestion, and the operation of urban bus-type MEDs, improving the realism and feasibility of system modeling. Third, it employs a multi-agent near-end strategy optimization algorithm, with a central controller acting as a global coordinator responsible for task scheduling and load balancing, and multiple agents jointly optimizing the selection of electric vehicle charging methods and driving paths. Finally, through a multi-agent reinforcement learning framework with centralized training and decentralized execution, the original complex mixed-integer nonlinear optimization problem is transformed into an intelligent decision-making problem that can be solved online. This invention can achieve coordinated scheduling between fixed charging stations and mobile charging resources in complex urban traffic environments, guiding electric vehicles to rationally allocate charging demand, reducing congestion at fixed charging stations, and reducing the additional charging costs of electric vehicles, thereby improving the overall operating efficiency of the hybrid charging system in multi-vehicle, large-scale application scenarios. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the system model of the present invention; Figure 2 This is a schematic diagram of the charging scheduling decision method of the present invention; Figure 3 This is a schematic diagram of the MED dynamic charging of the present invention; Figure 4 This is a schematic diagram of an implementation scenario of Example 2 of the present invention. Detailed Implementation

[0011] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0012] Example 1: As Figure 1 As shown, a hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization includes: Electric vehicles: Equipped with a wireless energy receiver that matches the wireless energy transmitter on the mobile energy distributor, used to send charging requests to roadside units and receive charging scheduling strategies from roadside units. In the system, electric vehicles need to compete with other vehicles for limited charging and path resources. Fixed charging stations: Infrastructure with power supply capabilities deployed in fixed locations in the city. They mainly provide static plug-in charging services for electric vehicles, send their own status information to aggregators, and receive charging scheduling strategies from aggregators.

[0013] Bus-type mobile energy distributor: A mobile charging device that uses a city bus as a carrier and carries a wireless energy transmitter to provide dynamic charging services for electric vehicles; Roadside unit: A communication infrastructure deployed along the road to collect electric vehicle charging requests and their own status, as well as the operating status information of mobile energy distributors. This information is then relayed to the central controller, and the charging scheduling decisions sent by the central controller are forwarded to the electric vehicles and mobile energy distributors. Aggregator: Responsible for collecting the status information sent by fixed charging stations, forwarding it to the central controller, and forwarding the charging scheduling decisions sent by the central controller to the fixed charging stations; Central Controller: As a global coordinator, it collects electric vehicle charging demands from roadside units, operational status of mobile energy distributors, and information on fixed charging stations from aggregators. It then generates the optimal charging scheduling strategy using the MAPPO algorithm and sends the generated optimal charging scheduling strategy to the roadside units and aggregators.

[0014] Among them, electric vehicles act as mobile communication and execution nodes, moving according to the car-following model, and the charging requests they generate include current battery level, target battery level, current location and destination information.

[0015] Fixed charging stations are fixed energy supply nodes equipped with traditional plug-in charging piles to provide electricity for electric vehicles. The status information of the charging station includes the location of the charging station, the usage status of the charging piles, the charging power, the charging price, and the load status of the charging station.

[0016] Among them, the mobile energy distributor is a mobile energy supply node, which is a city bus equipped with wireless charging devices. It operates periodically along a pre-set bus route, allowing electric vehicles to join it at bus stops to form a wireless charging fleet and to leave the wireless charging fleet at the same stops. Specifically, such as... Figure 3 As shown, the mobile energy distributor in this invention is equipped with a wireless energy transmitter on its vehicle body. The electric vehicle is equipped with a matching wireless energy receiver. When an electric vehicle is scheduled to use mobile charging, it meets the target bus mobile energy distributor at a designated section along the route planned by the central controller, maintaining a relatively stable charging platoon with the buses during the journey. During this process, the bus mobile energy distributor continuously provides power to the electric vehicle via wireless energy transmission, achieving dynamic charging. During dynamic charging, the discharge power and duration of the bus mobile energy distributor are constrained by its remaining dischargeable energy and its own operational tasks, ensuring that the normal operational needs of the buses are not affected. When the electric vehicle's battery level meets subsequent driving requirements or reaches a preset threshold, it leaves the platoon and continues to its destination or performs subsequent driving tasks.

[0017] Among them, the roadside unit is a fixed communication node on both sides of the road with a communication range of 500m, responsible for wireless communication with vehicles traveling into its service area.

[0018] The aggregator is a fixed communication node near a fixed charging station that communicates wirelessly with the fixed charging station.

[0019] A hybrid electric vehicle charging scheduling method based on multi-agent near-end strategy optimization is proposed, with the following specific steps: Step 1: Electric vehicles that need charging send a charging request to the roadside unit. The charging request includes the current battery level, target battery level, current location and destination information. Step 2: Fixed charging stations send their own status information to the aggregator, and mobile energy distributors send their operating status information to the roadside units; Step 3: The roadside unit sends the collected vehicle information and mobile energy distributor information to the central controller, and the aggregator sends the collected fixed charging station information to the central controller. Step 4: The central controller constructs the system state based on the collected information and establishes an optimization model with the goal of minimizing the overall cost of electric vehicles. Step 5: The central controller uses a multi-agent near-end strategy optimization algorithm to generate a hybrid charging scheduling strategy, determine whether each electric vehicle can choose to charge at a fixed charging station or wirelessly dynamically charge with the bus mobile energy distributor, and plan the corresponding driving path. Step 6: The central controller will generate the optimal hybrid charging scheduling strategy and distribute it to electric vehicles, fixed charging stations and mobile energy distributors through roadside units and aggregators, respectively. Step 7: The electric vehicle completes the charging process according to the scheduling strategy, and the scheduling ends.

[0020] Furthermore, the hybrid charging scheduling strategy includes selecting the charging method, determining the target fixed charging station or the target bus mobile energy distributor, and planning the corresponding driving route.

[0021] Furthermore, the comprehensive cost of electric vehicle charging includes the additional time cost of charging the electric vehicle, the additional energy consumption cost of driving, the additional driving distance cost, and the charging fee cost.

[0022] Furthermore, during the scheduling process, mutual exclusion constraints are set for each electric vehicle, ensuring that each electric vehicle is assigned to only one fixed charging station or one mobile energy distributor.

[0023] Furthermore, the multi-agent proximal policy optimization algorithm adopts a multi-agent reinforcement learning framework with centralized value function training and distributed policy execution.

[0024] like Figure 2 As shown, this invention models the electric vehicle charging scheduling problem as a multi-agent decision-making problem and uses a multi-agent near-end policy optimization algorithm for solution. In this embodiment, each electric vehicle with charging needs is considered an agent. The environment transitions based on the joint action state of all agents and feeds back corresponding reward information to each agent. Each agent constructs a policy network (Actor) to output charging scheduling actions based on its own observed state; simultaneously, it constructs a centralized value evaluation network (CCritic) to evaluate the overall system state and joint actions during the training phase. During training, the electric vehicle agents interact with the environment according to the current policy, obtaining state, action, reward, and next state information, and storing the interaction data in an experience replay buffer. The central controller samples small batches of data from the experience replay buffer, calculates the advantage function using the generalized advantage estimation method, and updates the policy network through a probability pruning mechanism to limit the policy update magnitude and improve the stability of the training process. During the execution phase, each electric vehicle makes decisions independently based only on its own policy network, without needing to obtain complete information from other agents, thus achieving a centralized training and distributed execution scheduling mode. In this way, the system can make stable and efficient charging scheduling decisions even when multiple vehicles are involved and the environment is dynamically changing.

[0025] Through the above implementation methods, the present invention realizes the coordinated scheduling between fixed charging stations and mobile energy distributors such as buses, enabling electric vehicles to flexibly select charging methods according to system status, reducing charging waiting time and overall travel costs while meeting energy constraints, and improving the operating efficiency and adaptability of the hybrid charging system in complex urban traffic environments.

[0026] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

[0027] Example 2: In this example, the road network of a 10km × 10km area in a certain city ( Figure 4 Using this as an application scenario, the specific execution process of the hybrid electric vehicle charging scheduling system and method based on multi-agent near-end strategy optimization described in this invention under the condition of large-scale vehicle access will be explained.

[0028] There are 200 electric vehicles with charging needs in this area. Each electric vehicle is equipped with a wireless communication module and a wireless energy receiving device. The vehicle's battery capacity is 50 kWh, and the initial battery level is between 10% and 90%. When an electric vehicle detects that its battery level is lower than a preset threshold while driving, it sends a charging request signal to the roadside unit within its communication range. The charging request signal includes at least the vehicle's current location, destination location, and current remaining battery level.

[0029] Six fixed charging stations are deployed in the area, each with 10 charging piles and a charging power of 50 kW. Each fixed charging station periodically reports its operating status information to the central controller through an aggregator, including charging pile occupancy, number of vehicles in queue, charging price, and station location.

[0030] Simultaneously, six bus routes are established within the area, with electric buses on each route operating periodically as mobile energy distributors. These mobile energy distributors travel at a constant speed along the pre-defined bus routes in dedicated bus lanes, and at bus stops, they allow electric vehicles to merge with them, forming a dynamic wireless charging fleet. The mobile energy distributors report their current location, subsequent stops, and remaining available discharge energy to the central controller via roadside units.

[0031] After receiving vehicle request information from roadside units and aggregators, fixed charging station status information, and mobile energy distributor operation information, the central controller constructs the overall system status and generates a hybrid charging scheduling strategy using a multi-agent near-end strategy optimization algorithm.

[0032] In this embodiment, the hybrid charging scheduling strategy generated by the central controller involves dispatching some electric vehicles to fixed charging stations with low loads for static charging and planning their travel routes. When the electric vehicles have sufficient charge, they leave the fixed charging stations according to the scheduling strategy and continue to their destination. Simultaneously, another group of electric vehicles is dispatched to bus-type mobile energy distributors, and their travel routes to the rendezvous point are planned. After rendezvous at designated bus stops, a wireless charging fleet is formed, and dynamic charging is completed during the journey. When the electric vehicles have sufficient charge for subsequent travel, they leave the charging fleet according to the scheduling strategy and continue to their destination.

[0033] Through the above scheduling process, the coordinated scheduling of fixed charging stations and mobile energy distributors is realized when multiple vehicles are present at the same time. This alleviates the charging load on fixed charging stations, reduces the additional time, driving distance, energy consumption and charging costs of electric vehicles due to charging, and improves the overall operating efficiency of the hybrid charging system in multi-vehicle scenarios.

[0034] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization, characterized in that, include: Electric vehicles: Equipped with a wireless energy receiver that matches the wireless energy transmitter on the mobile energy distributor, for sending charging requests to roadside units and receiving charging scheduling strategies from roadside units; Fixed charging stations: Infrastructure with power supply capabilities deployed in fixed locations in cities to provide static charging services for electric vehicles; Mobile energy distributor: A mobile charging device that uses city buses as carriers and carries wireless energy transmitters to provide dynamic charging services for electric vehicles; Roadside unit: A communication infrastructure deployed along the road to collect electric vehicle charging requests and their own status, as well as the operating status information of mobile energy distributors. This information is then relayed to the central controller, and the charging scheduling decisions sent by the central controller are forwarded to the electric vehicles and mobile energy distributors. Aggregator: Responsible for collecting the status information sent by fixed charging stations, forwarding it to the central controller, and forwarding the charging scheduling decisions sent by the central controller to the fixed charging stations; Central Controller: As a global coordinator, it collects electric vehicle charging demands from roadside units, operational status of mobile energy distributors, and information on fixed charging stations from aggregators. It then generates the optimal charging scheduling strategy using the MAPPO algorithm and sends the generated optimal charging scheduling strategy to the roadside units and aggregators.

2. The hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization according to claim 1, characterized in that: The electric vehicle acts as a mobile communication and execution node, moving according to a car-following model. The generated charging request includes the current battery level, target battery level, current location, and destination information.

3. The hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization according to claim 1, characterized in that: The fixed charging station is a fixed energy supply node, equipped with traditional plug-in charging piles to provide electricity for electric vehicles. The status information of the charging station includes the location of the charging station, the usage status of the charging piles, the charging power, the charging price, and the load status of the charging station.

4. The hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization according to claim 1, characterized in that: The mobile energy distributor is a mobile energy supply node, which is a city bus equipped with a wireless charging device. It runs periodically along a preset bus route and allows electric vehicles to join it at bus stops to form a wireless charging fleet and to leave the wireless charging fleet at bus stops.

5. The hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization according to claim 1, characterized in that: The roadside unit is a fixed communication node on both sides of the road, with a communication range of 500m, responsible for wireless communication with vehicles traveling within its service area.

6. The hybrid electric vehicle charging scheduling system based on multi-agent near-end strategy optimization according to claim 1, characterized in that: The aggregator is a fixed communication node near a fixed charging station that communicates wirelessly with the fixed charging station.

7. A hybrid electric vehicle charging scheduling method based on multi-agent near-end strategy optimization, characterized in that: The specific steps are as follows: Step 1: Electric vehicles that need charging send a charging request to the roadside unit. The charging request includes the current battery level, target battery level, current location and destination information. Step 2: Fixed charging stations send their own status information to the aggregator, and mobile energy distributors send their operating status information to the roadside units; Step 3: The roadside unit sends the collected vehicle information and mobile energy distributor information to the central controller, and the aggregator sends the collected fixed charging station information to the central controller. Step 4: The central controller constructs the system state based on the collected information and establishes an optimization model with the goal of minimizing the overall cost of electric vehicles; Step 5: The central controller uses a multi-agent near-end strategy optimization algorithm to generate a hybrid charging scheduling strategy, determine whether each electric vehicle can choose to charge at a fixed charging station or wirelessly dynamically charge with a mobile energy distributor, and plan the corresponding driving path. Step 6: The central controller will generate the optimal hybrid charging scheduling strategy and distribute it to electric vehicles, mobile energy distributors and fixed charging stations through roadside units and aggregators, respectively. Step 7: The electric vehicle completes the charging process according to the scheduling strategy, and the scheduling ends.

8. The hybrid electric vehicle charging scheduling method based on multi-agent proximal strategy optimization according to claim 7, characterized in that: The hybrid charging scheduling strategy includes selecting the charging method, determining the target fixed charging station or the target mobile energy distributor, and planning the corresponding driving path.

9. The hybrid electric vehicle charging scheduling method based on multi-agent proximal strategy optimization according to claim 7, characterized in that: The comprehensive cost of electric vehicle charging includes the additional time cost of charging, the additional energy consumption cost of driving, the additional driving distance cost, and the charging fee cost.

10. The hybrid electric vehicle charging scheduling method based on multi-agent proximal strategy optimization according to claim 7, characterized in that: During the scheduling process, mutual exclusion constraints are set for each electric vehicle, so that each electric vehicle is assigned to only one fixed charging station or one mobile energy distributor; the multi-agent proximal policy optimization algorithm adopts a multi-agent reinforcement learning framework with centralized value function training and distributed policy execution.

Images