Electric vehicle charging scheduling system and method based on deep reinforcement learning and renewable energy sources

Abstract

The invention relates to an electric vehicle charging scheduling system and method based on deep reinforcement learning and renewable energy sources, and belongs to the field of intelligent traffic systems. The system comprises an electric vehicle, a charging station, a roadside unit, an aggregator and a central controller, and the method comprises the following steps: firstly, predicting the production condition of renewable energy through a GAN, and constructing a short-term charging demand and energy supply prediction model in combination with the real-time state information of the charging station; and then, the forming process of the charging scheduling strategy is expressed as a Markov decision process (MDP), and path planning and charging mode optimization of multiple agents are carried out through an MADDPG algorithm. According to the invention, matching, path selection and charging mode selection of the charging station and the electric vehicle are optimized in real time. According to the method, multiple factors such as vehicle charging requirements, traffic conditions, renewable energy source supply and user green preference are comprehensively considered, the charging time and the waiting time of the electric vehicle are effectively shortened, and the overall utility of the charging scheduling system is remarkably improved.

Description

Technical Field

[0001] The present invention relates to an electric vehicle charging scheduling system and method based on deep reinforcement learning and renewable energy, and belongs to the field of intelligent transportation systems. Background Art

[0002] As the world's requirements for environmental protection and energy sustainability increase, the electric vehicle industry is experiencing rapid development, with more and more vehicles turning to electric power. However, with the popularity of electric vehicles, the deployment and management of charging infrastructure has become increasingly complex and challenging. Charging scheduling for electric vehicles is not just about meeting the energy needs of individual vehicles, it involves the efficient operation of the entire charging network and the optimal allocation of resources. First, a major challenge for charging networks at this stage is the volatility of renewable energy generation. The production of renewable energy sources such as solar energy and wind energy is highly susceptible to external factors such as weather and time, and is highly uncertain. This volatility makes it difficult to accurately predict energy supply, which may result in insufficient power supply at electric vehicle charging stations, or in some cases, excess energy that cannot be effectively utilized, resulting in reduced system utility and energy waste. Second, the load of charging stations in different geographical locations is often unbalanced, especially during peak hours in urban centers, when some charging stations may be overcrowded, resulting in long waiting times for vehicles, while in more remote areas, charging stations may face idle resources. This load imbalance not only reduces the operational efficiency of charging stations, but may also lead to traffic congestion and a decline in user experience. In addition, the path planning and charging mode selection of electric vehicles in practical applications also face great complexity. During the driving process, the vehicle needs to dynamically optimize the driving route and charging mode according to the real-time traffic conditions, the load conditions of the charging station, and the availability of renewable energy. The choice of fast charging and slow charging modes directly affects the charging time and energy consumption of the vehicle. How to optimize the switching between the two in different scenarios is also a major problem in charging scheduling. Most of the existing research focuses on a single optimization goal, such as the charging time of the vehicle or the layout of the charging station, and rarely considers the joint scheduling of multiple vehicles and multiple charging stations. Especially in the complex and dynamic urban traffic environment, the existing methods are difficult to effectively expand the application. In order to solve these problems, there is an urgent need for an intelligent scheduling solution that can simultaneously optimize the forecast of renewable energy supply, the dynamic matching of charging stations and electric vehicles, and path planning to ensure the adaptability and utility of the system in large-scale application scenarios. Summary of the invention

[0003] The technical problem solved by the present invention is that the electric vehicle charging scheduling system faces a variety of complex cooperation-competition relationship scenarios, in which electric vehicles compete for charging resources and path planning resources (including renewable energy supply, charging station load and charging mode) to deal with the highly sensitive demands of different vehicles for charging efficiency and driving time. In the absence of clear charging decisions of other electric vehicles and resource allocation of charging stations, each electric vehicle needs to minimize its own charging cost through the optimal charging scheduling strategy. This optimization problem is a typical NP problem. The complexity of this problem lies in the need to explore the optimal charging scheduling and path selection strategy under the multiple constraints of renewable energy uncertainty, dynamic changes in charging station load and vehicle path planning, so as to maximize vehicle charging efficiency, minimize energy consumption and optimize waiting time, thereby achieving long-term improvement in the overall utility of the system.

[0004] The technical solution adopted by the present invention is: first, the production of renewable energy is predicted by generating adversarial network model (GAN model), which solves the randomness problem of energy supply and provides reliable data for charging scheduling. Secondly, the MADDPG algorithm is used to optimize the matching of electric vehicles and charging stations, and the problem is modeled as a Markov decision process to achieve the optimal charging station allocation and path planning in a complex dynamic environment. At the same time, the vehicle path planning and charging mode selection are linked to optimize the driving time and charging waiting time by comprehensively considering the vehicle driving speed and remaining power. In addition, the user green preference model is introduced to optimize the charging decision according to the user's low-carbon needs and improve user satisfaction. Through the multi-agent framework of centralized training and decentralized execution, the system trains the optimal scheduling strategy in the centralized control center, and the agent makes autonomous decisions based on local and global information in the actual environment to ensure the efficient operation of the system. On the whole, with the goal of minimizing the total charging time, energy consumption and system cost, the utility of the electric vehicle charging scheduling system is maximized, and the charging efficiency and resource utilization are improved.

[0005] An electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy, comprising:

[0006] Electric vehicles, used to send charging requests and vehicle status information to roadside units and receive charging scheduling strategies sent by roadside units;

[0007] Charging stations provide electric energy for electric vehicles. Charging stations send their status information and historical renewable energy capacity data to the aggregator, and receive charging scheduling strategies sent by the aggregator;

[0008] Roadside units are responsible for collecting charging requests and their own status information sent by electric vehicles, and then forwarding them to the central controller, and forwarding the charging scheduling decisions sent by the central controller to the electric vehicles;

[0009] Aggregator, which is responsible for collecting the status information and historical renewable energy capacity data sent by the charging station, and then forwarding it to the central controller, and forwarding the charging scheduling decision sent by the central controller to the charging station;

[0010] The central controller collects the vehicle information sent by the roadside unit and the charging station information sent by the aggregator, generates the optimal charging scheduling strategy through the MADDPG algorithm, and sends the generated optimal charging scheduling strategy to the roadside unit and the aggregator.

[0011] Preferably, the electric vehicle serves as a mobile computing and execution node, and the vehicle status information includes vehicle location, user green preference, charging requirements and remaining power.

[0012] Preferably, the charging station is an energy supply node, and the status information of the charging station includes the location of the charging station, the usage of the charging piles, the renewable energy supply and the load of the charging station. The energy used by the charging station includes traditional power sources and renewable energy.

[0013] Preferably, the roadside units are stationary nodes fixed on both sides of the road, with a communication range of 500m. The coverage areas of various roadside units do not overlap, and electric vehicles traveling in a certain area can only communicate with roadside units in the current coverage area.

[0014] Preferably, the aggregator is a stationary node fixed near the charging station and communicates with the charging station via wireless communication.

[0015] A method for scheduling electric vehicle charging based on deep reinforcement learning and renewable energy, the specific steps are as follows:

[0016] Step 1: The electric vehicle to be charged sends a charging request and vehicle status information to the roadside unit. The vehicle status information includes vehicle location, user green preference, charging demand, and remaining power;

[0017] Step 2: The charging station sends its status information and historical renewable energy capacity data of the charging station to the aggregator, including the location of the charging station, the usage of charging piles, the renewable energy supply and the load of the charging station;

[0018] Step 3: The roadside units and aggregators send the collected vehicle information and charging station information to the central controller;

[0019] Step 4: The central controller establishes a multi-objective optimization function based on the collected vehicle information and charging station information;

[0020] Step 5: The central controller inputs the historical renewable energy capacity data of the charging station into the deployed generative adversarial network model to predict the renewable energy output in the short term in the future, and uses the prediction result as the state input of the MADDPG algorithm. Then, the multi-objective optimization function established in Step 4 is used as the target to generate the optimal charging scheduling strategy.

[0021] Step 6: The central controller forwards the generated optimal charging scheduling strategy to each electric vehicle and charging station through the roadside unit and aggregator respectively;

[0022] Step 7: The electric vehicle drives to the assigned charging station for charging according to the charging scheduling strategy of the central controller. When the vehicle arrives at the charging station, the charging station gives priority to providing services to the vehicle according to the assigned charging strategy, and the charging scheduling ends.

[0023] Specifically, the charging scheduling strategy includes a recommended charging station location, a recommended driving route, and a charging mode (fast charging or slow charging).

[0024] Specifically, the historical renewable energy production capacity data of the charging station includes atmospheric pressure, solar irradiance, and photovoltaic production capacity.

[0025] The beneficial effects of the present invention are: the production of renewable energy is effectively predicted through the GAN model, alleviating the uncertainty of energy supply; the optimal matching of electric vehicles and charging stations is achieved through the MADDPG algorithm, reducing the complexity of charging time and path planning; at the same time, the charging decision based on user green preference optimization significantly improves user satisfaction and the green utility of the system. The overall system achieves high efficiency in resource allocation, minimizes energy consumption, and maximizes system utility, and is suitable for large-scale electric vehicle charging scheduling scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the system scenario of the present invention;

[0027] Figure 2 It is a schematic diagram of the prediction method of the present invention;

[0028] Figure 3 It is a schematic diagram of the prediction method results of the present invention;

[0029] Figure 4 It is a schematic diagram of the charging scheduling decision method of the present invention;

[0030] Figure 5 It is a schematic diagram of the vehicle charging scheduling process of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0032] Example 1: Figure 1 As shown, an electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy includes:

[0033] Electric vehicles are used to send charging requests and vehicle status information to roadside units and receive charging scheduling strategies sent by roadside units. During the charging scheduling process, electric vehicles need to compete with other vehicles for limited charging resources and path resources;

[0034] Charging station: The main function of the charging station is to provide electric energy for electric vehicles. The charging station sends its status information and historical renewable energy capacity data of the charging station to the aggregator, and receives the charging scheduling strategy sent by the aggregator. In the system, the charging station not only uses traditional energy, but also gives priority to renewable energy (solar energy) to charge electric vehicles.

[0035] Roadside units are responsible for collecting charging requests and their own status information sent by electric vehicles, and then forwarding them to the central controller, and forwarding the charging scheduling decisions sent by the central controller to the electric vehicles;

[0036] Aggregator, which is responsible for collecting the status information and historical renewable energy capacity data sent by the charging station, and then forwarding it to the central controller, and forwarding the charging scheduling decision sent by the central controller to the charging station;

[0037] The central controller collects vehicle information sent by the roadside unit and charging station information sent by the aggregator, generates the optimal charging scheduling strategy through the MADDPG algorithm, and sends the generated optimal charging scheduling strategy to the roadside unit and the aggregator. That is, the central controller generates and distributes charging scheduling strategies respectively through the centralized training and decentralized execution framework of the MADDPG algorithm.

[0038] Preferably, the electric vehicle serves as a mobile computing and execution node, and the vehicle status information includes vehicle location, user green preference, charging requirements and remaining power.

[0039] Furthermore, the charging station is an energy supply node, and the status information of the charging station includes the location of the charging station, the usage of the charging piles, the renewable energy supply and the load of the charging station. The energy used by the charging station includes traditional power sources and renewable energy.

[0040] Furthermore, the roadside units are stationary nodes fixed on both sides of the road, with a communication range of 500m. The coverage areas of various roadside units do not overlap, and electric vehicles traveling in a certain area can only communicate with roadside units in the current coverage area.

[0041] Furthermore, the aggregator is a stationary node fixed near the charging station and communicates with the charging station via wireless communication.

[0042] Furthermore, the central controller has great computing and storage capabilities and stores historical data of all electric vehicles and charging stations.

[0043] Furthermore, the central controller can monitor the status of the charging station in real time, including the usage of charging piles, the supply of renewable energy, the load of the charging station, etc., and store this data in the cloud for long-term analysis. The system can dynamically adjust future charging strategies based on historical data, and improve the operating efficiency of the overall system by continuously optimizing the status information of charging stations and electric vehicles.

[0044] A method for scheduling electric vehicle charging based on deep reinforcement learning and renewable energy, the specific steps are as follows:

[0045] Step 1: The electric vehicle to be charged sends a charging request and vehicle status information to the roadside unit. The vehicle status information includes vehicle location, user green preference, charging demand, and remaining power;

[0046] Step 2: The charging station sends its status information and historical renewable energy capacity data of the charging station to the aggregator, including the location of the charging station, the usage of charging piles, the renewable energy supply and the load of the charging station;

[0047] Step 3: The roadside units and aggregators send the collected vehicle information and charging station information to the central controller;

[0048] Step 4: The central controller establishes a multi-objective optimization function based on the collected vehicle information and charging station information;

[0049] Step 5: The central controller inputs the historical renewable energy capacity data of the charging station into the deployed generative adversarial network model to predict the renewable energy output in the short term in the future, and uses the prediction result as the state input of the MADDPG algorithm. Then, the multi-objective optimization function established in Step 4 is used as the target to generate the optimal charging scheduling strategy.

[0050] Step 6: The central controller forwards the generated optimal charging scheduling strategy to each electric vehicle and charging station through the roadside unit and aggregator respectively;

[0051] Step 7: The electric vehicle drives to the assigned charging station for charging according to the charging scheduling strategy of the central controller. When the vehicle arrives at the charging station, the charging station gives priority to providing services to the vehicle according to the assigned charging strategy, and the charging scheduling ends.

[0052] Furthermore, the charging scheduling strategy includes a recommended charging station location, a recommended driving route, and a charging mode (fast charging or slow charging).

[0053] Furthermore, the historical renewable energy production capacity data of the charging station includes atmospheric pressure, solar irradiance, and photovoltaic energy production.

[0054] like Figure 2 As shown in Figure 1, the prediction model used in the system is a GAN model. The GAN model is deployed in the central controller to predict the production of renewable energy. The GAN model consists of a generator and a discriminator. The generator generates the energy production in the short term in the future based on the historical renewable energy production capacity data of the charging station (such as atmospheric pressure, solar irradiance, and photovoltaic production), and the discriminator evaluates its accuracy and continuously optimizes the performance of the generator by comparing the generated data with the real data. The GAN model is trained with the historical renewable energy production capacity data of the charging station collected by the central controller, and the trained GAN model is used to predict the future energy supply in real time. Through the adversarial mechanism of the generator and the discriminator, GAN can capture the complex patterns and changing trends in renewable energy production, help the system dynamically adjust the charging strategy of electric vehicles according to the future energy supply, and improve energy utilization efficiency and system performance.

[0055] like Figure 3 As shown in the figure, a day is divided into 96 time periods with a time interval of 15 minutes. The figure shows the comparison between the model's predicted value and the actual value of energy consumption in different time periods of the day. It can be seen from the figure that the model's predicted value (red curve) is basically consistent with the actual value (black curve), especially during peak and trough periods, the predicted curve is more consistent with the actual curve. This shows that the model has a high accuracy in predicting energy consumption.

[0056] like Figure 4As shown in the figure: In the MADDPG algorithm, four neural networks are used: two evaluation networks and two target networks. The evaluation network is used for real-time decision-making and training, and its weights are continuously updated based on empirical data; while the weights of the target network are delayed and regularly copied from the evaluation network to enhance the stability of training. Each agent has an independent actor network and critic network. The actor network outputs actions based on the local observation information of each agent (that is, the state of the environment perceived by the agent), and these actions determine the interaction between the agent and the environment. The critic network takes the observation information and actions of each agent as input and outputs a state-action value function (Q value) to evaluate the long-term return of the action and provide guidance for the strategy improvement of the actor network. The key to MADDPG lies in the centralized training of the critic network, that is, during the training process, the critic network not only uses the observations and actions of the current agent, but also combines the state and action information of other agents to provide feedback from a global perspective for each agent. This centralized critic takes into account the mutual influence of the entire multi-agent system during training, ensuring that each agent can optimize its strategy in a collaborative and competitive environment. In the execution phase, the decentralized actor network only relies on its own local observations to make action decisions, implementing a decentralized execution strategy. This centralized training and decentralized execution mechanism enables agents to better coordinate and cooperate in complex multi-agent scenarios and optimize the long-term effectiveness of the global system.

[0057] like Figure 5 As shown in the figure, the electric vehicle charging scheduling method based on deep reinforcement learning and renewable energy first uses the GAN model to predict the short-term solar power generation at each charging station, and the prediction results are used as part of the deep reinforcement learning state space. Next, the central controller collects the status information of electric vehicles and charging stations from roadside units and aggregators. This information is used as the input of the reinforcement learning model state space. Then, the MADDPG algorithm is used to determine the optimal charging station and driving route for any electric vehicle that wants to charge.

[0058] The present invention is used to optimize the charging path and charging station selection of electric vehicles through intelligent scheduling in a complex environment where renewable energy supply is unstable and vehicle charging demand changes dynamically. Due to the fluctuation of renewable energy supply and uncertainty of electric vehicle demand, it is a challenge to achieve efficient scheduling management. A decentralized charging scheduling scheme based on generative adversarial network (GAN) and multi-agent deep deterministic policy gradient (MADDPG) algorithm is proposed. The method first predicts the production of renewable energy through GAN, and combines the real-time status information of the charging station to build a short-term charging demand and energy supply prediction model. Then, the formation process of the charging scheduling strategy is expressed as a Markov decision process (MDP), and the multi-agent path planning and charging mode optimization are performed through the MADDPG algorithm. In this method, the matching and path selection of charging stations and electric vehicles are optimized in real time by using a multi-agent deep reinforcement learning algorithm through centralized training and decentralized execution. The scheme comprehensively considers multiple factors such as vehicle charging demand, traffic conditions, renewable energy supply, and user green preference, effectively reduces the charging time and waiting time of electric vehicles, and significantly improves the overall utility of the charging scheduling system.

[0059] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. An electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy, characterized by: include: Electric vehicles, used to send charging requests and vehicle status information to roadside units and receive charging scheduling strategies sent by roadside units; Charging stations provide electric energy for electric vehicles. Charging stations send their status information and historical renewable energy capacity data to the aggregator, and receive charging scheduling strategies sent by the aggregator; Roadside units are responsible for collecting charging requests and their own status information sent by electric vehicles, and then forwarding them to the central controller, and forwarding the charging scheduling decisions sent by the central controller to the electric vehicles; Aggregator, which is responsible for collecting the status information and historical renewable energy capacity data sent by the charging station, and then forwarding it to the central controller, and forwarding the charging scheduling decision sent by the central controller to the charging station; The central controller collects the vehicle information sent by the roadside unit and the charging station information sent by the aggregator, generates the optimal charging scheduling strategy through the MADDPG algorithm, and sends the generated optimal charging scheduling strategy to the roadside unit and the aggregator.

2. The electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy according to claim 1, characterized in that: The electric vehicle serves as a mobile computing and execution node, and the vehicle status information includes vehicle location, user green preference, charging requirements and remaining power.

3. The electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy according to claim 1, characterized in that: The charging station is an energy supply node. The status information of the charging station includes the location of the charging station, the usage of the charging piles, the renewable energy supply and the load of the charging station. The energy used by the charging station includes traditional power sources and renewable energy.

4. The electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy according to claim 1, characterized in that: The roadside units are stationary nodes fixed on both sides of the road, with a communication range of 500m. The coverage areas of various roadside units do not overlap, and electric vehicles traveling in a certain area can only communicate with roadside units in the current coverage area.

5. The electric vehicle charging scheduling system based on deep reinforcement learning and renewable energy according to claim 1, characterized in that: The aggregator is a stationary node fixed near the charging station and communicates with the charging station via wireless communication.

6. A method for scheduling electric vehicle charging based on deep reinforcement learning and renewable energy, characterized in that: The specific steps are as follows: Step 1: The electric vehicle to be charged sends a charging request and vehicle status information to the roadside unit. The vehicle status information includes vehicle location, user green preference, charging demand, and remaining power; Step 2: The charging station sends its status information and historical renewable energy capacity data of the charging station to the aggregator, including the location of the charging station, the usage of charging piles, the renewable energy supply and the load of the charging station; Step 3: The roadside units and aggregators send the collected vehicle information and charging station information to the central controller; Step 4: The central controller establishes a multi-objective optimization function based on the collected vehicle information and charging station information; Step 5: The central controller inputs the historical renewable energy capacity data of the charging station into the deployed generative adversarial network model to predict the renewable energy output in the short term in the future, and uses the prediction result as the state input of the MADDPG algorithm. Then, the multi-objective optimization function established in Step 4 is used as the target to generate the optimal charging scheduling strategy. Step 6: The central controller forwards the generated optimal charging scheduling strategy to each electric vehicle and charging station through the roadside unit and aggregator respectively; Step 7: The electric vehicle drives to the assigned charging station for charging according to the charging scheduling strategy of the central controller. When the vehicle arrives at the charging station, the charging station gives priority to providing services to the vehicle according to the assigned charging strategy, and the charging scheduling ends.

7. The electric vehicle charging scheduling method based on deep reinforcement learning and renewable energy according to claim 6 is characterized in that: The charging scheduling strategy includes recommended charging station locations, recommended driving routes, and charging modes.

8. The electric vehicle charging scheduling method based on deep reinforcement learning and renewable energy according to claim 7, characterized in that: The charging mode refers to fast charging or slow charging.

9. The electric vehicle charging scheduling method based on deep reinforcement learning and renewable energy according to claim 6, characterized in that: The historical renewable energy production capacity data of the charging station includes atmospheric pressure, solar irradiance, and photovoltaic energy production.

Images