Electric vehicle charging and power grid load optimization scheduling method based on reinforcement learning

Abstract

The invention is suitable for the technical field of electric vehicle and power grid interaction, and provides an electric vehicle charging and power grid load optimization scheduling method based on reinforcement learning, and the method comprises the steps: collecting power grid operation state data, electric vehicle user behavior data, weather environment data and traffic flow data; constructing a power grid dynamic model and a user state model; constructing a reinforcement learning scheduling model, wherein the reinforcement learning scheduling model comprises a state space, an action space and a reward function; a deep Q network is adopted to train and learn an initial strategy for the discrete action space, the continuous charging and discharging power is optimized based on a deep deterministic strategy gradient, and an optimized reinforcement learning scheduling model is obtained; the state space input of the optimized scheduling model is updated according to the collected data, the weight coefficient of the reward function is dynamically adjusted, a real-time charging and discharging scheduling instruction is generated, the electric vehicle charging requirement and the power grid load can be balanced, the power grid resource utilization rate can be optimized, and the system operation cost and the electric energy loss can be reduced.

Description

Technical Field

[0001] This invention relates to the field of electric vehicle and power grid interaction technology, and in particular to a method for optimizing electric vehicle charging and power grid load scheduling based on reinforcement learning. Background Technology

[0002] With the rapid development of the electric vehicle market, electric vehicles have become an important means of reducing carbon emissions and achieving low-carbon travel. However, large-scale electric vehicle charging can cause grid overload, power instability, and even affect the safe operation of the entire grid. Currently, most electric vehicle charging processes are based on a single mode of fixed time and fixed power, lacking dynamic coordination with the grid. This not only prevents electric vehicles from fully utilizing idle grid power during charging, resulting in energy waste, but also fails to effectively adjust charging strategies during peak grid load periods, further exacerbating grid load fluctuations.

[0003] In existing technologies, the interaction between electric vehicles and the power grid mostly relies on predetermined time periods or simple load management strategies, such as centralized dispatch systems based on grid load forecasting. However, these methods often overlook the flexibility and personalized needs of electric vehicle charging behavior, making it difficult to adapt to the real-time changes in grid load and the diverse charging demands of electric vehicles. Furthermore, in electric vehicle charging management, the charging demand of electric vehicles and the power supply capacity of the grid are dynamically changing, and traditional static dispatching cannot fully consider the real-time and dynamic nature of the system.

[0004] Therefore, a reinforcement learning-based method for optimizing electric vehicle charging and grid load scheduling is needed. Summary of the Invention

[0005] This application provides a reinforcement learning-based method for optimizing electric vehicle charging and grid load scheduling, which addresses the problem that scheduling strategies are difficult to adapt to real-time changes in grid load and the diverse charging needs of electric vehicles.

[0006] This application provides a reinforcement learning-based method for optimal scheduling of electric vehicle charging and power grid load, including: Collect power grid operation status data, electric vehicle user behavior data, weather environment data, and traffic flow data; A dynamic model of the power grid is constructed based on the distribution network topology and real-time load data in the power grid operation status data. At the same time, a user status model is generated based on the user's historical charging behavior in the electric vehicle user behavior data. A reinforcement learning scheduling model is constructed, which includes a state space, an action space, and a reward function. The state space integrates the power grid state model and the user state model, and superimposes real-time environmental data. The action space includes a discrete action set and a continuous power adjustment command. The parameters of the reward function include user revenue, power grid revenue, battery health reward, and carbon emission reward. A deep Q-network is used to train the discrete action space to learn the initial policy. The continuous charging and discharging power is optimized based on the gradient of the deep deterministic policy to obtain the optimized reinforcement learning scheduling model. The state space input of the optimized scheduling model is updated based on the real-time collected data, and the weight coefficients of the reward function are dynamically adjusted to generate real-time charging and discharging scheduling instructions.

[0007] Furthermore, the collection of power grid operation status data, electric vehicle user behavior data, weather environment data, and traffic flow data includes: The power grid operation status data includes the current load, real-time electricity price, voltage level, and line transmission capacity of the distribution network nodes; The electric vehicle user behavior data includes battery charge, charging and discharging costs, battery charging status, user charging preferences, and historical charging behavior. The weather and environmental data include temperature, precipitation, and extreme weather warning signals; The traffic flow data includes real-time road conditions and charging station availability.

[0008] Furthermore, the step of constructing a power grid dynamic model based on the distribution network topology and real-time load data in the power grid operation status data, and simultaneously generating a user status model based on the historical charging behavior of electric vehicle users in the user behavior data, includes: The constraints of the power grid dynamic model include node voltage deviation and line load rate, and the output is a real-time state vector that includes node voltage, load rate and the impact of extreme weather.

[0009] Furthermore, the step of constructing a power grid dynamic model based on the distribution network topology and real-time load data in the power grid operation status data, and simultaneously generating a user status model based on the historical charging behavior of electric vehicle users in the user behavior data, also includes: Time series clustering is performed on users' historical charging behavior data to classify charging time period preference categories and calculate the probability density function of charging demand for each time period. A Markov decision model is generated based on the parking duration and the availability of charging stations at the destination in the travel demand. After fusing the probability density function with the Markov decision model, the time and power joint distribution of user charging demand is output.

[0010] Furthermore, the state space integrates the power grid state model and the user state model, and overlays real-time environmental data, including: The time output by the power grid state model Current load of the power grid Real-time electricity price Voltage level The time output by the user state model Battery capacity Charging and discharging costs Battery charging status To merge and superimpose time. charging facility availability Weather conditions Traffic flow and user's historical charging behavior This forms a multidimensional state vector. : in: This indicates the number and power level of available charging stations at the user's current location. Includes temperature, precipitation, and extreme weather warning signals. The occupancy rate of charging stations and the road congestion index are obtained through the traffic monitoring system.

[0011] Furthermore, the action space includes a discrete action set and continuous power adjustment commands, including: The discrete action set is: in: This is a low-power charging mode, suitable for periods of low grid load. This is a high-power charging mode, suitable for emergency charging needs. In discharge mode, users feed electrical energy back to the grid through vehicle-to-grid interaction. To maintain the current state, no charging or discharging operations will be performed to preserve battery health; The charging power value of each charging pile is generated using a deep deterministic policy gradient algorithm.

[0012] Furthermore, the parameters of the reward function include user revenue, grid revenue, battery health reward, and carbon emission reward, including: reward function The expression is as follows: in: For user benefit, For the benefit of the power grid, As a reward for battery health, As a carbon emission incentive, , , and These are the corresponding weighting coefficients.

[0013] Furthermore, the method of training an initial strategy on a discrete action space using a deep Q-network, and optimizing the continuous charging and discharging power based on the gradient of a deep deterministic policy to obtain an optimized reinforcement learning scheduling model includes: The continuous charge / discharge power command is discretized into N power levels, and the discrete action set is: ; An offline training set is constructed based on historical data, and the Q-value function is updated using the Q-learning algorithm. in: In the state Select action value, For the current reward, For learning rate, As the discount factor, For the next state All possible actions The largest in value; Output a discrete action-policy mapping table, which serves as the initial policy input for the deep deterministic policy gradient algorithm.

[0014] Furthermore, the method of training an initial strategy on a discrete action space using a deep Q-network and optimizing continuous charging and discharging power based on the gradient of a deep deterministic policy to obtain an optimized reinforcement learning scheduling model also includes: Construct a network architecture that includes an Actor network and a Critic network; The Actor network is used to receive the state vector and output the continuous charge / discharge power value. or Satisfying the constraints or ,in This refers to the battery's maximum charge and discharge rate. The Critic network evaluates the Q-value of the state-action pair and updates the network parameters by minimizing the temporal difference error: in: For loss function, As the expected value, For the next state The action generated by the target Actor network, and For the output of the target network; A dual-delay deep deterministic policy gradient algorithm is introduced to suppress overfitting of the Actor network by delaying policy updates and smoothing the target network.

[0015] Furthermore, the step of updating the state space input of the optimized scheduling model based on real-time collected data, dynamically adjusting the weight coefficients of the reward function, and generating real-time charging and discharging scheduling instructions includes: The state space input of the reinforcement learning model is updated by periodically collecting power grid operation status parameters, user battery status and environmental data through smart sensors and electricity meters. Based on the deviation between real-time grid load fluctuation rate and user charging demand, the weight coefficients of user revenue, grid revenue, battery health and carbon emissions in the reward function are dynamically adjusted. The updated state space is input into the optimized reinforcement learning model to generate real-time charging and discharging power commands, which are then sent to the charging pile for execution via the V2G interface. When the grid load is detected to exceed a preset threshold or extreme weather events are detected, the system automatically switches to low-power charging mode and prioritizes the charging needs of critical users.

[0016] As can be seen from the above technical solutions, the present invention has the following advantages: This invention achieves dynamic optimization and real-time adjustment of electric vehicle charging scheduling through reinforcement learning algorithms. It can adaptively adjust charging strategies according to grid load fluctuations and changes in user demand, avoiding grid overload and optimizing the allocation of power resources. Employing a multi-objective collaborative optimization method, it balances grid load balance and user charging costs, maximizing the shared benefits for both the grid and users, avoiding the limitations of traditional technologies that focus on only a single objective. Based on the autonomous learning capability of deep reinforcement learning, the system can self-optimize through continuous interaction with the environment, improving scheduling accuracy and reducing human intervention. The optimized charging strategy improves electric vehicle charging efficiency, reduces user charging costs, and increases user participation through incentive mechanisms, further enhancing the overall system benefits. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of an embodiment of a reinforcement learning-based method for optimizing electric vehicle charging and grid load scheduling in this invention. Detailed Implementation

[0018] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] Example 1 The implementation method in this embodiment can be implemented in a system, on a server, or on a terminal; no specific limitation is made. The following section will describe the reinforcement learning-based electric vehicle charging and grid load optimization scheduling method of this application from the perspective of system implementation. Please refer to... Figure 1 The method provided in this application includes the following steps: S11. Collect power grid operation status data, electric vehicle user behavior data, weather environment data, and traffic flow data; In this embodiment, the power grid operation status data includes the current load, real-time electricity price, voltage level and line transmission capacity of the distribution network nodes; electric vehicle user behavior data includes battery power, charging and discharging costs, battery charging status, user charging preferences and historical charging behavior; weather environment data includes temperature, precipitation and extreme weather warning signals; traffic flow data includes real-time road traffic conditions and charging station availability.

[0020] S12. Construct a dynamic model of the power grid based on the distribution network topology and real-time load data in the power grid operation status data, and generate a user status model based on the user's historical charging behavior in the electric vehicle user behavior data. Specifically, a dynamic model of the power grid is constructed based on the distribution network topology and real-time load data from the power grid operation status data, including: The constraints of the power grid dynamic model include node voltage deviation and line load rate, and the output is a real-time state vector that includes node voltage, load rate and the impact of extreme weather.

[0021] Based on the distribution network topology, a node admittance matrix is ​​established, and the voltage deviation value of each node is calculated by combining real-time load data. And set the constraints as Based on line transmission capacity and real-time load factor Establish the dynamic equation of line load rate and constraints ,in This represents the maximum transmission capacity of the line. The output of the power grid dynamic equations is a real-time state vector that includes node voltages, load factors, and the effects of extreme weather. ,in, For the current load of the power grid, For electricity price, For voltage, These are weather state parameters. This refers to traffic flow parameters.

[0022] Specifically, a user state model is generated based on the user's historical charging behavior data in electric vehicle user behavior data, including: 1. Perform time series clustering on users' historical charging behavior data, classify charging time period preference categories, and calculate the probability density function of charging demand for each time period; Time-series cluster analysis was performed on users' historical charging behavior data to classify multiple charging time period preference categories, including peak charging periods, off-peak charging periods, and emergency charging periods. Based on the statistical distribution of charging demand in each time period, a probability density function of charging power demand was generated to characterize the typical charging power range of users in different time periods.

[0023] 2. A Markov decision model is generated based on the parking duration and the availability of charging stations at the destination in the travel demand to generate the user's preset charging time period; Based on the expected parking duration and destination charging station availability data from user travel needs, a Markov decision model is constructed. The state space is defined as the available charging time window, charging station load status, and user travel destination priority. A state transition probability matrix describes the user's behavioral switching patterns during different charging periods, and an immediate reward function is calculated by combining charging cost and waiting time.

[0024] 3. After fusing the probability density function with the Markov decision model, the time and power joint distribution of user charging demand is output.

[0025] The probability density function and the output of the Markov decision model are fused, and a time-power joint distribution function of user charging demand is generated through weighted superposition or Bayesian network ensemble. The joint distribution function output is a two-dimensional probability matrix, with the horizontal axis representing time segmentation and the vertical axis representing charging power level. The matrix elements represent the probability that a user will choose a specific power level for charging during a specific time period. Finally, the prediction accuracy of the joint distribution function is cross-validated using historical data, and the number of cluster categories and state transition probability parameters are adjusted based on real-time charging behavior feedback.

[0026] S13. Construct a reinforcement learning scheduling model. The reinforcement learning scheduling model includes a state space, an action space, and a reward function. The state space integrates the grid state model and the user state model, and overlays real-time environmental data. The action space includes a discrete action set and a continuous power adjustment command. The parameters of the reward function include user revenue, grid revenue, battery health reward, and carbon emission reward. In this embodiment, the reinforcement learning model is designed based on the joint state information of the power grid, users, and the environment, constructing a dynamic interactive optimization framework. The state space includes not only power grid load, electricity price, voltage, and user battery level, but also charging facility availability, user charging habits, and external environmental factors (such as weather and traffic flow) to improve the model's adaptability and optimization capabilities.

[0027] The time output of the power grid state model Current load of the power grid Real-time electricity price Voltage level The time output of the user state model Battery capacity Charging and discharging costs Battery charging status To merge and superimpose time. charging facility availability Weather conditions Traffic flow and user's historical charging behavior This forms a multidimensional state vector. : in: This indicates the number and power level of available charging stations at the user's current location. Includes temperature, precipitation, and extreme weather warning signals. The occupancy rate of charging stations and the road congestion index are obtained through the traffic monitoring system.

[0028] The discrete action set is: in: This is a low-power charging mode, suitable for periods of low grid load. This is a high-power charging mode, suitable for emergency charging needs. In discharge mode, users feed electrical energy back to the grid through vehicle-to-grid interaction. To maintain the current state, no charging or discharging operations will be performed to preserve battery health; The charging power value of each charging pile is generated using a deep deterministic policy gradient algorithm.

[0029] To enhance the model's practical application value, the optimized reward function comprehensively considers user benefits, grid benefits, battery health, and carbon emission factors, making it more aligned with actual needs.

[0030] reward function The expression is as follows: in: For user benefit, For the benefit of the power grid, As a reward for battery health, As a carbon emission incentive, , , and These are the corresponding weighting coefficients.

[0031] in, For incentive amount, For charging costs, User wait time (used to measure user experience) As a weighting factor, it balances user experience and economic cost.

[0032] in, For the current power grid load, For the target load (optimization target). As the power grid load balancing weighting factor, The weighting factor is the charging cost constraint.

[0033] in, Current battery status. The optimal SOC range (generally set at) To extend battery life. This is a weighting factor for battery health.

[0034] in, The proportion of fossil fuels currently used by the power grid, This represents the carbon emission cost coefficient.

[0035] S14. A deep Q-network is used to train the discrete action space to learn the initial strategy. The continuous charging and discharging power is optimized based on the gradient of the deep deterministic strategy to obtain the optimized reinforcement learning scheduling model. Deep Q-learning (DQN) and Deep Deterministic Policy Gradient (DDPG) algorithms are employed to optimize the charging and discharging strategy. The Q-learning algorithm evaluates the long-term reward of each state-action pair. DDPG optimizes the decision by generating charging and discharging power and evaluating the reward of the strategy. The reinforcement learning model is divided into offline and online phases. The offline phase uses historical data for pre-training, while the online phase adjusts the strategy based on real-time power grid and user status. Details are as follows: The continuous charge / discharge power command is discretized into N power levels, and the discrete action set is: ; An offline training set is constructed based on historical data, and the Q-value function is updated using the Q-learning algorithm. in: In the state Select action value, For the current reward, For learning rate, As the discount factor, For the next state All possible actions The largest in value; Output a discrete action-policy mapping table, which serves as the initial policy input for the deep deterministic policy gradient algorithm.

[0036] Construct a network architecture that includes an Actor network and a Critic network; The Actor network is used to receive the state vector and output the continuous charge / discharge power value. or Satisfying the constraints or ,in This refers to the battery's maximum charge and discharge rate. The Critic network evaluates the Q-value of the state-action pair and updates the network parameters by minimizing the temporal difference error: in: For loss function, As the expected value, For the next state The action generated by the target Actor network, and For the output of the target network; A dual-delay deep deterministic policy gradient algorithm is introduced to suppress overfitting of the Actor network by delaying policy updates and smoothing the target network.

[0037] S15. Update the state space input of the optimized scheduling model based on the real-time collected data, dynamically adjust the weight coefficient of the reward function, and generate real-time charging and discharging scheduling instructions.

[0038] In this embodiment, step S15 includes: 1. The state space input of the reinforcement learning model is updated by periodically collecting power grid operation status parameters, user battery status and environmental data through smart sensors and electricity meters; The power grid's operating status parameters, including the current load, are collected every 5 minutes via smart sensors and electricity meters. Node voltage Line transmission capacity and extreme weather warning signals Synchronously acquire user battery status parameters, including battery level. Charging and discharging costs Battery charging status and charging station availability The above parameters are integrated into a multidimensional state vector. Update the state space input of the reinforcement learning model.

[0039] 2. Based on the deviation between real-time grid load fluctuation rate and user charging demand, dynamically adjust the weight coefficients of user revenue, grid revenue, battery health and carbon emissions in the reward function; Based on real-time load With target load deviation Dynamically adjust the power grid revenue weight. Based on the difference between the user's actual charging power and the predicted demand Adjust user revenue weighting When extreme weather events (such as heavy rain or typhoons) are detected, the carbon emission weighting will be increased. In order to prioritize the response to low-carbon scheduling strategies.

[0040] 3. Input the updated state space into the optimized reinforcement learning model to generate real-time charging and discharging power commands, and send them to the charging pile for execution via the V2G interface; Update the state vector Input the optimized DDPG model, and output the charging and discharging power commands for each charging station through the Actor network. or And the power value meets the battery's maximum charge and discharge rate. Commands are sent to charging piles via the V2G interface for execution, and the changes in grid load and user charging response are monitored synchronously after the command is executed.

[0041] 4. When the grid load is detected to exceed the preset threshold or extreme weather events are detected, the system will automatically switch to low-power charging mode and prioritize the charging needs of critical users.

[0042] If the grid load rate is detected to exceed 95%, a forced switch to low-power charging mode will be implemented, and high-power charging requests from non-critical users will be suspended. When an extreme weather warning is received, priority will be given to ensuring the charging needs of emergency vehicles and critical users, while the charging power of ordinary users will be limited to a safe threshold.

[0043] Finally, the actual effect of the scheduling strategy is evaluated every 15 minutes through the online learning module. If the grid load balance or user satisfaction is lower than the preset threshold, the reward function weight coefficients are re-optimized. Abnormal events during the execution process (such as charging pile failures) are recorded and used as training data to input into the reinforcement learning model to improve the robustness of subsequent decisions.

[0044] The above embodiments can continuously learn and adjust scheduling strategies through reinforcement learning models, making the charging behavior of electric vehicles more intelligent, while mitigating grid load fluctuations and improving overall energy utilization efficiency.

[0045] Those skilled in the art will recognize that the units of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0046] In the embodiments provided by this invention, it should be understood that the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units may be combined into one unit, one unit may be split into multiple units, or some features may be ignored. Furthermore, the functional units in the various embodiments of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0047] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0048] It is understood that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning, characterized in that, include: Collect power grid operation status data, electric vehicle user behavior data, weather environment data, and traffic flow data; A dynamic model of the power grid is constructed based on the distribution network topology and real-time load data in the power grid operation status data. At the same time, a user status model is generated based on the user's historical charging behavior in the electric vehicle user behavior data. A reinforcement learning scheduling model is constructed, which includes a state space, an action space, and a reward function. The state space integrates the power grid state model and the user state model, and superimposes real-time environmental data. The action space includes a discrete action set and a continuous power adjustment command. The parameters of the reward function include user revenue, power grid revenue, battery health reward, and carbon emission reward. A deep Q-network is used to train the discrete action space to learn the initial policy. The continuous charging and discharging power is optimized based on the gradient of the deep deterministic policy to obtain the optimized reinforcement learning scheduling model. The state space input of the optimized scheduling model is updated based on the real-time collected data, and the weight coefficients of the reward function are dynamically adjusted to generate real-time charging and discharging scheduling instructions.

2. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 1, characterized in that, The collected power grid operation status data, electric vehicle user behavior data, weather environment data, and traffic flow data include: The power grid operation status data includes the current load, real-time electricity price, voltage level, and line transmission capacity of the distribution network nodes; The electric vehicle user behavior data includes battery charge, charging and discharging costs, battery charging status, user charging preferences, and historical charging behavior. The weather and environmental data include temperature, precipitation, and extreme weather warning signals; The traffic flow data includes real-time road conditions and charging station availability.

3. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 1, characterized in that, The process of constructing a power grid dynamic model based on the distribution network topology and real-time load data from the power grid operation status data, and simultaneously generating a user status model based on historical charging behavior data from electric vehicle user behavior data, includes: The constraints of the power grid dynamic model include node voltage deviation and line load rate, and the output is a real-time state vector that includes node voltage, load rate and the impact of extreme weather.

4. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 3, characterized in that, The process of constructing a power grid dynamic model based on the distribution network topology and real-time load data in the power grid operation status data, and generating a user status model based on the historical charging behavior of electric vehicle users in the user behavior data, also includes: Time series clustering is performed on users' historical charging behavior data to classify charging time period preference categories and calculate the probability density function of charging demand for each time period. A Markov decision model is generated based on the parking duration and the availability of charging stations at the destination in the travel demand. After fusing the probability density function with the Markov decision model, the time and power joint distribution of user charging demand is output.

5. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 1, characterized in that, The state space integrates the power grid state model and the user state model, and overlays real-time environmental data, including: The time output by the power grid state model Current load of the power grid Real-time electricity price Voltage level The time output by the user state model Battery capacity Charging and discharging costs Battery charging status To merge and superimpose time. charging facility availability Weather conditions Traffic flow and user's historical charging behavior This forms a multidimensional state vector. : in: This indicates the number and power level of available charging stations at the user's current location. Includes temperature, precipitation, and extreme weather warning signals. The occupancy rate of charging stations and the road congestion index are obtained through the traffic monitoring system.

6. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 5, characterized in that, The action space includes a discrete action set and continuous power adjustment commands, including: The discrete action set is: in: This is a low-power charging mode, suitable for periods of low grid load. This is a high-power charging mode, suitable for emergency charging needs. In discharge mode, users feed electrical energy back to the grid through vehicle-to-grid interaction. To maintain the current state, no charging or discharging operations will be performed to preserve battery health; The charging power value of each charging pile is generated using a deep deterministic policy gradient algorithm.

7. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 6, characterized in that, The parameters of the reward function include user revenue, grid revenue, battery health reward, and carbon emission reward, including: reward function The expression is as follows: in: For user benefit, For the benefit of the power grid, As a reward for battery health, As a carbon emission incentive, , , and These are the corresponding weighting coefficients.

8. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 1, characterized in that, The method employs a deep Q-network to train and learn an initial strategy in the discrete action space, and optimizes the continuous charging and discharging power based on the gradient of the deep deterministic policy to obtain an optimized reinforcement learning scheduling model, including: The continuous charge / discharge power command is discretized into N power levels, and the discrete action set is: ; An offline training set is constructed based on historical data, and the Q-value function is updated using the Q-learning algorithm. in: In the state Select action value, For the current reward, For learning rate, As the discount factor, For the next state All possible actions The largest in value; Output a discrete action-policy mapping table, which serves as the initial policy input for the deep deterministic policy gradient algorithm.

9. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 8, characterized in that, The method of training an initial strategy in a discrete action space using a deep Q-network, and optimizing the continuous charging and discharging power based on the gradient of the deep deterministic strategy to obtain an optimized reinforcement learning scheduling model, further includes: Construct a network architecture that includes an Actor network and a Critic network; The Actor network is used to receive the state vector and output the continuous charge / discharge power value. or Satisfying the constraints or ,in This refers to the battery's maximum charge and discharge rate. The Critic network evaluates the Q-value of the state-action pair and updates the network parameters by minimizing the temporal difference error: in: For loss function, As the expected value, For the next state The action generated by the target Actor network, and For the output of the target network; A dual-delay deep deterministic policy gradient algorithm is introduced to suppress overfitting of the Actor network by delaying policy updates and smoothing the target network.

10. The method for optimal scheduling of electric vehicle charging and power grid load based on reinforcement learning according to claim 1, characterized in that, The step of updating the state space input of the optimized scheduling model based on real-time collected data, dynamically adjusting the weight coefficients of the reward function, and generating real-time charging and discharging scheduling instructions includes: The state space input of the reinforcement learning model is updated by periodically collecting power grid operation status parameters, user battery status and environmental data through smart sensors and electricity meters. Based on the deviation between real-time grid load fluctuation rate and user charging demand, the weight coefficients of user revenue, grid revenue, battery health and carbon emissions in the reward function are dynamically adjusted. The updated state space is input into the optimized reinforcement learning model to generate real-time charging and discharging power commands, which are then sent to the charging pile for execution via the V2G interface. When the grid load is detected to exceed a preset threshold or extreme weather events are detected, the system automatically switches to low-power charging mode and prioritizes the charging needs of critical users.

Images