Scheduling methods, systems, equipment, and media for multi-objective electric vehicle charging
By using a hybrid algorithm combining deep reinforcement learning and chaotic simulated annealing particle swarm optimization, a multi-objective optimization model is constructed, which solves the problems of high computational overhead and poor real-time performance of electric vehicle charging in complex environments, and achieves comprehensive optimization of grid stability, user satisfaction and battery health.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANDONG JIAOTONG UNIV
- Filing Date
- 2026-01-29
- Publication Date
- 2026-06-02
AI Technical Summary
Existing electric vehicle charging optimization methods suffer from high computational overhead and poor real-time performance in complex environments. They also struggle to balance battery health, weather factors, and multi-objective optimization. Furthermore, reinforcement learning methods are slow to converge and prone to getting trapped in local optima, failing to effectively address grid stability and user satisfaction issues.
A hybrid algorithm combining deep reinforcement learning and chaotic simulated annealing particle swarm optimization is adopted. By combining battery state of charge, health state and grid load models, a multi-objective optimization model is constructed through Markov decision process. The model is pre-trained using chaotic simulated annealing particle swarm optimization algorithm and then interactively learned by deep reinforcement learning to optimize the charging scheduling strategy.
It has improved the stability and economy of power grid operation, adapted to weather changes, protected battery life, ensured user charging needs, and enhanced the intelligence and sustainability of dispatching strategies.
Smart Images
Figure CN122136916A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent charging and energy management technology for electric vehicles, and specifically to a scheduling method, system, device, and medium for multi-objective electric vehicle charging. Background Technology
[0002] With the rapid popularization of electric vehicles (EVs) and the widespread application of renewable energy, the large-scale integration of EVs has posed new challenges to grid operation safety and energy balance. Disorderly charging can easily lead to problems such as increased peak grid load, voltage fluctuations, and frequency deviations, thereby exacerbating grid pressure and affecting power supply stability.
[0003] Currently, researchers have proposed various methods for optimizing electric vehicle charging, including stochastic dynamic programming, mixed integer programming, genetic algorithms (GA), particle swarm optimization (PSO), and reinforcement learning (RL). The inventors have found that these methods can achieve partial optimization under specific conditions, but still have the following shortcomings: 1. Strong model limitations: Traditional methods require accurate modeling in complex environments, resulting in high computational overhead and poor real-time performance.
[0004] 2. Reinforcement learning has slow convergence and is prone to getting stuck in local optima: In large-scale dynamic scenarios, a single reinforcement learning algorithm is difficult to balance exploration and convergence.
[0005] 3. Insufficient environmental awareness: Existing methods generally ignore the impact of battery health (SOH) and weather factors, failing to achieve long-term economic and sustainability optimization.
[0006] 4. Insufficient multi-objective optimization: Existing research focuses more on electricity price and load balance, while lacking comprehensive consideration of multi-dimensional objectives such as user satisfaction, battery life and grid stability.
[0007] Therefore, there is an urgent need for a new multi-objective electric vehicle charging scheduling method that is environmentally adaptive and takes into account both battery health and grid constraints, in order to improve the economy, stability and sustainability of system operation. Summary of the Invention
[0008] The purpose of this invention is to provide a multi-objective electric vehicle charging scheduling method, system, device, and medium. By establishing a multi-objective optimization model and introducing a hybrid algorithm of deep reinforcement learning (DRL) and chaotic simulated annealing particle swarm optimization (CSA-PSO), the efficiency, stability, and intelligence of charging scheduling are achieved.
[0009] To achieve the above objectives, embodiments of the present invention provide a scheduling method for multi-objective electric vehicle charging, comprising: Collect basic power grid data, electric vehicle population data, and environmental weather data; Based on the aforementioned power grid basic data, electric vehicle population data, and environmental weather data, a dynamic coupled model is established, which includes a battery state of charge evolution model, a battery health state decay model, and a power grid load model. Within the Markov decision process framework, a multi-objective optimization model is constructed with the goal of maximizing the comprehensive reward. The comprehensive reward includes at least economic reward, battery health protection reward, and grid stability reward, and relevant power constraints, time constraints, and grid security constraints are set. The chaotic simulated annealing particle swarm optimization algorithm is used to iteratively pre-train the multi-objective optimization model to obtain the initial optimization strategy parameters. Based on the initial optimization strategy parameters, a deep reinforcement learning algorithm is used to interact with the environment and learn from it. PSO perturbation terms and chaotic perturbation terms are introduced in the action selection to optimize the charging scheduling strategy. Based on the optimized charging scheduling strategy, a charging power scheduling instruction for the electric vehicle group is generated and executed, wherein the charging power scheduling instruction responds to changes in electricity price, grid load, battery health status and weather conditions.
[0010] Optionally, the function corresponding to the battery state of charge evolution model is as follows:
[0011] In the formula, For charging efficiency, For charging power, For battery capacity, For the rate of change over time, This represents the state of charge of the battery at time t; The function corresponding to the battery health state degradation model is as follows:
[0012] In the formula, Based on the attenuation coefficient, SOC-related factors; The function corresponding to the power grid load model is as follows:
[0013] In the formula, Based on the basic load, Charging load for electric vehicle groups.
[0014] Optionally, the function corresponding to the multi-objective optimization model is as follows:
[0015] In the formula, This is a revenue item, reflecting the economic efficiency of scheduling. This is a health-related depreciation item, reflecting battery life. This is the load fluctuation term, reflecting the stability of the power grid. , , These are the weighting coefficients.
[0016] Optionally, the step of iteratively pre-training the multi-objective optimization model using the chaotic simulated annealing particle swarm optimization algorithm to obtain initial optimization strategy parameters includes: The initial positions of the particle swarm are mapped using a Logistic chaotic function; For each generation of particle swarms, perform the following steps: Calculate the fitness value for each particle, wherein the fitness value is calculated based on the multi-objective optimization function; Update the velocity and position of each particle based on its individual historical best position and the group's global best position; For the updated particle, a probability acceptance criterion based on simulated annealing is used to determine whether to accept it as a new position, wherein the probability of accepting a suboptimal solution gradually decreases as the iteration proceeds. After a preset number of iterations, the particle with the highest fitness value is selected from the final particle swarm, and its position vector is decoded into weight parameters, which are used as the initial parameters of the policy network in the deep reinforcement learning algorithm.
[0017] Optionally, the formula for calculating the probability acceptance criterion is as follows:
[0018] In the formula, The difference between the fitness value of the particle at the new location and the fitness value of the particle at the original location, where T is the current temperature parameter.
[0019] Optionally, the formula corresponding to the Logistic chaotic function is as follows:
[0020] In the formula, For control parameters, This represents the state variable at time k.
[0021] Optionally, based on the initial optimization strategy parameters, a deep reinforcement learning algorithm is used to interact with the environment, and PSO perturbation terms and chaotic perturbation terms are introduced into the action selection to optimize the charging scheduling strategy, including: A deep reinforcement learning model is constructed using the Actor-Critic network framework, where the Actor network is used to output action policies based on the state, and the Critic network is used to evaluate the value of the state. Define a state space, wherein the state space includes at least: the current time step, the current time-of-use electricity price, the real-time battery state of charge of the electric vehicle to be dispatched, the real-time battery health status of the electric vehicle to be dispatched, the current weather status, the current total load of the power grid, and the trend information of historical charging actions. Define an action space, wherein the action space is a discrete charging power command value, the value range of which is between a preset minimum power and a preset maximum power, and increases in increments of a preset step size; An experience replay pool is set up to store transferred samples. An experience replay sampling strategy is adopted, and a neural network learning rate, discount factor, and exploration rate decay plan are set to form a training mechanism. When the exploration rate is higher than the first preset exploration rate threshold, the perturbation action generated based on the particle swarm optimization algorithm is mixed with the original action output by the Actor network at a first ratio and used as the final action to be executed. When the exploration rate is lower than the second preset exploration rate threshold, the random perturbation action generated based on chaotic mapping is mixed with the original action output by the Actor network at a second ratio as the final execution action; wherein, in the first ratio, the proportion of perturbation action generated based on particle swarm optimization algorithm is higher than the proportion of perturbation action generated based on chaotic mapping, and the second ratio is the opposite.
[0022] The parameters of the Actor network and the Critic network are updated using the value function gradient and policy gradient calculated by the Critic network to maximize the expected cumulative reward.
[0023] Secondly, the present invention also provides a scheduling system for multi-target electric vehicle charging, comprising: The data acquisition module is used to collect basic power grid data, electric vehicle population data, and environmental weather data. The coupled model construction module is used to establish a dynamic coupled model based on the power grid basic data, electric vehicle group data and environmental weather data, which includes a battery state of charge evolution model, a battery health state decay model and a power grid load model. The optimization model building module is used to construct a multi-objective optimization model with the goal of maximizing the comprehensive reward under the Markov decision process framework. The comprehensive reward includes at least economic reward, battery health protection reward and grid stability reward, and sets relevant power constraints, time constraints and grid security constraints. The optimized model training module is used to iteratively pre-train the multi-objective optimization model using the chaotic simulated annealing particle swarm optimization algorithm to obtain initial optimization strategy parameters. The scheduling strategy optimization module is used to learn interactively with the environment using a deep reinforcement learning algorithm based on the initial optimization strategy parameters, and to introduce PSO perturbation terms and chaotic perturbation terms in action selection to optimize the charging scheduling strategy. The scheduling instruction generation module is used to generate and execute charging power scheduling instructions for electric vehicle groups based on the optimized charging scheduling strategy. The charging power scheduling instructions respond to changes in electricity price, grid load, battery health status and weather conditions.
[0024] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the multi-target electric vehicle charging scheduling method described above.
[0025] Fourthly, the present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described multi-target electric vehicle charging scheduling method.
[0026] Through the above technical solution, within the Markov Decision Process (MDP) framework, a multi-objective optimization model is constructed by comprehensively considering grid load, battery health, weather conditions, and user charging needs. A hybrid intelligent algorithm is then used to achieve synergy between global and local optimization. This improves grid operation stability and economic efficiency, while also adapting to weather changes, proactively protecting battery life, and ensuring user charging needs are met. Ultimately, this enhances the intelligence, sustainability, and overall robustness of the scheduling strategy.
[0027] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0028] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a multi-target electric vehicle charging scheduling method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a multi-objective intelligent charging scheduling system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a hybrid DRL-CSA-PSO algorithm architecture provided in an embodiment of the present invention; Figure 4 This is a comparison chart of the convergence speeds of a hybrid DRL-CSA-PSO and DRL provided in an embodiment of the present invention; Figure 5 This is a comparison chart of power grid load curves between hybrid DRL+CSA-PSO scheduling and DRL scheduling provided by an embodiment of the present invention; Figure 6 This is a comparison diagram of the grid stability of a hybrid DRL-CSA-PSO and a DRL provided in an embodiment of the present invention; Figure 7 This is a comparison chart of user satisfaction between hybrid DRL-CSA-PSO and DRL provided in an embodiment of the present invention; Figure 8 This is a comparison chart of the convergence of a hybrid DRL-CSA-PSO algorithm and a pure CSA-PSO algorithm provided in an embodiment of the present invention; Figure 9 This is a comparison chart of power grid stability between a hybrid DRL-CSA-PSO and a pure CSA-PSO provided by an embodiment of the present invention; Figure 10 This is a comparison chart of user satisfaction between hybrid DRL-CSA-PSO and pure CSA-PSO provided by an embodiment of the present invention; Figure 11 This is a comparison chart of basic load, mixed DRL-CSA-PSO and pure CSA-PSO load curves provided by an embodiment of the present invention; Figure 12 This is a schematic diagram of a multi-target electric vehicle charging scheduling system provided in an embodiment of the present invention; Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0029] Various embodiments of this disclosure will be described more fully in the following detailed description. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.
[0030] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions or operations and do not limit the addition of one or more functions or operations. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, or combination of the foregoing and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, or combinations of the foregoing, or the possibility of adding one or more features, numbers, steps, operations, or combinations of the foregoing.
[0031] In various embodiments of this disclosure, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.
[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0033] See Figure 1 The diagram shows a flowchart of a multi-target electric vehicle charging scheduling method in a specific embodiment, including the following execution steps: Step 100: Collect basic power grid data, electric vehicle population data, and environmental weather data.
[0034] Specifically, based on 500 electric vehicles, a city's regional power grid, and typical 24-hour load and electricity price curves, a multi-objective intelligent charging scheduling system is constructed. (See [reference needed]). Figure 2 As shown, the optimal scheduling strategy is generated using the hybrid algorithm proposed in this application. 24-hour urban basic load data is collected. The time granularity is 15 minutes (96 time periods in total); the electricity price adopts a time-of-use pricing model, including three price categories: peak, flat, and valley periods; the maximum allowable load of the power grid is set as follows: .
[0035] Electric vehicle group data generation: Randomly generated for 500 EVs: Arrival time (Follows a normal distribution, mean 18:00, variance 1.5h); Departure time (Mean 7:00, Variance 1h); Initial SOC distribution ranges from 20% to 60%; Battery capacity Set to a standard 60kWh; target SOC set at 90% (user expectation); initial SOH between 0.95 and 1.00.
[0036] Environment and Weather: Weather conditions are selected from {sunny, cloudy, rainy}, and vary randomly, affecting charging efficiency. Battery temperature-related degradation coefficient For example: sunny day: =0.95, =1.0; Cloudy: =0.93, =1.1; Rainy day: =0.92, =1.2.
[0037] Step 101: Based on the aforementioned power grid basic data, electric vehicle group data, and environmental weather data, establish a dynamic coupling model that includes a battery state of charge evolution model, a battery health state decay model, and a power grid load model.
[0038] Specifically, the function corresponding to the battery state of charge evolution model is as follows:
[0039] In the formula, For charging efficiency, For charging power, For battery capacity, For the rate of change over time, This represents the state of charge of the battery at time t.
[0040] For example, time step =15min=0.25h, maximum charging power Pmax=7kW (standard for home charging stations). The function corresponding to the battery health state degradation model is as follows:
[0041] In the formula, Based on the attenuation coefficient, These are SOC-related factors.
[0042] For example, the attenuation coefficient: =10 -5 (Basic attenuation rate) = (Depending on weather conditions).
[0043] The function corresponding to the power grid load model is as follows:
[0044] In the formula, Based on the basic load, Charging load for electric vehicle groups.
[0045] like This will result in penalties being imposed on the intelligent agent.
[0046] Step 102: Construct a multi-objective optimization model with the goal of maximizing the overall reward within the Markov decision process framework.
[0047] The comprehensive reward includes at least economic rewards, battery health protection rewards, and grid stability rewards, and sets relevant power constraints, time constraints, and grid security constraints.
[0048] Specifically, the function corresponding to the multi-objective optimization model is as follows:
[0049] In the formula, This is a revenue item, reflecting the economic efficiency of scheduling. This is a health-related depreciation item, reflecting battery life. This is the load fluctuation term, reflecting the stability of the power grid. , , These are the weighting coefficients.
[0050] In one specific implementation, ; The greater the decay, the greater the penalty; , = 0.6、 = 0.2、 = 0.2.
[0051] In one specific implementation, the relevant power constraints, time constraints, and grid security constraints are as follows: Charging power limit: Rechargeable time constraints: Charging is only permitted during the arrival and departure time intervals; Grid safety constraints: .
[0052] Step 103: Use the chaotic simulated annealing particle swarm optimization algorithm to iteratively pre-train the multi-objective optimization model to obtain the initial optimization strategy parameters.
[0053] In one specific embodiment, a schematic diagram of the hybrid DRL-CSA-PSO algorithm architecture constructed in this application is shown below. Figure 3 As shown, specifically, when executing step 103, the following steps can be performed: S1030: The initial position of the particle swarm is mapped using the Logistic chaotic function.
[0054] Specifically, the formula corresponding to the Logistic chaotic function is as follows:
[0055] In the formula, The parameter is preferably 4, which is used to enhance diversity. This represents the state variable at time k.
[0056] S1031: For each generation of particle swarm, perform the following steps: S10310: Calculate the fitness value for each particle, wherein the fitness value is calculated based on the multi-objective optimization function.
[0057] S10311: Update the velocity and position of each particle based on its historical best position and the global best position of the group.
[0058] S10312: For the updated particle, a probability acceptance criterion based on simulated annealing is used to determine whether to accept it as a new position, wherein the probability of accepting a suboptimal solution gradually decreases as the iteration proceeds.
[0059] Specifically, the formula for calculating the probability acceptance criterion is as follows:
[0060] In the formula, The difference between the fitness value of the particle at the new location and the fitness value of the particle at the original location, where T is the current temperature parameter.
[0061] Specifically, after the particle is updated, a worse solution is accepted based on probability: , It decreases in an exponential decay manner.
[0062] S1032: After a preset number of iterations, select the particle with the highest fitness value from the final particle swarm and decode its position vector into weight parameters, which are used as the initial parameters of the policy network in the deep reinforcement learning algorithm.
[0063] For example, after 50 iterations, the particle with the highest fitness is selected as the initial weight of the DRL Actor network.
[0064] Step 104: Based on the initial optimization strategy parameters, use a deep reinforcement learning algorithm to interact with the environment and introduce PSO perturbation terms and chaotic perturbation terms in action selection to optimize the charging scheduling strategy.
[0065] Specifically, when executing step 104, the following steps can be performed: S1040: A deep reinforcement learning model is constructed using the Actor-Critic network framework, where the Actor network is used to output action policies based on the state, and the Critic network is used to evaluate the value of the state.
[0066] S1041: Define a state space, wherein the state space includes at least: the current time step, the current time-of-use electricity price, the real-time battery state of charge of the electric vehicle to be dispatched, the real-time battery health status of the electric vehicle to be dispatched, the current weather status (encoded as one-hot), the current total load of the power grid, and the trend information of historical charging actions.
[0067] S1042: Define an action space, wherein the action space is a discrete charging power command value, the value of which is between a preset minimum power and a preset maximum power, and increases in increments of a preset step size.
[0068] For example, the action values range from 0 to 7 kW and are discrete values in 1 kW increments.
[0069] S1043: Set up an experience replay pool to store transferred samples, adopt an experience replay sampling strategy, and set up a neural network learning rate, discount factor, and exploration rate decay plan to form a training mechanism.
[0070] For example, the experience replay pool size is 20,000, the priority replay (PER) is sorted by TD-Error, the learning rate is 1e-4, and the discount factor is... =0.95, the exploration rate gradually decreases from 1 to 0.05.
[0071] S1044: When the exploration rate is higher than the first preset exploration rate threshold, the perturbation action generated based on the particle swarm optimization algorithm is mixed with the original action output by the Actor network at a first ratio and used as the final execution action.
[0072] S1045: When the exploration rate is lower than the second preset exploration rate threshold, the random perturbation action generated based on chaotic mapping is mixed with the original action output by the Actor network at a second ratio as the final execution action; wherein, in the first ratio, the proportion of perturbation action generated based on particle swarm optimization algorithm is higher than the proportion of perturbation action generated based on chaotic mapping, and the second ratio is the opposite.
[0073] For example, when the DRL exploration rate At higher rates: 50% of actions are generated by PSO perturbations, and 50% are output by the Actor network. At lower levels: 90% is dominated by the Actor network, and 10% introduces chaotic perturbations to prevent getting trapped in local optima.
[0074] S1046: Using the value function gradient and policy gradient calculated by the Critic network, update the parameters of the Actor network and the Critic network to maximize the expected cumulative reward.
[0075] It should be understood that the hybrid exploration mechanism is dominated by PSO in the early stage (strengthening global exploration) and dominated by DRL in the later stage (refining local optimization).
[0076] Step 105: Based on the optimized charging scheduling strategy, generate and execute charging power scheduling instructions for the electric vehicle group.
[0077] The charging power scheduling command responds to changes in electricity price, grid load, battery health status, and weather conditions.
[0078] For example, the final generated charging scheduling strategy has the following characteristics: 1. Actively charge during periods of low electricity price and reduce charging power during periods of high electricity price; 2. Actively adjust charging behavior during peak load periods to avoid grid overload; 3. Automatically reduce charging power in extreme weather (rainy days) to protect the battery; 4. Prioritize charging vehicles that leave early to improve user satisfaction; 5. All vehicles' SOC reaches the target value before the required time.
[0079] In one specific implementation, simulations were performed under different weather and load conditions, and compared with the traditional DRL and pure CSA-PSO algorithms. The comparison results are shown in [link to relevant documentation]. Figures 4-11 , Figure 4 The convergence speed curves of the proposed method both show an upward trend with increasing iterations (the vertical axis value approaches 0), indicating that both algorithms can achieve convergence through iteration (the optimization objective gradually improves), which aligns with the fundamental characteristic of reinforcement learning algorithms: "iterative optimization." The hybrid algorithm Hybrid DRL+CSA-PSO vs DRL converges faster: In the early iterations (Episode ≤ 200), the hybrid algorithm curve is significantly higher than the DRL curve, indicating that it can quickly approach a better solution in the early training phase, while DRL requires more iterations to achieve a similar effect. Better convergence performance: Throughout the entire iteration cycle (0-1000 iterations), the hybrid algorithm curve remains above the DRL curve (with the same number of iterations, the reward value of the hybrid algorithm is closer to 0), indicating that its final convergence optimization performance is superior to that of DRL alone. Even in the later iterations (Episode ≥ 800), when the curves tend to stabilize, the difference between the two remains stable, verifying the advantage of the hybrid algorithm in "avoiding local optima" (the chaotic perturbation mechanism suppresses the later convergence stagnation of DRL). Figure 5 By comparing the grid load curves of the hybrid DRL+CSA-PSO dispatching and the traditional DRL dispatching, it can be verified whether the optimization strategy can achieve peak shaving and valley filling (reducing load peaks and smoothing load fluctuations), thereby improving the stability of grid operation and reducing power supply costs.
[0080] Figure 6This study intuitively demonstrates the significant superiority of the core technical solution presented in this application compared to a single deep reinforcement learning (DRL) method in improving the key objective of grid stability. At the initial training stage (around iteration count 0), the stability score of the hybrid algorithm (approximately 0.45) is already higher than that of traditional DRL (approximately 0.15), reflecting the effectiveness of the "CSA-PSO pre-training stage." The initial strategy obtained through global search ensures the system operates in a more stable and optimal state from the outset. The hybrid algorithm's curve reaches a stable plateau after approximately 400 iterations, while the traditional DRL algorithm requires approximately 800 iterations. This verifies that the hybrid architecture design aims to accelerate the overall convergence process and solves the problem of slow convergence in traditional DRL. After convergence, the hybrid algorithm stabilizes the grid stability score at around 0.8, while traditional DRL only reaches approximately 0.65. This demonstrates that the hybrid algorithm, through collaborative global search (PSO) and local refinement (DRL), ultimately finds a scheduling strategy that more effectively mitigates load fluctuations and ensures the safe operation of the grid.
[0081] Figure 7 As the curves show, the hybrid algorithm (Hybrid DRL+CSA-PSO) initially performed slightly worse than the traditional DRL algorithm, but gradually improved during training, eventually reaching a level comparable to or even better than deep reinforcement learning algorithms, while ensuring long-term user satisfaction. This indicates that the hybrid algorithm not only has higher learning efficiency, but also achieves a superior level in meeting user needs. This validates the effectiveness of "accelerating convergence through CSA-PSO pre-training" and "dynamically optimizing the charging scheduling strategy to balance multiple objectives."
[0082] Figure 8 By combining the global exploration of CSA-PSO with the online learning optimization of DRL, not only are the bottlenecks of each individual algorithm addressed, but a significant performance leap is also achieved. The starting point of the hybrid algorithm is approximately -4.25 × 10⁻⁶. 6 It is far superior to pure CSA-PSO (approximately -5.4 × 10⁻⁶). 6 The study verified the effectiveness of "CSA-PSO pre-training": it provides a high-quality, high-starting-point initial strategy for subsequent online DRL learning, avoiding the inefficient process of exploring DRL from scratch. The hybrid algorithm converged rapidly to its performance plateau after approximately 400 iterations. In contrast, the curve of pure CSA-PSO showed a slow and continuous decline within 1000 iterations, without exhibiting a clear convergence trend. This confirms the invention's objective of "improving the global search capability and convergence speed of the DRL algorithm" and reveals the shortcomings of single CSA-PSO in solving such complex dynamic optimization problems, namely, low search efficiency and susceptibility to stagnation in the later stages of the search. After convergence, the stable reward value of the hybrid algorithm (approximately -3.75 × 10⁻⁶) is... 6The value is significantly higher than that of pure CSA-PSO (approximately -5.1 × 10⁻⁶). 6 It has been demonstrated that through online refinement and adaptation based on pre-training using DRL, the hybrid algorithm finds an optimal strategy that is far superior to that found by a single optimization algorithm.
[0083] Figure 9 At the start of training (iteration count 0), the hybrid algorithm's grid stability index (approximately 0.018) was already superior to that of pure CSA-PSO (approximately 0.02). This demonstrates the value of the "pre-training phase." CSA-PSO's global search provides the system with an initial, relatively better scheduling scheme, making it more grid-friendly from the first generation of policies. The hybrid algorithm's curve rapidly decreased and stabilized within 400 iterations. In contrast, the pure CSA-PSO curve decreased slowly and did not reach a stable plateau within 1000 iterations. This verifies that utilizing DRL's online self-learning capability allows for rapid refinement and adaptive adjustment of the pre-trained policy, overcoming the shortcomings of low optimization efficiency and slow response of single CSA-PSO in dynamic environments. The hybrid algorithm optimized and stabilized the grid stability index at approximately 0.015. Pure CSA-PSO, with the same training time, could only achieve 0.008. The results demonstrate that the hybrid algorithm, through collaborative exploration and learning, ultimately found a scheduling strategy that can more effectively "shave peaks and fill valleys" and suppress load fluctuations, achieving a more profound optimization of power grid stability.
[0084] Figure 10 In the results, the hybrid algorithm maintained a stable value of approximately 0.38, while the pure CSA-PSO algorithm, initially valued at 0.9, dropped sharply to zero after 800 rounds, indicating poor stability. This demonstrates that the DRL online learning mechanism in the hybrid algorithm can more effectively and stably refine the strategy based on interaction experience, quickly adapt to user behavior patterns, and thus reliably improve satisfaction.
[0085] Figure 11 In this paper, the average load curves of different methods are compared. The proposed hybrid algorithm significantly reduces peak load demand from 17:00 to 22:00, while slightly increasing load during off-peak hours (0:00 to 8:00). This demonstrates its effective peak shaving and valley filling effect, improving load balance and supporting grid stability. In contrast, pure CSA-PSO maintains a higher peak level, but its compensation for off-peak load is limited, resulting in poor demand regulation capability. Overall, the hybrid strategy achieves a more balanced daily load curve while meeting the charging demand of electric vehicles, demonstrating its practical value in sustainable charging scheduling.
[0086] In summary, by Figures 4-11As can be seen, compared with traditional baseline methods such as DRL and CSA-PSO, the method in this application has a faster convergence speed (see [reference]). Figure 4 It demonstrates significant advantages in terms of power grid stability, user satisfaction, and load balance, fully verifying its superior performance and practical engineering value in complex application scenarios. This application has the following advantages: 1. Multi-objective integrated optimization: For the first time, user satisfaction, battery life, grid stability, and economic efficiency are modeled in a unified manner. 2. Strong environmental and health awareness: Weather conditions and State of Health (SOH) modeling are introduced to achieve environmentally adaptive scheduling. 3. Superior algorithm performance: Convergence is accelerated through CSA-PSO pre-training, and chaotic perturbations avoid getting trapped in local optima. Overall, it shows significant advantages over traditional DRL and CSA-PSO baseline methods in terms of convergence speed, grid stability, user satisfaction, and load balance, fully demonstrating its significant improvement over existing methods. 4. Strong robustness and generalization: It maintains stable performance in stochastic scenarios and adapts to different loads and environmental conditions. 5. High engineering application value: The method is suitable for deployment in smart grid management platforms and can be widely applied to the optimization of new energy vehicle charging networks, distributed energy dispatch, and battery life management.
[0087] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0088] like Figure 12 As shown, the following are embodiments of the multi-target electric vehicle charging scheduling system provided in this disclosure. It belongs to the same inventive concept as the multi-target electric vehicle charging scheduling method in the above embodiments. For details not described in detail in the embodiments of the multi-target electric vehicle charging scheduling system, please refer to the embodiments of the multi-target electric vehicle charging scheduling method described above.
[0089] A multi-objective electric vehicle charging scheduling system includes: The data acquisition module is used to collect basic power grid data, electric vehicle population data, and environmental weather data. The coupled model construction module is used to establish a dynamic coupled model based on the power grid basic data, electric vehicle group data and environmental weather data, which includes a battery state of charge evolution model, a battery health state decay model and a power grid load model. The optimization model building module is used to construct a multi-objective optimization model with the goal of maximizing the comprehensive reward under the Markov decision process framework. The comprehensive reward includes at least economic reward, battery health protection reward and grid stability reward, and sets relevant power constraints, time constraints and grid security constraints. The optimized model training module is used to iteratively pre-train the multi-objective optimization model using the chaotic simulated annealing particle swarm optimization algorithm to obtain initial optimization strategy parameters. The scheduling strategy optimization module is used to learn interactively with the environment using a deep reinforcement learning algorithm based on the initial optimization strategy parameters, and to introduce PSO perturbation terms and chaotic perturbation terms in action selection to optimize the charging scheduling strategy. The scheduling instruction generation module is used to generate and execute charging power scheduling instructions for electric vehicle groups based on the optimized charging scheduling strategy. The charging power scheduling instructions respond to changes in electricity price, grid load, battery health status and weather conditions.
[0090] Figure 13 This is a schematic diagram of the hardware structure of an electronic device that implements various embodiments of the present invention.
[0091] The multi-target electric vehicle charging scheduling method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.
[0092] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.
[0093] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0094] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.
[0095] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.
[0096] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.
[0097] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.
[0098] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
[0099] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.
[0100] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.
[0101] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.
[0102] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.
[0103] Electronic devices can achieve display functions through GPUs, displays, and application processors.
[0104] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.
[0105] A display screen is used to display images, videos, etc. A display screen includes a display panel.
[0106] The storage medium provided in this application stores a program product capable of implementing a scheduling method for multi-target electric vehicle charging.
[0107] In some possible implementations, the subject matter of this disclosure, scheduling method and system for multi-target electric vehicle charging, can be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.
[0108] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A scheduling method for multi-objective electric vehicle charging, characterized in that, include: Collect basic power grid data, electric vehicle population data, and environmental weather data; Based on the aforementioned power grid basic data, electric vehicle population data, and environmental weather data, a dynamic coupled model is established, which includes a battery state of charge evolution model, a battery health state decay model, and a power grid load model. Within the Markov decision process framework, a multi-objective optimization model is constructed with the goal of maximizing the comprehensive reward. The comprehensive reward includes at least economic reward, battery health protection reward, and grid stability reward, and relevant power constraints, time constraints, and grid security constraints are set. The chaotic simulated annealing particle swarm optimization algorithm is used to iteratively pre-train the multi-objective optimization model to obtain the initial optimization strategy parameters. Based on the initial optimization strategy parameters, a deep reinforcement learning algorithm is used to interact with the environment and learn from it. PSO perturbation terms and chaotic perturbation terms are introduced in the action selection to optimize the charging scheduling strategy. Based on the optimized charging scheduling strategy, a charging power scheduling instruction for the electric vehicle group is generated and executed, wherein the charging power scheduling instruction responds to changes in electricity price, grid load, battery health status and weather conditions.
2. The scheduling method for multi-target electric vehicle charging according to claim 1, characterized in that, The function corresponding to the battery state of charge evolution model is as follows: In the formula, For charging efficiency, For charging power, For battery capacity, The rate of change over time, This represents the state of charge of the battery at time t; The function corresponding to the battery health state degradation model is as follows: In the formula, Based on the attenuation coefficient, SOC-related factors; The function corresponding to the power grid load model is as follows: In the formula, Basic load, Charging load for electric vehicle groups.
3. The scheduling method for multi-target electric vehicle charging according to claim 1, characterized in that, The function corresponding to the multi-objective optimization model is as follows: In the formula, This is a revenue item, reflecting the economic efficiency of scheduling. This is a health-related depreciation item, reflecting battery life. This is the load fluctuation term, reflecting the stability of the power grid. , , These are the weighting coefficients.
4. The scheduling method for multi-target electric vehicle charging according to claim 1, characterized in that, The multi-objective optimization model is iteratively pre-trained using the chaotic simulated annealing particle swarm optimization algorithm to obtain initial optimization strategy parameters, including: The initial positions of the particle swarm are mapped using a Logistic chaotic function; For each generation of particle swarms, perform the following steps: Calculate the fitness value for each particle, wherein the fitness value is calculated based on the multi-objective optimization function; Update the velocity and position of each particle based on its individual historical best position and the group's global best position; For the updated particle, a probability acceptance criterion based on simulated annealing is used to determine whether to accept it as a new position, wherein the probability of accepting a suboptimal solution gradually decreases as the iteration proceeds. After a preset number of iterations, the particle with the highest fitness value is selected from the final particle swarm, and its position vector is decoded into weight parameters, which are used as the initial parameters of the policy network in the deep reinforcement learning algorithm.
5. The scheduling method for multi-target electric vehicle charging according to claim 4, characterized in that, The formula for calculating the probability acceptance criterion is as follows: In the formula, The difference between the fitness value of the particle at the new location and the fitness value of the particle at the original location, where T is the current temperature parameter.
6. The scheduling method for multi-target electric vehicle charging according to claim 4, characterized in that, The formula corresponding to the Logistic chaotic function is as follows: In the formula, For control parameters, This represents the state variable at time k.
7. The scheduling method for multi-target electric vehicle charging according to claim 1, characterized in that, Based on the initial optimization strategy parameters, a deep reinforcement learning algorithm is used to interact with the environment, and PSO perturbation terms and chaotic perturbation terms are introduced into the action selection to optimize the charging scheduling strategy, including: A deep reinforcement learning model is constructed using the Actor-Critic network framework, where the Actor network is used to output action policies based on the state, and the Critic network is used to evaluate the value of the state. Define a state space, wherein the state space includes at least: the current time step, the current time-of-use electricity price, the real-time battery state of charge of the electric vehicle to be dispatched, the real-time battery health status of the electric vehicle to be dispatched, the current weather status, the current total load of the power grid, and the trend information of historical charging actions. Define an action space, wherein the action space is a discrete charging power command value, the value range of which is between a preset minimum power and a preset maximum power, and increases in increments of a preset step size; An experience replay pool is set up to store transferred samples. An experience replay sampling strategy is adopted, and a neural network learning rate, discount factor, and exploration rate decay plan are set up to form a training mechanism. When the exploration rate is higher than the first preset exploration rate threshold, the perturbation action generated based on the particle swarm optimization algorithm is mixed with the original action output by the Actor network at a first ratio and used as the final action to be executed. When the exploration rate is lower than the second preset exploration rate threshold, the random perturbation action generated based on chaotic mapping is mixed with the original action output by the Actor network at a second ratio to serve as the final execution action; wherein, in the first ratio, the proportion of perturbation actions generated based on particle swarm optimization algorithm is higher than the proportion of perturbation actions generated based on chaotic mapping, and the second ratio is the opposite. The parameters of the Actor network and the Critic network are updated using the value function gradient and policy gradient calculated by the Critic network to maximize the expected cumulative reward.
8. A scheduling system for multi-objective electric vehicle charging, characterized in that, include: The data acquisition module is used to collect basic power grid data, electric vehicle population data, and environmental weather data. The coupled model construction module is used to establish a dynamic coupled model based on the power grid basic data, electric vehicle group data and environmental weather data, which includes a battery state of charge evolution model, a battery health state decay model and a power grid load model. The optimization model building module is used to construct a multi-objective optimization model with the goal of maximizing the comprehensive reward under the Markov decision process framework. The comprehensive reward includes at least economic reward, battery health protection reward and grid stability reward, and sets relevant power constraints, time constraints and grid security constraints. The optimized model training module is used to iteratively pre-train the multi-objective optimization model using the chaotic simulated annealing particle swarm optimization algorithm to obtain initial optimization strategy parameters. The scheduling strategy optimization module is used to learn interactively with the environment using a deep reinforcement learning algorithm based on the initial optimization strategy parameters, and to introduce PSO perturbation terms and chaotic perturbation terms in action selection to optimize the charging scheduling strategy. The scheduling instruction generation module is used to generate and execute charging power scheduling instructions for electric vehicle groups based on the optimized charging scheduling strategy. The charging power scheduling instructions respond to changes in electricity price, grid load, battery health status and weather conditions.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the multi-target electric vehicle charging scheduling method as described in any one of claims 1 to 7.
10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-target electric vehicle charging scheduling method as described in any one of claims 1 to 7.