Electric vehicle intelligent scheduling method and system, and storage medium

Abstract

The invention discloses an electric vehicle intelligent scheduling method and system, and a storage medium, and relates to the technical field of electric vehicle charging. The method comprises the following steps: initializing training parameters of an upper-layer optimization module; initializing a training environment of an upper-layer optimization module; the training environment comprises a state space and an action space; determining a charging strategy adopted by each agent according to the state space and the Actor-Critic strategy network, and calculating a reward value corresponding to the charging strategy according to a reward function; a state space is updated, an upper layer optimization module optimizes a charging strategy through a PPO reinforcement learning algorithm, multiple times of training are carried out, and a charging model is obtained; the upper-layer optimization module provides a charging strategy for the to-be-charged electric vehicle through the charging model, and the lower-layer optimization module optimizes the charging cost and the user satisfaction degree through an objective function according to the charging strategy and distributes a charging station for the to-be-charged electric vehicle. According to the method, flexible resource scheduling and management can be realized, and the overall efficiency of the system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicle charging, and in particular, to an intelligent scheduling method, system, and storage medium for electric vehicles. Background Art

[0002] With the popularization of electric vehicles and the rapid development of renewable energy technologies, how to optimize the charging process of electric vehicles to improve energy utilization efficiency and user satisfaction has become a research hotspot in academia and industry. Traditional electric vehicle charging optimization systems usually adopt a single-level optimization method, which is difficult to balance the coordination between global strategies and specific executions. This method may lead to a mismatch between charging strategies and actual resource allocations, reducing the overall efficiency of the system. In addition, existing systems lack real-time and flexibility in charging station resource allocation, unable to quickly respond to changes in grid load and dynamic adjustments of user demands, which may result in overloading of charging stations or waste of resources, affecting the user experience. Traditional methods often ignore the personalized needs and preferences of users during the optimization process, lacking a personalized recommendation mechanism for user preferences and unable to provide the best charging experience. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art. For this purpose, the present invention provides an intelligent scheduling method, system, and storage medium for electric vehicles, which can balance the coordination between global strategies and specific executions, make the charging strategy match the actual resource allocation, improve the user's charging experience, and avoid waste of resources.

[0004] On the one hand, according to an embodiment of the present invention, an intelligent scheduling method for electric vehicles includes the following steps:

[0005] Applied to an intelligent scheduling system for electric vehicles, the intelligent scheduling system for electric vehicles includes an upper-layer optimization module, a lower-layer optimization module, and an integration module. The upper-layer optimization module and the lower-layer optimization module perform data interaction and decision coordination through the integration module; the intelligent scheduling method for electric vehicles includes the following steps:

[0006] Initialize the training parameters of the upper-layer optimization module;

[0007] Initialize the training environment of the upper-layer optimization module; the training environment includes a state space and an action space. The state space includes the state information of each agent, and the action space includes the charging strategies that each agent can adopt. Each agent represents an electric vehicle;

[0008] According to the state space, the Actor-Critic policy network determines the charging strategy adopted by each agent and calculates the reward value corresponding to the charging strategy according to the reward function;

[0009] The state space is updated, and the upper optimization module optimizes the charging strategy using a PPO reinforcement learning algorithm, performing multiple trainings until the training is completed to obtain a charging model;

[0010] The upper optimization module provides a charging strategy for the electric vehicle to be charged through the charging model, and the lower optimization module optimizes the charging cost and user satisfaction through the objective function according to the charging strategy, and allocates a charging station to the electric vehicle to be charged.

[0011] According to some embodiments of the present invention, the state space includes the battery charging state of the intelligent agent, the remaining power requirement, the expected departure time, the current electricity price signal and the number of available charging piles of the charging station.

[0012] According to some embodiments of the present invention, the Actor-Critic strategy network includes an Actor network and a Critic network, the Actor network adopts a Transformer architecture, and the Critic network adopts a multi-layer perceptron structure.

[0013] According to some embodiments of the present invention, the step of initializing the training parameters of the upper-layer optimization module includes:

[0014] Initialize the number of training rounds, number of agents, grid load limit, and maximum charging time;

[0015] According to the actual electricity price policy, a day is divided into three periods: peak and valley, flat peak and peak, and different electricity prices are set for each period.

[0016] According to some embodiments of the present invention, the state space is:

[0017] S t =[SOC 1,t ,…,SOC N,t ,D 1,t ,…,D N,t ,T 1,t ,…,T N,t ,Y t ,N c,t ] T

[0018] Among them, SOC i,t is the battery charge state of the ith agent at time t, D i,t is the remaining power requirement of the ith agent, T i,t is the expected departure time of the i-th agent, Y t is the electricity price at time t, N c,t is the number of available charging piles at the charging station, and N is the total number of agents;

[0019] The action space is as follows:

[0020] A t = [d t , q t , r t T ;

[0021] where d t is the charging strategy variable, q t is the charging amount, and r t is the charging rate;

[0022] The state transition function of the agent is as follows:

[0023]

[0024] where η is the charging rate and C i is the battery capacity of the i-th agent;

[0025] L i,t+1 = L i,t + Vt;

[0026] where L i,t is the charging time of the i-th agent and Vt is the fixed time interval;

[0027] P t+1 = ∑(q t * r t )

[0028] where P t is the total load at time t.

[0029] According to some embodiments of the present invention, the reward function is as follows:

[0030] R(S t , A t ) = w1Rusr(S t , A t ) + w2Rload(S t , A t ) + w3Rcost(Ss t , A t );

[0031] where w1, w2, and w3 are weight coefficients;

[0032] Ruser(S t , A t ) = α∑SOC(S t ) - β∑L(S t );

[0033] Among them, α and β are weight coefficients related to user satisfaction, and SOC(S t ) is the battery charge state of the agent in the corresponding state, and L(S t ) is the charging time of the agent in the corresponding state;

[0034]

[0035] Among them, l t is the actual load at time t, l target is the target load, and l max is the maximum load limit;

[0036] Rcost(S t ,A t )=-Y t *d t *q t *r t .

[0037] According to some embodiments of the present invention, the objective function is:

[0038]

[0039] Among them, C i is the charging cost of the i-th charging station, P i is the charging power of the i-th charging station, S i is the service cost of the i-th charging station, α is the weight coefficient related to user satisfaction, and U is the user satisfaction score.

[0040] According to some embodiments of the present invention, the constraint conditions of the objective function are:

[0041]

[0042] 0≤SOC i ≤1;

[0043] Among them, represents the total charging power, GridLimit represents the maximum grid load limit, x ij represents the charging allocation of the i-th agent at the j-th charging station, and C j represents the capacity limit of the j-th charging station.

[0044] On the other hand, the electric vehicle intelligent scheduling system according to the embodiments of the present invention includes an upper-layer optimization module, a lower-layer optimization module, and an integration module. The upper-layer optimization module and the lower-layer optimization module perform data interaction and decision collaboration through the integration module. The electric vehicle intelligent scheduling system is used to execute the electric vehicle intelligent scheduling method described in the above embodiments.

[0045] On the other hand, according to the storage medium of the embodiment of the present invention, the storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute the electric vehicle intelligent scheduling method described in the above embodiment.

[0046] The electric vehicle intelligent scheduling method, system, and storage medium according to the embodiment of the present invention at least have the following beneficial effects: The upper-layer optimization module optimizes the charging strategy based on the PPO reinforcement learning algorithm to determine the overall adjustable load and charging time allocation of the system; the lower-layer optimization module optimizes the charging station allocation based on the Web service and is responsible for the specific charging station allocation and real-time scheduling management. At the same time, the upper-layer optimization module and the lower-layer optimization module realize data interaction and decision collaboration between the two layers through the integration module to ensure a high degree of consistency between the global strategy and the specific execution. The lower-layer optimization module can quickly respond to system changes by monitoring the usage status of the charging station and the grid load in real time and dynamically adjusting in combination with the strategy of the upper-layer optimization module, realizing flexible resource scheduling and management, improving the overall efficiency of the system. The system comprehensively considers the personalized needs and preferences of users in the allocation strategy, calculates the satisfaction score through multiple factors such as geographical distance, cost structure, and charging type matching, and provides personalized charging station recommendations, significantly improving user satisfaction.

[0047] The additional aspects and advantages of the present invention will be partly given in the following description, partly will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, wherein:

[0049] Figure 1 is a schematic structural diagram of the electric vehicle intelligent scheduling system according to the embodiment of the present invention;

[0050] Figure 2 is a flowchart of the steps of the electric vehicle intelligent scheduling method according to the embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of the training process of the upper-layer optimization module according to the embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of data management of the lower-layer optimization module according to the embodiment of the present invention;

[0053] Figure 5 is a flowchart of the front-end interface operation of the lower-layer optimization module according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and should not be construed as a limitation to the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation and illustration, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0055] In the description of the present invention, it should be understood that with respect to the orientation description, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc., is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention.

[0056] The terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0057] Referring to "embodiments" in the present invention means that the specific features, structures or characteristics described in connection with the embodiments may be included in at least one embodiment of the present invention. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0058] With the popularization of electric vehicles and the rapid development of renewable energy technologies, how to optimize the charging process of electric vehicles to improve energy utilization efficiency and user satisfaction has become a research hotspot in academia and industry. Traditional electric vehicle charging optimization systems usually adopt a single-level optimization method, which is difficult to balance the coordination of global strategies and specific executions simultaneously. This method may lead to a mismatch between charging strategies and actual resource allocations, reducing the overall efficiency of the system. In addition, existing systems lack real-time performance and flexibility in charging station resource allocation and cannot quickly respond to changes in grid loads and dynamic adjustments of user demands, which may cause charging stations to be overloaded or resources to be wasted, affecting the user experience. Traditional methods often ignore the personalized needs and preferences of users during the optimization process and lack a personalized recommendation mechanism for user preferences, unable to provide the best charging experience.

[0059] Existing charging optimization technical solutions mainly focus on single-level optimization methods and usually formulate charging strategies and allocate resources through simple rules or static models. These solutions are unable to cope when faced with complex grid load changes and diverse user demands.

[0060] Therefore, the embodiments of the present invention provide an intelligent scheduling method, system, and storage medium for electric vehicles. The system adopts a two-layer optimization structure. The upper-layer optimization module optimizes the charging strategy based on the PPO reinforcement learning algorithm to determine the overall adjustable load and charging time allocation of the system. The lower-layer optimization module optimizes the charging station allocation based on Web services and is responsible for specific charging station allocation and real-time scheduling management. At the same time, the upper-layer optimization module and the lower-layer optimization module achieve data interaction and decision-making coordination between the two layers through an integration module to ensure a high degree of consistency between global strategies and specific executions. By monitoring the usage status of charging stations and grid loads in real time and dynamically adjusting in combination with the strategies of the upper-layer optimization module, the lower-layer optimization module can quickly respond to system changes, achieve flexible resource scheduling and management, and improve the overall efficiency of the system. The system comprehensively considers the personalized needs and preferences of users in the allocation strategy, calculates the satisfaction score through multiple factors such as geographical distance, cost structure, and charging type matching, and provides personalized charging station recommendations, significantly improving user satisfaction.

[0061] The following explains the terms that appear in the embodiments of the present invention:

[0062] PPO: Proximal Policy Optimization, a policy optimization algorithm in the field of reinforcement learning. It improves the stability and sample efficiency of learning by restricting the range of policy updates.

[0063] Actor-Critic: The Actor represents the policy network, which is used to select actions. The Actor gives the probability of each action a in state s through the parameterized policy function π(a|s,θ). This policy function can be any function approximation model. The goal of the Actor is to maximize the expected cumulative reward and update the policy through the gradient of parameter θ. The Critic represents the value function network, which is used to evaluate the value of the state. The Critic is responsible for estimating the value function V(s) of the current state, that is, the expected cumulative reward that can be obtained by executing the policy starting from this state. The value function can be used to evaluate the quality of the current policy. Therefore, the goal of the Critic is to minimize the error of the value function;

[0064] Transformer: A deep learning model based on the self-attention mechanism. It uses the self-attention mechanism and multi-head attention to capture the dependencies within the sequence and introduces position information through position encoding. The essence of the Transformer is an encoder-decoder architecture based on the self-attention mechanism. It can effectively handle sequence-to-sequence tasks and capture long-range dependencies in the input sequence.

[0065] Next, in combination with the accompanying drawings, the electric vehicle intelligent scheduling method, system, and storage medium of the embodiments of the present invention will be described in detail.

[0066] On the one hand, the embodiments of the present invention propose an electric vehicle intelligent scheduling system, as Figure 1 shown. The electric vehicle intelligent scheduling system includes an upper-layer optimization module 100, a lower-layer optimization module 200, and an integration module 300. The upper-layer optimization module 100 and the lower-layer optimization module 200 perform data interaction and decision coordination through the integration module 300. The system uses a two-layer optimization structure to optimize electric vehicle charging. The upper-layer optimization module 100 optimizes the charging strategy based on the PPO reinforcement learning algorithm and is mainly responsible for determining the overall adjustable load and charging time allocation of the system. The lower-layer optimization module 200 is the optimization of charging station allocation based on Web services and is responsible for specific charging station allocation and real-time scheduling management. The two layers perform data interaction and decision coordination through the integration module 300, realizing a complete optimization process from the global policy to the specific execution. The system is also equipped with an intuitive Web visualization interface, supporting real-time monitoring and human-computer interaction, making the entire charging optimization process controllable and visible.

[0067] Specifically, the upper-layer optimization module 100 adopts the PPO reinforcement learning algorithm and realizes intelligent decision-making through the Actor-Critic policy network. Among them, the Actor-Critic policy network includes an Actor network and a Critic network. The Actor network adopts a Transformer architecture with a self-attention mechanism, which can better capture the temporal characteristics of the charging strategy. The Critic network adopts a multi-layer perceptron structure (MLP) to evaluate the state value and provide a reference for policy optimization.

[0068] Environmental design of the upper-layer optimization module 100: For the process of optimizing the charging schedule of electric vehicles, each electric vehicle is regarded as an agent, and an environment simulating the charging of electric vehicles is designed. The state space of this environment includes state information such as the battery charge state (SOC) of each electric vehicle, the remaining power demand, the expected departure time, the current electricity price signal, and the number of available charging piles in the charging station. These state information help the model comprehensively understand the current environment in order to make reasonable decisions. The action space defines the charging strategies that the agent can take, including charging decisions (whether to charge), the amount of charge, and the charging rate, etc. The agent selects these actions through the Actor-Critic policy network, which directly affects the charging state of the vehicle and the grid load. The state transition of the vehicle is realized through the state transition matrix, simulating the change of the vehicle's charging state. The Actor-Critic policy network is also set with a reward function, which calculates the reward value according to factors such as the adjustable load amount, the electricity price signal, and user satisfaction, comprehensively considering factors such as grid load limits, user preferences, and battery life.

[0069] Implementation process of the PPO reinforcement learning algorithm: The core logic of the PPO algorithm includes policy update and advantage function calculation. The policy update limits the update amplitude through policy clipping to ensure stable training. The policy loss function is:

[0070] L CLIP (θ) = E t [min(rt(θ)A t , clip(r t (θ), 1 - ∈, 1 + ∈)A t )];

[0071] Among them, L CLIP (θ) is the clipping loss function of the PPO algorithm, θ represents the parameters of the policy network, E t is the empirical expectation value at time step t, r t (θ) is the ratio of the new and old policy probabilities, A tis the advantage function estimation, clip() is the clipping function of the PPO algorithm, and ∈ is the policy clipping parameter. The advantage function is used to measure how good an action is relative to the average:

[0072] A t = δ t +(γλ)δ t+1 +…+(γλ) T-t+1 δ T-1 ;

[0073] where δ t is the temporal difference error, γ is the discount factor, λ is the parameter used to balance bias and variance, and T is the trajectory length (the total number of time steps in an episode. An episode is a complete sequence of interactions between the agent and the environment. Each episode is like a complete "game" or "trial", and the agent learns the optimal policy by repeatedly conducting such trials). The value function is used to estimate the long-term return of a state:

[0074]

[0075] where V(s t ) is the value function of state s t , is the mathematical expectation, r t+k is the immediate reward obtained at time step t + k, γ is the discount factor, and k is the time step from the current time onwards.

[0076] On the other hand, based on the electric vehicle intelligent scheduling system in the above-mentioned embodiment, the embodiment of the present invention also proposes an electric vehicle intelligent scheduling method, as Figure 2 shown, the method includes the following steps:

[0077] Step S100: Initialize the training parameters of the upper-layer optimization module 100;

[0078] Step S200: Initialize the training environment of the upper-layer optimization module 100; The training environment includes a state space and an action space. The state space includes the state information of each agent, and the action space includes the charging strategies that each agent can take. Each agent represents an electric vehicle;

[0079] Step S300: According to the state space, the Actor-Critic policy network determines the charging strategy taken by each agent, and calculates the reward value corresponding to the charging strategy according to the reward function;

[0080] Step S400: Update the state space. The upper-layer optimization module 100 optimizes the charging strategy through the PPO reinforcement learning algorithm, conducts multiple trainings until the training ends, and obtains a charging model;

[0081] Step S500: The upper - layer optimization module 100 provides a charging strategy for the electric vehicle to be charged through a charging model. The lower - layer optimization module 200 optimizes the charging cost and user satisfaction through an objective function according to the charging strategy, and allocates a charging station for the electric vehicle to be charged.

[0082] Specifically, in order to enable the upper - layer optimization module 100 to provide an optimal charging strategy, the upper - layer optimization module 100 needs to be trained. During training, multiple scenarios are simulated to optimize the strategy. The agent learns the optimal strategy through continuous trial - and - error. The experience storage is used to record the state, action, reward, and policy probability. These experiences are used for policy update to help the agent learn the optimal strategy through continuous trial - and - error. By updating the policy multiple times, the convergence and stability of the model are ensured. The result evaluation verifies the effectiveness of the model in different scenarios by analyzing the daily total adjustable load, the average hourly adjustable load, and the total reward over time.

[0083] During training, first, the training parameters are initialized. In this stage, it is necessary to set the basic parameters required for training, including key parameters such as the number of training episodes, the number of agents, the grid load limit, and the maximum charging time. At the same time, according to the actual electricity price policy, a day is divided into three periods: peak - valley, flat - peak, and peak, and different electricity price levels are set respectively to provide a price - guiding signal for the agents.

[0084] Then, the training environment is initialized. The training environment includes a state space and an action space. The state space includes the state information of each agent, and the action space includes the charging strategies that each agent can adopt. Each agent represents an electric vehicle. The system first initializes the state of charge (SOC) of each electric vehicle's battery to its initial value. At the same time, a random remaining power demand value is generated for each vehicle, and the expected departure time is set to simulate the diverse charging needs of users in the real scenario. In addition, the system also needs to initialize a charging - time counter to record and control the charging process. The action space defines the charging strategies that the agent can adopt, including charging decisions (whether to charge), the charging amount, and the charging rate, etc.

[0085] The constructed state space and action space are as follows:

[0086] S t =[SOC 1,t ,…,SOC N,t ,D 1,t ,…,D N,t ,T 1,t ,…,T N,t ,Y t ,N c,t T ;

[0087] ​Among them, SOC i,t is the battery charging state of the i-th agent at time t, D i,t is the remaining power demand of the i-th agent, T i,t is the estimated departure time of the i-th agent, Y t is the electricity price at time t, N c,t is the number of available charging piles at the charging station; N is the total number of agents;

[0088] The action space is:

[0089] A t = [d t , q t , r t T ;

[0090] Among them, d t is the charging strategy variable, q t is the charging amount, r t is the charging rate;

[0091] The state transition function of the agent is:

[0092]

[0093] Among them, η is the charging rate, C i is the battery capacity of the i-th agent;

[0094] L i,t+1 = L i,t + Vt;

[0095] Among them, L i,t is the charging time of the i-th agent, Vt is a fixed time interval (such as 5 minutes, 10 minutes, etc.);

[0096] P t+1 = ∑(q t * r t );

[0097] Among them, P t is the total load at time t.

[0098] ​State-action interaction is a core part of the training process. At each time step, the agent first observes the current environmental state, including information such as the vehicle's SOC, remaining electricity demand, estimated departure time, current electricity price, and the number of available charging piles at the charging station. Based on this state information, the Actor network outputs action probabilities to decide whether to charge the vehicle, the amount of charge, and the charging rate. After receiving these actions, the environment updates the state and returns the corresponding reward value. Reward calculation is the key mechanism to guide the agent's learning. The system comprehensively considers multiple factors such as the adjustable load, electricity price signal, user satisfaction, and battery life, and balances various optimization objectives by setting different weight coefficients. This multi-objective reward design can guide the agent to learn a charging strategy that not only meets user needs but also benefits grid operation. The reward function is as follows:

[0099] R(S t ,A t ) = w1Ruser(S t ,A t ) + w2Rload(S t ,A t ) + w3Rcost(S t ,A t );

[0100] where w1, w2, and w3 are weight coefficients;

[0101] Ruser(S t ,A t ) = α∑SOC(S t ) - β∑L(S t );

[0102] where α and β are weight coefficients related to user satisfaction, and SOC(S t ) is the battery charging state and charging time of the agent in the corresponding state;

[0103]

[0104] [[ID=4,4]]where l t is the actual load at time t, l target is the target load, and l max is the maximum load limit;

[0105] Rcost(S t ,A t ) = -Y t *d t *q t *r t .

[0106] The state transition matrix of the agent is constructed as:

[0107]

[0108] Among them, G ij is the probability of transferring from state i to state j, where i, j ∈ {0, 1}, 0 represents the non - charging state, and 1 represents the charging state. The calculation of the transition probability is based on the following rules:

[0109] When SOC < SOC max G 01 = 0.9, G 00 = 0.1, tending to start charging.

[0110] When SOC > SOC max G 01 = 0.2, G 00 = 0.8, tending to stop charging.

[0111] When in the high - electricity - price period, G 11 = 0.3, G 10 = 0.7, tending to stop charging.

[0112] In other cases, G 11 = 0.7, G 00 = 0.3, tending to continue charging.

[0113] By designing a reasonable reward function, multiple objectives such as grid constraints, user requirements, and economy are integrated into the optimization process, realizing the intelligent control of the charging process. The overall process is as Figure 3 shown.

[0114] The training and evaluation measure the performance of the model through multiple metrics. The system records metrics such as the total daily adjustable load, the average hourly adjustable load distribution, the change trend of the cumulative reward value, the user charging satisfaction, and the charging cost. By analyzing the changes in these metrics, the performance of the model in different scenarios can be evaluated to verify the effectiveness of the training. After multiple rounds of training, the model can learn to formulate reasonable charging strategies according to the time - period and electricity - price characteristics, achieving a balance among grid load, charging cost, and user requirements.

[0115] After the training is completed, the upper - layer optimization module 100 can provide the optimal charging strategy for the electric vehicles to be charged, while the lower - layer optimization module 200, according to the charging strategy, optimizes the charging cost and user satisfaction through the objective function and allocates charging stations for the electric vehicles to be charged. The objective function is as follows:

[0116]

[0117] Among them, C i is the charging cost of the i - th charging station, P iis the charging power of the i-th charging station, S i is the service cost of the i-th charging station, α is the weight coefficient related to user satisfaction, and U is the user satisfaction score.

[0118] The constraint conditions of the objective function are:

[0119]

[0120] 0 ≤ SOC i ≤ 1;

[0121] Among them, represents the total charging power, GridLimit represents the maximum grid load limit, and x ij represents the charging allocation of the i-th agent at the j-th charging station (a binary variable that can be 0 or 1, indicating whether to charge at this charging station), and C j represents the capacity limit of the j-th charging station (i.e., the upper limit of the number of vehicles that can be served simultaneously).

[0122] Through the above constraint conditions, 1) ensure that the total charging power does not exceed the maximum grid load limit and ensure that the use of each charging station does not exceed its capacity; 3) ensure that the battery charge state of each vehicle is within a reasonable range.

[0123] At the same time, in the lower-level optimization module 200, the charging station allocation strategy selects the best charging station for allocation by calculating the satisfaction score between each vehicle and the available charging stations. The satisfaction score comprehensively considers multiple key factors. First, geographical distance is an important factor affecting satisfaction. Generally, the closer the distance, the higher the satisfaction. Therefore, the system preferentially selects the charging station with a shorter distance. Second is the cost structure, including charging cost and service cost, which directly affects the user's choice. The system will preferentially select the charging station with a lower cost to improve user satisfaction. In addition, the matching of charging types is also an important consideration. Different vehicles may require different types of charging piles (such as fast charging or slow charging), and the system will match the appropriate charging station type according to the vehicle's charging needs. The availability of the charging station is also a key factor. The system will check the current state of the charging station to ensure its availability. User preferences are also taken into account. Some users may have specific preferences, such as a faster charging speed or a specific brand of charging station. The system can make personalized recommendations based on the user's historical selection and preference data. Finally, to avoid overloading of certain charging stations, the system will try to balance the load of each charging station to ensure the rational use of resources. By comprehensively considering the above factors, the system calculates the satisfaction score between each vehicle and the charging station and selects the charging station with the highest score for allocation.

[0124] In addition, the lower-layer optimization module 200 also maintains a complete database structure, which contains detailed information about charging stations and vehicles, such as charging station status, charging type, fee standard, and key data such as vehicle SOC and adjustable capacity. During the optimization process, according to the adjustable load constraints provided by the upper-layer optimization module 100, combined with the geographical location, usage status, and service capabilities of the charging stations, the most suitable charging station is assigned to each vehicle that needs to be charged. At the same time, the system also implements a complete data management function, supporting functions such as adding, deleting, modifying, querying, and data export of charging station and vehicle information, as Figure 4 and Figure 5 shown

[0125] According to the electric vehicle intelligent scheduling method of the embodiment of the present invention, during operation, the upper-layer optimization module 100 first generates a charging strategy and an adjustable load upper limit according to the system state, and these decision results are transmitted to the lower-layer optimization module in real time. The lower-layer optimization module 200 then executes specific charging station allocation tasks according to these constraint conditions, combined with the actual distribution of charging stations and vehicle demands. The system continuously runs the optimization process through background threads and displays the optimization results in real time through a Web interface, including information such as vehicle charging status and charging station usage, forming a complete closed-loop optimization system.

[0126] The electric vehicle intelligent scheduling method according to the embodiment of the present invention has the following advantages:

[0127] (1) It combines the charging strategy optimization of the upper-layer optimization module 100 based on the PPO reinforcement learning algorithm and the charging station allocation optimization of the lower layer based on Web services. The upper layer realizes intelligent decision-making through the Actor-Critic network structure to ensure the effectiveness and stability of the global strategy; the lower layer is responsible for the real-time allocation and management of charging station resources, realizing a complete optimization process from the global strategy to the specific execution.

[0128] (2) Through the real-time response and user satisfaction improvement mechanism, the lower-layer optimization module 200 realizes flexible resource scheduling and management by real-time monitoring the usage status of charging stations and grid load, combined with the dynamic adjustment of the strategy of the upper-layer optimization module 100, improving the overall efficiency and response speed of the system. At the same time, the system comprehensively considers the personalized needs and preferences of users in the allocation strategy, calculates the satisfaction score through multiple factors, and provides personalized charging station recommendations, significantly improving user satisfaction.

[0129] (3) Through the integrated module 300, data interaction and decision collaboration between the upper and lower layers are realized, ensuring a high degree of consistency between the global strategy and the specific execution. The system forms a closed-loop optimization structure, supports real-time monitoring and human-computer interaction, and provides an efficient, flexible, and user-friendly electric vehicle charging optimization solution.

[0130] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned intelligent scheduling method for electric vehicles is implemented.

[0131] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include memories remotely located relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] Those of ordinary skill in the art can understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or can be implemented as hardware, or can be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory, or other memory technologies, CD-ROM, digital versatile disk (DVD), or other optical disk storage, magnetic cassette, tape, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium generally includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0133] The above is a specific description of the preferred embodiment of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of the present invention.

Claims

1. An intelligent scheduling method for electric vehicles, characterized in that, Applied to an electric vehicle intelligent scheduling system, the electric vehicle intelligent scheduling system includes an upper-layer optimization module, a lower-layer optimization module, and an integration module. The upper-layer optimization module and the lower-layer optimization module perform data interaction and decision-making collaboration through the integration module. The electric vehicle intelligent scheduling method includes the following steps: Initialize the training parameters of the upper-layer optimization module; Initialize the training environment of the upper-layer optimization module; the training environment includes a state space and an action space. The state space includes the state information of each agent, and the action space includes the charging strategies that each agent can adopt. Each agent represents an electric vehicle; According to the state space, the Actor-Critic policy network determines the charging strategy adopted by each agent and calculates the reward value corresponding to the charging strategy according to the reward function; Update the state space. The upper-layer optimization module optimizes the charging strategy through the PPO reinforcement learning algorithm, conducts multiple trainings until the training ends, and obtains a charging model; The upper-layer optimization module provides a charging strategy for the electric vehicle to be charged through the charging model. The lower-layer optimization module optimizes the charging cost and user satisfaction through the objective function according to the charging strategy and allocates a charging station for the electric vehicle to be charged.

2. The intelligent scheduling method for electric vehicles according to claim 1, wherein The state space includes the battery charging state, remaining power demand, estimated departure time, current electricity price signal of the agent, and the number of available charging piles at the charging station.

3. The intelligent scheduling method for electric vehicles according to claim 1, wherein The Actor-Critic policy network includes an Actor network and a Critic network. The Actor network adopts a Transformer architecture, and the Critic network adopts a multi-layer perceptron structure.

4. The intelligent scheduling method for electric vehicles according to claim 1, characterized in that The step of initializing the training parameters of the upper-layer optimization module includes: Initialize the number of training rounds, the number of agents, the grid load limit, and the maximum charging time; According to the actual electricity price policy, divide a day into three periods: peak-valley, flat peak, and peak, and set different electricity prices for each period.

5. The intelligent scheduling method for electric vehicles according to claim 1, characterized in that The state space is: S t = [SOC 1,t ,…,SOC N,t ,D 1,t ,…,D N,t ,T 1,t ,…,T N,t ,Y t ,N c,t T ​ Among them, SOC i,t is the battery charge state of the i-th agent at time t, D i,t is the remaining power demand of the i-th agent, T i,t is the expected departure time of the i-th agent, Y t is the electricity price at time t, N c,t is the number of available charging piles at the charging station, and N is the total number of agents; The action space is: A t = [d t , q t , r t T ;​ Among them, d t is the charging strategy variable, q t is the charging amount, r t is the charging rate; The state transition function of the agent is: where η is the charging rate, and C i is the battery capacity of the i-th agent; L i,t+1 = L i,t + Vt; Among them, L i,t is the charging time of the i-th agent, and Vt is a fixed time interval; P t+1 = ∑(q t * r t )); Among them, P t is the total load at time t.

6. The intelligent scheduling method for electric vehicles according to claim 5, wherein, The reward function is: R(S t ,A t ) = w1Ruser(S t ,A t ) + w2Rload(S t ,A t ) + w3Rcost(S t ,A t ); Wherein, w1, w2, and w3 are weight coefficients; Ruser(S t ,A t ) = α∑SOC(S t ) - β∑L(S t ); where α and β are weight coefficients related to user satisfaction, and SOC(S t ) is the battery charge state of the agent in the corresponding state, and L(S t ) is the charging time of the agent in the corresponding state; where, l t is the actual load at time t, l target is the target load, and l max is the maximum load limit; Rcost(S t ,A t ) = -Y t *d t *q t *r t 。 7. The intelligent scheduling method for electric vehicles according to claim 5, characterized in that The objective function is: Among them, C i is the charging cost of the i-th charging station, P i is the charging power of the i-th charging station, S i is the service cost of the i-th charging station, α is the weight coefficient related to user satisfaction, and U is the user satisfaction score.

8. The intelligent scheduling method for electric vehicles according to claim 7, characterized in that, The constraint conditions of the objective function are: 0 ≤ State of Charge i ≤ 1; Among them, represents the total charging power, GridLimit represents the maximum grid load limit, and x ij represents the charging allocation of the i-th agent at the j-th charging station, and C j represents the capacity limit of the j-th charging station.

9. An intelligent scheduling system for electric vehicles, characterized in that, Including an upper-layer optimization module, a lower-layer optimization module, and an integration module. The upper-layer optimization module and the lower-layer optimization module perform data interaction and decision-making collaboration through the integration module. The electric vehicle intelligent scheduling system is used to execute the electric vehicle intelligent scheduling method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute the electric vehicle intelligent scheduling method according to any one of claims 1 to 8.

Images