An optimization method for charging scheduling of electric vehicles

Abstract

The present invention discloses an electric vehicle charging scheduling optimization method, which relates to the technical field of power resource scheduling optimization, including constructing an electric vehicle charging problem model; initializing a state; obtaining and updating an MPC-based strategy and an NN-based strategy; obtaining a baseline action and an action generated by machine learning, and calculating a perception radius; obtaining a new scheduling action, sampling to obtain the next state, and obtaining an estimated value of the system state at time t through a formula; storing information in a replay buffer D, and updating a charging environment according to the scheduling action; the present invention adopts the above-mentioned electric vehicle charging scheduling optimization method, which can optimize the electric vehicle charging strategy and improve the overall performance of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of power resource scheduling optimization, and in particular to an electric vehicle charging scheduling optimization method. Background Art

[0002] In the era of electric vehicles (EVs), developing efficient charging solutions to optimize EV operations has become a key challenge, especially for EV charging stations that integrate renewable energy. This adaptive charging problem is further complicated by the uncertainty introduced by renewable energy generation and human behavior. For example, the widespread installation of rooftop photovoltaic (PV) solar systems at workplace charging stations improves sustainability but also introduces variability in the operation of solar-enabled EV charging systems. The heterogeneous charging characteristics of non-commercial EVs determined by human driver behavior and battery specifications further exacerbate the uncertainty of charging dynamics.

[0003] To cope with the resulting uncertainty, there has been increasing interest in recent years in developing advanced charging schemes using data-driven and machine learning methods. In simulation scenarios, these techniques have demonstrated superior performance over traditional scheduling algorithms such as Earliest Deadline First (EDF), Least Slack First (LLF), and predictive scheduling, especially when these traditional methods have difficulty learning renewable energy generation and human behavior statistics. However, existing learning-based charging strategies require high-quality training data and may fail when the data distribution changes over time. This highlights the concept of statistical distribution shift (Out-Of-Distribution / OOD), which is a key issue in machine learning, where the data faced by the model is significantly different from its training set.

[0004] In general, both advanced learning-based algorithms and control-based algorithms are still some distance away from the optimal solution in actual electric vehicle charging tasks, which is specifically manifested in the following aspects: (1) Learning-based algorithms can achieve higher scheduling efficiency on average, but cannot guarantee the worst-case performance due to distribution offset; (2) Model predictive control (MPC) scheduling is more robust in non-stationary environments, but is still suboptimal. Summary of the invention

[0005] The present invention proposes an electric vehicle charging scheduling optimization method to solve the problems mentioned in the above background technology.

[0006] To achieve the above object, the present invention provides an electric vehicle charging scheduling optimization method, comprising the following steps:

[0007] S1. Build an electric vehicle charging problem model and define the state vector Represents the state and action vector of the system at time t represents the scheduling action and scheduling strategy π of the system at time t;

[0008] S2, initialization state , MPC (Model Predictive Control)-based strategy and NN (Neural Network)-based strategies Go to the replay buffer D to collect the preliminary data input by the system and the user ;

[0009] S3, obtaining and updating the MPC-based strategy and the NN-based strategy;

[0010] S4. Obtain baseline actions based on MPC strategy and system status , based on NN’s strategy and system status, we get machine learning-generated actions , calculate the perception radius ;

[0011] S5. Projection to baseline motion Center, radius On the sphere, get a new dispatch action , sample to get the next state ;

[0012] S6, store information to replay buffer D, according to the scheduling action Update charging environment.

[0013] Preferably, the specific steps of S1 include:

[0014] S11. Define the problem and parameters: Define the number of charging piles as m, and each discrete time step ; Each new electric car j will trigger a charging task, represented as a tuple , where at time , the electric car arrives at the charging station , the battery capacity is , and at time Leave, use Represents the set of all charging tasks. is random and non-stationary, defined as represents the battery charging status of m charging piles, Indicates at time The electric energy allocated to m charging piles at the time is defined as the state vector and motion vector ,symbol Represents the concatenation of two vectors, and the state space is , the action space is ;

[0015] S12, State Space and action space It is used to define the constraints of battery capacity and charging rate. The constraint formula is as follows:

[0016] ;

[0017] ;

[0018] The charging status is updated according to the dynamic changes of the battery;

[0019] S13. Construct an OOD (Out Of Distribution) electric vehicle charging problem model. The dynamic equation of charging is as follows:

[0020] ;

[0021] in, Indicates battery dynamics, represents user behavior / solar disturbance, is a safe function when and hour, ,otherwise ,function Used to project the state into the state space On, Event Indicates that the current active charging record ends and the electric vehicle will leave; therefore The i-th entry of is reset to 0; is the action projection function, Represents the time interval between two charging controls, , is the matrix of charging dynamics, the system matrix used to model the time-varying behavior of non-ideal batteries, , is a block matrix, defined as follows:

[0022] ;

[0023] ;

[0024] Where m is the number of rows in the matrix. and represent the charging efficiency and control efficiency coefficient respectively, is a matrix whose elements are all 0. is a random disturbance caused by human charging behavior, so that when Is a charging record and When a new charging session starts, , is a disturbance caused by the sun, Indicates time The solar energy input of the i-th charging pile at time ; Represents a matrix with m rows and m columns, with all diagonal elements being 1 and all other elements being 0.

[0025] S14. Design a secondary charging cost function to optimize the overall charging strategy, which is as follows:

[0026] ;

[0027] in, For charging cost, and is a positive definite matrix of time-varying charging costs;

[0028] This total cost penalizes excessive use of installed batteries while encouraging the use of solar power. This quadratic cost function can effectively balance the use of batteries and solar input and optimize the overall charging strategy.

[0029] Preferably, the specific steps of S3 are as follows:

[0030] S31. Obtaining an MPC-based strategy: Assuming the predicted length is k, Indicates the maximum predicted departure time based on user input. is min{t+k,T-1}, receiving the state parameter , by solving the MPC optimization problem, we can obtain the MPC-based strategy , that is, a given state Can get action , and the corresponding strategy is recorded as ;

[0031] In practice, this method can effectively deal with the uncertainty and disturbance factors in electric vehicle charging through dynamic prediction and optimization, and provide an optimized scheduling strategy for the charging process;

[0032] S32. Obtain NN-based strategy:

[0033] First, define the Q value function estimated at time t , ε is the metric error between the Q-value functions, using The norm is used to measure the difference between the trained Q-value function and the optimal Q-value function, as shown below:

[0034] ;

[0035] is the optimal Q value function, which is defined as follows:

[0036] ;

[0037] Initialize hyper parameters and learning rate η, randomly select a small batch of data B from the dataset D, use the trajectory record h and gradient in batch B to update the estimated Q value function by gradient descent ,

[0038] in , where s represents the state and a represents the action. represents the state at the next moment, c represents the cost, which is used to quantify the cost or benefit of action a in state s. represents a trajectory / history that records the state of the system at a particular time step, the actions taken, the subsequent states, and the costs or rewards obtained, where Describes a complete interaction fragment, which helps analyze state transitions and policy effects;

[0039] The gradient looks like this:

[0040] ,

[0041] Update hyperparameters ;

[0042] Update to get NN-based strategy ;

[0043] S33, obtain .

[0044] Preferably, the MPC-based strategy in S31 is specifically as follows:

[0045] First define For in time The random charging behavior disturbance received at A series of predictions, defining is the forecast of future solar power generation received at time t, with a forecast horizon of k, and ; At each time t, define the predicted disturbance ,in and They represent the predicted values ​​of charging behavior disturbance and solar power generation, is the time interval between two charging controls;

[0046] The MPC's estimate of the current state at time t is as follows:

[0047] ;

[0048] Describes how the system moves from its previous state By performing actions and subject to prediction disturbance The impact of transfer to the current state .

[0049] The departure time entered is and battery capacity Probably related to the charging record ancestor The true value of is different. A safety projection function based on the input departure time Estimates.

[0050] The MPC optimization problem is as follows:

[0051] ;

[0052] ;

[0053] ;

[0054] in,( : ) is the time from time t to Future disturbance predictions, is a customized terminal cost used to regularize the final predicted state, is the terminal matrix, is the estimated value of the current state;

[0055] The goal of this optimization problem is to adjust the action sequence from time t to t'-1 given the future disturbance prediction and the terminal cost. , minimizing the cumulative cost of the system.

[0056] Preferably, the specific steps of S4 are as follows:

[0057] S41. Obtain baseline actions by executing MPC-based strategies ;

[0058] S42, by executing the NN-based strategy to minimize the Q-value function Get the action generated by machine learning ;

[0059] S43, calculate the perception radius ;

[0060] ;

[0061] Estimate the temporal difference (TD) error based on the historical trajectory in the replay buffer D:

[0062] ;

[0063] in, is a tuning parameter used to adjust the impact of the estimated temporal difference error on the overall update.

[0064] Preferably, the specific content of S5 is as follows:

[0065] Get the new dispatch action The formula is as follows:

[0066] ;

[0067] in, , Defines a As the center, is the Euclidean sphere with radius

[0068] Sample to get the next state ;

[0069] Preferably, the information stored in S6 is .

[0070] Therefore, the present invention adopts the above-mentioned electric vehicle charging scheduling optimization method, which has the following beneficial effects:

[0071] (1) An OOD-aware strategy is introduced, which can not only achieve near-optimal average performance under unstable conditions, but also ensure robust performance when the distribution changes, which has performance improvements compared to MPC-based scheduling;

[0072] (2) The application scope of the learning enhancement strategy is expanded to solve the complexity of electric vehicle scheduling in the specific environment of solar electric vehicle charging. Compared with MPC, it is a scheduling strategy that is more suitable for real scenarios.

[0073] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The present invention is a flow chart of an electric vehicle charging scheduling optimization method. DETAILED DESCRIPTION

[0075] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] Example

[0077] like Figure 1 As shown, a method for optimizing charging scheduling of electric vehicles comprises the following steps:

[0078] S1. Build an electric vehicle charging problem model and define the state vector Represents the state and action vector of the system at time t Represents the scheduling action and scheduling strategy π of the system at time t.

[0079] S11. Define the problem and parameters: Define the number of charging piles as m, and at each discrete time step , new electric vehicles may arrive, and the vehicles that previously occupied the charging pile may leave. Each new electric vehicle j will trigger a charging task, represented as a tuple , where at time , the electric car arrives at the charging station , the battery capacity is , and at time Leave, use Represents the set of all charging tasks. is random and non-stationary, which is caused by the out-of-distribution (OOD) problem. Assume that the charging task set is known.

[0080] definition Represents the battery charging state of m charging piles, (SoC, State-of-Charge), that is:

[0081] When an electric vehicle is charging at the i-th charging pile, if >0, then Indicates the amount of electricity that needs to be delivered (unit: kilowatt-hour, kWh);

[0082] like <0, indicating the battery is charging;

[0083] like =0, indicating that there is no charging task at present.

[0084] Indicates at time The power allocated to m charging piles at a time, each charging pile has a charging limit > 0, so for all i and t, ≤ In addition, the charging rate must also meet the total energy constraint, that is, ,here Represents the upper bound of the charging rate.

[0085] Define the state vector ,in Contains the charging rates of all m charging piles, and n=2m, symbol Represents the concatenation of two vectors.

[0086] Defining the action vector , where each component Controls changes in charging rate from connection to the grid.

[0087] S12, State Space and action space It is used to define the constraints of battery capacity and charging rate. The constraint formula is as follows:

[0088] ;

[0089] ;

[0090] The charging status is updated based on the dynamic changes of the battery.

[0091] However, due to the plug-in and unplug operations of electric vehicle charging piles and the uncertain disturbance of solar energy input, the system is susceptible to out-of-distribution (OOD) conditions. Based on this, a model of electric vehicle charging problem considering OOD problems is introduced below.

[0092] S13. Construct an OOD problem model for electric vehicle charging. The dynamic equation for charging is as follows:

[0093] ;

[0094] in, Indicates battery dynamics, represents user behavior / solar disturbance, is a security function, which is defined as: and hour, ,otherwise Here, reuse the symbol ,when Appears in a tuple of charging records, function Used to project the state into the state space On, Event Indicates that the current active charging record ends and the electric vehicle will leave; therefore The i-th entry of is reset to 0; similarly, ,This projection operation is used to ensure that states and actions are within their ,restricted spaces, thereby ensuring the safety and feasibility of the system at ,runtime.

[0095] This function is used to ensure the status Satisfy the safety constraints of the system. Specifically, it will input the state Projection to the state space that meets safety requirements Such safety functions are often used in reinforcement learning or control systems to ensure that the system operates within acceptable limits and does not violate any physical or safety constraints.

[0096] Represents the time interval between two charging controls, , is the matrix of charging dynamics, the system matrix used to model the time-varying behavior of non-ideal batteries, , is a block matrix, defined as follows:

[0097] ;

[0098] ;

[0099] Where m is the number of rows in the matrix. and represent the charging efficiency and control efficiency coefficient respectively, is a matrix whose elements are all 0. is a random disturbance caused by human charging behavior, so that when Is a charging record and When a new charging session starts, , is a disturbance caused by the sun, Indicates time is the solar energy input of the i-th charging pile.

[0100] At every time The future disturbance in the charging dynamics equation ( : t) is unknown, and the previous solar power generation situation ( ) are known, and in addition, the parameters of the dynamic system , Security Function and action function All given in advance.

[0101] S14. Design a secondary charging cost function to optimize the overall charging strategy, which is as follows:

[0102] ;

[0103] in, For charging cost, and is a positive definite matrix of time-varying charging costs, applicable to each time step .

[0104] This total cost penalizes excessive use of installed batteries while encouraging the use of solar power. This quadratic cost function can effectively balance battery use with solar input and optimize the overall charging strategy.

[0105] S2, initialization state , MPC-based strategy and NN-based strategies Go to the replay buffer D to collect the preliminary data input by the system and users .

[0106] S3. Obtain and update the MPC-based strategy and the NN-based strategy.

[0107] S31. Obtaining an MPC-based strategy: Assuming the predicted length is k, Indicates the maximum predicted departure time based on user input. is min{t+k,T-1}, receiving the state parameter , by solving the MPC optimization problem, we can obtain the MPC-based strategy , that is, a given state Can get action , and the corresponding strategy is recorded as ; The details are as follows:

[0108] First define For in time The random charging behavior disturbance received at A series of predictions, defining is the forecast of future solar power generation received at time t, with a forecast horizon of k, and ; At each time t, define the predicted disturbance ,in and They represent the predicted values ​​of charging behavior disturbance and solar power generation, is the time interval between two charging controls;

[0109] The MPC's estimate of the current state at time t is as follows:

[0110] ;

[0111] Describes how the system moved from its previous state By performing actions and subject to prediction disturbance The impact of transfer to the current state In this way, MPC can make a real-time estimate of the system state based on the latest prediction information at each time step, and optimize the next control decision accordingly, thereby improving the system's response speed and control accuracy.

[0112] The departure time entered is and battery capacity Probably related to the charging record ancestor The true value of is different. A safety projection function based on the input departure time Estimates.

[0113] Given an estimate of the current system state , from time t to Future disturbance predictions , assuming >t, and the terminal matrix .

[0114] The MPC optimization problem is as follows:

[0115] ;

[0116] ;

[0117] ;

[0118] in,( : ) is the time from time t to Future disturbance predictions, is a customized terminal cost used to regularize the final predicted state, is the terminal matrix, is the estimated value of the current state;

[0119] The goal of this optimization problem is to adjust the action sequence from time t to t'-1 given the future disturbance prediction and the terminal cost. , minimizing the cumulative cost of the system.

[0120] At each time step, MPC solves a finite-time optimization problem with k-step forecasts and selects the first control action from the optimal solution to execute. The terminal cost matrix It is pre-set.

[0121] In practice, this method can effectively deal with the uncertainties and disturbances in electric vehicle charging through dynamic prediction and optimization, and provide an optimized scheduling strategy for the charging process.

[0122] S32. Obtain NN-based strategy:

[0123] First, define the Q value function estimated at time t , ε is the metric error between the Q-value functions, using The norm is used to measure the difference between the trained Q-value function and the optimal Q-value function, as shown below:

[0124] ;

[0125] is the optimal Q value function, which is defined as follows:

[0126] ;

[0127] And satisfy the Bellman optimality equation.

[0128] The process of generating an estimated Q-value function becomes the NN-update process, that is, obtaining a NN-based strategy.

[0129] Using the Deep Deterministic Policy Gradient (DDPG) algorithm as an example of this process, we show how to update the hyperparameters of the neural network policy and the Q-value function and provide an estimate of , by the parameter However, the method is not limited to the specific policy update rules in S32. Any reinforcement learning algorithm that can update the value function can be used as the NN-update process.

[0130] Initialize hyper parameters and learning rate η, randomly select a small batch of data B from the dataset D, use the trajectory record h and gradient in batch B to update the estimated Q value function by gradient descent ,

[0131] in , where s represents the state and a represents the action. represents the state at the next moment, c represents the cost, which is used to quantify the cost or benefit of action a in state s. represents a trajectory / history that records the state of the system at a particular time step, the actions taken, the subsequent states, and the costs or rewards obtained, where A complete interaction fragment is described, which is helpful for analyzing state transitions and policy effects.

[0132] The gradient looks like this:

[0133] ,

[0134] Update hyperparameters ;

[0135] Update to get NN-based strategy ;

[0136] S33, obtain .

[0137] definition for , are the parameters required by the MPC strategy.

[0138] Store the previous charging trajectory in the replay buffer D, and at each time step t, use it for input prediction will be collected.

[0139] S4. Obtain baseline actions based on MPC strategy and system status , based on NN’s strategy and system status, we get machine learning-generated actions , calculate the perception radius .

[0140] S41. Obtaining baseline actions by executing MPC-based strategies ;

[0141] S42, by executing the NN-based strategy to minimize the Q-value function Get the action generated by machine learning ;

[0142] S43, calculate the perception radius ;

[0143] ;

[0144] Estimate the temporal difference (TD) error based on the historical trajectories in the replay buffer D:

[0145] ;

[0146] in, is a tuning parameter used to adjust the impact of the estimated temporal difference error on the overall update.

[0147] S5. Projection to baseline motion Center, radius On the sphere, get a new dispatch action , sample to get the next state , the estimated value of the system state at time t is obtained through the formula .

[0148] Get the new dispatch action The formula is as follows:

[0149] ;

[0150] in, , Defines a As the center, is the Euclidean sphere with radius;

[0151] Sample to get the next state ;

[0152] S6. Storage Information To the replay buffer D, according to the scheduling action Update charging environment.

[0153] This method can be called repeatedly in an episodic setting to update the deep value-based policy.

[0154] Therefore, the present invention adopts the above-mentioned electric vehicle charging scheduling optimization method, which combines the advantages of model predictive control (MPC) and reinforcement learning, and can optimize the electric vehicle charging strategy and improve the overall performance of the system when dealing with uncertain system states and predictions based on user input.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for optimizing charging scheduling of electric vehicles, characterized in that: The following steps are involved: S1. Build an electric vehicle charging problem model and define the state vector Represents the state and action vector of the system at time t Represents the scheduling action and scheduling strategy π of the system at time t; the specific steps include: S11. Define the problem and parameters: Define the number of charging piles as m, and each discrete time step ; Each new electric car j will trigger a charging task, represented as a tuple , where at time , the electric car arrives at the charging station , the battery capacity is , and at time Leave, use Represents the set of all charging tasks, definition represents the battery charging status of m charging piles, Indicates at time The electric energy allocated to m charging piles at the time is defined as the state vector and motion vector ,symbol Represents the concatenation of two vectors, and the state space is , the action space is ; S12, State Space and action space It is used to define the constraints of battery capacity and charging rate. The constraint formula is as follows: ； ； S13. Construct an OOD problem model for electric vehicle charging. The dynamic equation for charging is as follows: ； in, Indicates battery dynamics, Indicates user behavior. is a safe function. is the action projection function, Represents the time interval between two charging controls, , For the matrix of charging dynamics, , is a block matrix, defined as follows: ； ； Where m is the number of rows in the matrix. and Represent the charging efficiency and control efficiency coefficient respectively, is a matrix whose elements are all 0. It is a random disturbance caused by human charging behavior. is a disturbance caused by the sun, Represents a matrix with m rows and m columns, the diagonal is 1, and the rest are 0; S14. Design a secondary charging cost function to optimize the overall charging strategy, which is as follows: ； in, For charging cost, and is a positive definite matrix of time-varying charging costs; S2, initialization state , MPC-based strategy and NN-based strategies Go to the replay buffer D to collect the preliminary data input by the system and users ; S3, obtaining and updating the MPC-based strategy and the NN-based strategy; S4. Obtain baseline actions based on MPC strategy and system status , based on NN’s strategy and system status, we get machine learning-generated actions , calculate the perception radius ; S5. Projection to baseline motion Center, radius On the sphere, get a new dispatch action , sample to get the next state ; S6, store information to replay buffer D, according to the scheduling action Update charging environment.

2. The method for optimizing charging scheduling of electric vehicles according to claim 1, characterized in that: The specific steps of S3 are as follows: S31. Obtaining an MPC-based strategy: Assuming the predicted length is k, Indicates the maximum predicted departure time based on user input. is min{t+k,T-1}, receiving the state parameter , by solving the MPC optimization problem, we can obtain the MPC-based strategy , for a given state Can get action , and the corresponding strategy is recorded as ; S32. Obtain NN-based strategy: First, define the Q value function estimated at time t , and define ε as the metric error between the Q-value functions, using The norm is used to measure the difference between the trained Q-value function and the optimal Q-value function, as shown below: ； is the optimal Q value function, which is defined as follows: ； Initialize hyper parameters and learning rate η, randomly select a small batch of data B from the dataset D, use the trajectory record h and gradient in batch B to update the estimated Q value function by gradient descent , in , where s represents the state and a represents the action. represents the state at the next moment, c represents the cost, Represents a trajectory, and the gradient is as follows: ； Update hyperparameters ; Update to get NN-based strategy ; S33, obtain .

3. The method for optimizing charging scheduling of electric vehicles according to claim 2, characterized in that: The MPC-based strategy in S31 is as follows: First define For in time The random charging behavior disturbance received at A series of predictions, defining is the forecast of future solar power generation received at time t, with a forecast horizon of k, and ; At each time t, define the predicted disturbance ,in and They represent the predicted values ​​of charging behavior disturbance and solar power generation, is the time interval between two charging controls; The MPC's estimate of the current state at time t is as follows: ； in, A safety projection function based on the input departure time estimates; The MPC optimization problem is as follows: ； ; ; in, : From time t to Future disturbance predictions, is a customized terminal cost used to regularize the final predicted state, is the terminal matrix, is an estimate of the current state.

4. The method for optimizing charging scheduling of electric vehicles according to claim 1, characterized in that: The specific steps of S4 are as follows: S41. Obtain baseline actions by executing MPC-based strategies ; S42, by executing the NN-based strategy to minimize the Q-value function Get the action generated by machine learning ; S43, calculate the perception radius ; ； Estimate the time difference error based on the historical trajectory in the replay buffer D: ； in, is a tuning parameter used to adjust the impact of the estimated temporal difference error on the overall update.

5. The method for optimizing charging scheduling of electric vehicles according to claim 4, characterized in that: The specific content of S5 is as follows: Get the new dispatch action The formula is as follows: ； in, , Defines a As the center, is the Euclidean sphere with radius; Sample to get the next state .

6. The method for optimizing charging scheduling of electric vehicles according to claim 1, characterized in that: The information stored in S6 is .

Images