A method for estimating electric vehicle travel demand and placing traffic sensors

Abstract

The present invention discloses a method for estimating electric vehicle travel demand and placing traffic sensors. First, historical departure-destination travel demand data, historical charging load and road section flow are collected; second, the collected data is used to train the deep reinforcement learning agent and the Transformer of the decoder; then, the selected typical day of charging load is input into the trained agent to generate a traffic sensor placement plan, and then the traffic road flow data observed by the sensor is input into the Transformer of the encoder, and the Transformer of the encoder is trained according to the regression loss to obtain the final departure-destination accurate estimation value; finally, the obtained departure-destination estimation value is used as input, and the trained fitting traffic distribution network is used to obtain the final traffic distribution and charging load estimation value. Through the above steps, the present invention obtains the departure-destination demand estimation and traffic sensor placement method for electric vehicle traffic flow in smart grid and intelligent transportation system, which overcomes the inherent complexity of the three-level framework, accurately estimates the departure-destination demand of electric vehicles and the key locations for placing traffic sensors, thereby effectively managing smart grid and intelligent transportation system.

Description

Technical Field

[0001] The invention belongs to the field of smart grid and traffic network, and in particular relates to a method for estimating the departure-destination demand of electric vehicle traffic flow and placing traffic sensors. Background Art

[0002] With the acceleration of urbanization and the increasing popularity of electric vehicles, making full use of data collected by smart grids and intelligent transportation networks has become key to effectively manage transportation and smart grids. Accurately estimating origin-destination demand and placing electric vehicle traffic sensors have become crucial. However, solving the complexity of coordinating electric vehicle origin-destination demand estimation and traffic sensor placement faces important research challenges. The interdependence between these two factors requires a comprehensive framework to capture the intricate relationship between traffic flow distribution and demand estimation.

[0003] Although some studies have attempted to analyze this complex problem, there are still some limitations. Previous studies have either ignored the consideration of electric vehicle charging load in smart grids and failed to consider the impact of electric vehicle charging stations, resulting in suboptimal results; or relied on traditional algorithms that are difficult to handle the increasing size and complexity of large-scale transportation networks, affecting scalability and computational efficiency. Summary of the invention

[0004] To solve this problem, the present invention proposes a method for estimating the departure-destination demand and placing traffic sensors for electric vehicle traffic flow. The electric vehicle traffic sensor layout optimization and departure-destination demand estimation are integrated into a comprehensive three-layer structure. At the upper layer, the focus is on determining the optimal layout strategy for electric vehicle traffic sensors. The goal of this layer is to capture the complex distribution of traffic flow by strategically deploying sensors at key locations in the traffic network. Subsequently, the two-layer model acts as an intermediate layer to solve the inverse problem of traffic assignment. Given the limitations of traditional methods in dealing with the inherent complexity of this problem, the present invention introduces a deep learning-based solution to overcome these challenges. Specifically, deep reinforcement learning is used to determine the optimal location layout of traffic sensors to maximize the accuracy of departure-destination demand estimation under a preset budget. In addition, the present invention designs a Transformer-based encoder-decoder framework to approximate the solution to the two-layer inverse optimization problem, while utilizing the multi-head attention mechanism provided by the Transformer to capture the complex relationships therein.

[0005] The present invention adopts the following technical solutions:

[0006] A method for estimating electric vehicle travel demand and placing traffic sensors comprises: first, the present invention collects historical start-end point travel demand data and historical charging load and road section flow, and selects a typical day of charging load according to historical experience distribution; second, uses the collected data to train a deep reinforcement learning agent and a Transformer of a decoder, specifically: inputs the travel demand data, historical charging load and road section flow into the Transformer of the decoder, and trains the network to fit the traffic distribution problem; uses the historical charging load as observation, and the mean square error loss of travel demand estimation and traffic flow distribution as a reward, and the agent outputs a placement plan of traffic sensors to train the deep reinforcement learning agent; then, inputs the selected typical day of charging load into the trained agent to generate a traffic sensor placement plan, and inputs the traffic road flow data observed by the sensor into the Transformer of the encoder, and trains the Transformer of the encoder according to the regression loss to obtain the final departure-destination accurate estimation value; finally, uses the obtained departure-destination estimation value as input, and uses the trained fitting traffic distribution network to obtain the final traffic distribution and charging load estimation value. Through the above steps, the present invention obtains a method for estimating the departure-destination demand and placing traffic sensors for electric vehicle traffic flow in a smart grid and smart transportation system, which can accurately calculate the optimal location for placing traffic sensors and the travel demand of electric vehicles.

[0007] As a further improvement of the present invention, the historical charging load is used as the observation of the deep reinforcement learning agent, the error between the traffic distribution and charging load estimation output by the travel demand estimation and traffic assignment problem and the true value is used as a reward, and the location of the traffic sensor is decoupled from the electric vehicle travel demand estimation problem, thereby facilitating the solution process of training the deep reinforcement learning agent to output the sensor placement plan, including:

[0008] A deep genetic policy with no gradient training algorithm is established. The initial population is evaluated by randomly assigning parameters. The top T individuals are selected as the parents of the next generation to apply additive Gaussian noise to their network parameters. After multiple generations of iterations, the selection and deployment of traffic sensors based on expected rewards are improved. The training algorithm of the deep reinforcement learning agent is as follows:

[0009] (1) Initialize the first generation g0;

[0010] (2) For the gth generation, cycle from 1 to G;

[0011] (3) Evaluate the gth i-1 generation;

[0012] (4) From g i-1 In the generation, select T top individuals as the gthi The parent generation of the generation;

[0013] (5) For the population cycle from 1 to P;

[0014] (6) Randomly select a parent and add Gaussian noise to its parameters:

[0015] θ′=θ+σεθ~N(0,1)(A-3)

[0016] As a further improvement of the present invention, the selected typical day of charging load is input into the trained agent to generate a traffic sensor placement plan, including: establishing an objective function to minimize the overall reconstruction loss between the estimation and actual observation of traffic flow and electric vehicle charging load, and the objective function is:

[0017]

[0018] where u a Indicates whether a traffic sensor is installed in route a∈A, x a and They represent the estimated traffic flow of the route and the corresponding actual observation results, respectively. and are the estimated and actual observation of the electric vehicle charging load at the electric vehicle charging station k∈K, U d is the maximum number of traffic sensors.

[0019] As a further improvement of the present invention, the encoder Transformer is trained based on the observed incomplete traffic road flow data to obtain the final accurate estimate of the electric vehicle travel demand. It includes:

[0020] According to the gap between the estimated value and the initial value of travel demand, the gap between the observed value and the true value of road flow and charging load is used as the error, and the encoder Transformer is trained according to the regression loss to make the observed data and the simulation results as close as possible. The objective function is:

[0021]

[0022] 0≤d w ≤d max ,w∈W (A-5)

[0023] Where x a is the estimated traffic flow, is the actual observed value of traffic flow, u a Indicates whether a∈A link is equipped with a traffic sensor (u a =1), d w is the final estimate of travel demand, is the initial estimate of travel demand, is the estimated value of charging load, is the actual observed value of charging load, and the β factor represents the initial estimate of the function matching travel demand. The weight of , || || is a measure of the similarity of any data.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention innovatively proposes a three-layer framework for coordinating the estimation of electric vehicle travel demand and traffic sensor placement, proposes the interdependent relationship between the two parts, and improves the overall estimation accuracy in smart grids and smart transportation networks under the overall framework approach. The present invention proposes a deep learning-based solution, including the use of deep reinforcement learning to optimize the placement of traffic sensors, and the use of a Transformer-based encoding-decoding framework to approximate the solution of the two-layer inverse optimization problem, which can capture the complex relationships in the collected electric vehicle traffic data and significantly improve performance. The present invention can successfully solve the departure-destination demand estimation of electric vehicle traffic flow and calculate the optimal solution for traffic sensor placement, thereby effectively managing traffic and smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the method of the present invention.

[0027] Figure 2 Comparison of the training process of a gradient-free deep genetic strategy and a SAC algorithm.

[0028] Figure 3 Comparison of the encoder training process using a Transformer network and a fully convolutional neural network.

[0029] Figure 4 Comparison of mean square error in the same batch scenario.

[0030] Figure 5 Comparison of mean square error at different uncertainty levels. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0032] The present invention proposes a method for estimating electric vehicle travel demand and placing traffic sensors, comprising:

[0033] Collect historical start-end point travel demand data and historical charging load and road section flow;

[0034] Use the collected data to train deep reinforcement learning agents and Transformer networks;

[0035] The selected typical day of charging load is input into the trained agent to generate a traffic sensor placement plan, and the traffic road flow data observed by the sensor is input into the Transformer to obtain the final accurate estimate of the departure-destination.

[0036] The obtained origin-destination estimates are used as input and the trained fitted traffic assignment network is used to obtain the final traffic distribution and charging load estimates.

[0037] Example 1

[0038] The experimental setup used for verification of the present invention is as follows:

[0039] The present invention uses the solver to perform simulation based on the above model framework and solution method. Under the same test conditions, the present invention compares the training process of the gradient-free deep genetic strategy and the SA algorithm. Figure 2 shown.

[0040] The results show that compared with the SAC (Soft Actor-Critic) based deep genetic strategy, the gradient-free deep genetic strategy significantly outperforms the existing methods by continuously reducing the mean square error in each period and is more suitable for optimizing the location of traffic sensors.

[0041] The present invention uses the solver to perform simulation based on the above model framework and solution method. Under the same test conditions, the present invention compares the training process of the encoder using the Transformer network and the fully convolutional neural network. Figure 3 shown.

[0042] The results show that compared with the fully convolutional neural network, the advanced multi-head attention mechanism adopted by the Transformer network enhances its ability to capture complex patterns in the data, thus having faster convergence speed and lower mean square error value.

[0043] The Transformer-based encoder successfully estimates the EV travel demand, tightly integrating it with the true value and reducing the uncertainty introduced by the initial travel demand.

[0044] Based on the above model framework and solution method, the present invention calls the solver for simulation. Under the same test conditions, the present invention compares the performance of the Transformer network and the fully connected network. The mean square error in different batch scenarios is as follows: Figure 3 , the mean square error at different uncertainty levels is as follows Figure 4 :

[0045] The results show that Transformer networks can effectively capture multimodal patterns and estimate EV demand in parallel. Transformer networks consistently outperform fully connected networks in terms of accuracy and resilience to data uncertainty, highlighting their suitability for handling complex and diverse traffic scenarios.

[0046] Based on the above model framework and solution method, the present invention calls the solver for simulation. Under the same test conditions, the scenario of coordinating smart grid and smart transportation network and the scenario of one of the network data being blocked are compared. The mean square error of the electric vehicle travel demand estimation is as follows: Figure 5 .

[0047] The results show that integrating the information of smart grid and intelligent transportation network can significantly reduce the mean square error and improve the accuracy of EV travel demand estimation.

Claims

1. A method for estimating electric vehicle travel demand and placing traffic sensors, characterized in that: include: Obtain historical travel demand data, historical charging load and road traffic, and select typical days for charging load based on historical experience distribution; Input travel demand data, historical charging load, and road traffic into the decoder's Transformer to train the network to fit the traffic assignment problem; The historical charging load is used as observation, and the mean square error loss of travel demand estimation and traffic flow distribution is used as reward. The intelligent agent outputs the placement plan of traffic sensors to train the deep reinforcement learning intelligent agent. The selected typical day of charging load is input into the trained agent to generate a traffic sensor placement plan; The traffic flow data observed by the sensor is input into the encoder's Transformer, and the encoder's Transformer is trained based on the regression loss. The training network obtains the final accurate estimation of travel demand. Using the trained fitted traffic assignment network and inputting the estimated travel demand, we can get accurate estimates of traffic distribution and charging load.

2. The method for estimating electric vehicle travel demand and placing traffic sensors according to claim 1, characterized in that: The historical charging load is used as the observation of the deep reinforcement learning agent, the error between the estimated traffic distribution and charging load output by the travel demand estimation and traffic assignment problem and the actual value is used as the reward, and the location of traffic sensors is decoupled from the electric vehicle travel demand estimation problem, thereby facilitating the solution process to train the deep reinforcement learning agent to output the sensor placement plan, including: A deep genetic policy with no gradient training algorithm is established. The initial population is evaluated by randomly assigning parameters, and the top T individuals are selected as the parents of the next generation to apply additive Gaussian noise to their network parameters. After multiple generations of iterations, the selection and deployment of traffic sensors based on expected rewards are improved. The training algorithm of deep reinforcement learning agents includes the following steps: (1) Initialize the first generation g0; (2) For the gth generation, cycle from 1 to G; (3) Evaluate the gth i-1 generation; (4) From g i-1 In the generation, select T top individuals as the gth i The parent generation of the generation; (5) For the population Population, it cycles from 1 to P; (6) Randomly select a parent and add Gaussian noise to its parameters: θ′=θ+σεθ~N(0,1) (A-3).

3. The method for estimating electric vehicle travel demand and placing traffic sensors according to claim 1, characterized in that: The selected typical day of charging load is input into the trained agent to generate a traffic sensor placement plan, including: The objective function is established to minimize the overall reconstruction loss between the estimated and actual observations of traffic flow and electric vehicle charging load. The objective function is: where u a Indicates whether a traffic sensor is installed in route a∈A, x a and They represent the estimated traffic flow of the route and the corresponding actual observation results, respectively. and are the estimated and actual observation of the electric vehicle charging load at the electric vehicle charging station k∈K, U d is the maximum number of traffic sensors.

4. The method for estimating electric vehicle travel demand and placing traffic sensors according to claim 1, characterized in that: The encoder Transformer is trained based on the observed incomplete traffic road flow data to obtain an accurate estimate of the final electric vehicle travel demand; including: Based on the gap between the estimated value and the initial value of travel demand, and the gap between the observed value and the true value of road flow and charging load as errors, the encoder Transformer is trained according to the regression loss to make the observed data and the simulation results as close as possible; The objective function is: 0≤d w ≤d max ,w∈W (A-5) Where x a is the estimated traffic flow, is the actual observed value of traffic flow, u a Indicates whether a∈A link is equipped with a traffic sensor (u a =1), d w is the final estimate of travel demand, is the initial estimate of travel demand, is the estimated value of charging load, is the actual observed value of charging load, and the β factor represents the initial estimate of the function matching travel demand. The weight of .

Images