A vehicle trajectory optimization control method for expressway under a vehicle-infrastructure integrated traffic flow environment

Abstract

The application provides a vehicle trajectory optimization control method for a fast road under a vehicle-road cooperation mixed traffic flow environment, and aims at a mixed driving scene of a networked automatic driving vehicle and a traditional human-driven vehicle at a signal intersection of a fast road.Based on the application of mobile networking and automatic driving technology, on one hand, road side equipment and vehicle-mounted equipment are used to collect and transmit front signal and vehicle flow operation information;on the other hand, a vehicle trajectory optimization model is constructed based on a deep reinforcement learning algorithm, the model adopts an actor-critic framework for optimization, takes intersection safety as an optimization target, obeys road speed limit and red light rule constraints, sets a safety detector to forcibly correct potential unsafe behaviors adopted in the control process, realizes optimization design of the trajectory of the networked automatic driving vehicle to avoid collision, and improves the passing efficiency of the fast road.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic management and control technology, and in particular relates to a method for optimizing vehicle trajectory control on expressways in a vehicle-road cooperative mixed traffic flow environment. Background Technology

[0002] Traffic oscillations are a major cause of traffic congestion and pollution. In urban road environments, traffic lights require vehicles to follow red and green light restrictions at intersections to avoid conflicts between traffic flows from different directions. Frequent acceleration and deceleration at signalized intersections cause traffic oscillations that propagate upstream, leading to severe queuing and hindering rapid traffic flow on urban arterial roads, significantly reducing road safety and efficiency. Therefore, optimizing vehicle trajectories to absorb intersection oscillations is a crucial task in solving urban traffic problems.

[0003] The development of vehicle-to-everything (V2X) and autonomous driving technologies provides crucial technical support for addressing the oscillation problem at urban intersections. Connected vehicles with communication capabilities, working in conjunction with roadside units (RSUs) via vehicle-to-infrastructure (V2I) communication networks, can acquire traffic flow status information within road segments, providing more accurate information for generating control actions for controlled vehicles. Changes in the behavior of connected vehicles also constrain and guide other conventional vehicles, thereby improving the overall traffic environment. Compared to traditional methods, onboard ad hoc networking technology offers advantages such as more continuous cross-sections, more stable effects, and richer information for drivers when providing connected vehicles with beyond-line-of-sight traffic information or control commands. Autonomous vehicles possess high-resolution perception systems and highly controllable central control systems, enabling them to accurately perceive surrounding road information and strictly execute target control actions. Their reaction time is shorter than that of traditional human-driven vehicles, and they can tolerate smaller following distances, which is beneficial for solving the oscillation problem at urban intersections and improving road capacity.

[0004] Mixed traffic flow environments exhibit higher stochasticity, placing greater demands on the computational performance of control methods. Unlike traditional linear or nonlinear control and model predictive control methods, deep reinforcement learning-based control methods are more suitable for systems with highly stochastic characteristics, offer faster computation speeds, and are better suited for online control of autonomous vehicles. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to propose a vehicle trajectory optimization control method for expressways in a vehicle-road cooperative mixed traffic flow environment. By optimizing the trajectory of autonomous vehicles, the method absorbs traffic oscillation waves at signalized intersections, thereby reducing the risk of traffic accidents and traffic congestion caused by oscillation waves.

[0006] Technical Solution: To achieve the objectives of this invention, this invention proposes a method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment. This method includes the following steps:

[0007] Step 1) Design a simulation scenario for an intelligent connected signalized intersection and determine the simulation parameters, and determine the composition and definition of the information acquired by the connected autonomous vehicle in the simulation;

[0008] Step 2) Construct a networked autonomous vehicle intelligent agent based on reinforcement learning. During the simulation process, the agent interacts with the environment, acquires experience data and stores it in the experience dataset. Data is extracted from the experience dataset to construct the training dataset.

[0009] Step 3) Train and test the vehicle trajectory optimization control model for signalized intersections, and save the final model parameters after training;

[0010] Step 4) Based on the trained vehicle trajectory optimization control model, output the acceleration of the connected autonomous vehicle for the next simulation step. Perform safety risk detection on the acceleration output by the control model, and forcibly correct the acceleration with safety risks. The autonomous vehicle executes the corrected acceleration action in the next simulation and performs distributed control of the connected autonomous vehicle in the signalized intersection entrance area.

[0011] Furthermore, in step 1), a simulation scenario for a single-signal intersection in an intelligent connected environment is designed, and the basic parameters of the simulated vehicle are determined, including the maximum speed v. max Maximum acceleration a max Vehicle length l car Minimum parking distance s, reaction time τ; determine intersection signal timing parameters, including cycle time t. cycle Red light timing r Green light timing g Yellow light timing y Each autonomous vehicle acquires operational data during the simulation, including environmental status information. Action information and reward information The specific definitions are as follows:

[0012] Environmental state information acquired by connected autonomous vehicle n at time t Including local state information S local and global state information S global S local Data obtained from the onboard equipment of the connected vehicle, including the vehicle speed of vehicle n at time t. Speed ​​of vehicle n-1 at time t Distance between vehicles n and n-1 at time t S local Data obtained from roadside facilities, including the distance between vehicle n and the intersection at time t. Current signal period t cycleCurrent phase P, remaining time t of the current phase p Estimated number of vehicles in queue N platoon ;

[0013] Action information Let n be the acceleration of the connected autonomous vehicle n at time t, i.e., the vehicle acceleration.

[0014] Reward Information For connected autonomous vehicles n The reward value obtained in the state, including the speed reward r speed Safety Rewards safe Time reward r time Smoothness reward r a .

[0015] Furthermore, the number N of vehicles in the queue platoon The number of connected autonomous vehicles and vehicles at intersections reaching the stop line within the queuing time window is estimated using the collected traffic state information. The specific calculation method is as follows:

[0016] Step 1: Calculate the vehicle queuing time window [t] based on the traffic conditions at time t obtained by the connected autonomous vehicle n. pla_begin , t pla_end ]:

[0017]

[0018]

[0019]

[0020] In the formula, Let vehicle n be at time t with its current speed. The time t takes to reach the stop line at the intersection g_begin t g_end t y_begin t r_end These represent the start time of the green light, the end time of the green light, the start time of the yellow light, and the end time of the red light, respectively, where k is the number of signal cycles;

[0021] Step 2: Obtain the number K and status information of other vehicles between the connected autonomous vehicle n and the stop line at the signalized intersection using roadside video information. Calculate the estimated number N of vehicles N that pass through the intersection within the same green light phase as connected autonomous vehicles n. platoon :

[0022]

[0023]

[0024]

[0025] In the formula, i represents the number of the connected autonomous vehicle n and the vehicle between the intersection stop line. Let be the distance between vehicle i and the intersection at time t. Let be the speed of vehicle i at time t. Let i be the number of vehicles in the queue corresponding to vehicle i.

[0026] Furthermore, speed reward r speed Safety Rewards safe Time reward r time Acceleration bonus r a The specific calculation method is as follows:

[0027] (1) Speed ​​reward r speed For the connected autonomous vehicle n to perform acceleration at time t Vehicle speed at time t+1 The corresponding reward is given within the speed limit range [0, v]. max ]Inside, The larger r is speed The higher;

[0028]

[0029] (2) Safety reward r safe Based on the distance between connected autonomous vehicle n and its preceding vehicle n-1 at time t+1 Does it meet the expected safety distance? Calculations are required;

[0030]

[0031]

[0032] (3) Time reward r time Used to encourage vehicles during the green light phase [t] g_begin , t g_end Pass through the intersection stop line and are not affected by the queue ahead;

[0033] L platoon =N platoon *(l car +s)

[0034]

[0035]

[0036] In the formula, L platoonLet ω be the queue length, ω be the dissipation wave velocity (m / s) of the queue at the intersection after the green light phase begins, and a be the queue length. ave The average acceleration of queue dissipation at the intersection (m / s²) 2 ), t free The green light time occupied by queuing vehicles;

[0037] (4) Acceleration bonus r a To ensure smooth vehicle operation, acceleration is calculated as follows:

[0038]

[0039] (5) Total reward value:

[0040] R t+1 =ω1r speed +ω2r safe +ω3r time +ω4r a

[0041] In the formula, ω1, ω2, ω3, and ω4 are the reward weights.

[0042] Furthermore, in step 2), a connected autonomous vehicle intelligent agent is constructed, which adopts a vehicle trajectory optimization model based on the deep reinforcement learning PPO algorithm. It consists of two sub-models, Critic and Actor, both of which are neural network models constructed from multiple fully connected layers. The input layer is the environmental state obtained by the intelligent agent. The Critic model outputs the agent's policy value V(s), used to evaluate the Actor. The Actor model outputs the agent's policy, i.e., acceleration. The probability distribution P(s).

[0043] Furthermore, in step 2), each connected autonomous vehicle acts as an independent intelligent agent, interacting with the simulation environment during the simulation operation, acquiring state, action, and reward-related data and storing them in the experience dataset M. Data is then extracted from the experience dataset to construct a training dataset, as detailed below:

[0044] Agent n obtains the environment state at time t. Based on state The trajectory optimization model outputs the acceleration probability distribution P(s) of agent n, and the acceleration action is randomly obtained from the probability distribution. and their corresponding probabilities The action is executed and the simulation runs to time t+1, at which point the environment state is updated. According to the state Get action reward value Based on the above process, we obtain one piece of empirical data. Each agent acquires experience data through continuous interaction with the environment during the simulation process, and stores it in a centralized experience dataset M. The maximum data capacity C of M is set, and experience data acquisition stops when the maximum capacity is reached. A training set is constructed by randomly sampling a fixed number of experience data from M.

[0045] Furthermore, in step 3), the vehicle trajectory optimization control model for the signalized intersection is trained and tested, and the final model parameters after training are saved. The specific training process is as follows:

[0046] Step 5.1 Set the total number of training rounds N and determine the hyperparameters, including the learning rate LR, the pruning factor ε, the size of the empirical dataset C, the size of the training dataset c, and the discount factor.

[0047] Step 5.2 Set the initial simulation parameters, including simulation duration, traffic flow parameters, and traffic light parameters;

[0048] Step 5.3 Initialize and run the simulation, collect empirical data and store it in the empirical dataset M. When the simulation ends, update the simulation random seed, reset the simulation and continue collecting data until the maximum capacity C is reached and then stop collecting empirical data.

[0049] Step 5.4 Randomly select a fixed number c of empirical data from M as the training set, and update the model iteratively x times, where x = C / c;

[0050] Step 5.5 Perform multiple simulation tests on the updated control model and output the average reward value of the vehicle.

[0051] Step 5.6 Check if the total number of rounds N has been reached. If it has, end the training and save the model parameters; if it has not been reached, clear the experience dataset and return to step 5.3.

[0052] Furthermore, in step 4), the multiple connected autonomous vehicles at the signalized intersection entrance are controlled in a distributed manner based on the trajectory optimization model trained in step 3), with a control step size of 0.5s. The trajectory optimization model is based on the environmental states acquired by each connected autonomous vehicle n. Output control action Based on the expected safety distance Action Perform security risk assessment and correction to obtain the final execution acceleration.

[0053]

[0054] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0055] 1. The vehicle trajectory optimization control method for expressways proposed in this study under vehicle-road cooperative mixed traffic flow environment takes the vehicle safety risk at intersections as the optimization objective, which can significantly eliminate queuing at intersections and reduce the potential safety risks brought by intersection shock waves.

[0056] 2. Estimating the intersection shock wave boundary: Based on traffic state information collected by sensing facilities, the queue length of the intersection in the future is estimated. The minimum boundary of the intersection shock wave is estimated through traffic shock wave theory. Combined with the exploration capabilities of deep reinforcement learning, it can effectively solve the difficulty of predicting the trajectory of people, vehicles and drivers in mixed traffic flow environments.

[0057] 3. Considering the occasional rear-end collision risk in the control strategy, this study sets up a safety detector before the control strategy is implemented to further assess the rear-end collision risk brought about by the control strategy and to forcibly correct the risky actions, thereby eliminating potential error risks that may exist in the neural network model. This safety detector can guide the training of the control model during the model training stage and also detect the actions executed during the control execution stage.

[0058] 4. By optimizing the trajectory of connected autonomous vehicles upstream of the intersection, subsequent traffic flows can smoothly enter the signalized intersection, absorbing the shock wave caused by traffic lights, thereby reducing the potential accident risk caused by the shock wave. At the same time, it can reduce fuel consumption and pollution at the signalized intersection, thus improving urban road traffic congestion. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the control model training and decision-making mechanism of the present invention.

[0060] Figure 2 This is a schematic diagram of the simulation scenario of the present invention.

[0061] Figure 3 This is a schematic diagram of queue length prediction in this invention.

[0062] Figure 4 This is a flowchart of the training process for the reinforcement learning model in this invention. Detailed Implementation

[0063] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. Figure 1 This is a schematic diagram of the control model training and decision-making mechanism of the present invention. The specific steps are as follows:

[0064] Step 1) Design a simulation scenario for an intelligent connected signalized intersection and determine the simulation parameters, including the composition and definition of the information acquired by the connected autonomous vehicle in the simulation; such as... Figure 2 As shown, it displays a simulation scene diagram;

[0065] Step 2) Construct a networked autonomous vehicle intelligent agent based on reinforcement learning. During the simulation process, the agent interacts with the environment, acquires experience data and stores it in the experience dataset. Data is extracted from the experience dataset to construct the training dataset.

[0066] Step 3) Train and test the vehicle trajectory optimization control model for the signalized intersection, and save the final model parameters after training; such as Figure 4 As shown, it illustrates the training and testing process of the model;

[0067] Step 4) Based on the trained vehicle trajectory optimization control model, output the acceleration of the connected autonomous vehicle for the next simulation step. Perform safety risk detection on the acceleration output by the control model, and forcibly correct the acceleration with safety risks. The autonomous vehicle executes the corrected acceleration action in the next simulation and performs distributed control of the connected autonomous vehicle in the signalized intersection entrance area.

[0068] Furthermore, in step 1), a simulation scenario for a single-signal intersection under an intelligent connected environment is designed, and the basic parameters of the simulated vehicle are determined, including the maximum speed v. max Maximum acceleration a max Vehicle length l car Minimum parking distance s, reaction time τ; determine intersection signal timing parameters, including cycle time t. cycle Red light timing r Green light timing g Yellow light timing y Each autonomous vehicle acquires operational data during the simulation, including environmental status information. Action information and reward information The specific definitions are as follows:

[0069] Environmental state information acquired by connected autonomous vehicle n at time t Including local state information S local and global state information S global S local Data obtained from the onboard equipment of the connected vehicle, including the vehicle speed of vehicle n at time t. Speed ​​of vehicle n-1 at time t Distance between vehicles n and n-1 at time t S local Data obtained from roadside facilities, including the distance between vehicle n and the intersection at time t. Current signal period t cycle Current phase P, remaining time t of the current phase p Estimated number of vehicles in queue N platoon ;like Figure 3As shown, it illustrates a schematic diagram of queue length prediction in this invention.

[0070] Action information Let n be the acceleration of the connected autonomous vehicle n at time t, i.e., the vehicle acceleration.

[0071] Reward Information For connected autonomous vehicles n The reward value obtained in the state, including the speed reward r speed Safety Rewards safe Time reward r time Smoothness reward r a .

[0072] Furthermore, the number N of vehicles in the queue platoon The number of connected autonomous vehicles and vehicles at intersections reaching the stop line within the queuing time window is estimated using the collected traffic state information. The specific calculation method is as follows:

[0073] Step 1: Calculate the vehicle queuing time window [t] based on the traffic conditions at time t obtained by the connected autonomous vehicle n. pla_begin , t pla_end ]:

[0074]

[0075]

[0076]

[0077] In the formula, Let vehicle n be at time t with its current speed. The time t takes to reach the stop line at the intersection g_begin t g_end t y_begin t r_end These represent the start time of the green light, the end time of the green light, the start time of the yellow light, and the end time of the red light, respectively, where k is the number of signal cycles;

[0078] Step 2: Obtain the number K and status information of other vehicles between the connected autonomous vehicle n and the stop line at the signalized intersection using roadside video information. Calculate the estimated number N of vehicles N that pass through the intersection within the same green light phase as connected autonomous vehicles n. platoon :

[0079]

[0080]

[0081]

[0082] In the formula, i represents the number of the connected autonomous vehicle n and the vehicle between the intersection stop line. Let be the distance between vehicle i and the intersection at time t. Let be the speed of vehicle i at time t. Let i be the number of vehicles in the queue corresponding to vehicle i.

[0083] Furthermore, speed reward r speed Safety Rewards safe Time reward r time Acceleration bonus r a The specific calculation method is as follows:

[0084] (1) Speed ​​reward r speed For the connected autonomous vehicle n to perform acceleration at time t Vehicle speed at time t+1 The corresponding reward is given within the speed limit range [0, v]. max ]Inside, The larger r is speed The higher;

[0085]

[0086] (2) Safety reward r safe Based on the distance between connected autonomous vehicle n and its preceding vehicle n-1 at time t+1 Does it meet the expected safety distance? Calculations are required;

[0087]

[0088]

[0089] (3) Time reward r time Used to encourage vehicles during the green light phase [t] g_begin , t g_end Pass through the intersection stop line and are not affected by the queue ahead;

[0090] L platoon =N platoon *(l car +s)

[0091]

[0092]

[0093] In the formula, L platoonLet ω be the queue length, ω be the dissipation wave velocity (m / s) of the queue at the intersection after the green light phase begins, and a be the queue length. ave The average acceleration of queue dissipation at the intersection (m / s²) 2 ), t free The green light time occupied by queuing vehicles;

[0094] (4) Acceleration bonus r a To ensure smooth vehicle operation, acceleration is calculated as follows:

[0095]

[0096] (5) Total reward value:

[0097] R t+1 =ω1r speed +ω2r safe +ω3r time +ω4r a

[0098] In the formula, ω1, ω2, ω3, and ω4 are the reward weights.

[0099] Furthermore, in step 2), a connected autonomous vehicle intelligent agent is constructed, which adopts a vehicle trajectory optimization model based on the deep reinforcement learning PPO algorithm. It consists of two sub-models, Critic and Actor, both of which are neural network models constructed from multiple fully connected layers. The input layer is the environmental state obtained by the intelligent agent. The Critic model outputs the agent's policy value V(s), used to evaluate the Actor. The Actor model outputs the agent's policy, i.e., acceleration. The probability distribution P(s).

[0100] Furthermore, in step 2), each connected autonomous vehicle acts as an independent intelligent agent, interacting with the simulation environment during the simulation operation, acquiring state, action, and reward-related data and storing them in the experience dataset M. Data is then extracted from the experience dataset to construct a training dataset, as detailed below:

[0101] Agent n obtains the environment state at time t. Based on state The trajectory optimization model outputs the acceleration probability distribution P(s) of agent n, and the acceleration action is randomly obtained from the probability distribution. and their corresponding probabilities The action is executed and the simulation runs to time t+1, at which point the environment state is updated. According to the state Get action reward value Based on the above process, we obtain one piece of empirical data. Each agent acquires experience data through continuous interaction with the environment during the simulation process, and stores it in a centralized experience dataset M. The maximum data capacity C of M is set, and experience data acquisition stops when the maximum capacity is reached. A training set is constructed by randomly sampling a fixed number of experience data from M.

[0102] Furthermore, in step 3), the vehicle trajectory optimization control model for the signalized intersection is trained and tested, and the final model parameters after training are saved. The specific training process is as follows:

[0103] Step 5.1 Set the total number of training rounds N and determine the hyperparameters, including the learning rate LR, the pruning factor ε, the size of the empirical dataset C, the size of the training dataset c, and the discount factor.

[0104] Step 5.2 Set the initial simulation parameters, including simulation duration, traffic flow parameters, and traffic light parameters;

[0105] Step 5.3 Initialize and run the simulation, collect empirical data and store it in the empirical dataset M. When the simulation ends, update the simulation random seed, reset the simulation and continue collecting data until the maximum capacity C is reached and then stop collecting empirical data.

[0106] Step 5.4 Randomly select a fixed number c of empirical data from M as the training set, and update the model iteratively x times, where x = C / c;

[0107] Step 5.5 Perform multiple simulation tests on the updated control model and output the average reward value of the vehicle.

[0108] Step 5.6 Check if the total number of rounds N has been reached. If it has, end the training and save the model parameters; if it has not been reached, clear the experience dataset and return to step 5.3.

[0109] Furthermore, in step 4), the multiple connected autonomous vehicles at the signalized intersection entrance are controlled in a distributed manner based on the trajectory optimization model trained in step 3), with a control step size of 0.5s. The trajectory optimization model is based on the environmental states acquired by each connected autonomous vehicle n. Output control action Based on the expected safety distance Action Perform security risk assessment and correction to obtain the final execution acceleration.

[0110]

[0111] This method, applied in urban road traffic flow environments with vehicle-road cooperative systems, designs longitudinal vehicle trajectories based on traffic state information, aiming at vehicle safety and stability. This guides vehicles to smoothly pass through intersections within the green light time, reducing stopping time and mitigating potential accident risks from traffic shock waves. Compared to existing methods, this approach, in addition to directly acquiring traffic state information from traditional sensing devices and network communication technologies, incorporates traffic shock wave theory to further estimate intersection shock wave boundaries, providing more detailed state information for trajectory optimization. Furthermore, it ensures the target trajectory meets safety requirements before execution, reducing the impact of control model errors in practical applications. In summary, this method addresses the issue of ensuring traffic flow safety at signalized intersections while simultaneously reducing queue length and vehicle delays, improving the efficiency of urban arterial roads. It has practical engineering application value in reducing urban road traffic accident risks and solving urban congestion problems.

Claims

1. A method for optimizing vehicle trajectory control on a rapid transit road under a vehicle-road cooperative mixed traffic flow environment, characterized in that, The method includes the following steps: Step 1) Design a simulation scenario for an intelligent connected signalized intersection, determine the simulation parameters, and define the composition and definition of the information acquired by the connected autonomous vehicle in the simulation; Step 2) Construct a networked autonomous vehicle intelligent agent based on reinforcement learning. During the simulation process, the agent interacts with the environment, acquires experience data and stores it in the experience dataset. Data is extracted from the experience dataset to construct the training dataset. Step 3) Train and test the vehicle trajectory optimization control model for signalized intersections, and save the final model parameters after training; Step 4) Based on the trained vehicle trajectory, optimize the control model to output the acceleration of the connected autonomous vehicle for the next simulation step. Perform safety risk detection on the acceleration output by the control model and forcibly correct the acceleration with safety risks. The autonomous vehicle executes the corrected acceleration action in the next simulation and performs distributed control of the connected autonomous vehicle in the signalized intersection entrance area. In step 1), a simulation scenario of a single-signal intersection under an intelligent connected environment is designed, and the basic parameters of the simulated vehicle are determined, including maximum speed. Maximum acceleration Vehicle length Minimum parking distance s, reaction time τ; determine intersection signal timing parameters, including cycle time t. cycle Red light timing r Green light timing g Yellow light timing y Each autonomous vehicle acquires operational data during the simulation, including environmental status information. Action information and reward information The specific definition is as follows: Environmental state information acquired by connected autonomous vehicle n at time t Including local state information and global state information , Data obtained from the onboard equipment of the connected vehicle, including the vehicle speed of vehicle n at time t. The speed of the preceding vehicle n-1 at time t The distance between vehicles n and n-1 at time t , Data obtained from roadside facilities, including the distance between vehicle n and the intersection at time t. Periodic time t cycle Current phase P, remaining time t of the current phase p Estimated number of vehicles in queue N platoon ; Action information Let n be the acceleration of the connected autonomous vehicle n at time t, i.e., the vehicle acceleration. ; Reward Information For connected autonomous vehicles n The reward value obtained in the state, including the speed reward r speed Safety Rewards safe Time reward r time Smoothness reward r a ; The estimated number of vehicles in the queue The number of connected autonomous vehicles and vehicles at intersections reaching the stop line within the queuing time window is estimated using the collected traffic state information. The specific calculation method is as follows: Step 1: Calculate the vehicle queuing time window based on the traffic conditions at time t obtained by the connected autonomous vehicle n. : ； In the formula, Let vehicle n be at time t with its current speed. Time to reach the stop line at the intersection These are the start and end times of the queuing time window, respectively. , , , These represent the start time of the green light, the end time of the green light, the start time of the yellow light, and the end time of the red light, respectively, where k is the number of signal cycles; Step 2: Obtain the number K and status information of other vehicles between the connected autonomous vehicle n and the stop line at the signalized intersection using roadside video information. Calculate the estimated number of vehicles queuing at the intersection that pass through the same green light phase as connected autonomous vehicles n. : ； ； ； In the formula, i represents the number of the connected autonomous vehicle n and the vehicle between the intersection stop line. Let be the distance between vehicle i and the intersection at time t. Let be the speed of vehicle i at time t. Let i be the estimated number of vehicles in the queue corresponding to vehicle i.

2. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 1, characterized in that, Speed ​​reward r speed Safety Rewards safe Time reward r time Acceleration bonus r a The specific calculation method is as follows: (1) Speed ​​bonus r speed Let n be the speed of the connected autonomous vehicle at time t. The corresponding reward is shown within the speed limit range. Inside, The larger r is speed The higher; ； (2) Safety reward r safe Based on the distance between connected autonomous vehicle n and its preceding vehicle n-1 at time t Does it meet the expected safety distance? Calculations are required; ； ； (3) Time reward r time Used to encourage vehicles during the green light phase Pass through the intersection stop line and are not affected by the queue ahead; ； In the formula, Let ω be the queue length, ω be the dissipation wave velocity (m / s) of the queue at the intersection after the green light phase begins, and a be the queue length. ave The average acceleration of queue dissipation at the intersection (m / s²) 2 ), t free The green light time occupied by queuing vehicles; (4) Acceleration bonus r a acceleration at time t The corresponding reward value is used to ensure the smoothness of vehicle operation, and its calculation method is as follows: 。 3. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 2, characterized in that, The total reward value is calculated as follows: ； In the formula, ω1, ω2, ω3, and ω4 are the reward weights.

4. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 1, characterized in that, In step 2), a connected autonomous vehicle intelligent agent is constructed, which adopts a vehicle trajectory optimization model based on the deep reinforcement learning PPO algorithm. It consists of two sub-models, Critic and Actor, both of which are neural network models constructed from multiple fully connected layers. The input layer is the environmental state obtained by the intelligent agent. The Critic model outputs the agent's policy value V(s), used to evaluate the Actor. The Actor model outputs the agent's policy, i.e., acceleration. The probability distribution P(s).

5. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 1, characterized in that, In step 2), each connected autonomous vehicle acts as an independent intelligent agent, interacting with the simulation environment during the simulation process, acquiring state, action, and reward-related data, storing them in the experience dataset M, and extracting data from the experience dataset to construct a training dataset, as detailed below: Agent n obtains the environment state at time t. Based on state The trajectory optimization model outputs the acceleration probability distribution P(s) of agent n, and the acceleration action is randomly obtained from the probability distribution. and their corresponding probabilities The action is executed and the simulation runs to time t+1, at which point the environment state is updated to... According to the state Get action reward value Based on the above process, an empirical data point is obtained { , , , , Each agent acquires experience data through continuous interaction with the environment during the simulation process, and stores it in the experience dataset M. The maximum data capacity C of M is set, and experience data acquisition stops when the maximum capacity is reached. A training set is constructed by randomly sampling a fixed number of experience data from M.

6. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 1, characterized in that, In step 3), the vehicle trajectory optimization control model for the signalized intersection is trained and tested, and the final model parameters after training are saved. The specific training process is as follows: Step 5.1 Set the total number of training rounds N and determine the hyperparameters, including the learning rate LR, the pruning factor ε, the size of the empirical dataset C, the size of the training dataset c, and the discount factor. Step 5.2 Set the initial simulation parameters, including simulation duration, traffic flow parameters, and traffic light parameters; Step 5.3 Initialize and run the simulation, collect empirical data and store it in the empirical dataset M. When the simulation ends, update the simulation random seed, reset the simulation and continue collecting data until the maximum capacity C is reached and then stop collecting empirical data. Step 5.4 Randomly select a fixed number c of empirical data from M as the training set, and update the model iteratively x times, where x = C / c; Step 5.5 Perform multiple simulation tests on the updated control model and output the average reward value of the vehicle; Step 5.6 Check if the total number of rounds N has been reached. If it has, end the training and save the model parameters; if it has not been reached, clear the experience dataset and return to step 5.

3.

7. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 1, characterized in that, In step 4), the multiple connected autonomous vehicles at the signalized intersection entrance are controlled in a distributed manner based on the trajectory optimization model trained in step 3). The control step size is 0.5s. The trajectory optimization model is based on the environmental state acquired by each connected autonomous vehicle n. Output control action Based on the expected safety distance Action Perform security risk assessment and correction to obtain the final execution acceleration. .

8. The method for optimizing vehicle trajectory control on expressways under a vehicle-road cooperative mixed traffic flow environment according to claim 7, characterized in that, Execution acceleration The calculation method is as follows: 。

Images