A dual-layer ecological city vehicle-road cooperative control method and system under mixed traffic environment

Abstract

The present invention proposes a double-layer ecological urban vehicle-road cooperative control method and system in a mixed traffic environment, the method comprising: regarding the traffic light at each intersection as an intelligent agent, and obtaining the real-time driving information of the vehicle through infrastructure communication; the intelligent agent calculates the number of vehicles and the driving speed of each convoy according to the headway between the vehicles; the upper-layer perceptron predicts the time required for the convoy to reach the stop line of the intersection by stacking a fully connected layer and a long short-term memory network; the predicted time information is input into the policy network and value network of the intelligent agent to generate an action, and the policy network is updated by delayed feedback rewards; the optimized recommended driving speed is sent to the head vehicle of the convoy to optimize the driving trajectory of the convoy, alleviate road network congestion and reduce pollutant emissions; the present invention improves the vehicle-road cooperative control capability and traffic efficiency of the urban road network by using intelligent agents to perform cooperative control through distributed decision-making and cooperative learning.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic data processing, and in particular to a double-layer ecological city vehicle-road collaborative control method and system in a mixed traffic environment. Background Art

[0002] In recent years, with the rapid development of urban scale, traffic congestion has become one of the main challenges facing urban development; as a node where traffic flows in different directions meet, signalized intersections are regarded as the bottleneck of urban traffic congestion; the frequent acceleration, deceleration and idling of vehicles when approaching intersections is one of the main causes of traffic congestion and environmental pollution; therefore, formulating a reasonable traffic signal control method can not only alleviate traffic congestion, but also reduce vehicle pollution emissions. However, with the continuous expansion of urban road network scale, traditional traffic signal control methods often lead to increased traffic congestion and increased pollutant emissions due to low control efficiency when dealing with dynamic traffic flow growth.

[0003] However, the existing technology only counts the number of vehicles on the lane through roadside units, so the state information obtained by the intelligent agent is not comprehensive and accurate, which easily leads to inefficient decision-making and aggravated traffic congestion. Compared with traditional control methods, some single-agent frameworks have been proposed based on deep reinforcement learning for vehicle-road cooperative control near isolated signal intersections. However, the optimization method of single-agent is not suitable for large-scale urban traffic road networks. Therefore, it is necessary to focus on vehicle-road cooperative control methods based on multi-agent deep reinforcement learning. In order to promote the application of intelligent connected vehicles in large-scale road networks, it is still necessary to develop more effective methods to improve the overall optimization of vehicle-road cooperative control. Summary of the invention

[0004] In view of the above situation, the main purpose of the present invention is to propose a double-layer ecological city vehicle-road cooperative control method and system in a mixed traffic environment to solve the above technical problems.

[0005] The present invention proposes a double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment, the method comprising the following steps:

[0006] Step 1: Consider the traffic lights at each intersection as an intelligent agent, which obtains real-time driving information of vehicles through communication between vehicles and infrastructure;

[0007] Step 2: The agent identifies the headway distance between vehicles based on the real-time driving information of the vehicles, and divides different vehicles into different fleets based on the headway distance to calculate the number of vehicles and driving speed in the fleet;

[0008] Step 3: Input the number of vehicles in the convoy and the driving speed into a multi-layer perceptron to obtain the time required for the convoy to reach the stop line of the intersection;

[0009] Step 4: The predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent. The policy network generates the status of each vehicle in the convoy, and obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate an execution action; the value network generates a delayed feedback reward according to the execution action generated by the policy network, and uses positive and negative feedback to update the policy network to obtain an updated policy network;

[0010] Step 5: Use the updated policy network to generate optimized execution actions. The agent generates a recommended driving speed based on the optimized execution actions and sends it to the front of the fleet to dynamically optimize the driving trajectory of the fleet.

[0011] The intelligent agent continuously interacts and learns with the environment to obtain the optimal collaborative control strategy that responds to the dynamic characteristics of traffic flow, thereby alleviating road network congestion and reducing pollutant emissions.

[0012] The present invention also proposes a double-layer ecological urban vehicle-road cooperative control system in a mixed traffic environment, the system comprising:

[0013] Intersection Agent Building Blocks for:

[0014] The traffic lights at each intersection are regarded as an intelligent agent, which obtains the real-time driving information of vehicles through communication between vehicles and infrastructure;

[0015] Fleet identification and segmentation module, used for:

[0016] The intelligent agent identifies the headway distance between vehicles based on the real-time driving information of the vehicles, and divides different vehicles into different fleets based on the headway distance to calculate the number of vehicles and driving speed in the fleet;

[0017] Fleet arrival time prediction module, used for:

[0018] The number of vehicles in the convoy and the driving speed are input into the multi-layer perceptron to obtain the time required for the convoy to reach the stop line of the intersection;

[0019] Model training module, used to:

[0020] The predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent. The policy network generates the status of each vehicle in the convoy, obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate execution actions; the value network generates delayed feedback rewards based on the execution actions generated by the policy network, and uses positive and negative feedback to update the policy network to obtain an updated policy network;

[0021] The vehicle-road cooperative control module is used to:

[0022] The updated policy network is used to generate optimized execution actions. The agent generates a recommended driving speed based on the optimized execution actions and sends it to the front of the fleet to dynamically optimize the driving trajectory of the fleet.

[0023] The intelligent agent continuously interacts and learns with the environment to obtain the optimal collaborative control strategy that responds to the dynamic characteristics of traffic flow, thereby alleviating road network congestion and reducing pollutant emissions.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The present invention divides vehicles on the road into fleets and accurately predicts the arrival time of the fleet, so that each intelligent agent can perform collaborative control through distributed decision-making and cooperative learning, thereby significantly improving the vehicle-road collaborative control capability of large-scale urban road networks, improving urban traffic efficiency, and reducing vehicle pollutant emissions;

[0026] 2. The present invention introduces a spatial discount factor and a divergence control strategy to update the step size, thereby providing the model with the best vehicle trajectory and signal phase for optimizing each intersection and effectively handling different traffic scenarios.

[0027] 3. The present invention is aimed at vehicle-road collaborative control at large-scale network-level intersections, and is applicable to isolated intersections and trunk-level intersections. It is committed to reducing pollution emissions, incorporates green and environmentally friendly transportation concepts, and provides innovative solutions for future urban traffic management.

[0028] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flowchart of the steps of a double-layer ecological city vehicle-road collaborative control method in a mixed traffic environment proposed by the present invention.

[0030] Figure 2 This is a model framework diagram of a double-layer ecological city vehicle-road collaborative control method in a mixed traffic environment proposed by the present invention.

[0031] Figure 3 This is a structural diagram of a double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention.

[0032] Figure 4 This is a schematic diagram of a simulated road network experimental setup for a double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention.

[0033] Figure 5 This is a heat map of simulated road network traffic flow of a double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention.

[0034] Figure 6 This is a schematic diagram of the NEMA eight-ring four-phase signal lights in a simulated road network of a double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention.

[0035] Figure 7 This is a training curve diagram of five RL-based vehicle-road cooperative control models for a double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention.

[0036] Figure 8 This is a model training stability curve diagram of a double-layer ecological city vehicle-road cooperative control method and system in a mixed traffic environment of the present invention.

[0037] Fig. 9 This is a curve diagram of the average queue length at intersections in a simulated road network for six RL-based vehicle-road cooperative control models of a double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention.

[0038] Fig.10 The spatiotemporal trajectory of the model in the double-layer ecological city vehicle-road cooperative control method and system under different CAV presence rates in a mixed traffic environment of the present invention Figure 1 .

[0039] Fig.11 The spatiotemporal trajectory of the model in the double-layer ecological city vehicle-road cooperative control method and system under different CAV presence rates in a mixed traffic environment of the present invention Figure 2 .

[0040] Fig.12 This is a bar chart showing the different modules of the double-layer ecological city vehicle-road cooperative control system in a mixed traffic environment proposed by the present invention. DETAILED DESCRIPTION

[0041] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout are the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0042] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to provide some ways to implement the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0043] See also Figure 1 , an embodiment of the present invention proposes a double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment, the method comprising the following steps:

[0044] Step 1: Treat the traffic lights at each intersection as an intelligent agent, which obtains real-time driving information of vehicles through communication between vehicles and infrastructure.

[0045] Step 2: The intelligent agent identifies the headway between vehicles based on the real-time driving information of the vehicles, and divides different vehicles into different fleets based on the headway to calculate the number of vehicles and driving speed in the fleet.

[0046] See also Figure 2 In step 2, different vehicles are divided into different convoys according to the headway distance, so as to calculate the number of vehicles and the driving speed in the convoy, which specifically includes the following sub-steps:

[0047] The intelligent agent calculates the headway of the vehicles in the identification area, and determines the vehicle's belonging to the fleet based on the calculation result to obtain a judgment result;

[0048] Among them, the relationship between the headway distance in the identification area is:

[0049] ;

[0050] in, represents the current presence ratio of automated vehicles on the road, Indicates the maximum headway, Indicates the headway between vehicles.

[0051] It should be noted that if the headway between two adjacent connected autonomous vehicles is less than or equal to , then the two connected autonomous vehicles are considered to belong to the same fleet; otherwise, they belong to different fleets. After the mixed fleet is identified, the following model is set to reasonably control the manually driven vehicles within the fleet.

[0052] The judgment result is passed through the intelligent driver model set by the agent to simulate the driving behavior of human drivers;

[0053] Based on the intelligent driver model, the judgment results are combined to obtain the number of vehicles in the fleet and the driving speed;

[0054] In the step of obtaining the number of vehicles and driving speed in the fleet based on the intelligent driver model and combining the judgment results, the definition of the intelligent driver model is as follows:

[0055] ;

[0056] in, Indicates vehicle In time The acceleration of represents the maximum acceleration of the vehicle, represents the desired speed of the vehicle; Indicates at time Time Vehicle speed; represents the expected headway between the vehicle and the vehicle ahead, Indicates the actual headway; Indicates the minimum headway; Indicates a safe time interval; Indicates the speed difference between the vehicle and the vehicle ahead. represents the acceptable deceleration related to comfort, Indicates the speed of the vehicle.

[0057] Step 3: Input the number of vehicles in the convoy and the driving speed into the multi-layer perceptron to obtain the predicted time required for the convoy to reach the stop line of the intersection.

[0058] In step 3, the number of vehicles in the convoy and the driving speed are input into the multi-layer perceptron to obtain the predicted time required for the convoy to reach the stop line of the intersection, which specifically includes the following sub-steps:

[0059] Obtain the speed, location and size of the fleet at the preset time, and input them into the fully connected layer for feature combination to obtain the combined feature vector;

[0060] Among them, the relationship between the corresponding process is:

[0061] ;

[0062] in, represents the feature vector, Indicates the fleet at time speed, Indicates the fleet at time location, Indicates the fleet at time fleet size.

[0063] The combined feature vector is input into the LSTM layer to predict the time required for the convoy to reach the stop line at the intersection.

[0064] Furthermore, the agent communicates with the fleet in the lane to obtain the speed, position and fleet size of the fleet at the preset time. The obtained information is input into the fully connected layer and combined into a feature vector. Then, all hidden units are combined and input into the LSTM layer. There are three gate units in LSTM: input gate, forget gate and output gate. The unit receives input data, output data of the previous time step and state data of the previous time step, obtains the previous prediction time, prediction state and current fleet data, and inputs them into the input gate unit as input. The forget gate determines how much information in the memory unit of the previous time step will be retained to the current time step based on the arrival time prediction effect of the previous step. The output gate determines the value of the hidden state of the current time step. Finally, the accurate arrival time of the fleet is output to the agent through the hidden state. At the beginning of the agent decision-making process, the fleet arrival time prediction module is added, and the agent can obtain more accurate state data than ordinary observations, which provides the agent with more stable and vehicle-road cooperative control capabilities that conform to the current road network environment.

[0065] Step 4: Input the predicted time required for the convoy to reach the stop line of the intersection into the policy network of the intelligent agent. The policy network generates the status of each vehicle in the convoy, obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate an execution action; the value network generates a delayed feedback reward based on the execution action generated by the policy network, and uses positive and negative feedback to update the policy network to obtain an updated policy network.

[0066] In step 4, the predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent, the policy network generates the state of each vehicle in the convoy, and obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate an execution action; the value network generates a delayed feedback reward according to the execution action generated by the policy network, which specifically includes the following sub-steps:

[0067] The prediction results are input into the agent's policy network to obtain the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light;

[0068] The acceleration change of the convoy and the signal phase change of the traffic light are obtained by using the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light, and the acceleration change of the convoy and the signal phase change of the traffic light are used to generate the actions of the connected autonomous vehicle, the actions of the manually driven vehicle, the actions of the pilot connected autonomous vehicle, and the changes of the traffic light to form an action selection as a whole;

[0069] The value network performs action selection and generates delayed feedback rewards. When the convoy passes the intersection, the agent receives delayed feedback rewards.

[0070] In the step of inputting the prediction results into the agent's policy network to obtain the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light, the corresponding relationships in the process are as follows:

[0071] ;

[0072] in, express , , and Status information, Indicates the state of the connected autonomous vehicle. Indicates the state of a manually driven car. Indicates the status of the pilot network connected autonomous driving car. represents the traffic signal phase and duration of the traffic light, Indicates that connected self-driving cars will be The distance from the stop line of the downstream intersection to the Indicates that connected self-driving cars will be The speed of time, Indicates Car in time The speed of time, Indicates the fleet at time The moment The distance from the vehicle to the stop line, Indicates the current time One-hot encoding of time, Indicates the current intersection time The remaining time of the moment phase.

[0073] In the step of obtaining the acceleration change of the convoy and the signal phase change of the traffic light through the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light, and using the acceleration change of the convoy and the signal phase change of the traffic light to generate the actions of the connected autonomous vehicle, the actions of the manually driven vehicle, the actions of the pilot connected autonomous vehicle, and the changes of the traffic light to form an overall action selection, the acceleration change of the convoy is calculated, and the relationship between the corresponding process is:

[0074] ;

[0075] ;

[0076] in, represents the maximum deceleration of the vehicle, represents the maximum acceleration of the vehicle, Indicates the action of speed change. represents the original acceleration, Indicates time Time by Calculated acceleration.

[0077] It should be noted that in this way, the acceleration action space of the connected autonomous driving vehicle state can be stabilized within a safe range.

[0078] The value network performs action selection and generates delayed feedback rewards. When the convoy passes the intersection, the agent obtains delayed feedback rewards. The calculation of delayed feedback rewards corresponds to the following relationship:

[0079] ;

[0080] in, represents delayed feedback reward, represents the total number of vehicles passing through the intersection, Indicates vehicle Total fuel consumption, and Both represent weight parameters, Indicates the end of the training phase. Indicates vehicle travel delays.

[0081] The value network performs action selection and generates delayed feedback rewards. When the convoy passes the intersection, the agent obtains the delayed feedback reward. The calculation of the vehicle's travel delay is calculated by the following relationship:

[0082] ;

[0083] in, Indicates the arrival time of the last vehicle in the fleet. represents the arrival time of the fleet leader's connected autonomous vehicle, represents the length of the intersection, Indicates the maximum speed of the vehicle.

[0084] It should be noted that the values ​​related to fuel consumption and delay are calculated only when the time is equal to a certain time; at other time points, the result of the reward function is zero.

[0085] Furthermore, all mixed convoys and individual vehicles on the lane should be considered as part of the state. At the same time, the safe distance between the convoy and other vehicles should be considered, and the traffic conditions ahead should be predicted. In order to reduce the frequent acceleration and deceleration of the convoy during driving, the agent also needs to obtain information about traffic signals at nearby intersections.

[0086] Each agent acts as a central processing unit, collecting observable and predicted state information for processing and selecting actions, which consist of changes in the acceleration of the fleet and changes in the signal phase.

[0087] Forced acceleration and deceleration as actions are prone to traffic accidents, which often occur during training. In addition, as the speed increases, the vehicle is also prone to speeding, so the speed change action is modified.

[0088] See also Figure 6 There are four action spaces for signal phase change, corresponding to Figure 4 Shown: East-West turns left (first phase, ), east-west straight (second phase, ), north-south turn left (third phase, ) and north-south straight (the fourth phase, ). Since the right turn can be freely passed without the control of the traffic light, the right turn signal phase is always green in this embodiment. The hot coding is composed of "0" and "1", which respectively represent the signal as "red light" and "green light".

[0089] For example, in Time, Action , representing the first phase, is encoded by the agent as In this case, the traffic signal sequence corresponding to each lane can be expressed as "101100101100". At the intersection, there are 12 entrance lanes. The sequence "110100110100" can be divided into subsequences of 3 characters. The first three characters "110" indicate that the straight right turn in the east direction is green, and the left turn is red. "100" indicates that the right turn in the north direction is green, and the rest are red, as shown in Table 1 below:

[0090] Table 1: Relationship between signalized intersection actions, phases, and hot encoding

[0091] ;

[0092] The optimization objectives include waiting time, driving speed, pollution emissions, and travel delays, which are fed back in the reward function. However, travel delays can only be calculated when the vehicle passes through an intersection. Distributing delay rewards to each training stage is a complex problem. Other studies have used cumulative energy consumption and driving distance as the two parts of delay rewards. However, the cumulative driving distance cannot accurately represent the travel delay of the fleet. A more intuitive approach is to use delay rewards, which can directly reflect the optimization objectives.

[0093] Compared with other definitions, this definition of delay reward value is related to the state and action, and emphasizes travel delay and fuel consumption at the same time. While improving traffic efficiency, it also achieves the goal of energy conservation and environmental protection, providing important help for ecological traffic signal control.

[0094] Step 5: Use the updated policy network to generate optimized execution actions. The agent generates a recommended driving speed based on the optimized execution actions and sends it to the front of the fleet to dynamically optimize the driving trajectory of the fleet.

[0095] The intelligent agent continuously interacts and learns with the environment to obtain the optimal collaborative control strategy that responds to the dynamic characteristics of traffic flow, thereby alleviating road network congestion and reducing pollutant emissions.

[0096] See also Figure 3 The present invention also provides a double-layer ecological urban vehicle-road cooperative control system in a mixed traffic environment, the system comprising:

[0097] Intersection Agent Building Blocks for:

[0098] The traffic lights at each intersection are regarded as an intelligent agent, which obtains the real-time driving information of vehicles through communication between vehicles and infrastructure;

[0099] Fleet identification and segmentation module, used for:

[0100] The intelligent agent identifies the headway distance between vehicles based on the real-time driving information of the vehicles, and divides different vehicles into different fleets based on the headway distance to calculate the number of vehicles and driving speed in the fleet;

[0101] Fleet arrival time prediction module, used for:

[0102] The number of vehicles in the convoy and the driving speed are input into the multi-layer perceptron to obtain the time required for the convoy to reach the stop line of the intersection;

[0103] Model training module, used to:

[0104] The predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent. The policy network generates the status of each vehicle in the convoy, obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate execution actions; the value network generates delayed feedback rewards based on the execution actions generated by the policy network, and uses positive and negative feedback to update the policy network to obtain an updated policy network;

[0105] The vehicle-road cooperative control module is used to:

[0106] The updated policy network is used to generate optimized execution actions. The agent generates a recommended driving speed based on the optimized execution actions and sends it to the front of the fleet to dynamically optimize the driving trajectory of the fleet.

[0107] The intelligent agent continuously interacts and learns with the environment to obtain the optimal collaborative control strategy that responds to the dynamic characteristics of traffic flow, thereby alleviating road network congestion and reducing pollutant emissions.

[0108] See also Figure 4 Schematic diagram of the simulated road network experiment setup. In order to verify the effectiveness of the model proposed in the present invention, a 5×5 large-scale network-level road network was constructed on the traffic simulation simulator. The road network consists of 25 signalized intersections, with two lanes in the east-west direction and a single lane in the north-south direction. In order to simulate a safe driving environment, the speed limits of the single lane and the double lane are 45km / h and 70km / h respectively. Unlike the single straight traffic flow in most studies, the present invention constructs a complex traffic environment containing 6 mixed traffic flows, which include straight, left and right turns. And some overlapping traffic flows are included to increase the challenge of the training task.

[0109] See also Figure 5 The heat map of the simulated road network traffic flow. In the experiment, increasing the traffic flow in the road network can better test the stability of the model proposed by the present invention. Therefore, the hourly traffic flow is set to 400 per lane. and dual lane 800 . Figure 5 The heat map of the total traffic flow correlation within one hour in the road network is displayed. The heat map can intuitively reflect the mutual influence and traffic distribution of different roads under high traffic conditions, thereby verifying the stability and reliability of the model proposed in this invention in complex traffic environments.

[0110] See also Figure 7 Five based on RL As shown in the training curve of the vehicle-road cooperative control model, in order to better observe the long-term trend of the model performance, this section sets the number of training times to one million times, which helps to evaluate the convergence ability of the model. As the number of training times increases, the model learns through accumulated experience to achieve the optimal reward. The solid line in the figure represents the average training reward curve, and the shaded part represents the standard deviation of the reward value of the model. By comparison, it can be seen that as the number of training times increases, the average reward curve of the model proposed in the present invention also rises steadily, and the optimal training effect is achieved at the end of training, which fully demonstrates the effectiveness and superiority of the model proposed in the present invention in a large-scale road network environment. In contrast, other reinforcement learning models failed to show the same stability and convergence under the same number of training times, thereby verifying the advantages of the model proposed in the present invention in dealing with complex mixed traffic flow scenarios.

[0111] See also Figure 8 The model training stability curve of the double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment proposed in the present invention is shown in the figure. In order to verify the stability of the model proposed in the present invention, 5 independent trainings were performed by setting different random seeds to evaluate the stability of the model proposed in the present invention under different initial conditions. Figure 8 The result curves of these independent trainings are shown. It can be seen from the reward curve that the reward curve finally converges to around -400. Therefore, it can be proved that the model proposed in the present invention has better stability and robustness.

[0112] See also Fig. 9 Six based on RL The curve of the average queue length at the intersection of the vehicle-road cooperative control model in the simulated road network shows that the average queue length in the road network can well reflect the overall congestion situation. Therefore, this section uses the average queue length as the key indicator for model evaluation. Fig. 9 As shown, the reinforcement learning model is shown in the figure MDPPO, MA2C, IA2C, PPO and DQN Models, and traditional algorithms GLOSAThe average queue length curve of vehicles changing over time in a simulated road network environment. The results show that the algorithm in the framework proposed by the present invention exhibits excellent optimization control capabilities. Compared with other models, the model proposed by the present invention performs better in alleviating traffic congestion. The average queue length of other models also gradually increases with time, while the model proposed by the present invention can maintain a stable queue length after 1800 seconds, showing its stability and effectiveness in long-term operation. Other parameter indicators are shown in Table 2:

[0113] Table 2: Evaluation results of other indicators

[0114] ;

[0115] See also Fig.10 and Fig.11 The model of the dual-layer ecological city vehicle-road cooperative control method in a mixed traffic environment proposed by the present invention is shown in the spatiotemporal trajectory diagram of different networked autonomous driving vehicle presence rates. In order to study the impact of the presence ratio of different networked autonomous driving vehicle states on traffic congestion, tests were conducted in three different experimental environments: 20% networked autonomous driving vehicles and 60% networked autonomous driving vehicles. The specific results are shown in Fig.10 and Fig.11 shown.

[0116] In the above three environments, when there are only manually driven cars, it is difficult to accurately predict the arrival time of the vehicle due to the inability to interact with the central controller, resulting in long parking waiting time and traffic congestion. When there are 20% connected autonomous driving car states in the road network, mixed vehicles can be seen forming fleets. Since the connected autonomous driving car state has a strong communication capability, it can interact with the central controller and reasonably predict the arrival time of the fleet for trajectory control, thereby reducing the parking waiting time of the fleet and improving traffic efficiency. In contrast, when there are more connected autonomous driving car states on the road network, the model proposed by the present invention exhibits excellent control capabilities, which can greatly control the driving trajectory of the fleet, allowing more vehicles to pass within a signal cycle, avoiding long parking waits for vehicles.

[0117] See also Fig.12As shown in the bar graph of the performance of different modules in the model, different modules in the model proposed by the present invention are removed respectively to form three different variants: no formation of convoy and no control, formation of convoy but no cooperative control, cooperative control but no formation of convoy, and the complete model proposed by the present invention, that is, formation of convoy and cooperative control. The first variant is set as the baseline of this experiment and used as a reference. The travel delays of these variants are evaluated under the ratio of different connected autonomous vehicle states, so as to verify the importance of convoy identification, arrival time prediction and vehicle-road cooperative control modules in the model proposed by the present invention.

[0118] From the figure, we can see that as the presence ratio of connected autonomous vehicles increases, the overall travel delay in the road network decreases. Compared with the baseline, P-NC By dividing the mixed vehicles into several fleets, a certain travel delay optimization effect is achieved. NP-C The vehicles are not divided into fleets for control, but a significant optimization is also achieved. However, the model proposed by the present invention, which includes the complete module, performs best in terms of travel delay optimization. It can be seen that the method proposed by the present invention is feasible by dividing the vehicles into fleets for arrival time prediction and then performing vehicle-road cooperative control, and has excellent cooperative control capabilities.

[0119] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0121] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A dual-layer ecological city vehicle-road cooperative control method in a mixed traffic environment, characterized in that: The method comprises the following steps: Step 1: Consider the traffic lights at each intersection as an intelligent agent, which obtains real-time driving information of vehicles through communication between vehicles and infrastructure; Step 2: The agent identifies the headway distance between vehicles based on the real-time driving information of the vehicles, and divides different vehicles into different fleets based on the headway distance to calculate the number of vehicles and driving speed in the fleet; Step 3: Input the number of vehicles in the convoy and the driving speed into a multi-layer perceptron to obtain the time required for the convoy to reach the stop line of the intersection; Step 4: The predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent. The policy network generates the status of each vehicle in the convoy, and obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate an execution action; the value network generates a delayed feedback reward according to the execution action generated by the policy network, and uses positive and negative feedback to update the policy network to obtain an updated policy network; Step 5: Use the updated policy network to generate optimized execution actions. The agent generates a recommended driving speed based on the optimized execution actions and sends it to the front of the fleet to dynamically optimize the driving trajectory of the fleet. The intelligent agent continuously interacts with the environment to learn and obtain the optimal collaborative control strategy that responds to the dynamic characteristics of traffic flow, so as to alleviate road network congestion and reduce pollutant emissions; In step 4, the predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent, the policy network generates the state of each vehicle in the convoy, and obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate an execution action; the value network generates a delayed feedback reward according to the execution action generated by the policy network, which specifically includes the following sub-steps: The prediction results are input into the agent's policy network to obtain the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light; The acceleration change of the vehicle fleet and the signal phase change of the traffic light are obtained through the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light; and the acceleration change of the vehicle fleet and the signal phase change of the traffic light are used to generate the actions of the connected autonomous vehicle, the actions of the manually driven vehicle, the actions of the pilot connected autonomous vehicle, and the changes of the traffic light to form an overall action selection; The value network performs action selection and generates delayed feedback rewards. When the convoy passes the intersection, the agent receives delayed feedback rewards. In the step of inputting the prediction results into the agent's policy network to obtain the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light, the corresponding relationship is as follows: ； in, express , , and Status information, Indicates the state of the connected autonomous vehicle. Indicates the state of a manually driven car. Indicates the status of the pilot network connected autonomous driving car. represents the traffic signal phase and duration of the traffic light, Indicates that connected self-driving cars will be The distance from the stop line of the downstream intersection to the Indicates that connected self-driving cars will be The speed of time, Indicates Car in time speed, Indicates the fleet at time The moment The distance from the vehicle to the stop line, Indicates the current time One-hot encoding of time, Indicates the current intersection time The remaining time of the moment phase.

2. According to the double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to claim 1, it is characterized in that: In step 2, different vehicles are divided into different convoys according to the headway distance, so as to calculate the number of vehicles and the driving speed in the convoy. The specific steps include the following: The intelligent agent calculates the headway of the vehicles in the recognition area, and determines the vehicle's belonging to the fleet based on the calculation result to obtain a judgment result; Among them, the relationship between the headway distance in the identification area is: ； in, represents the current presence ratio of automated vehicles on the road, Indicates the maximum headway distance, Indicates the headroom; The judgment result is passed through the intelligent driver model set by the agent to simulate the driving behavior of human drivers; Based on the intelligent driver model, the judgment results are combined to obtain the number of vehicles and driving speed in the fleet.

3. The double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to claim 2 is characterized in that: In the step of obtaining the number of vehicles in the convoy and the driving speed by combining the judgment results based on the intelligent driver model, the definition of the intelligent driver model is as follows: ； in, Indicates vehicle In time The acceleration of represents the maximum acceleration of the vehicle, represents the desired speed of the vehicle; Indicates at time vehicle speed; represents the expected headway between the vehicle and the vehicle ahead, Indicates the actual headway; Indicates the minimum headway; Indicates a safe time interval; Indicates the speed difference between the vehicle and the vehicle ahead. represents the acceptable deceleration related to comfort, Indicates the speed of the vehicle.

4. The double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to claim 3 is characterized in that: In step 3, the number of vehicles in the convoy and the driving speed are input into a multi-layer perceptron to obtain the time required for the convoy to reach the stop line of the intersection, which specifically includes the following sub-steps: Obtain the speed, location and size of the fleet at the preset time, and input them into the fully connected layer for feature combination to obtain the combined feature vector; Among them, the relationship between the corresponding process is: ； in, represents the feature vector, Indicates the fleet at time speed, Indicates the fleet at time location, Indicates the fleet at time The size of the fleet; The combined feature vector is input into the LSTM layer to predict the time required for the convoy to reach the stop line at the intersection.

5. The double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to claim 1 is characterized in that: In the step of obtaining the acceleration change of the convoy and the signal phase change of the traffic light through the state of the connected autonomous vehicle, the state of the manually driven vehicle, the state of the pilot connected autonomous vehicle, and the traffic signal phase and duration of the traffic light, and using the acceleration change of the convoy and the signal phase change of the traffic light to generate the action of the connected autonomous vehicle, the action of the manually driven vehicle, the action of the pilot connected autonomous vehicle and the change of the traffic light to form the action selection as a whole, the acceleration change of the convoy is calculated, and the corresponding process has the relationship formula: ； ； in, represents the maximum deceleration of the vehicle, represents the maximum acceleration of the vehicle, Indicates the action of speed change. represents the original acceleration, Indicates time Time by Calculated acceleration.

6. The double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to claim 5 is characterized in that: The value network performs action selection and generates delayed feedback rewards. When the convoy passes the intersection, the agent obtains delayed feedback rewards. The calculation of delayed feedback rewards corresponds to the following relationship: ； in, represents delayed feedback reward, represents the total number of vehicles passing through the intersection, Indicates vehicle Total fuel consumption, and Both represent weight parameters, Indicates the end of the training phase. Indicates vehicle travel delays.

7. The double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to claim 6 is characterized in that: The value network performs action selection and generates delayed feedback rewards. When the convoy passes the intersection, the agent obtains the delayed feedback reward. The calculation of the vehicle's travel delay is calculated by the following relationship: ； in, Indicates the arrival time of the last vehicle in the fleet. represents the arrival time of the fleet leader's connected autonomous vehicle, represents the length of the intersection, Indicates the maximum speed of the vehicle.

8. A dual-layer ecological urban vehicle-road cooperative control system in a mixed traffic environment, characterized in that: The system applies the double-layer ecological city vehicle-road cooperative control method in a mixed traffic environment according to any one of claims 1 to 7, and the system comprises: Intersection Agent Building Blocks for: The traffic lights at each intersection are regarded as an intelligent agent, which obtains the real-time driving information of vehicles through communication between vehicles and infrastructure; Fleet identification and segmentation module, used for: The intelligent agent identifies the headway distance between vehicles based on the real-time driving information of the vehicles, and divides different vehicles into different fleets based on the headway distance to calculate the number of vehicles and driving speed in the fleet; Fleet arrival time prediction module, used for: The number of vehicles in the convoy and the driving speed are input into the multi-layer perceptron to obtain the time required for the convoy to reach the stop line of the intersection; Model training module, used to: The predicted time required for the convoy to reach the stop line of the intersection is input into the policy network of the intelligent agent. The policy network generates the status of each vehicle in the convoy, obtains the acceleration change and signal phase change of the convoy through the intelligent agent, and processes the acceleration change and signal phase change of the convoy to generate execution actions; the value network generates delayed feedback rewards based on the execution actions generated by the policy network, and uses positive and negative feedback to update the policy network to obtain an updated policy network; The vehicle-road cooperative control module is used to: The updated policy network is used to generate optimized execution actions. The agent generates a recommended driving speed based on the optimized execution actions and sends it to the front of the fleet to dynamically optimize the driving trajectory of the fleet. The intelligent agent continuously interacts and learns with the environment to obtain the optimal collaborative control strategy that responds to the dynamic characteristics of traffic flow, thereby alleviating road network congestion and reducing pollutant emissions.

Images