Hard shoulder driving and variable speed limit cooperative control method and system

Abstract

The invention relates to the technical field of intelligent traffic systems, and discloses a hard shoulder driving and variable speed limit cooperative control method and system.The method comprises the steps that traffic data are collected, and the traffic data comprise traffic information and speed limit values; according to the hard shoulder lane-borrowing driving motivation, the lane-borrowing safety condition and the emergency or rescue vehicle requirement of the networked autonomous vehicle, whether the networked autonomous vehicle can run through the hard shoulder lane or not is judged; inputting the collected traffic data into a pre-constructed DDPG algorithm to determine a state, an action space and a reward function of the DDPG algorithm, and outputting an optimal speed limit value; and the optimal speed limit value of each lane of the control road section is generated through a DDPG algorithm, and the network-connected automatic driving vehicle adjusts the vehicle speed according to the optimal speed limit value. The method can be used for highway entrance ramp sections with serious congestion, the average travel time and carbon dioxide emission of the road network can be reduced, and a new method is provided for efficient and low-carbon targets of smart highways in China.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent transportation systems, and relates to a method and system for coordinated control of hard shoulder driving and variable speed limit, and specifically to a method and system for coordinated control of hard shoulder driving and variable speed limit in a mixed traffic flow environment. Background Art

[0002] Highways often have traffic bottlenecks due to factors such as merging and emergencies, and often bear tremendous transportation pressure during special periods such as holidays and commuting rush hours. The emergence of efficiency and safety issues in bottleneck areas restricts the transportation capacity of the entire highway. Currently, hard shoulders and variable speed limits are popular active traffic management measures.

[0003] The Hard Shoulder Running (HSR) strategy is an active management measure to solve traffic congestion on bottleneck sections of highways. It opens the hard shoulder lanes of highways according to conditions and allows vehicles on the driving lanes to use the lanes. However, at present, hard shoulder running is mostly based on non-intelligent networked environments, that is, manual vehicles are allowed to use the hard shoulder lanes. However, the width of the hard shoulder lanes is often smaller than the driving lanes, and the speed of highway vehicles is high. It is difficult for manual vehicles to control the lateral direction on the hard shoulder lanes. The safety of hard shoulder running is difficult to ensure, and the real-time control problem of hard shoulder driving strategies is difficult to solve. In the future mixed traffic flow environment, only allowing connected and automated vehicles (CAV) to use the hard shoulder will help solve the above problems. This is because CAV has high driving stability, which can ensure the safety of hard shoulder driving, and CAV can receive emergency vehicle or rescue vehicle demand information to ensure the basic needs of hard shoulder lanes.

[0004] Similar to general lanes, if speed limit measures are not taken for hard shoulder lanes, safety problems are likely to occur. Therefore, it is necessary to stipulate the speed limit value for hard shoulder lanes. At present, the speed limit measures for highways are generally fixed speed limits, which are not flexible enough and cannot respond to real-time changes in the traffic system. Traditional variable speed limits (VSL) include rule-based methods and feedback-based methods. These methods are suitable for sections with relatively regular traffic conditions, but cannot show good results in sections with complex and changeable traffic conditions. Deep reinforcement learning (DRL) combines the perception ability of deep learning and the decision-making ability of reinforcement learning, and provides a new method for active traffic management measures including variable speed limits. With the increasing demand for high-quality road service levels and the increasingly complex traffic flow environment, only taking one active traffic management measure shows that the optimization effect is not obvious. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for coordinated control of hard shoulder driving and variable speed limit, which can be used for seriously congested highway entrance ramp sections, can reduce the average travel time and carbon dioxide emissions of the road network, and provide a new method for my country's high-efficiency and low-carbon smart highway goals.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0007] In a first aspect, the present invention provides a method for coordinated control of hard shoulder driving and variable speed limit, comprising the following steps:

[0008] Collecting traffic data, the traffic data including traffic information and speed limit values; the traffic information including the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving vehicle; the speed limit value includes the speed limit value of the control section;

[0009] Determine whether the connected autonomous vehicle can use the hard shoulder lane based on the connected autonomous vehicle's motivation for using the hard shoulder lane, safety conditions for using the hard shoulder lane, and the need for emergency or rescue vehicles;

[0010] Input the collected traffic data into the pre-built DDPG algorithm to determine the state, action space, and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool, store the collected traffic information and speed limit value in the experience replay pool, and use the priority experience replay method for sampling in subsequent training;

[0011] The result of being able to drive on the hard shoulder lane is combined with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit collaborative control method. The optimal speed limit value for each lane of the controlled section is generated by the DDPG algorithm. The networked autonomous driving car adjusts the speed according to the optimal speed limit value, and the driver of the manually driven car also adjusts the speed according to the optimal speed limit value.

[0012] In combination with the first aspect, further, the roadside unit transmits the optimal speed limit value to the variable information board for display, and the driver of the manually driven car adjusts the vehicle speed according to the optimal speed limit value displayed on the variable information board.

[0013] In combination with the first aspect, further, the specific method for collecting traffic data is:

[0014] Several detectors are respectively set up in the control section, the bottleneck section and the entrance ramp, and the detectors are used to obtain the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving car; among them, the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the position, speed and acceleration of the networked autonomous driving car are directly obtained through the detector collection; the average speed of all vehicles in the bottleneck section is calculated by collecting the speeds of all vehicles in the bottleneck section; the carbon dioxide emissions are directly calculated by the simulation software through the data collected by the detector;

[0015] Roadside units are deployed on the roadside, and the networked autonomous driving vehicles can exchange information with the roadside units; the roadside units are used to collect the position, speed and acceleration of the networked autonomous driving vehicles, and send the speed limit value of the controlled road section to the networked autonomous driving vehicles; the networked autonomous driving vehicles can obtain the position, speed and acceleration of other networked autonomous driving vehicles through the roadside units;

[0016] The detector and the roadside unit send the collected traffic information to the traffic control center, and the traffic control center calculates the received traffic information to obtain the optimal speed limit value of the controlled road section.

[0017] Combined with the first aspect, the specific method for determining whether the connected autonomous driving vehicle has the motivation to drive in the hard shoulder lane is as follows:

[0018] The motivation for driving on the hard shoulder means that the connected autonomous vehicle cannot drive at the expected speed in the current lane, and the hard shoulder lane conditions can meet the current conditions for the connected autonomous vehicle to drive at the expected speed; the specific method for determining the motivation for driving on the hard shoulder is:

[0019] ;

[0020] ;

[0021] in: is the distance between the connected autonomous vehicle and the vehicle in front. is the length of the connected autonomous vehicle, Refers to the speed of connected autonomous vehicles. Refers to the acceleration of connected autonomous vehicles. Refers to the maximum speed of a connected autonomous vehicle. It is the headway between the connected autonomous vehicle and the vehicle ahead on the hard shoulder lane.

[0022] In combination with the first aspect, further, a specific method for determining whether a vehicle with the intention of using the hard shoulder for driving meets the safety conditions for using the hard shoulder is as follows:

[0023] The lane-changing behavior will not interfere with the normal driving of the vehicle behind in the hard shoulder lane, and the lane-changing behavior meets the safety clearance requirements; the specific judgment method for meeting the safety conditions for lane-changing is:

[0024] ;

[0025] ;

[0026] in: is the distance between the current connected autonomous vehicle and the vehicle behind it in the hard shoulder lane. is the speed of the vehicle behind on the hard shoulder lane, Refers to the safe gap between the front and rear vehicles on the hard shoulder lane; For safe headway, Refers to the acceleration of the vehicle behind on the hard shoulder lane. It is the maximum braking deceleration of the connected autonomous vehicle.

[0027] If the traffic control center receives a request from an emergency or rescue vehicle to enter the hard shoulder lane, the networked autonomous vehicle on the driving lane cannot use the hard shoulder lane through communication between the roadside unit and the connected autonomous vehicle, and the non-emergency networked autonomous vehicle on the hard shoulder lane must leave the hard shoulder lane as soon as possible. If there is no demand from an emergency or rescue vehicle to enter the hard shoulder lane, the networked autonomous vehicle that meets the motivation and safety conditions for using the lane can use the hard shoulder lane.

[0028] Combined with the first aspect, further, the specific method for determining the state, action space and reward function of the DDPG algorithm is:

[0029] Construct the value network and policy network of the DDPG algorithm. The input layer of the policy network is based on the state, the output layer is based on the action, and the policy network parameters are ; The input layer of the value network is based on state and action, and the value network parameters are The states based on the input layer of the strategy network and the input layer of the value network are both the occupancy rates of the controlled sections and bottleneck sections. The actions based on the output layer of the strategy network are the speed limit values, and the action space is the sum of the number of lanes of the driving lane and the hard shoulder lane.

[0030] The detector is used to collect the occupancy rate of the control section and the bottleneck section, which is recorded as state ;

[0031] The average speed of the bottleneck section is collected by the detector and recorded as the first reward ; Use the detector to collect the carbon dioxide emissions of the road network, which is recorded as the second reward ; According to the reward weight coefficient and the standardized coefficient Calculate total rewards , then the reward function is:

[0032] ;

[0033] The continuous action space generated by the DDPG algorithm is converted into a discrete action space through mapping;

[0034] Establish an experience replay pool and collect the state at each time step t ,action ,award The next moment state , recorded as an experience , and store it in the experience replay pool; in the experience replay pool, the sequence of each experience sample Absolute value of TD error Negatively correlated, Indicates the first Experience samples;

[0035] When the experience in the experience replay pool exceeds the minimum training capacity requirement, sampling is performed in the experience replay pool for training. The sampling method is the priority experience replay method. The probability of an experience sample being drawn .

[0036] Combined with the first aspect, further, the result of being able to drive on the hard shoulder lane is combined with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit coordinated control method, and the value network and strategy network parameters of the DDPG algorithm are optimized using the collected traffic data to generate the optimal speed limit value. The specific method is:

[0037] Initialize policy network parameters and value network parameters , get the initialization target strategy network parameters Network parameters with target value , initialize the experience replay pool;

[0038] Create an iterative loop for DDPG algorithm training, where each loop contains M time steps;

[0039] Variable speed limit agent obtains state ;

[0040] The policy network is based on the state Generate Action ;in As the strategy function, add normal distribution when generating actions Noise ,Right now ;

[0041] Set the speed limit for the controlled section and get rewards The next moment state ;

[0042] Experience Stored in the experience replay pool;

[0043] If the minimum training capacity of the experience replay pool is reached, W experience samples are extracted for training using the priority experience replay method. The probability of each experience sample being extracted is , Represents the sequence of experience samples in the experience replay pool; Indicates the first Experience samples;

[0044] Using value network calculation The action value of the experience sample , and predict the next moment The action value of the experience sample ,in Indicates The state of the experience sample, Indicates Actions of experience samples; Indicates the next moment of a sample of experience;

[0045] Calculate the TD error of empirical samples ,in, represents the discount factor; Indicates Experience sample reward;

[0046] Update the value network parameters:

[0047] ;

[0048] in, is the sample size, ;

[0049] Update the policy network parameters, where the policy network gradient calculation method is as follows:

[0050] ;

[0051] According to TD error Update the probability of experience being extracted, the absolute value of TD error With sample sequence Negative correlation;

[0052] Update target value network parameters , update the target strategy network parameters ;in, Indicates soft update parameters;

[0053] The optimal speed limit value for each lane of the controlled road section is generated through the strategy network and sent to the roadside unit via the traffic control center. The networked autonomous driving car adjusts the speed according to the optimal speed limit value sent by the roadside unit, and the driver of the manually driven car adjusts the speed according to the optimal speed limit value.

[0054] In a second aspect, the present invention proposes a hard shoulder driving and variable speed limit coordinated control system, comprising:

[0055] A traffic data collection module is configured to collect traffic data, wherein the traffic data includes traffic information and speed limit values; the traffic information includes the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving vehicle; the speed limit value includes the speed limit value of the control section;

[0056] A lane-passing judgment module is configured to judge whether the connected autonomous driving vehicle can drive in the hard shoulder lane according to the driving motivation of the connected autonomous driving vehicle to drive in the hard shoulder lane, the lane-passing safety conditions, and the demand for emergency or rescue vehicles;

[0057] The DDPG algorithm module is configured to input the collected traffic data into the pre-built DDPG algorithm to determine the state, action space and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit value in the experience replay pool;

[0058] The collaborative control module is configured to combine the result of being able to drive on the hard shoulder lane with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit collaborative control method, generate the optimal speed limit value for each lane of the controlled section through the DDPG algorithm, the networked autonomous driving car adjusts the speed according to the optimal speed limit value, and the driver of the manually driven car adjusts the speed according to the optimal speed limit value.

[0059] In a third aspect, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned collaborative control method.

[0060] In a fourth aspect, the present invention provides a computer device, comprising:

[0061] Memory for storing computer programs;

[0062] A processor is used to execute the computer program to implement the steps of the above-mentioned collaborative control method.

[0063] In a fifth aspect, a computer program product includes a computer program, which implements the steps of the above-mentioned collaborative control method when executed by a processor.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] (1) The present invention can be used for seriously congested highway entrance ramp sections, which can reduce the average travel time and carbon dioxide emissions of the road network, and provide a new method for achieving my country's high-efficiency and low-carbon smart highway goals.

[0066] (2) The networked autonomous driving vehicles that meet the requirements of the present invention can travel on the hard shoulder, which helps to alleviate traffic congestion on the main line of the expressway and make full use of road resources.

[0067] (3) The present invention proposes a variable speed limit control strategy based on the DDPG algorithm to improve traffic efficiency and reduce carbon dioxide emissions, realize the combined control of speed limits of different lanes, standardize the speed of upstream vehicles, and adjust the supply and demand relationship in the bottleneck area.

[0068] (4) The collaborative control method of the present invention overcomes the problem that traditional methods have poor performance in complex traffic scenarios, and provides a new method for active traffic management on my country's smart highways. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 Schematic diagram of the process of the collaborative control method in Example 1 of the present invention;

[0070] Figure 2 A schematic diagram illustrating a road network according to Embodiment 1 of the present invention;

[0071] Figure 3 Schematic diagram of average travel time of a road network under different penetration rates of connected autonomous driving vehicles in Example 1 of the present invention;

[0072] Figure 4 Schematic diagram of carbon dioxide emissions from a road network at different penetration rates of connected autonomous driving vehicles in Example 1 of the present invention. DETAILED DESCRIPTION

[0073] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0074] The term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the related objects are in an "or" relationship.

[0075] Example 1

[0076] like Figure 1 As shown, this embodiment introduces a method for coordinated control of hard shoulder driving and variable speed limit in a mixed traffic flow environment, including the following steps:

[0077] Step S1: Collect traffic data, which includes traffic information and speed limit values; traffic information includes the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of vehicles in the bottleneck section, and carbon dioxide emissions; the speed limit value includes the speed limit value of the control section; the networked autonomous driving vehicle can exchange information with the roadside unit through V2I communication technology.

[0078] The specific steps of step S1 are as follows:

[0079] Step S11: Figure 2 As shown in the figure, the road section includes the main line section and the entrance ramp section. The main line includes several lanes and hard shoulder lanes. For example, when there are three lanes, the lanes are numbered from right to left as lane 1, lane 2, and lane 3 according to the vehicle's forward direction. The entrance ramp is a single lane. The upstream of the entrance ramp is a controlled section. Connected autonomous vehicles that meet the conditions can drive on the hard shoulder lane. The area around the entrance ramp is a bottleneck section.

[0080] Step S12: a plurality of detectors are respectively arranged on the control section, the bottleneck section and the entrance ramp, corresponding to the control section detector, the bottleneck section detector and the ramp detector respectively, and the detectors are used to obtain the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions and the position, speed and acceleration of the networked autonomous driving vehicle; wherein, the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the position, speed and acceleration of the networked autonomous driving vehicle are directly obtained by collecting the detectors; the average speed of all vehicles in the bottleneck section is calculated by collecting the speeds of all vehicles in the bottleneck section; the carbon dioxide emissions are directly calculated by the simulation software through the data collected by the detectors;

[0081] Step S13: Roadside units are deployed on the roadside. The networked autonomous driving vehicle can use V2I communication technology to exchange information with the roadside unit. The roadside unit is used to collect the position, speed, acceleration and other traffic information of the networked autonomous driving vehicle, and send the speed limit value of the controlled section to the networked autonomous driving vehicle. The networked autonomous driving vehicle can obtain the position, speed, acceleration and other traffic information of other surrounding networked autonomous driving vehicles through the roadside unit.

[0082] Step S14: The detector and the roadside unit send the collected traffic information to the traffic control center TCC, and the traffic control center TCC calculates the received traffic information to obtain the optimal speed limit value of the controlled road section.

[0083] Step S2: Determine whether the networked autonomous driving vehicle can drive on the hard shoulder lane based on the driving motivation of the networked autonomous driving vehicle in using the hard shoulder lane, the safety conditions for using the hard shoulder lane, and the need for emergency or rescue vehicles.

[0084] The specific steps of step S2 are as follows:

[0085] Step S21: Determine whether there is a motivation to drive on the hard shoulder. The motivation means that the networked autonomous driving vehicle cannot drive at the expected speed in the current lane, and the hard shoulder lane conditions can meet the current networked autonomous driving vehicle to drive at the expected speed. The specific method for determining whether there is a motivation to drive on the hard shoulder is:

[0086]

[0087]

[0088] in: is the distance between the connected autonomous vehicle and the vehicle in front. is the length of the connected autonomous vehicle, Refers to the speed of connected autonomous vehicles. Refers to the acceleration of connected autonomous vehicles. Refers to the maximum speed of a connected autonomous vehicle. It is the headway between the connected autonomous vehicle and the vehicle ahead on the hard shoulder lane.

[0089] Step S22: Determine whether the vehicle that has the motivation to use the hard shoulder meets the safety conditions for using the lane, that is, the lane-changing behavior will not interfere with the normal driving of the vehicle behind on the hard shoulder lane, and the lane-changing behavior meets the safety gap requirements. The specific method for determining whether the safety conditions for using the lane are met is:

[0090]

[0091]

[0092] in: is the distance between the current connected autonomous vehicle and the vehicle behind it on the hard shoulder lane. is the speed of the vehicle behind on the hard shoulder lane, Refers to the safe gap between the front and rear vehicles on the hard shoulder lane; For safe headway, Refers to the acceleration of the vehicle behind on the hard shoulder lane. It is the maximum braking deceleration of the connected autonomous vehicle.

[0093] Step S23: If the traffic control center TCC receives a request from an emergency or rescue vehicle to enter the hard shoulder lane, it communicates with the networked autonomous driving vehicle through the roadside unit. At this time, the networked autonomous driving vehicle on the driving lane cannot use the hard shoulder lane, and the non-emergency networked autonomous driving vehicle on the hard shoulder lane leaves the hard shoulder lane as soon as possible. If there is no demand for an emergency or rescue vehicle to enter the hard shoulder lane, the networked autonomous driving vehicle that meets the driving motivation and safety conditions for using the hard shoulder lane can use the hard shoulder lane.

[0094] As long as the connected autonomous driving car on the controlled road section meets the three conditions in step S2 (i.e., the motivation for using the lane, the safety conditions for using the lane, and the need for emergency or rescue vehicles), it can choose to use the hard shoulder lane.

[0095] Step S3: Use the traffic data collected in step S1 to establish a DDPG algorithm for variable speed limits, determine the parameters, state, action space, and reward function of the DDPG algorithm; build an experience replay pool, store the collected traffic information and speed limit values ​​in the experience replay pool, and use the priority experience replay method for sampling in subsequent training.

[0096] The specific steps of step S3 are as follows:

[0097] Step S31: Construct the value network and policy network of the DDPG algorithm. The policy network input layer is based on the state, the output layer is based on the action, and the policy network parameters are ; The value network input layer is based on state and action, and the value network parameters are , the action based on the value network input layer is the speed limit value collected by the detector, the state based on the strategy network input layer and the state based on the value network input layer are the occupancy rates of the control section and the bottleneck section. When there are three lanes, the action space dimension of the DDPG algorithm is 4, including the speed limits of the three lanes and the speed limit of the hard shoulder lane. After reaching the hard shoulder lane, the speed needs to be adjusted according to the speed limit of the hard shoulder lane.

[0098] The DDPG algorithm is based on the Actor-Critic framework, including a value network (Critic network), a policy network (Actor network), an experience replay pool, and a target network; the value network is responsible for learning deterministic policies, that is, directly outputting action values ​​under a given state; the policy network is responsible for learning value functions and evaluating the value of the current state; the experience replay pool is used to store the experience gained from the interaction between the agent and the environment, and is used for random sampling during training to solve sample correlation and stability problems; the target network includes a target policy network (Target Actor network) and a target value network (Target Critic network), which are used to estimate the target Q value and target policy, which are gradually updated from the parameters of the main network through soft updates to stabilize the training process.

[0099] The update strategy of the DDPG algorithm includes the following steps:

[0100] Experience replay: Randomly sample a batch of experience data from the experience replay pool to update the parameters of the Actor network and Critic network.

[0101] Critic network update: Use the sampled empirical data to update the parameters of the Critic network by minimizing the loss function, which is usually defined as the square of the difference between the current Q value and the target Q value.

[0102] Actor network update: Use the gradient information evaluated by the Critic network to update the parameters of the Actor network through the gradient ascent method to maximize the expected return.

[0103] Target network update: Use soft update to gradually copy the parameters of the main network to the target network to maintain the stability of the target network.

[0104] In addition, in order to increase the exploration ability of the agent, noise is added to the actions output by the Actor network during the training phase.

[0105] When a connected autonomous vehicle meets the conditions for driving in the hard shoulder lane, the lane changing process can be achieved through the lane changing model of the simulation software.

[0106] Step S32: Setting DDPG algorithm parameters. The DDPG algorithm parameter setting method is to determine the final parameters according to the parameter setting range and after training. For example, in this embodiment, the learning rate of the Actor strategy network is 0.0005, the learning rate of the Critic value network is 0.001, and the optimization method of the two neural network parameters uses the Adam optimizer; the discount coefficient is 0.98, the soft update parameter is 0.005; the noise standard deviation is 0.1; the experience replay pool size is 5000, the minimum training capacity is 100, the number of experiences sampled each time is 64; the training rounds are 500.

[0107] Step S33: Use the detector to collect the occupancy rate of the control section and the bottleneck section, and record it as state ;

[0108] Step S34: Use the detector to collect and calculate the average speed of the bottleneck section, which is recorded as the first reward , calculate the carbon dioxide emissions of the road network based on the carbon emission model, which is recorded as the second reward , according to the reward weight coefficient and the standardized coefficient Calculate total rewards , then the reward function is as follows.

[0109]

[0110] Step S35: In step S3, the action of the DDPG algorithm is a continuous action space, which needs to be converted into a discrete action space that meets the speed limit requirements. The continuous action generated by the algorithm is mapped to the discrete action space. The mapping method is: divide the agent action into (0,13) and obtain its integer part discrete value Considering the stability of the driver's response to the speed limit, the speed limit is set to an integer multiple of 5km / h and a minimum speed limit is specified. and maximum speed limit , specifically:

[0111]

[0112]

[0113] in, Indicates the optimized speed limit value. represents discrete values, Represents the action output by the policy network (hereinafter referred to as action).

[0114] Step S36: Establish an experience replay pool. At each time step t, the variable speed limit agent (referring to the variable speed limit control module of the traffic control center) interacts with the traffic environment and collects the state ,action ,award The next moment state , recorded as an experience , and store it in the experience replay pool. The sequence of each experience sample in the experience replay pool Absolute value of TD error Negatively correlated, Indicates the first An experience sample.

[0115] Step S37: When the experience in the experience replay pool exceeds the minimum training capacity requirement, samples can be drawn from the experience replay pool for training. The sampling method is the priority experience replay method. The probability of each experience sample being drawn is .

[0116] Step S4: Based on the hard shoulder driving method proposed in step S2 and the DDPG algorithm with variable speed limit in step S3, a collaborative control method is established and the collaborative control agent training is performed. The traffic data sampled in step S1 is used to continuously optimize the value network and strategy network parameters of the DDPG algorithm and generate the optimal speed limit value.

[0117] The specific steps of establishing the collaborative control method in step S4 are as follows:

[0118] Step S41: Initialize policy network parameters and value network parameters , get the initialization target strategy network parameters Network parameters with target value , initialize the experience replay pool.

[0119] Step S42: creating an iterative loop for DDPG algorithm training, each loop containing M time steps;

[0120] Step S43: The agent interacts with the traffic environment to obtain the state ;

[0121] Step S44: The policy network is based on the state Generate Action ;in As the strategy function, add normal distribution when generating actions Noise ,Right now .

[0122] Step S45: Set a speed limit for the controlled section and obtain a reward The next moment state ;

[0123] Step S46: Experience Stored in the experience replay pool;

[0124] Step S47: If the minimum training capacity of the experience replay pool is reached, extract using the priority experience replay method experience samples for training (preferably =64), the probability of each experience sample being drawn , Represents the sequence of experience samples in the experience replay pool;

[0125] Step S48: Calculate the value network The action value of the experience sample , and predict the action value at the next moment ,in, Indicates The state of the experience sample, Indicates Actions of experience samples; Indicates the next moment of a sample of experience;

[0126] Step S49 calculates the TD error of empirical samples ,in represents the discount factor; Indicates the first Experience samples; Indicates Experience sample reward.

[0127] Step S410: Update value network parameters;

[0128]

[0129] in, is the sample size, ;

[0130] Step S411: Update the policy network parameters, where the policy network gradient calculation method is as follows:

[0131]

[0132] Step S412: According to the TD error Update the probability of experience being extracted, the absolute value of TD error With sample sequence Negative correlation;

[0133] Step S413: Update target value network parameters , update the target strategy network parameters ;in, Indicates soft update parameters;

[0134] Step S414: Generate the optimal speed limit value for each lane of the controlled road section through the strategy network, and send it to the roadside unit via the traffic control center. The networked autonomous driving vehicle adjusts the speed according to the optimal speed limit value sent by the roadside unit, and the driver of the manually driven vehicle adjusts the speed according to the optimal speed limit value displayed on the variable information board.

[0135] A typical highway section with an entrance ramp is designed in the traffic simulation software, where the initial speed limit of the main lane is 120km / h, the speed limit of the hard shoulder lane is 100km / h, and the speed limit of the entrance ramp is 60km / h. The vehicles on the road include manually driven vehicles and networked self-driving cars. The hard shoulder lane is open in the control section and variable speed limit control is adopted. The hard shoulder lane is also open in the bottleneck section. The mainline vehicle input is 4950 vehicles / hour, the ramp vehicle input is 743 vehicles / hour, and the penetration rate of networked self-driving cars is 40%. A total of 500 rounds of simulation are carried out, and each round of simulation lasts 3600 seconds.

[0136] The hard shoulder driving and variable speed limit collaborative control method was applied to the controlled section. Compared with not using the collaborative control method of the present invention, the average travel time of the road network was reduced by 15.6% and the carbon dioxide emissions were reduced by 6.7%. A comparative experiment was conducted with 20%, 40%, 60%, 80%, and 100% penetration rates of intelligent networked vehicles. Figure 3 and Figure 4 Compared with not taking active traffic management measures, the collaborative control method achieves optimization results at different penetration rates. The best efficiency improvement effect is achieved at a penetration rate of 40%, which can reduce the average road network time by 15.6%. The best emission reduction effect is achieved at a penetration rate of 80%, which reduces carbon dioxide emissions by 9.0%.

[0137] Example 2

[0138] Based on the same inventive concept as that of Example 1, this embodiment introduces a hard shoulder driving and variable speed limit coordinated control system, including:

[0139] The traffic data collection module is configured to collect traffic data, the traffic data including traffic information and speed limit values; the traffic information includes the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving vehicle; the speed limit value includes the speed limit value of the control section;

[0140] A lane-passing judgment module is configured to judge whether the connected autonomous driving vehicle can drive in the hard shoulder lane according to the driving motivation of the connected autonomous driving vehicle to drive in the hard shoulder lane, the lane-passing safety conditions, and the demand for emergency or rescue vehicles;

[0141] The DDPG algorithm module is configured to input the collected traffic data into the pre-built DDPG algorithm to determine the state, action space and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit value in the experience replay pool;

[0142] The collaborative control module is configured to combine the result of being able to drive on the hard shoulder lane with the DDPG algorithm to obtain a collaborative control method for hard shoulder driving and variable speed limit. The optimal speed limit value for each lane of the controlled section is generated through the DDPG algorithm. The networked autonomous driving vehicle adjusts the vehicle speed according to the optimal speed limit value, and the driver of the manually driven vehicle adjusts the vehicle speed according to the optimal speed limit value displayed on the variable information board.

[0143] Example 3

[0144] Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned collaborative control method are implemented.

[0145] Example 4

[0146] Based on the same inventive concept as other embodiments, this embodiment introduces a computer device, including: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the above-mentioned collaborative control method.

[0147] Example 5

[0148] Based on the same inventive concept as other embodiments, this embodiment introduces a computer program product, including a computer program, which implements the steps of the above-mentioned collaborative control method when executed by a processor.

[0149] In the controlled section of the highway where the collaborative control method of the present invention is applied, the connected autonomous driving vehicle that meets the conditions can drive on the hard shoulder and adopt a variable speed limit control strategy based on the DDPG algorithm on the hard shoulder section to achieve combined control of speed limits in different lanes, standardize the driving speed of upstream vehicles, and adjust the supply and demand relationship in the bottleneck area. It can reduce the average travel time and carbon dioxide emissions of the road network, and provide a new method for my country's high-efficiency and low-carbon goals of smart highways.

[0150] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0154] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make many forms under the guidance of the present invention, which all fall within the protection of the present invention.

Claims

1. A method for coordinated control of hard shoulder driving and variable speed limit, characterized in that: The following steps are involved: Collecting traffic data, the traffic data including traffic information and speed limit values; the traffic information including the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving vehicle; the speed limit value includes the speed limit value of the control section; Determine whether the connected autonomous vehicle can use the hard shoulder lane based on the connected autonomous vehicle's motivation for using the hard shoulder lane, safety conditions for using the hard shoulder lane, and the need for emergency or rescue vehicles; Input the collected traffic data into the pre-built DDPG algorithm to determine the state, action space and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit value in the experience replay pool. The result of being able to drive on the hard shoulder lane is combined with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit collaborative control method. The optimal speed limit value for each lane of the controlled section is generated by the DDPG algorithm, and the networked autonomous driving vehicle adjusts the vehicle speed according to the optimal speed limit value.

2. The hard shoulder driving and variable speed limit coordinated control method according to claim 1, characterized in that: The specific methods for collecting traffic data are as follows: Several detectors are respectively set up in the control section, bottleneck section and entrance ramp. The detectors are used to obtain the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving car; Roadside units are deployed on the roadside, and the networked autonomous driving vehicles can exchange information with the roadside units; the roadside units are used to collect the position, speed and acceleration of the networked autonomous driving vehicles, and send the speed limit value of the controlled road section to the networked autonomous driving vehicles; the networked autonomous driving vehicles can obtain the position, speed and acceleration of other networked autonomous driving vehicles through the roadside units; The detector and the roadside unit send the collected traffic information to the traffic control center, and the traffic control center calculates the received traffic information to obtain the optimal speed limit value of the controlled road section.

3. The hard shoulder driving and variable speed limit coordinated control method according to claim 1, characterized in that: The specific method for judging whether a connected autonomous vehicle has the motivation to drive in the hard shoulder lane is: The motivation for driving on the hard shoulder means that the connected autonomous vehicle cannot drive at the expected speed in the current lane, and the hard shoulder lane conditions can meet the current conditions for the connected autonomous vehicle to drive at the expected speed; the specific method for determining the motivation for driving on the hard shoulder is: ； ； in: is the distance between the connected autonomous vehicle and the vehicle in front. is the length of the connected autonomous vehicle, Refers to the speed of connected autonomous vehicles. Refers to the acceleration of connected autonomous vehicles. Refers to the maximum speed of a connected autonomous vehicle. It is the headway between the connected autonomous vehicle and the vehicle ahead on the hard shoulder lane.

4. The hard shoulder driving and variable speed limit coordinated control method according to claim 3, characterized in that: The specific method for judging whether a vehicle with the intention of using the hard shoulder meets the safety conditions for using the hard shoulder is as follows: The lane-changing behavior will not interfere with the normal driving of the vehicle behind in the hard shoulder lane, and the lane-changing behavior meets the safety clearance requirements; the specific judgment method for meeting the safety conditions for lane-changing is: ； ； in: is the distance between the current connected autonomous vehicle and the vehicle behind it in the hard shoulder lane. is the speed of the vehicle behind on the hard shoulder lane, Refers to the safe gap between the front and rear vehicles on the hard shoulder lane; For safe headway, Refers to the acceleration of the vehicle behind on the hard shoulder lane. It is the maximum braking deceleration of the connected autonomous vehicle.

5. The hard shoulder driving and variable speed limit coordinated control method according to claim 1, characterized in that: The specific method for determining the state, action space, and reward function of the DDPG algorithm is: Construct the value network and policy network of the DDPG algorithm. The input layer of the policy network is based on the state, the output layer is based on the action, and the policy network parameters are ; The input layer of the value network is based on state and action, and the value network parameters are ; The detector is used to collect the occupancy rate of the control section and the bottleneck section, which is recorded as state ; The average speed of the bottleneck section is collected by the detector and recorded as the first reward ; The carbon dioxide emissions of the road network are collected by the detector and recorded as the second reward. ; According to the reward weight coefficient and the standardized coefficient Calculate total rewards , then the reward function is: ； The continuous action space generated by the DDPG algorithm is converted into a discrete action space through mapping; Establish an experience replay pool and collect the state at each time step t ,action ,award The next moment state , recorded as an experience , and store it in the experience replay pool; in the experience replay pool, the sequence of each experience sample Absolute value of TD error There is a negative correlation, among which Indicates the first Experience samples; When the experience in the experience replay pool exceeds the minimum training capacity requirement, samples are taken from the experience replay pool for training. The probability of an experience sample being drawn .

6. The hard shoulder driving and variable speed limit coordinated control method according to claim 1, characterized in that: The result of being able to drive on the hard shoulder lane is combined with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit coordinated control method. The value network and strategy network parameters of the DDPG algorithm are optimized using the collected traffic data to generate the optimal speed limit value. The specific method is as follows: Initialize policy network parameters and value network parameters , get the initialization target strategy network parameters Network parameters with target value , initialize the experience replay pool; Create an iterative loop for DDPG algorithm training, where each loop contains M time steps; Variable speed limit agent obtains state ; The policy network is based on the state Generate Action ;in As the strategy function, add normal distribution when generating actions Noise ,Right now ; Set the speed limit for the controlled section and get rewards The next moment state ; Experience Stored in the experience replay pool; If the minimum training capacity of the experience replay pool is reached, W experience samples are extracted for training using the priority experience replay method. The probability of each experience sample being extracted is , Represents the sequence of experience samples in the experience replay pool; Indicates the first Experience samples; Using value network calculation The action value of the experience sample , and predict the next moment The action value of the experience sample ,in Indicates The state of the experience sample, Indicates Actions of experience samples; Indicates the next moment of a sample of experience; Calculate the TD error of empirical samples ,in, represents the discount factor; Indicates Experience sample reward; Update the value network parameters: ； in, is the sample size, ; Update the policy network parameters, where the policy network gradient calculation method is as follows: ； According to TD error Update the probability of experience being extracted, the absolute value of TD error With sample sequence Negative correlation; Update target value network parameters , update the target strategy network parameters ;in, Indicates soft update parameters; The optimal speed limit value for each lane of the controlled road section is generated through the strategy network and sent to the roadside unit via the traffic control center. The networked autonomous driving car adjusts the speed according to the optimal speed limit value sent by the roadside unit, and the driver of the manually driven car adjusts the speed according to the optimal speed limit value.

7. A hard shoulder driving and variable speed limit coordinated control system, characterized in that: include: A traffic data collection module is configured to collect traffic data, wherein the traffic data includes traffic information and speed limit values; the traffic information includes the traffic volume of the entrance ramp section, the occupancy rate of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, the carbon dioxide emissions, and the position, speed and acceleration of the networked autonomous driving vehicle; the speed limit value includes the speed limit value of the control section; A lane-passing judgment module is configured to judge whether the connected autonomous driving vehicle can drive in the hard shoulder lane according to the driving motivation of the connected autonomous driving vehicle to drive in the hard shoulder lane, the lane-passing safety conditions, and the demand for emergency or rescue vehicles; The DDPG algorithm module is configured to input the collected traffic data into the pre-built DDPG algorithm to determine the state, action space and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit value in the experience replay pool; The collaborative control module is configured to combine the result of being able to drive on the hard shoulder lane with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit collaborative control method, generate the optimal speed limit value for each lane of the controlled section through the DDPG algorithm, and the networked autonomous driving vehicle adjusts the vehicle speed according to the optimal speed limit value.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cooperative control method described in any one of claims 1 to 6 are implemented.

9. A computer device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program to implement the steps of the collaborative control method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the cooperative control method according to any one of claims 1 to 6 are implemented.

Images