A method and system for collaborative control of hard shoulder driving and variable speed limit

Abstract

The present invention relates to the technical field of intelligent transportation systems and discloses a method and system for coordinated control of hard-shoulder driving and variable speed limits. The method comprises collecting traffic data, including traffic information and speed limits; determining whether a connected autonomous vehicle can use the hard-shoulder lane based on the vehicle's motivation for using the hard-shoulder lane, safety requirements, and emergency or rescue vehicle needs; inputting the collected traffic data into a pre-built DDPG algorithm to determine the DDPG algorithm's state, action space, and reward function, and outputting an optimal speed limit; generating an optimal speed limit for each lane in the controlled section using the DDPG algorithm, and the connected autonomous vehicle adjusting its speed based on the optimal speed limit. The present invention can be used on heavily congested highway on-ramp sections, reducing average travel time and carbon dioxide emissions across the road network, providing a new approach to achieving my country's goal of high efficiency and low carbon emissions for smart highways.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent transportation systems, and relates to a method and system for collaborative control of hard shoulder driving and variable speed limits, and specifically to a method and system for collaborative control of hard shoulder driving and variable speed limits in a mixed traffic flow environment. Background Art

[0002] Highways often experience traffic bottlenecks due to factors such as merging and emergencies. These bottlenecks often bear immense transportation pressure during special periods such as holidays and rush hour. The emergence of efficiency and safety issues in bottleneck areas restricts the transportation capacity of the entire highway. Currently, hard shoulders and variable speed limits are popular active traffic management measures.

[0003] The hard shoulder running (HSR) strategy is a proactive management measure to address traffic congestion on bottleneck sections of highways. It opens the hard shoulder lanes of highways according to conditions, allowing vehicles on the driving lanes to use them. However, currently, hard shoulder running is mostly based on non-intelligent connected environments, allowing manual vehicles to use the hard shoulder. However, hard shoulder lanes are often narrower than driving lanes, and highway vehicle speeds are high, making lateral control of manual vehicles on the hard shoulder difficult. This makes it difficult to ensure the safety of hard shoulder driving, and the real-time control of hard shoulder driving strategies remains difficult to resolve. In future mixed traffic environments, allowing only connected and automated vehicles (CAVs) to use the hard shoulder will help address these issues. This is because CAVs have high driving stability, which can ensure hard shoulder driving safety. CAVs can also receive information about the needs of emergency or rescue vehicles, ensuring the basic requirements of hard shoulder lanes.

[0004] Similar to general lanes, failure to implement speed limits on hard shoulder lanes can easily lead to safety issues, necessitating a specific speed limit. Currently, fixed speed limits are common on highways, which are inflexible and unable to respond to real-time changes in the traffic system. Traditional variable speed limits (VSLs) include rule-based and feedback-based approaches. These methods are suitable for sections with relatively regular traffic conditions but are ineffective on sections with complex and changing traffic conditions. Deep reinforcement learning (DRL) combines the perception capabilities of deep learning with the decision-making capabilities of reinforcement learning, providing a new approach for active traffic management measures, including variable speed limits. With the increasing demand for high-quality road service levels and the increasingly complex traffic environment, adopting only one active traffic management measure has shown limited optimization effects. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for coordinated control of hard-shoulder driving and variable speed limits, which can be used on severely congested highway entrance ramp sections, reduce the average travel time and carbon dioxide emissions of the road network, and provide a new method for achieving my country's high-efficiency and low-carbon goals for smart highways.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0007] In a first aspect, the present invention provides a method for coordinated control of hard shoulder driving and variable speed limit, comprising the following steps:

[0008] Collecting traffic data, including traffic information and speed limits; the traffic information includes traffic volume on on-ramp sections, occupancy rates of controlled sections and bottleneck sections, average speed of all vehicles on bottleneck sections, carbon dioxide emissions, and the location, speed, and acceleration of connected autonomous vehicles; and the speed limits include speed limits on controlled sections.

[0009] Determine whether the connected autonomous vehicle can use the hard shoulder lane based on its motivation for using the hard shoulder lane, safety conditions for using the lane, and the need for emergency or rescue vehicles.

[0010] The collected traffic data is fed into a pre-built DDPG algorithm to determine the state, action space, and reward function of the DDPG algorithm and output the optimal speed limit. An experience replay pool is constructed to store the collected traffic information and speed limit values. Subsequent training uses the prioritized experience replay method for sampling.

[0011] The results of being able to drive in the hard shoulder lane are combined with the DDPG algorithm to obtain the collaborative control method of hard shoulder driving and variable speed limit. The DDPG algorithm is used to generate the optimal speed limit value for each lane in the controlled section. The connected autonomous driving vehicle adjusts its speed according to the optimal speed limit value, and the driver of the manually driven vehicle also adjusts its speed according to the optimal speed limit value.

[0012] In combination with the first aspect, further, the roadside unit transmits the optimal speed limit value to the variable information board for display, and the driver of the manually driven car adjusts the vehicle speed according to the optimal speed limit value displayed on the variable information board.

[0013] In combination with the first aspect, further, the specific method of collecting traffic data is:

[0014] Several detectors are installed in the control section, bottleneck section, and on-ramp. The detectors are used to obtain the traffic volume of the on-ramp section, the occupancy rate of the control section and bottleneck section, the average speed of all vehicles in the bottleneck section, CO2 emissions, and the position, speed, and acceleration of the connected autonomous vehicle. The traffic volume of the on-ramp section, the occupancy rate of the control section and bottleneck section, and the position, speed, and acceleration of the connected autonomous vehicle are directly obtained through the detectors. The average speed of all vehicles in the bottleneck section is calculated by collecting the speeds of all vehicles in the bottleneck section. CO2 emissions are directly calculated using simulation software using the data collected by the detectors.

[0015] Roadside units are deployed on the roadside, and connected autonomous vehicles can exchange information with the roadside units; the roadside units are used to collect the position, speed, and acceleration of the connected autonomous vehicles and send speed limits for controlled road sections to the connected autonomous vehicles; connected autonomous vehicles can obtain the position, speed, and acceleration of other connected autonomous vehicles through the roadside units;

[0016] The detector and the roadside unit send the collected traffic information to the traffic control center, which calculates the received traffic information to obtain the optimal speed limit value of the controlled road section.

[0017] In conjunction with the first aspect, the specific method for determining whether a connected autonomous vehicle has the motivation to drive in the hard shoulder lane is as follows:

[0018] The motivation to drive on the hard shoulder means that the connected autonomous vehicle cannot travel at the expected speed in the current lane, and the hard shoulder lane conditions can meet the current conditions for the connected autonomous vehicle to travel at the expected speed. The specific method for determining the motivation to drive on the hard shoulder is:

[0019] ;

[0020] ;

[0021] in: is the distance between the connected autonomous vehicle and the vehicle in front. is the length of the connected autonomous vehicle, Refers to the speed of connected autonomous vehicles. Refers to the acceleration of connected autonomous vehicles. Refers to the maximum speed of a connected autonomous vehicle. It is the headway between a connected autonomous vehicle and the vehicle ahead on the hard shoulder lane.

[0022] In combination with the first aspect, further, a specific method for determining whether a vehicle intending to use the hard shoulder for driving meets the safety conditions for using the hard shoulder is as follows:

[0023] The lane change behavior will not interfere with the normal driving of the vehicle behind in the hard shoulder lane, and the lane change behavior meets the safety clearance requirements. The specific method for determining whether the lane change safety conditions are met is:

[0024] ;

[0025] ;

[0026] in: The distance between the current connected autonomous vehicle and the vehicle behind it on the hard shoulder lane, is the speed of the vehicle behind on the hard shoulder lane, Refers to the safe gap between the front and rear vehicles on the hard shoulder lane; For safe headway, Refers to the acceleration of the vehicle behind on the hard shoulder lane. It is the maximum braking deceleration of the connected autonomous vehicle.

[0027] If the traffic control center receives a request from an emergency or rescue vehicle to enter the hard shoulder lane, the roadside unit communicates with the connected autonomous vehicle. Connected autonomous vehicles in the driving lane are prohibited from using the hard shoulder lane, and non-emergency connected autonomous vehicles on the hard shoulder lane must exit the hard shoulder lane as soon as possible. If there is no emergency or rescue vehicle requesting to enter the hard shoulder lane, connected autonomous vehicles that meet both the motivation and safety requirements for using the hard shoulder lane may use the hard shoulder lane.

[0028] Combined with the first aspect, the specific method for determining the state, action space, and reward function of the DDPG algorithm is as follows:

[0029] Construct the value network and policy network of the DDPG algorithm. The input layer of the policy network is based on the state, the output layer is based on the action, and the policy network parameters are The input layer of the value network is based on state and action, and the value network parameters are The states based on the input layer of the policy network and the input layer of the value network are both the occupancy rates of the controlled and bottleneck sections. The actions based on the output layer of the policy network are the speed limits, and the action space is the sum of the number of lanes, including the driving lanes and the hard shoulder lanes.

[0030] The detector is used to collect the occupancy rate of the control section and the bottleneck section, which is recorded as state ;

[0031] The average speed of the bottleneck section is collected by the detector and recorded as the first reward ; Use the detector to collect the carbon dioxide emissions of the road network, which is recorded as the second reward ; According to the reward weight coefficient and standardized coefficients Calculate total rewards , then the reward function is:

[0032] ;

[0033] The continuous action space generated by the DDPG algorithm is converted into a discrete action space through mapping;

[0034] Establish an experience replay pool and collect the state at each time step t ,action ,award and the next moment state , recorded as an experience , and store it in the experience replay pool; the sequence of each experience sample in the experience replay pool Absolute value of TD error There is a negative correlation, among which Indicates the first Experience samples;

[0035] When the experience in the experience replay pool exceeds the minimum training capacity requirement, sampling is performed in the experience replay pool for training. The sampling method is the priority experience replay method. The probability that an experience sample is drawn .

[0036] In combination with the first aspect, the results of being able to drive in the hard shoulder lane are further combined with the DDPG algorithm to obtain the collaborative control method of hard shoulder driving and variable speed limit. The collected traffic data is used to optimize the value network and strategy network parameters of the DDPG algorithm to generate the optimal speed limit value. The specific method is as follows:

[0037] Initialize policy network parameters and value network parameters , get the initialization target strategy network parameters Network parameters with target values , initialize the experience replay pool;

[0038] Create an iterative loop for DDPG algorithm training, where each loop contains M time steps;

[0039] Variable speed limit agent obtains state ;

[0040] The policy network is based on the state Generate Action ;in As the strategy function, add normal distribution when generating actions Noise ,Right now ;

[0041] Set the speed limit for the controlled road section and get rewards and the next moment state ;

[0042] The experience Stored in the experience replay pool;

[0043] If the minimum training capacity of the experience replay pool is reached, W experience samples are extracted for training using the priority experience replay method. The probability of each experience sample being extracted is , Represents the sequence of experience samples in the experience replay pool; Indicates the first Experience samples;

[0044] Using the value network to calculate The action value of an experience sample , and predict the next moment The action value of an experience sample ,in Indicates the The state of the experience sample, Indicates the Actions of experience samples; Indicates the the next moment of an experience sample;

[0045] Calculate the TD error of empirical samples ,in, represents the discount factor; Indicates the Reward for experience samples;

[0046] Update the value network parameters:

[0047] ;

[0048] in, is the sample size, ;

[0049] Update the policy network parameters, where the policy network gradient calculation method is as follows:

[0050] ;

[0051] According to TD error Update the probability of experience being extracted, the absolute value of TD error With sample sequence There is a negative correlation;

[0052] Update target value network parameters , update the target strategy network parameters ;in, Indicates soft update parameters;

[0053] The optimal speed limit for each lane of the controlled road section is generated through the strategy network and sent to the roadside unit via the traffic control center. The connected autonomous driving vehicle adjusts its speed according to the optimal speed limit sent by the roadside unit, and the driver of the manually driven vehicle adjusts its speed according to the optimal speed limit.

[0054] In a second aspect, the present invention proposes a hard shoulder driving and variable speed limit coordinated control system, comprising:

[0055] a traffic data collection module configured to collect traffic data, the traffic data including traffic information and speed limits; the traffic information including traffic volume on an on-ramp section, occupancy ratios of controlled sections and bottleneck sections, average speed of all vehicles on the bottleneck section, carbon dioxide emissions, and the position, speed, and acceleration of connected autonomous vehicles; and the speed limits including speed limits on controlled sections.

[0056] a lane-passing determination module configured to determine whether the connected autonomous vehicle can use the hard shoulder lane based on the connected autonomous vehicle's motivation for using the hard shoulder lane, lane-passing safety conditions, and emergency or rescue vehicle needs;

[0057] The DDPG algorithm module is configured to input the collected traffic data into a pre-built DDPG algorithm to determine the state, action space, and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit values ​​in the experience replay pool;

[0058] The collaborative control module is configured to combine the result of being able to drive in the hard shoulder lane with the DDPG algorithm to obtain the collaborative control method of hard shoulder driving and variable speed limit. The DDPG algorithm is used to generate an optimal speed limit value for each lane of the controlled section. The connected autonomous driving vehicle adjusts the vehicle speed according to the optimal speed limit value, and the driver of the manually driven vehicle adjusts the vehicle speed according to the optimal speed limit value.

[0059] In a third aspect, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned collaborative control method.

[0060] In a fourth aspect, the present invention provides a computer device, comprising:

[0061] memory for storing computer programs;

[0062] A processor is used to execute the computer program to implement the steps of the above-mentioned collaborative control method.

[0063] In a fifth aspect, a computer program product includes a computer program, which implements the steps of the above-mentioned collaborative control method when executed by a processor.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] (1) The present invention can be used on heavily congested highway entrance ramp sections, reducing the average travel time and carbon dioxide emissions of the road network, and providing a new approach to achieving my country's high-efficiency and low-carbon smart highway goals.

[0066] (2) The connected autonomous driving vehicle that meets the requirements of the present invention can travel on the hard shoulder, which helps to alleviate traffic congestion on the main line of the expressway and make full use of road resources.

[0067] (3) The present invention proposes a variable speed limit control strategy based on the DDPG algorithm to improve traffic efficiency and reduce carbon dioxide emissions, realize the combined control of speed limits of different lanes, standardize the speed of upstream vehicles, and adjust the supply and demand relationship in the bottleneck area.

[0068] (4) The collaborative control method of the present invention overcomes the problem of poor performance of traditional methods in complex traffic scenarios and provides a new method for active traffic management of my country's smart highways. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 Schematic diagram of the process of the collaborative control method in Example 1 of the present invention;

[0070] Figure 2 This is a schematic diagram illustrating a road network according to Example 1 of the present invention;

[0071] Figure 3 This is a schematic diagram of average travel time on a road network at different penetration rates of connected autonomous vehicles in Example 1 of the present invention;

[0072] Figure 4 Schematic diagram of road network carbon dioxide emissions at different penetration rates of connected autonomous vehicles in Example 1 of the present invention. DETAILED DESCRIPTION

[0073] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0074] The term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " generally indicates an "or" relationship between the related objects.

[0075] Example 1

[0076] like Figure 1 As shown, this embodiment introduces a method for coordinated control of hard shoulder driving and variable speed limit in a mixed traffic flow environment, including the following steps:

[0077] Step S1: Collect traffic data, which includes traffic information and speed limits. Traffic information includes the traffic volume of the entrance ramp section, the occupancy rate of the controlled section and the bottleneck section, the average speed of vehicles in the bottleneck section, and carbon dioxide emissions. The speed limit includes the speed limit of the controlled section. Connected autonomous vehicles can exchange information with roadside units through V2I communication technology.

[0078] The specific steps of step S1 are as follows:

[0079] Step S11: Figure 2 As shown, a road section consists of a mainline section and an on-ramp section. The mainline section consists of several lanes and a hard shoulder lane. For example, if there are three lanes, they are numbered from right to left as Lane 1, Lane 2, and Lane 3, according to the vehicle's direction of travel. The on-ramp is a single lane. The section upstream of the on-ramp is a controlled section, where qualified connected autonomous vehicles can use the hard shoulder lane. The area around the on-ramp is a bottleneck section.

[0080] Step S12: Several detectors are respectively installed on the control section, the bottleneck section, and the on-ramp, corresponding to the control section detector, the bottleneck section detector, and the ramp detector, respectively. The detectors are used to obtain the on-ramp traffic volume, the occupancy ratio of the control section and the bottleneck section, the average speed of all vehicles in the bottleneck section, carbon dioxide emissions, and the position, speed, and acceleration of the connected autonomous vehicle. The on-ramp traffic volume, the occupancy ratio of the control section and the bottleneck section, and the position, speed, and acceleration of the connected autonomous vehicle are directly obtained through the detectors; the average speed of all vehicles in the bottleneck section is calculated by collecting the speeds of all vehicles in the bottleneck section; and carbon dioxide emissions are directly calculated using simulation software using the data collected by the detectors.

[0081] Step S13: Roadside units are deployed on the roadside. Connected autonomous vehicles can use V2I communication technology to exchange information with the roadside units. The roadside units are used to collect traffic information such as the position, speed, acceleration, etc. of the connected autonomous vehicles, and send speed limit values ​​of controlled sections to the connected autonomous vehicles. Connected autonomous vehicles can obtain traffic information such as the position, speed, acceleration, etc. of other connected autonomous vehicles around them through the roadside units.

[0082] Step S14: The detector and the roadside unit send the collected traffic information to the traffic control center TCC. The traffic control center TCC calculates the received traffic information and obtains the optimal speed limit value of the controlled road section.

[0083] Step S2: Determine whether the connected autonomous vehicle can drive on the hard shoulder based on the vehicle's motivation for driving on the hard shoulder, safety conditions for driving on the hard shoulder, and the need for emergency or rescue vehicles.

[0084] The specific steps of step S2 are as follows:

[0085] Step S21: Determine whether there is a motivation to drive on the hard shoulder. A motivation means that the connected autonomous vehicle cannot drive at the expected speed in the current lane, and the hard shoulder lane conditions can meet the current conditions for the connected autonomous vehicle to drive at the expected speed. The specific method for determining whether there is a motivation to drive on the hard shoulder is as follows:

[0086]

[0087]

[0088] in: is the distance between the connected autonomous vehicle and the vehicle in front. is the length of the connected autonomous vehicle, Refers to the speed of connected autonomous vehicles. Refers to the acceleration of connected autonomous vehicles. Refers to the maximum speed of a connected autonomous vehicle. It is the headway between a connected autonomous vehicle and the vehicle ahead on the hard shoulder lane.

[0089] Step S22: Determine whether the vehicle that intends to use the hard shoulder meets the lane-changing safety conditions, that is, whether the lane-changing behavior will not interfere with the normal driving of the vehicle behind on the hard shoulder lane, and whether the lane-changing behavior meets the safety clearance requirements. The specific method for determining whether the lane-changing safety conditions are met is as follows:

[0090]

[0091]

[0092] in: The distance between the current connected autonomous vehicle and the vehicle behind it on the hard shoulder lane, is the speed of the vehicle behind on the hard shoulder lane, Refers to the safe gap between the front and rear vehicles on the hard shoulder lane; For safe headway, Refers to the acceleration of the vehicle behind on the hard shoulder lane. It is the maximum braking deceleration of the connected autonomous vehicle.

[0093] Step S23: If the Traffic Control Center (TCC) receives a request from an emergency or rescue vehicle to enter the hard shoulder lane, it communicates with the connected autonomous vehicle via the roadside unit. Connected autonomous vehicles in the driving lane are prohibited from using the hard shoulder lane, and non-emergency connected autonomous vehicles in the hard shoulder lane must exit the hard shoulder lane as soon as possible. If there are no emergency or rescue vehicles requesting to enter the hard shoulder lane, connected autonomous vehicles that meet both the motivation and safety requirements for using the hard shoulder lane may use the hard shoulder lane.

[0094] As long as the connected autonomous driving vehicle on the controlled road section meets the three conditions in step S2 (i.e., the motivation for using the lane, the safety conditions for using the lane, and the need for emergency or rescue vehicles), it can choose to use the hard shoulder lane.

[0095] Step S3: Use the traffic data collected in step S1 to establish a DDPG algorithm for variable speed limits, determine the parameters, state, action space, and reward function of the DDPG algorithm; build an experience replay pool, store the collected traffic information and speed limit values ​​in the experience replay pool, and use the priority experience replay method for sampling in subsequent training.

[0096] The specific steps of step S3 are as follows:

[0097] Step S31: Construct the value network and policy network of the DDPG algorithm. The policy network input layer is based on the state, the output layer is based on the action, and the policy network parameters are ; The value network input layer is based on state and action, and the value network parameters are The action based on the value network input layer is the speed limit value collected by the detector. The state based on the policy network input layer and the state based on the value network input layer are the occupancy ratio of the controlled section and the bottleneck section. When there are three lanes, the DDPG algorithm's action space dimension is four, including the speed limits of the three lanes and the speed limit of the hard shoulder lane. After reaching the hard shoulder lane, the speed must be adjusted according to the speed limit of the hard shoulder lane.

[0098] The DDPG algorithm is based on the Actor-Critic framework, including a value network (Critic network), a policy network (Actor network), an experience replay pool, and a target network; the value network is responsible for learning deterministic policies, that is, directly outputting action values ​​under a given state; the policy network is responsible for learning value functions and evaluating the value of the current state; the experience replay pool is used to store the experience gained by the interaction between the agent and the environment, and is used for random sampling during training to solve sample correlation and stability problems; the target network includes a target policy network (Target Actor network) and a target value network (Target Critic network), which are used to estimate the target Q value and target policy, and are gradually updated from the parameters of the main network through soft updates to stabilize the training process.

[0099] The update strategy of the DDPG algorithm includes the following steps:

[0100] Experience replay: Randomly sample a batch of experience data from the experience replay pool to update the parameters of the Actor network and Critic network.

[0101] Critic network update: Using the sampled empirical data, the parameters of the Critic network are updated by minimizing the loss function, which is usually defined as the square of the difference between the current Q value and the target Q value.

[0102] Actor network update: Use the gradient information evaluated by the Critic network to update the parameters of the Actor network through the gradient ascent method to maximize the expected return.

[0103] Target network update: Use soft update to gradually copy the parameters of the main network to the target network to maintain the stability of the target network.

[0104] In addition, in order to increase the exploration ability of the agent, noise is added to the actions output by the Actor network during the training phase.

[0105] When a connected autonomous vehicle meets the conditions for driving in the hard shoulder lane, the lane changing process can be achieved through the lane changing model of the simulation software.

[0106] Step S32: Setting DDPG algorithm parameters. The DDPG algorithm parameter setting method is to determine the final parameters based on the parameter setting range and training. For example, in this embodiment, the learning rate of the Actor strategy network is 0.0005, the learning rate of the Critic value network is 0.001, and the Adam optimizer is used for optimizing the parameters of both neural networks. The discount coefficient is 0.98, the soft update parameter is 0.005, the noise standard deviation is 0.1, the experience replay pool size is 5000, the minimum training capacity is 100, the number of experiences sampled each time is 64, and the number of training rounds is 500.

[0107] Step S33: Use the detector to collect the occupancy rate of the control section and the bottleneck section, and record it as state ;

[0108] Step S34: Use the detector to collect and calculate the average speed of the bottleneck section, which is recorded as the first reward , calculate the carbon dioxide emissions of the road network based on the carbon emission model, recorded as the second reward , according to the reward weight coefficient and standardized coefficients Calculate total rewards , then the reward function is as follows.

[0109]

[0110] Step S35: In step S3, the action of the DDPG algorithm is a continuous action space, which needs to be converted into a discrete action space that meets the speed limit requirements. The continuous action generated by the algorithm is mapped to the discrete action space. The mapping method is: divide the agent action into (0,13) and obtain the discrete value of its integer part Considering the stability of the driver's response to the speed limit, the speed limit is set to an integer multiple of 5km / h and a minimum speed limit is specified. and maximum speed limit , specifically:

[0111]

[0112]

[0113] in, Indicates the optimized speed limit value. represents discrete values, Represents the action output by the policy network (hereinafter referred to as action).

[0114] Step S36: Establish an experience replay pool. At each time step t, the variable speed limit agent (referring to the variable speed limit control module of the traffic control center) interacts with the traffic environment and collects state ,action ,award and the next moment state , recorded as an experience , and store it in the experience replay pool. The sequence of each experience sample in the experience replay pool Absolute value of TD error There is a negative correlation, among which Indicates the first An experience sample.

[0115] Step S37: When the experience in the experience replay pool exceeds the minimum training capacity requirement, sampling can be performed in the experience replay pool for training. The sampling method is the priority experience replay method. The probability of each experience sample being sampled is .

[0116] Step S4: Based on the hard shoulder driving method proposed in step S2 and the DDPG algorithm with variable speed limits in step S3, a collaborative control method is established and the collaborative control agent is trained. The traffic data sampled in step S1 is used to continuously optimize the value network and policy network parameters of the DDPG algorithm and generate the optimal speed limit.

[0117] The specific steps for establishing the collaborative control method in step S4 are as follows:

[0118] Step S41: Initialize policy network parameters and value network parameters , get the initialization target strategy network parameters Network parameters with target values , initialize the experience replay pool.

[0119] Step S42: Create an iterative loop for DDPG algorithm training, where each loop contains M time steps;

[0120] Step S43: The agent interacts with the traffic environment to obtain the state ;

[0121] Step S44: The policy network is based on the state Generate Action ;in As the strategy function, add normal distribution when generating actions Noise ,Right now .

[0122] Step S45: Set the speed limit for the controlled road section and obtain rewards and the next moment state ;

[0123] Step S46: Experience Stored in the experience replay pool;

[0124] Step S47: If the minimum training capacity of the experience replay pool is reached, extract using the priority experience replay method Experience samples are used for training (preferably =64), the probability of each experience sample being drawn , Represents the sequence of experience samples in the experience replay pool;

[0125] Step S48: Calculate the value network The action value of an experience sample , and predict the action value at the next moment ,in, Indicates the The state of the experience sample, Indicates the Actions of experience samples; Indicates the the next moment of an experience sample;

[0126] Step S49 calculates the TD error of empirical samples ,in represents the discount factor; Indicates the first Experience samples; Indicates the Experience sample reward.

[0127] Step S410: Update value network parameters;

[0128]

[0129] in, is the sample size, ;

[0130] Step S411: Update the policy network parameters, where the policy network gradient calculation method is as follows:

[0131]

[0132] Step S412: According to the TD error Update the probability of experience being extracted, the absolute value of TD error With sample sequence There is a negative correlation;

[0133] Step S413: Update target value network parameters , update the target strategy network parameters ;in, Indicates soft update parameters;

[0134] Step S414: The optimal speed limit value for each lane of the controlled road section is generated through the strategy network and sent to the roadside unit via the traffic control center. The networked autonomous driving vehicle adjusts its speed according to the optimal speed limit value sent by the roadside unit, and the driver of the manually driven vehicle adjusts its speed according to the optimal speed limit value displayed on the variable information board.

[0135] A typical highway section with an on-ramp was designed using traffic simulation software. The mainline lanes had an initial speed limit of 120 km / h, the hard shoulder lanes had a speed limit of 100 km / h, and the on-ramp speed limit was 60 km / h. Vehicles on the road included both human-driven vehicles and connected autonomous vehicles. The hard shoulder lanes were open on the control section and implemented with variable speed limits. The hard shoulder lanes were also open on the bottleneck section. The mainline vehicle input was 4,950 vehicles / hour, the ramp vehicle input was 743 vehicles / hour, and the penetration rate of connected autonomous vehicles was 40%. A total of 500 simulation rounds were conducted, each lasting 3,600 seconds.

[0136] The collaborative control method of hard shoulder driving and variable speed limit was applied to the controlled road section. Compared with not using the collaborative control method of the present invention, the average travel time of the road network was reduced by 15.6% and the carbon dioxide emissions were reduced by 6.7%. The comparative experiments were conducted at 20%, 40%, 60%, 80% and 100% penetration rates of intelligent connected vehicles. Figure 3 and Figure 4 Compared with no active traffic management measures, the collaborative control approach achieved optimal results at all penetration rates. The most effective efficiency improvement was achieved at a 40% penetration rate, reducing the average network travel time by 15.6%. The most effective emission reduction was achieved at an 80% penetration rate, reducing carbon dioxide emissions by 9.0%.

[0137] Example 2

[0138] Based on the same inventive concept as Example 1, this embodiment introduces a hard shoulder driving and variable speed limit coordinated control system, including:

[0139] a traffic data collection module configured to collect traffic data, the traffic data including traffic information and speed limits; the traffic information including traffic volume on an on-ramp section, occupancy ratios of controlled sections and bottleneck sections, average speed of all vehicles on the bottleneck section, carbon dioxide emissions, and the position, speed, and acceleration of connected autonomous vehicles; and the speed limits including speed limits on controlled sections.

[0140] a lane-passing determination module configured to determine whether the connected autonomous vehicle can use the hard shoulder lane based on the connected autonomous vehicle's motivation for using the hard shoulder lane, lane-passing safety conditions, and emergency or rescue vehicle needs;

[0141] The DDPG algorithm module is configured to input the collected traffic data into a pre-built DDPG algorithm to determine the state, action space, and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit values ​​in the experience replay pool;

[0142] The collaborative control module is configured to combine the results of being able to drive in the hard shoulder lane with the DDPG algorithm to obtain a collaborative control method for hard shoulder driving and variable speed limits. The DDPG algorithm is used to generate an optimal speed limit value for each lane in the controlled section. The connected autonomous driving vehicle adjusts its speed according to the optimal speed limit value, and the driver of the manually driven vehicle adjusts its speed according to the optimal speed limit value displayed on the variable information board.

[0143] Example 3

[0144] Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned collaborative control method are implemented.

[0145] Example 4

[0146] Based on the same inventive concept as other embodiments, this embodiment introduces a computer device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the above-mentioned collaborative control method.

[0147] Example 5

[0148] Based on the same inventive concept as other embodiments, this embodiment introduces a computer program product, including a computer program, which implements the steps of the above-mentioned collaborative control method when executed by a processor.

[0149] In the controlled sections of highways where the collaborative control method of the present invention is applied, connected autonomous vehicles that meet the conditions can use the hard shoulder to travel, and adopt a variable speed limit control strategy based on the DDPG algorithm on the hard shoulder sections to achieve combined control of speed limits in different lanes, standardize the speed of upstream vehicles, and adjust the supply and demand relationship in bottleneck areas. This can reduce the average travel time and carbon dioxide emissions of the road network, providing a new method for achieving my country's high-efficiency and low-carbon goals for smart highways.

[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0154] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make many forms under the guidance of the present invention, which are all protected by the present invention.

Claims

1. A method for coordinated control of hard shoulder driving and variable speed limit, characterized in that: The following steps are involved: Collecting traffic data, including traffic information and speed limits; the traffic information includes traffic volume on on-ramp sections, occupancy rates of controlled sections and bottleneck sections, average speed of all vehicles on bottleneck sections, carbon dioxide emissions, and the location, speed, and acceleration of connected autonomous vehicles; and the speed limits include speed limits on controlled sections. Determine whether the connected autonomous vehicle can use the hard shoulder lane based on its motivation for using the hard shoulder lane, safety conditions for using the lane, and the need for emergency or rescue vehicles. The collected traffic data is input into the pre-built DDPG algorithm to determine the state, action space, and reward function of the DDPG algorithm and output the optimal speed limit value; an experience replay pool is constructed and the collected traffic information and speed limit values ​​are stored in the experience replay pool; The results of being able to drive in the hard shoulder lane are combined with the DDPG algorithm to obtain the hard shoulder driving and variable speed limit coordinated control method. The DDPG algorithm is used to generate the optimal speed limit value for each lane in the controlled section, and the connected autonomous vehicle adjusts its speed according to the optimal speed limit value. The specific method for determining the state, action space, and reward function of the DDPG algorithm is: Construct the value network and policy network of the DDPG algorithm. The input layer of the policy network is based on the state, the output layer is based on the action, and the policy network parameters are The input layer of the value network is based on state and action, and the value network parameters are ; The detector is used to collect the occupancy rate of the control section and the bottleneck section, which is recorded as state ; The average speed of the bottleneck section is collected by the detector and recorded as the first reward ; The carbon dioxide emissions of the road network are collected using the detector and recorded as the second reward. ; According to the reward weight coefficient and standardized coefficients Calculate total rewards , then the reward function is: ； The continuous action space generated by the DDPG algorithm is converted into a discrete action space through mapping; Establish an experience replay pool and collect the state at each time step t ,action ,award and the next moment state , recorded as an experience , and store it in the experience replay pool; the sequence of each experience sample in the experience replay pool Absolute value of TD error There is a negative correlation, among which Indicates the first Experience samples; When the experience in the experience replay pool exceeds the minimum training capacity requirement, samples are taken from the experience replay pool for training. The probability that an experience sample is drawn ; The results of being able to drive in the hard shoulder lane are combined with the DDPG algorithm to obtain the collaborative control method of hard shoulder driving and variable speed limit. The collected traffic data is used to optimize the value network and strategy network parameters of the DDPG algorithm to generate the optimal speed limit value. The specific method is as follows: Initialize policy network parameters and value network parameters , get the initialization target strategy network parameters Network parameters with target values , initialize the experience replay pool; Create an iterative loop for DDPG algorithm training, where each loop contains M time steps; Variable speed limit agent obtains state ; The policy network is based on the state Generate Action ;in As the strategy function, add normal distribution when generating actions Noise ,Right now ; Set the speed limit for the controlled road section and get rewards and the next moment state ; The experience Stored in the experience replay pool; If the minimum training capacity of the experience replay pool is reached, W experience samples are extracted for training using the priority experience replay method. The probability of each experience sample being extracted is , Represents the sequence of experience samples in the experience replay pool; Indicates the first Experience samples; Using the value network to calculate The action value of an experience sample , and predict the next moment The action value of an experience sample ,in Indicates the The state of the experience sample, Indicates the Actions of experience samples; Indicates the the next moment of an experience sample; Calculate the TD error of empirical samples ,in, represents the discount factor; Indicates the Reward for experience samples; Update the value network parameters: ； in, is the sample size, ; Update the policy network parameters, where the policy network gradient calculation method is as follows: ； According to TD error Update the probability of experience being extracted, the absolute value of TD error With sample sequence There is a negative correlation; Update target value network parameters , update the target strategy network parameters ;in, Indicates soft update parameters; The optimal speed limit for each lane of the controlled road section is generated through the strategy network and sent to the roadside unit via the traffic control center. The connected autonomous vehicle adjusts its speed based on the optimal speed limit sent by the roadside unit, and the driver of the manually driven vehicle adjusts its speed based on the optimal speed limit.

2. The method for coordinated control of hard shoulder driving and variable speed limit according to claim 1, characterized in that: The specific methods for collecting traffic data are as follows: Several detectors are installed in the control section, bottleneck section, and on-ramp. The detectors are used to obtain the traffic volume of the on-ramp section, the occupancy rate of the control section and bottleneck section, the average speed of all vehicles in the bottleneck section, carbon dioxide emissions, and the position, speed, and acceleration of the connected autonomous vehicle. Roadside units are deployed on the roadside, and connected autonomous vehicles can exchange information with the roadside units; the roadside units are used to collect the position, speed, and acceleration of the connected autonomous vehicles and send speed limits for controlled road sections to the connected autonomous vehicles; connected autonomous vehicles can obtain the position, speed, and acceleration of other connected autonomous vehicles through the roadside units; The detector and the roadside unit send the collected traffic information to the traffic control center, which calculates the received traffic information to obtain the optimal speed limit value of the controlled road section.

3. A hard shoulder driving and variable speed limit coordinated control system, characterized in that: A collaborative control method for implementing any one of claims 1 to 2, comprising: a traffic data collection module configured to collect traffic data, the traffic data including traffic information and speed limits; the traffic information including traffic volume on an on-ramp section, occupancy ratios of controlled sections and bottleneck sections, average speed of all vehicles on the bottleneck section, carbon dioxide emissions, and the position, speed, and acceleration of connected autonomous vehicles; and the speed limits including speed limits on controlled sections. a lane-passing determination module configured to determine whether the connected autonomous vehicle can use the hard shoulder lane based on the connected autonomous vehicle's motivation for using the hard shoulder lane, lane-passing safety conditions, and emergency or rescue vehicle needs; The DDPG algorithm module is configured to input the collected traffic data into a pre-built DDPG algorithm to determine the state, action space, and reward function of the DDPG algorithm and output the optimal speed limit value; build an experience replay pool and store the collected traffic information and speed limit values ​​in the experience replay pool; The collaborative control module is configured to combine the result of being able to drive in the hard shoulder lane with the DDPG algorithm to obtain the collaborative control method of hard shoulder driving and variable speed limit. The DDPG algorithm is used to generate the optimal speed limit value for each lane of the controlled section, and the connected autonomous driving vehicle adjusts the vehicle speed according to the optimal speed limit value.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cooperative control method described in any one of claims 1 to 2 are implemented.

5. A computer device, characterized in that: include: memory for storing computer programs; A processor is used to execute the computer program to implement the steps of the collaborative control method according to any one of claims 1 to 2.

6. A computer program product comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the cooperative control method according to any one of claims 1 to 2 are implemented.

Images