Automatic driving vehicle special lane dynamic control method, system, equipment and medium
Through the combination of the deep reinforcement learning model and the cellular update model, the dedicated lanes for autonomous driving vehicles are dynamically controlled, which solves the problems of low efficiency and poor stability in hybrid traffic, and achieves more efficient and stable traffic flow management.
Patent Information
- Application Number
- CN202411238748.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-13
AI Technical Summary
In hybrid traffic, the conservative behavior of autonomous driving vehicles leads to a reduction in operating efficiency, and the dedicated lanes for autonomous driving vehicles are not effectively utilized in a low proportion, which instead reduces road traffic efficiency, and the lane change behavior of artificially driven vehicles reduces traffic stability.
Using deep reinforcement learning model combined with cell update model, by obtaining road traffic data, building vehicle demand flow data through demand flow and lane change, setting reward functions, training dedicated lane control model, dynamically controlling switches for dedicated lanes for autonomous driving vehicles, and optimizing Traffic flow.
Improve the stability and traffic efficiency of hybrid traffic flow, avoid the chaotic problem of deep reinforcement learning models in complex traffic systems, solve the harm of dimensional explosion and immature learning traffic control to the actual system, and enhance the stability of traffic flow Sex and efficiency.
Smart Images
Figure CN119132085A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a method for dynamically controlling a dedicated lane for an autonomous driving vehicle. Background Art
[0002] In recent years, self-driving cars have gradually entered the market, and the development of self-driving technology has attracted the attention of many consumers and researchers. At present, due to the limitations of technological development and other factors, mixed traffic of connected and autonomous vehicles (CAV) and human-driven vehicles (HDV) (hereinafter referred to as mixed traffic) will continue to exist for a long time in the future.
[0003] Research shows that the conservative behavior of autonomous vehicles in mixed traffic reduces the efficiency of autonomous vehicles; in mixed traffic, the efficiency of manually driven vehicles cannot be improved. Based on this, research on the setting of dedicated lanes for autonomous vehicles (hereinafter referred to as dedicated lanes) has emerged, hoping to improve traffic efficiency by separating the two types of vehicles. However, when the proportion of autonomous vehicles in mixed traffic is low, the dedicated lanes for autonomous driving are not effectively utilized, but will reduce the overall traffic efficiency of the road; in addition, when the dedicated lanes are open to manually driven vehicles, the lane change behavior generated by manually driven vehicles will reduce the stability of mixed traffic.
[0004] In summary, how to flexibly control the switching of autonomous driving lanes under different ratios of autonomous driving vehicles is an urgent problem that needs to be solved. Summary of the invention
[0005] In order to overcome the shortcomings of low traffic efficiency and poor stability of mixed traffic, the present invention provides a dynamic control method for a dedicated lane for an autonomous driving vehicle, comprising the following steps:
[0006] Obtain road traffic data for the controlled section of the dedicated lane for autonomous vehicles;
[0007] Divide the controlled road section into a plurality of cells of equal length, construct a cell update model according to the road traffic data, and obtain vehicle passing demand flow data and vehicle lane change demand flow data in each cell within an equal time step through the cell update model;
[0008] According to the vehicle passing demand flow data and the vehicle lane changing demand flow data, the lane changing congestion impact, the speed change rate of each lane and the speed variance value of each lane of the controlled section are obtained. The comprehensive state of the lane changing congestion impact, the speed change rate of each lane and the speed variance value of each lane is used as the state space, and whether the controlled section is open to the dedicated lane for autonomous driving vehicles is used as the action space, and the reward function is set; the state space and action space are used to train the parameters of the deep reinforcement learning model, and the reward function is used to determine whether the trained deep reinforcement learning model converges. If so, the dedicated lane control model is obtained;
[0009] The road traffic data of the section where the dedicated lane for the autonomous driving vehicle is controlled is input into the dedicated lane control model, and the switching of the dedicated lane for the autonomous driving vehicle is dynamically controlled through the dedicated lane control model.
[0010] Preferably, the road traffic data of the controlled section of the dedicated lane for autonomous driving vehicles is obtained, specifically: according to the layout distance of the road monitoring system, the controlled section is discretized into multiple network intervals, each network interval has 2-3 road monitoring cameras, and the road traffic data is collected through the road monitoring cameras.
[0011] Preferably, the road traffic data includes traffic flow characteristics and vehicle speed change values within the controlled section, wherein the traffic flow characteristics include the current number of vehicles, the number of lane-changing vehicles and the vehicle speed within the controlled section.
[0012] Preferably, there are multiple control sections, and the action space is whether to open a dedicated lane for autonomous driving vehicles for each control section.
[0013] Preferably, the cell update model is:
[0014]
[0015] in, represents the number of autonomous driving vehicles on the cell (i, j) at the initial time in the tth time interval; represents the number of autonomous driving vehicles on the cell (i, j) at the initial time in the t+1th time interval; represents the number of autonomous vehicles flowing from cell (i, j) to cell (i+1, j) in the tth time interval; represents the number of autonomous vehicles flowing from cell (i-1, j) to cell (i, j) in the tth time interval; represents the number of autonomous driving vehicles flowing from cell (i, j) into cell (i+1, j') in the tth time interval, i is the cell index on a lane, j is the index of the lane, and j' represents the other two lanes adjacent to lane j.
[0016] Preferably, the reward function is:
[0017]
[0018] Among them, γ1, γ2, γ3, and γ4 are the weight values of each influencing factor; n and n max Indicates the number of vehicles and the maximum number of vehicles in one lane of the controlled section; It indicates the average speed of one lane at time T in the controlled section; It indicates the average speed of one lane at time T+1 in the controlled section; It represents the average speed of two lanes at time T in the controlled section; It indicates the average speed of two lanes in the controlled section at time T+1; It represents the average speed of the three lanes at time T in the controlled section; represents the average speed of the three lanes in the controlled section at time T+1; σ k represents the speed variance of the k lanes in the controlled section; f i,j,j' Represents the number of vehicles flowing from cell (i,j) into cell (i+1,j') at time T.
[0019] Preferably, the deep reinforcement learning model is a DQN algorithm.
[0020] The present invention also provides a dynamic control system for a dedicated lane for an autonomous driving vehicle, comprising:
[0021] A data acquisition module, used to acquire road traffic data of the controlled section of the dedicated lane for autonomous driving vehicles;
[0022] A cellular update model construction module is used to divide the controlled road section into a plurality of cells of equal length, construct a cellular update model according to the road traffic data, and obtain vehicle passing demand flow data and vehicle lane change demand flow data in each cell within an equal time step through the cellular update model;
[0023] The dedicated lane control model construction module is used to obtain the lane change congestion impact of the controlled section, the speed change rate of each lane, and the speed variance value of each lane based on the vehicle passing demand flow data and the vehicle lane change demand flow data, and use the comprehensive state of the lane change congestion impact, the speed change rate of each lane, and the speed variance value of each lane as the state space, and use whether the controlled section is open to the dedicated lane for autonomous driving vehicles as the action space, and set the reward function; use the state space and action space to train the parameters of the deep reinforcement learning model, and use the reward function to determine whether the trained deep reinforcement learning model converges. If so, obtain the dedicated lane control model;
[0024] The lane control module is used to input the road traffic data of the dedicated lane control section of the autonomous driving vehicle into the dedicated lane control model, and dynamically control the switching of the dedicated lane of the autonomous driving vehicle through the dedicated lane control model.
[0025] The present invention also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the dynamic control method of the dedicated lane of the autonomous driving vehicle.
[0026] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for loading by a processor to execute the dynamic control method of the dedicated lane for the autonomous driving vehicle.
[0027] The method for dynamic control of dedicated lanes for autonomous driving vehicles provided by the present invention has the following beneficial effects:
[0028] The present invention can obtain the vehicle passing demand flow data and the vehicle lane change demand flow data in the current cell by constructing a cellular update model according to the road traffic data, thereby intuitively showing the traffic flow information; the lane change congestion impact of the controlled section, the speed change rate of each lane and the speed variance value of each lane can be obtained through the vehicle passing demand flow data and the vehicle lane change demand flow data, and the comprehensive state of the lane change congestion impact, the speed change rate of each lane and the speed variance value of each lane is used as the state space, and whether the controlled section is open to the automatic driving vehicle dedicated lane is used as the action space, so that the deep reinforcement learning model can be used to learn and optimize the complex nonlinear traffic system, and a better mixed traffic flow automatic driving vehicle dedicated lane control model is obtained, so as to solve the traffic system chaos problem caused by the direct use of the deep reinforcement learning model; by combining the cellular update model with the dedicated lane control model to dynamically control the switch of the automatic driving vehicle dedicated lane, it can not only solve the problem of dimensional explosion when facing a complex nonlinear traffic system, but also avoid the harm of immature traffic control in the learning stage of the reinforcement learning model to the actual traffic system, enhance the stability of the mixed traffic flow, and improve the traffic flow efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0030] Figure 1 This is a flow chart of a method for dynamic control of a dedicated lane for an autonomous driving vehicle according to an embodiment of the present invention;
[0031] Figure 2 To lay out a map for the controlled section;
[0032] Figure 3 Lay out monitoring areas;
[0033] Figure 4 Lay out a map of the cells in the region;
[0034] Figure 5 Update the relationship graph for the cell;
[0035] Figure 6 It is a theoretical schematic diagram and a basic principle diagram; Figure 6 (a) Schematic diagram of cell transport capacity and maximum vehicle receiving capacity; Figure 6 (b) Basic principle diagram under different permeabilities;
[0036] Figure 7 This is the result diagram of the dedicated lane control for the autonomous driving vehicle; Figure 7 (a) Reward function result; Figure 7 (b) indicates the dedicated lane is open. DETAILED DESCRIPTION
[0037] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.
[0038] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the technical solutions of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0039] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.
[0040] Example
[0041] The present invention provides a method for dynamically controlling a dedicated lane for an autonomous driving vehicle, specifically: Figure 1 As shown, the following steps are included:
[0042] Step 1: Obtain road traffic data for the section of the dedicated lane for autonomous driving vehicles.
[0043] The scenario of the present invention is a one-way three-lane road, and the driving lanes are the 1st to 3rd lanes from left to right along the road direction. Figure 2 As shown, the first lane is a dedicated lane for autonomous vehicles, and the rest are ordinary lanes. The dedicated lanes are only for autonomous vehicles.
[0044] In order to reduce the impact of frequent lane changes on mixed traffic flow, the present invention sets the length of the dedicated lane control section to 3000m, and sets a signal display screen 200m in front of each control section to control the opening of the dedicated lane. Considering the vehicle-road collaborative scenario, a road test unit (Roadside unit, RSU) is deployed to achieve vehicle-road communication and provide active management and control decisions through its communication module, edge computing module and vehicle status monitoring system. In the mixed traffic flow scenario, the average deployment interval of RSU is set to 1000m, which is the monitoring area. Figure 3 shown.
[0045] In order to better display traffic flow information, the present invention processes the road network into a grid, and sets the discrete network interval to 200m according to the layout distance of the road monitoring system. The road traffic data is collected through 2-3 road monitoring cameras within the grid, and the traffic flow status in this section is described based on the traffic flow characteristics in the controlled section and the speed change values in each monitoring area.
[0046] Step 2: Divide the controlled road section into a plurality of cells of equal length, construct a cell update model according to the road traffic data, and obtain the vehicle passing demand flow data and vehicle lane change demand flow data in each cell within equal time steps through the cell update model.
[0047] Divide the control section into multiple cells of equal length, such as Figure 4 shown.
[0048] In the environment of autonomous vehicles, the vehicles transmitted by the cells of the basic road section are a mixture of HDVs and CAVs. A cell can be represented by (i, j), where i is the cell index on a lane and j is the index of the lane. For example, if there are three lanes and one lane is divided into 100 cells, the cell (3, 2) is the third cell of the second lane. Figure 5 As shown in the figure, the mathematical recursive relationship for cell update on the basic road segment is:
[0049]
[0050] in, represents the number of autonomous driving vehicles on the cell (i, j) at the initial time in the tth time interval; represents the number of autonomous driving vehicles on the cell (i, j) at the initial time in the t+1th time interval; represents the number of autonomous vehicles flowing from cell (i, j) to cell (i+1, j) in the tth time interval; represents the number of autonomous vehicles flowing from cell (i-1, j) to cell (i, j) in the tth time interval; It represents the number of autonomous driving vehicles flowing from cell (i,j) to cell (i+1,j') in the tth time interval.
[0051] n i,j (t+1)=n i,j (t)+y i-1,j (t)-y i,j (t)+Σ j' f i-1,j',j (t)-Σ j f i,j,j' (t),
[0052] Among them, n i,j (t) represents the number of vehicles on the cell (i, j) at the initial time in the tth time interval; n i,j (t+1) represents the number of vehicles on the cell (i, j) at the initial time in the t+1th time interval; y i,j (t) represents the number of vehicles flowing from cell (i, j) to cell (i+1, j) in the tth time interval; y i-1,j (t) represents the number of vehicles flowing from cell (i-1, j) to cell (i, j) in the tth time interval; f i,j,j'(t)(j'∈{j-1,j+1}) represents the number of vehicles flowing from cell (i,j) to cell (i+1,j') in the tth time interval; f i-1,j',j (t)(j'∈{j-1,j+1}) represents the number of vehicles flowing from cell (i-1,j') into cell (i,j) in the tth time interval.
[0053] In addition to considering the propagation of traffic in the same lane, the cell of mixed traffic flow also needs to consider the propagation of traffic in lane change. Since the head distance of lane change vehicles is greater than the head distance of following vehicles, the present invention establishes an amplification coefficient α (α>1) to represent this effect:
[0054] y i,j (t)+∑ j f i,j,j' (t)≤S i,j (t),
[0055] y i,j (t)+α∑ j' f i,j',j (t)≤R i,j (t),
[0056] In the formula, S i,j (t) is the maximum number of vehicles outflowing from the cell, R i,j (t) is the maximum number of vehicles flowing into the cell.
[0057] Figure 6 (a) is the maximum number of vehicles that can flow out of the cell S i,j (t) and the maximum number of vehicles flowing into the cell R i,j (t) Schematic diagram, which can be described as:
[0058]
[0059] Among them, n i,j (t) represents the number of vehicles on the cell (i, j) at the initial time in the tth time interval, Q i,j represents the maximum number of vehicles that the cell (i, j) can accommodate; ω1 and ω2 represent S i,j (t) and R i,j (t) is the reverse wave propagation speed; k c and k j denote the critical density and crowding density respectively; Δx denotes the cell length; v f Indicates free flow speed.
[0060] Figure 6 (b) shows the basic principle diagram of flow-density under different penetration rates. Under appropriate headway, the increase in the penetration rate of autonomous driving vehicles can improve the capacity and reverse wave propagation speed of roads with mixed traffic flows.
[0061] When the traffic occupancy rate of the adjacent lane is low, the vehicle can consider changing lanes. At the same time, other factors, such as road layout, traffic signs and route selection, may also affect the lane changing behavior. The present invention only considers the relative difference in occupancy rate, and other factors are collectively referred to as hidden factors. The lane changing utility can be written as:
[0062]
[0063] in, Indicates the hidden factors that may affect lane-changing behavior, represents the relative difference coefficient between the occupancy rates of cells (i,j) and (i,j'); n i,j (t) represents the number of vehicles on the cell (i, j) at the initial time in the tth time interval; n i,j' (t) represents the number of vehicles on the cell (i, j') at the initial time in the tth time interval.
[0064] The probability of lane change can be expressed as:
[0065]
[0066] The present invention refers to the traffic that needs to change lanes as demand flow, which can be divided into passing demand flow and lane change demand flow, which are represented by A i,j,j (t) and A i,j,j' (t),j'∈{j-1,j+1}.
[0067] The lane change demand flow is determined by the following formula:
[0068] A i,j,j' (t) = S i,j (t)·P i,j,j' (t),
[0069] The demand flow is determined by the following formula:
[0070] A i,j,j (t) = S i,j (t)-∑ j' A i,j,j' (t).
[0071] The lane change demand flow and the through demand flow may be limited because the available space in the downstream cell may not be enough to receive all the demand flows. In this case, the available receiving space will be allocated to the lane change demand flow and the through demand flow according to their proportion. Therefore, a coefficient δ is introduced to scale the demand flow. The allocation of the receiving part is described by the following equation:
[0072]
[0073] yi,j (t) = δ·A i,j,j (t),
[0074] f i,j+1,j (t) = δ·A i,j+1,j (t),
[0075] f i,j-1,j (t) = δ·A i,j-1,j (t),
[0076] In the formula, R i+1,j (t) is the maximum number of vehicles flowing into the cell (i+1,j) in the tth time interval; A i,j',j (t) is the sum of the number of vehicles flowing from cells (i,j-1), (i,j), and (i,j+1) into cell (i+1,j) during the tth time interval; A i,j,j (t) represents the demand flow; y i,j (t) represents the number of vehicles flowing from cell (i, j) to cell (i+1, j) in the tth time interval; f i,j+1,j (t) represents the number of vehicles flowing from cell (i, j+1) to cell (i+1, j) in the tth time interval; f i,j-1,j (t) represents the number of vehicles flowing from cell (i, j-1) to cell (i+1, j) in the tth time interval; A i,j+1,j (t) represents the maximum number of vehicles flowing from cell (i, j+1) to cell (i+1, j) in the tth time interval; A i,j-1,j (t) represents the maximum number of vehicles flowing from cell (i, j-1) into cell (i+1, j) during the tth time interval.
[0077] Step 3: According to the vehicle passing demand flow data and the vehicle lane changing demand flow data, the lane changing congestion impact, the speed change rate of each lane and the speed variance value of each lane are obtained on the controlled section. The comprehensive state of the lane changing congestion impact, the speed change rate of each lane and the speed variance value of each lane is used as the state space, and whether the controlled section is open to the dedicated lane for autonomous driving vehicles is used as the action space, and the reward function is set; the state space and action space are used to train the parameters of the deep reinforcement learning model, and the reward function is used to determine whether the trained deep reinforcement learning model converges. If so, the dedicated lane control model is obtained.
[0078] The present invention uses the deep reinforcement learning DQN (DeepQ-learning) model, and first transforms the research scenario into the DQN model. It is assumed that the road vehicles are cars with consistent performance and appearance, there are no emergencies on the road, and there is no delay in communication between vehicles.
[0079] (1) State space
[0080] When constructing the deep reinforcement learning model, the output results of CTM are used, and the traffic flow state is used as the system state space. The approximate value function Qs,a;θ is used to determine the optimal action, and the parameters are updated through continuous training. The experience replay mechanism is used to reduce the correlation between samples and improve the efficiency and stability of training.
[0081] In order to better represent the dynamic characteristics of traffic flow, the present invention uses the comprehensive state of the lane change congestion impact s1t, the speed change rate of each lane s2t and the speed variance value s3t of each lane in the road section as the current system state.
[0082] Define the traffic flow state s at time t t for:
[0083] S=s1t,s2t,s3t,
[0084]
[0085] s3(t)={-∑σ k |k=1,2,3},
[0086] Among them, k represents the lane number in the controlled section, n and n max Indicates the number of vehicles and the maximum number of vehicles in lane j of the controlled section; It represents the average speed of lane k at time T in the controlled section; represents the average speed of lane k in the controlled section at time T+1; σ k represents the speed variance of the k lanes in the controlled section; f i,j,j' represents the number of vehicles flowing from cell (i, j) into cell (i+1, j') at time T, J represents all lanes, and J\{j} represents all lanes except lane j.
[0087] When t When s is larger, it indicates that the road capacity within the section is better, the vehicle speed increases faster, and the speed change within the controlled section is smaller, the mixed traffic flow is more stable, and the control of the dedicated lanes can effectively improve the efficiency of the traffic system; if s t When it is smaller, it indicates that the mixed traffic flow in the road section is unstable, and there may be traffic congestion or the spread of traffic congestion.
[0088] (2) Action Space
[0089] The present invention takes three consecutive road sections as traffic control objects and sets the action space A = {a k (T)|k=1,2,3,a k (T) represents the opening action of the kth road segment at time T. k When (T) = 0, section k is closed. kWhen (T) = 1, the road section k is open, so the agent has 8 actions to choose from at time T. At the same time, in order to reduce the frequent lane changes caused by changing the open state, the action interval ΔT is set to 30 minutes in the present invention.
[0090] (3) Reward Function
[0091] In the system designed by the present invention, the agent learns by interacting with the dynamic traffic environment. Each time it takes an action, it will receive a reward as feedback from the environment. The present invention sets a reward function r(t) to evaluate the decision-making behavior. The specific formula of the reward function r(t) is as follows:
[0092]
[0093] Among them, γ1, γ2, γ3, and γ4 are the weight values of each influencing factor; n and n max Indicates the number of vehicles and the maximum number of vehicles in one lane of the controlled section; It indicates the average speed of one lane at time T in the controlled section; It indicates the average speed of one lane at time T+1 in the controlled section; It represents the average speed of two lanes at time T in the controlled section; It indicates the average speed of two lanes in the controlled section at time T+1; It represents the average speed of the three lanes at time T in the controlled section; represents the average speed of the three lanes in the controlled section at time T+1; σ k represents the speed variance of the k lanes in the controlled section; f i,j,j' represents the number of vehicles flowing from cell (i, j) into cell (i+1, j') at time T; J represents all lanes, and J\{j} represents all lanes except lane j.
[0094] Considering the impact of vehicle lane changing on upstream traffic under congestion, the lane changing behavior of vehicles is constrained under a given coefficient γ1 to avoid vehicles changing lanes in congestion; through the speed change rate of each lane within ΔT, γ2 and γ3 are used to reflect the improvement of the control strategy on the traffic efficiency of dedicated lanes and ordinary lanes; in order to improve road traffic safety, under a given coefficient γ4, the standard deviation of vehicle speed in each section is evaluated to ensure the stability of traffic flow.
[0095] After the environment modeling is completed, the traffic system observation data (CTM simulation data is used instead in the theoretical stage) is used to perform model training cycles, calculate the loss function and update the parameters of the Q network, stabilize the training process, and end the training when the reward converges and stabilizes. Use the trained model for testing, evaluate its performance in the environment, and optimize and adjust the model parameters and training process based on the test results, and derive the optimal spatiotemporal control strategy for dedicated lanes under a given mixed traffic flow state.
[0096] Step 4: Input the road traffic data of the autonomous driving vehicle dedicated lane control section into the dedicated lane control model, and dynamically control the switching of the autonomous driving vehicle dedicated lane through the dedicated lane control model.
[0097] The present invention also provides a dynamic control system for dedicated lanes for autonomous driving vehicles, including a data acquisition module, a cellular update model construction module, a dedicated lane control model construction module and a lane control module. The data acquisition module is used to acquire road traffic data of the dedicated lane control section of the autonomous driving vehicle; the cellular update model construction module is used to divide the control section into a plurality of cells of equal length, construct a cellular update model according to the road traffic data, and obtain vehicle passing demand flow data and vehicle lane change demand flow data in each cell within equal time steps through the cellular update model; the dedicated lane control model construction module is used to acquire the lane change congestion impact of the control section, the speed change rate of each lane and the speed variance value of each lane according to the vehicle passing demand flow data and the vehicle lane change demand flow data, and calculate the lane change congestion impact. The comprehensive state of the lane conditions, the speed change rate of each lane and the speed variance value of each lane is used as the state space, and whether the control section is open to the dedicated lane for autonomous driving vehicles is used as the action space, and a reward function is set; the state space and action space are used to train the parameters of the deep reinforcement learning model, and the reward function is used to determine whether the trained deep reinforcement learning model converges. If so, a dedicated lane control model is obtained; the lane control module is used to input the road traffic data of the dedicated lane control section for autonomous driving vehicles into the dedicated lane control model, and the dedicated lane control model is used to dynamically control the switch of the dedicated lane for autonomous driving vehicles.
[0098] The present invention also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute a dynamic control method for a dedicated lane of an autonomous driving vehicle.
[0099] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute a dynamic control method for a dedicated lane for an autonomous driving vehicle.
[0100] The present invention takes the case where the penetration rate of autonomous driving vehicles is 0.4 during the morning peak period of 8:00-9:30 as an example. Under a given training cycle (epoch), the dedicated lane control strategy is continuously optimized. The convergence result of the reward function and the open state of the dedicated lane are shown in Figure 2. Figure 7 (a) Figure 7 As shown in (b), from Figure 7 As can be seen from (a), as the number of training times increases, the reward function gradually converges. At this time, the optimal dedicated lane control strategy can be obtained, such as Figure 7 (b) as shown.
[0101] The above embodiments are only preferred specific implementation methods of the present invention, and the protection scope of the present invention is not limited thereto. Any simple changes or equivalent replacements of the technical solutions that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belong to the protection scope of the present invention.
Claims
1. A method for dynamic control of a dedicated lane for an autonomous driving vehicle, characterized in that: The steps include: Obtain road traffic data for the controlled section of the dedicated lane for autonomous vehicles; Divide the controlled road section into a plurality of cells of equal length, construct a cell update model according to the road traffic data, and obtain vehicle passing demand flow data and vehicle lane change demand flow data in each cell within an equal time step through the cell update model; According to the vehicle passing demand flow data and the vehicle lane changing demand flow data, the lane changing congestion impact, the speed change rate of each lane and the speed variance value of each lane of the controlled section are obtained. The comprehensive state of the lane changing congestion impact, the speed change rate of each lane and the speed variance value of each lane is used as the state space, and whether the controlled section is open to the dedicated lane for autonomous driving vehicles is used as the action space, and the reward function is set; the parameters of the deep reinforcement learning model are trained using the state space and the action space, and the reward function is used to determine whether the trained deep reinforcement learning model converges. If so, a dedicated lane control model is obtained; The road traffic data of the section where the dedicated lane for the autonomous driving vehicle is controlled is input into the dedicated lane control model, and the switching of the dedicated lane for the autonomous driving vehicle is dynamically controlled through the dedicated lane control model.
2. The method for dynamic control of a dedicated lane for an autonomous driving vehicle according to claim 1, characterized in that: Obtain the road traffic data of the controlled section of the dedicated lane for autonomous driving vehicles. Specifically, based on the layout distance of the road monitoring system, the controlled section is discretized into multiple network intervals. There are 2-3 road monitoring cameras in each network interval. The road traffic data is collected through the road monitoring cameras.
3. The method for dynamic control of a dedicated lane for an autonomous driving vehicle according to claim 1, characterized in that: The road traffic data includes traffic flow characteristics and vehicle speed change values within the controlled section, wherein the traffic flow characteristics include the current number of vehicles, the number of lane-changing vehicles, and the vehicle speed within the controlled section.
4. The method for dynamic control of a dedicated lane for an autonomous driving vehicle according to claim 1, characterized in that: There are multiple control sections, and the action space is whether to open a dedicated lane for autonomous driving vehicles for each control section.
5. The method for dynamic control of a dedicated lane for an autonomous driving vehicle according to claim 1, characterized in that: The cell update model is: in, represents the number of autonomous driving vehicles on the cell (i, j) at the initial time in the tth time interval; represents the number of autonomous driving vehicles on the cell (i, j) at the initial time in the t+1th time interval; represents the number of autonomous vehicles flowing from cell (i, j) to cell (i+1, j) in the tth time interval; represents the number of autonomous vehicles flowing from cell (i-1, j) to cell (i, j) in the tth time interval; represents the number of autonomous driving vehicles flowing from cell (i, j) into cell (i+1, j') in the tth time interval, i is the cell index on a lane, j is the index of the lane, and j' represents the other two lanes adjacent to lane j.
6. The method for dynamic control of a dedicated lane for an autonomous driving vehicle according to claim 1, characterized in that: The reward function is: Among them, γ1, γ2, γ3, and γ4 are the weight values of each influencing factor; n and n max Indicates the number of vehicles and the maximum number of vehicles in one lane of the controlled section; It indicates the average speed of one lane at time T in the controlled section; It indicates the average speed of one lane at time T+1 in the controlled section; It represents the average speed of two lanes at time T in the controlled section; It indicates the average speed of two lanes in the controlled section at time T+1; It represents the average speed of the three lanes at time T in the controlled section; represents the average speed of the three lanes in the controlled section at time T+1; σ k represents the speed variance of the k lanes in the controlled section; f i,j,j' Represents the number of vehicles flowing from cell (i,j) into cell (i+1,j') at time T.
7. The method for dynamic control of a dedicated lane for an autonomous driving vehicle according to claim 1, characterized in that: The deep reinforcement learning model is the DQN algorithm.
8. A dynamic control system for a dedicated lane for an autonomous driving vehicle, characterized in that: include: A data acquisition module, used to acquire road traffic data of the controlled section of the dedicated lane for autonomous driving vehicles; A cellular update model construction module is used to divide the controlled road section into a plurality of cells of equal length, construct a cellular update model according to the road traffic data, and obtain vehicle passing demand flow data and vehicle lane change demand flow data in each cell within an equal time step through the cellular update model; The dedicated lane control model construction module is used to obtain the lane change congestion impact of the controlled section, the speed change rate of each lane, and the speed variance value of each lane based on the vehicle passing demand flow data and the vehicle lane change demand flow data, and use the comprehensive state of the lane change congestion impact, the speed change rate of each lane, and the speed variance value of each lane as the state space, and use whether the controlled section is open to the dedicated lane for autonomous driving vehicles as the action space, and set the reward function; use the state space and action space to train the parameters of the deep reinforcement learning model, and use the reward function to determine whether the trained deep reinforcement learning model converges. If so, obtain the dedicated lane control model; The lane control module is used to input the road traffic data of the dedicated lane control section of the autonomous driving vehicle into the dedicated lane control model, and dynamically control the switching of the dedicated lane of the autonomous driving vehicle through the dedicated lane control model.
9. A computer device, characterized in that: It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the dynamic control method for a dedicated lane for an autonomous driving vehicle as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for loading by a processor to execute the dynamic control method for a dedicated lane for an autonomous driving vehicle as described in any one of claims 1-7.