Intelligent network connection vehicle no-signal-control intersection passing optimization method based on coupling algorithm

Abstract

The invention discloses an intelligent network connection vehicle non-signal-control intersection passing optimization method based on a coupling algorithm, and the acceleration of a vehicle is adjusted and controlled based on a mode of coupling a deep reinforcement learning algorithm and a microscopic traffic flow model. Constructing a maximum acceleration passing strategy, a minimum acceleration passing strategy and a dynamic adjustment acceleration passing strategy; and selecting a corresponding passing strategy according to the actual scene to control the intelligent network connection vehicle to pass at the signal-control-free intersection. The traffic efficiency can be effectively improved and the safety can be considered in a complex traffic scene without a signal control intersection.

Description

Technical Field

[0001] The present invention relates to an intersection passing method for an intelligent network-connected vehicle, in particular to an intelligent network-connected vehicle intersection passing optimization method based on a coupling algorithm. Background Art

[0002] In existing traffic simulation and control technologies, traffic optimization of intelligent connected vehicles has received extensive attention, especially in the scenario of signal-controlled intersections. However, there are relatively few studies on traffic optimization and simulation of intersections without signal control. Most existing technologies rely on a single algorithm for optimization and fail to effectively combine deep reinforcement learning with microscopic traffic flow models to improve traffic efficiency and safety. In addition, although deep reinforcement learning is increasingly widely used in the field of autonomous driving, its application in complex traffic scenarios, especially at intersections without signal control, is still limited.

[0003] The currently published technical solutions for information quality effectiveness analysis include the following:

[0004] 1. Intelligent networked hybrid vehicle multi-system dynamic coordination control system and method (invention patent number: CN202111153172.9) proposes a multi-system dynamic coordination control method for intelligent networked hybrid vehicles. This method obtains vehicle status information through on-board sensors and transmits it to the multi-system dynamic coordination control module to optimize the allocation of engine power and motor power to improve the vehicle's fuel economy and system coordination capabilities. However, this method mainly focuses on vehicle internal energy management, does not involve traffic optimization at uncontrolled intersections, and does not combine microscopic traffic flow models and deep reinforcement learning algorithms, making it difficult to cope with traffic problems in complex traffic environments.

[0005] 2. A reinforcement learning-based economic passage method for connected vehicles at signal-controlled intersections (invention patent number: CN202110554621.4) provides an economic passage strategy for connected vehicles at signal-controlled intersections based on reinforcement learning. This method uses a deep deterministic policy gradient algorithm (DDPG) to model vehicle and roadside environmental state information to optimize the traffic efficiency and energy consumption performance of signal-controlled intersections. However, this method is only applicable to signal-controlled intersections, and does not consider the complex traffic flow scenarios of non-signal-controlled intersections. It cannot solve the problem of vehicle collisions under high traffic volume using a single reinforcement learning algorithm, nor does it combine microscopic traffic flow models with deep reinforcement learning, making it difficult to improve the traffic efficiency and safety of non-signal-controlled intersections.

[0006] 3. A method for coordinated optimization control of vehicle speed and torque of an intelligent connected PHEV (patent number: CN202310707932.9) relates to a method for coordinated optimization control of vehicle speed and torque of an intelligent connected PHEV vehicle. This method solves the coupling problem of autonomous vehicle speed selection and power source torque by establishing a coordinated optimization model in the time domain and the space domain, thereby improving the energy economy of the vehicle. Although this method optimizes the internal power system of the vehicle, it does not involve traffic optimization at intersections without signal control, nor does it combine microscopic traffic flow models for coordinated control, making it difficult to meet the needs of intelligent connected vehicles in complex traffic environments.

[0007] 4. A method for solving the traffic order of intelligent connected vehicles at intersections based on dynamic division of the time domain (invention patent number: CN202210945297.3) provides a method for solving the traffic order of intelligent connected vehicles based on dynamic division of the time domain. This method optimizes the traffic order of vehicles at intersections by dynamically dividing the time domain, reduces the probability of vehicle conflicts, and improves the traffic efficiency of intersections. Although this method reduces the probability of vehicle conflicts and improves traffic efficiency, it is only applicable to the optimization of vehicle traffic order. It does not combine deep reinforcement learning with microscopic traffic flow models to solve the trade-off between traffic efficiency and safety, and it is difficult to cope with complex traffic scenarios at intersections without signal control.

[0008] 5. Intelligent connected vehicle traffic coordination method and system at non-signalized intersections (invention patent number: CN202111451373.7) involves a traffic coordination method for intelligent connected vehicles at non-signalized intersections. The system improves the safety and traffic efficiency of vehicles at non-signalized intersections by optimizing vehicle sorting and speed recommendations, while reducing the use of complex algorithms and reducing the implementation cost of the system. Although this method improves traffic efficiency, it is difficult to achieve a more efficient and safer way of passing without combining the microscopic traffic flow model with the deep reinforcement learning algorithm for optimization.

[0009] 6. Traffic coordination method and system for intelligent connected vehicles at non-signalized intersections (patent number: CN202111451373.7) proposes a traffic coordination method for intelligent connected vehicles at non-signalized intersections. This method ensures safe and efficient passage at non-signalized intersections by real-time sorting and speed recommendations for vehicles in the intersection, avoiding the high cost and cumbersomeness of using complex algorithms. Although traffic is optimized through sorting and speed recommendations, the combination of microscopic traffic flow models and deep reinforcement learning is also not considered, and there is a lack of comprehensive optimization of intelligent connected vehicles in complex traffic environments.

[0010] Therefore, most of the existing technical solutions have optimized traffic for specific vehicle optimization or signal light control scenarios. However, in complex traffic scenarios without signal-controlled intersections, there is a common problem that both traffic efficiency and driving safety cannot be taken into account at the same time. Summary of the invention

[0011] The purpose of the present invention is to provide a method for optimizing the passage of intelligent networked vehicles at non-signaled intersections based on a coupling algorithm. The present invention can not only effectively improve the passage efficiency but also take into account safety in complex traffic scenarios at non-signaled intersections.

[0012] The technical solution of the present invention is: an optimization method for the passage of intelligent connected vehicles at non-signaled intersections based on a coupling algorithm, which adjusts and controls the acceleration of the vehicle by coupling a deep reinforcement learning algorithm and a microscopic traffic flow model, and thereby constructs a maximum acceleration passage strategy, a minimum acceleration passage strategy and a dynamically adjusted acceleration passage strategy; according to the actual scenario, the corresponding passage strategy is selected to control the passage of intelligent connected vehicles at non-signaled intersections.

[0013] In the aforementioned intelligent connected vehicle traffic optimization method for non-signaled intersections based on coupling algorithm, the control equation of the microscopic traffic flow model is:

[0014] Where: v n is the vehicle speed; Δv n is the speed difference between the front and rear vehicles; Δx n is the distance between the front and rear vehicles; V[Δx n (t)] = V1 + V2 tanh [C1 (Δx n -l c )-C2], V(·) is the optimized speed function identified by measured data; a is the sensitivity coefficient; k is the speed difference coefficient; V1, V2, C1, C2 are the constant terms of the optimized speed function; l c is the length of the car body; tanh(·) is the hyperbolic tangent function;

[0015] The minimum acceleration passing strategy in the aforementioned intelligent connected vehicle non-signaled intersection passing optimization method based on the coupling algorithm includes the following steps:

[0016] S11, initializing actual vehicle driving acceleration a drive , the acceleration a generated by the deep learning algorithm DRL , the acceleration a generated by the microscopic traffic flow model TF ; Minimum acceleration a m , maximum acceleration a M ;

[0017] S12, initializing the status information of the target vehicle i;

[0018] S13. For each vehicle i, obtain the status information of the target vehicle i;

[0019] S14, interact with the environment and calculate the corresponding acceleration a i DRL and

[0020] S15, determine and select the actual driving acceleration:

[0021] S16. Standardize the actual driving acceleration:

[0022] S17: Output the actual driving acceleration of vehicle i And use that acceleration to update the state.

[0023] The maximum acceleration passing strategy in the aforementioned intelligent connected vehicle non-signaled intersection passing optimization method based on the coupling algorithm includes the following steps:

[0024] S21, initializing actual vehicle driving acceleration a drive , the acceleration a generated by the deep learning algorithm DRL , the acceleration a generated by the microscopic traffic flow model TF ; Minimum acceleration a m , maximum acceleration a M ;

[0025] S22, initializing the status information of the target vehicle i;

[0026] S23. For each vehicle i, obtain the status information of the target vehicle i;

[0027] S24, interact with the environment and calculate the corresponding acceleration a i DRL and

[0028] S25, determine and select the actual driving acceleration:

[0029] S26. Standardize the actual driving acceleration:

[0030] S27: Output the actual driving acceleration of vehicle i And use that acceleration to update the state.

[0031] In the aforementioned intelligent connected vehicle traffic optimization method for non-signaled intersections based on coupling algorithm, the adjustment process of dynamically adjusting the acceleration traffic strategy is: the two factors of the front and rear vehicle distance Δx and the speed difference Δv are linearly combined to generate a safety index S to measure the safety level under the current driving conditions, as follows:

[0032]

[0033] Where Δx is the distance between the front and rear vehicles; Δv is the speed difference between the front and rear vehicles; X s is the safe distance threshold of the front and rear vehicles; V s is the speed difference safety threshold of the front and rear vehicles; S is the safety index, ranging from 0 to 1;

[0034] When S≤α1, the system adopts a conservative driving strategy and selects the minimum acceleration passing strategy; when S≥α2, the system adopts an aggressive driving strategy and selects the maximum acceleration passing strategy; when α1<S<α2, the linear combination acceleration passing strategy is selected; α1 is the conservative driving strategy threshold; α2 is the aggressive driving strategy threshold, α1 and α2 are 0.3 and 0.6 respectively.

[0035] In the aforementioned intelligent networked vehicle non-signaled intersection traffic optimization method based on coupling algorithm, the linear combination acceleration traffic strategy includes the following steps:

[0036] S31, initializing actual vehicle driving acceleration a drive , the acceleration a generated by the deep learning algorithm DRL , the acceleration a generated by the microscopic traffic flow model TF ; Minimum acceleration a m , maximum acceleration a M ;

[0037] S32, initializing the status information of the target vehicle i;

[0038] S33, for each vehicle i, obtaining the state information of the target vehicle i;

[0039] S34, interact with the environment and calculate the corresponding acceleration a i DRL and

[0040] S35, calculating the actual driving acceleration according to the rules

[0041] a drive =(1-β)·a DRL +β·a TF ,

[0042] S36. Standardize the actual driving acceleration:

[0043] S37: Output the actual driving acceleration of vehicle i And use that acceleration to update the state.

[0044] Compared with the prior art, the present invention couples the microscopic traffic flow model with the deep reinforcement learning algorithm, so that the corresponding acceleration optimization traffic strategy (three acceleration optimization traffic strategies) can be dynamically selected according to the changes in real-time traffic flow, so as to better balance the traffic efficiency and safety issues in the scenario of non-signaled intersections.

[0045] Specifically, this application considers the coupling algorithm of deep reinforcement learning and microscopic traffic flow model at the same time, and optimizes the traffic efficiency and safety of intelligent connected vehicles at non-signaled intersections by integrating the advantages of both. The core of deep reinforcement learning is to be able to learn the best behavior strategy in an interactive environment, while the microscopic traffic flow model provides the system with more accurate environmental constraints to simulate vehicle behavior in complex traffic flows. By combining these two vehicle motion control algorithms, vehicle collisions can be avoided while greatly improving the efficiency and speed of intelligent connected vehicles passing through non-signaled intersections.

[0046] In summary, the present invention can not only effectively improve traffic efficiency but also take safety into consideration in complex traffic scenarios without signalized intersections. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a deep reinforcement learning algorithm framework diagram;

[0048] Figure 2 This is the DDPG algorithm structure diagram;

[0049] Figure 3 It is the minimum acceleration passing strategy flow chart;

[0050] Figure 4 It is the maximum acceleration passing strategy flow chart;

[0051] Figure 5 It is a flow chart for dynamically adjusting the acceleration traffic strategy;

[0052] Figure 6 This is the Matplotlib visualization effect diagram in training mode;

[0053] Figure 7 The simulation image (left) and video (right) display in the test mode;

[0054] Figure 8 It is a graph showing the change of the average vehicle speed (a) and acceleration rate (b) with the number of training steps;

[0055] Fig. 9 The figure shows the change of vehicle collision rate (a) and the cumulative number of vehicles in collision (b) with the number of training steps. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.

[0057] Embodiment. A method for optimizing the passage of intelligent networked vehicles at intersections without signal control based on a coupling algorithm, adjusting and controlling the acceleration of vehicles by coupling a deep reinforcement learning algorithm and a microscopic traffic flow model, and thereby constructing a maximum acceleration passage strategy, a minimum acceleration passage strategy, and a dynamically adjusted acceleration passage strategy; selecting corresponding passage strategies according to actual scenarios to control the passage of intelligent networked vehicles at intersections without signal control.

[0058] Deep reinforcement learning algorithms

[0059] To improve the accuracy and efficiency of deep reinforcement learning algorithm model simulation. This application uses advanced deep reinforcement learning technology to build a highly complex neural network model, further improving the intelligence level of the system. The core of deep reinforcement learning technology is that it can learn the optimal behavior strategy through interaction, so as to make the best decision in different environments. Using deep reinforcement learning technology (deep deterministic policy gradient (DDPG) algorithm, the structure is as follows Figure 2 As shown in Figure 2, the neural network is constructed. The overall deep learning framework is as follows: Figure 1 As shown in the figure, a deep neural network based on the Actor-Critic framework is constructed. The deep reinforcement learning algorithm consists of two core parts: the actor network and the critic network. The actor network is used to generate specific action strategies, and the critic network is responsible for evaluating the quality of the generated strategies. In order to effectively solve the problem of the limitation of reinforcement learning on environmental exploration, the DDPG algorithm introduces the experience pool (ReplayBuffer). The agent will obtain the samples obtained through the exploration strategy in the form of tuples (s t ,a t ,r t ,s t+1 ) in the experience pool, and small batch random sampling is used for exploration training. At the same time, in order to avoid the robustness problem caused by the use of the experience pool, the DDPG algorithm introduces the target network and soft target update mechanism, and softly updates the parameters θ in the target network through the learning rate τ, thereby avoiding overfitting and improving the robustness and generalization ability of the model.

[0060] Specifically, the DDPG algorithm pseudo code is as follows:

[0061]

[0062] The DDPG algorithm is mainly composed of the original network, the target network and the experience pool, as shown below: Figure 2 As shown. In the whole system, the actor network is responsible for the parameter θ μ The critic network is responsible for the iterative update of the parameter θ Q It is used to iteratively update the output value Q at the current moment. The critic target network is responsible for updating the target parameter θ of the value network Q′ The experience pool module is responsible for storing samples and providing sampling data. During the training process, in order to update the parameters of the original network, samples are randomly drawn from the experience pool each time.

[0063] The Actor-Critic (AC) network is a reinforcement learning algorithm that combines the policy gradient method with the value function approximation. It aims to solve the efficiency and variance problems existing in traditional reinforcement learning methods by separating the strategy (behavior) and the evaluation (value) and learning them simultaneously to achieve faster learning speed and higher stability. The AC network mainly consists of two parts: Actor and Critic. Among them,

[0064] Actor network: responsible for selecting actions based on the current state of the environment, that is, it directly controls the behavior of the learning body. The purpose of the actor is to learn a policy function that can map the current state of the environment to one or more possible actions to maximize future rewards.

[0065] ·Critic network: responsible for evaluating the quality of the actions selected by the Actor, that is, evaluating the expected return of the current action. Critic estimates the sum of future rewards that can be obtained by taking a specific action based on the current state by learning a value function. This evaluation helps the Actor understand the long-term impact of its behavior and adjust the strategy to optimize the behavior.

[0066] The process is as follows:

[0067]

[0068]

[0069] Microscopic traffic flow model

[0070] In the simulation of intelligent connected vehicles, the lack of environmental constraints is an important reason why the deep reinforcement learning algorithm cannot effectively avoid vehicle collisions. In order to effectively solve this problem, the present invention proposes a coupling algorithm based on an improved microscopic traffic flow model, which is used as a key environmental constraint to further optimize the traffic performance of intelligent connected vehicles in non-signaled intersection scenarios. Through the dynamic interaction between vehicles, the microscopic traffic flow model can provide accurate traffic information in a more complex traffic flow environment, thereby forming an effective synergy mechanism with the strategy generated by deep reinforcement learning, reducing possible collisions and unrealistic behaviors in the simulation.

[0071] The control equation proposed in this invention is improved on the basis of the traditional microscopic traffic flow model, focusing on solving the problem of excessive deceleration and emergency stop collision that may be caused by the speed difference model and the optimized speed model in practical applications. Specifically, the improved model dynamically balances the influence of vehicle speed difference and spacing change by introducing a new acceleration calculation formula, ensuring the reaction speed and safety in emergency situations, and proposes a speed difference model with a reciprocal term of spacing difference and a speed difference term. The control equation is as follows:

[0072]

[0073] Where: v n is the vehicle speed; Δv n =v n+1 -v n ,Δv n is the speed difference between the front and rear vehicles; Δx n =x n+1 -x n ,Δx n is the distance between the front and rear vehicles; V[Δx n (t)] = V1 + V2tanh[C1(Δx n -l c )-C2], V(·) is the optimized speed function identified by measured data; a is the sensitivity coefficient; k is the speed difference coefficient; V1, V2, C1, C2 are the constant terms of the optimized speed function; l c is the length of the car body; tanh(·) is the hyperbolic tangent function; the parameter value list is as follows

[0074] Table 1 Parameters

[0075]

[0076] This control equation adds the inverse term of the distance difference to the original speed difference model, so that when the distance between the front and rear vehicles is close, the vehicle can react to deceleration more promptly, thereby effectively reducing the occurrence of collisions. At the same time, the control equation reduces unnecessary excessive braking when the distance between the vehicles is far, thereby improving traffic efficiency.

[0077] In order to solve the problem of lack of environmental constraints for intelligent connected vehicles in complex traffic scenarios without signalized intersections, this paper proposes a driving strategy based on a coupled microscopic traffic flow model. By combining the improved microscopic traffic flow model and the deep reinforcement learning algorithm, the vehicle can more accurately adjust its driving behavior according to the real-time traffic environment to achieve efficient and safe passage. The following three acceleration passage strategies are set:

[0078] 1) High-security coupled microscopic traffic flow model driving strategy (minimum acceleration strategy):

[0079] Safety is an important prerequisite in vehicle simulation. It is a common method to select a smaller acceleration to achieve low-speed safe passage of vehicles. The specific high-safety coupled microscopic traffic flow model driving strategy is as follows: Figure 3 As shown:

[0080]

[0081]

[0082] 2) High traffic efficiency coupled with microscopic traffic flow model driving strategy (maximum acceleration traffic strategy):

[0083] The improvement of traffic efficiency helps vehicles to achieve smoother traffic flow, and effectively reflects the urgency of drivers passing through intersections in daily life. By selecting a larger acceleration, high-speed and high-traffic efficiency of vehicles can be achieved. The specific high-traffic efficiency coupled micro-traffic flow model driving strategy is as follows: Figure 4 As shown:

[0084]

[0085] 3) Dynamically adjust the acceleration strategy:

[0086] Only using high-safety coupled driving strategies often results in vehicles traveling too slowly, thus causing vehicle congestion; only using high-traffic efficiency coupled driving strategies will result in unrealistic vehicle collisions. To this end, the present invention proposes a dynamic adjustment coupled micro-traffic model driving strategy, which achieves safety and traffic efficiency by dynamically coordinating the driving strategies of vehicles. Specifically:

[0087] Safety and traffic efficiency are mainly determined by the front and rear vehicle distance Δx and the speed difference Δv. The two factors are linearly combined to generate a safety index S to measure the safety level under current driving conditions, as follows:

[0088]

[0089] Where Δx is the distance between the front and rear vehicles; Δv is the speed difference between the front and rear vehicles; X s is the safe distance threshold of the front and rear vehicles; V s is the speed difference safety threshold of the front and rear vehicles; S is the safety index, ranging from 0 to 1. The closer the value is to 1, the higher the vehicle driving safety. According to the safety index S, the conservative driving strategy threshold α1 and the aggressive driving strategy threshold α2, the detailed driving behavior is defined:

[0090] 1. When S≤α1, the system adopts a conservative driving strategy and selects the minimum acceleration to ensure the driving safety of the vehicle:

[0091] a drive =min(a DRL ,a TF )

[0092] 2. When S ≥ α2, the system adopts an aggressive driving strategy and selects the maximum acceleration to improve the vehicle's traffic efficiency:

[0093] a drive =max(a DRL ,a TF )

[0094] 3. When α1<S<α2, the system linearly combines the acceleration generated by the deep reinforcement algorithm and the acceleration generated by the microscopic traffic flow model according to the safety index S, and adopts a linear combination acceleration driving strategy considering the driving effects of the two strategies:

[0095] a drive =(1-β)·a DRL +β·a TF

[0096]

[0097]

[0098]

[0099] Intelligent connected vehicle non-signaled intersection passing system based on coupling algorithm

[0100] It includes the main control module, the maximum acceleration passing module, the minimum acceleration passing module and the linear combination acceleration passing module. The main control module realizes the constraints and operation of the entire system by calling the other three modules. The main functions are divided into two parts: training mode (Train) and test mode (Test).

[0101] To facilitate system operation and modification of related parameters, the system has set up a program interface for adjusting related parameters so that users can better adapt to independent project modules.

[0102] Training Mode (Train)

[0103] The training mode mainly provides users with virtual training to assist in completing the behavior simulation of intelligent connected vehicles at cross-no-signal controlled intersections. In this mode, after running the program, the terminal will display real-time simulation results. Every 100 steps, the terminal will output specific operating parameters, including but not limited to:

[0104] Table 2 Symbols and meanings (I)

[0105]

[0106]

[0107] Each round (epoch) has a total of 6000 steps. After running a round, the system will write the running information into x.cptk.index, x.cptk.meta and x.cptk.data (x represents the current round number). During the continuous operation of the simulation system, the deep reinforcement learning model continues to advance towards the preset "zero collision rate" goal. When the user thinks the simulation effect is satisfactory, the experiment can be stopped in advance. The system will automatically search for the simulation information of the current round and record it in the three files of best.cptk.index, best.cptk.meta and best.cptk.data, so that the subsequent test mode can call the data set.

[0108] When the training mode is enabled, you can call Tensorboard to view the simulation training results in real time. Compared with reading data through the terminal, it is more convenient to use Tensorboard to view the simulation effect. After the simulation is completed, the user can download the experimental data. The system provides Matplotlib visualization code for comparative research of multiple groups of simulation experiments. The following is a visualization effect diagram, such as Figure 6 , This is for reference only. The specific visualization effect and number of comparison groups can be adjusted according to user needs.

[0109] Test mode

[0110] The test mode is mainly used to verify whether the simulation data obtained in the training mode (Train) meets the predetermined goals, especially whether the "zero collision rate" requirement is achieved. In this mode, when the user runs the program, the system will pop up a simulation image (which can be closed) and display the simulation effect on the terminal. The test mode runs a total of 1000 steps.

[0111] Table 3 Symbols and meanings (II)

[0112]

[0113] During the running process, the system will automatically capture the information of the pop-up simulation image and store it in the result_imgs file by default, which contains the running pictures of each step and the complete simulation video (from step 0 to step 999). Figure 7 As shown in the figure, test.avi is the simulation video, and test is the default name, which can be modified by the user during operation.

[0114] Training Experiment Results and Analysis

[0115] In order to verify the effectiveness of the intelligent networked vehicle non-signaled intersection traffic device and system based on the coupling algorithm, the experiment was carried out from two aspects: vehicle driving conditions and vehicle collision conditions. By comparing the performance in the three modes of "dynamic adjustment", "safety" and "traffic efficiency", the advantages and disadvantages of each model were analyzed respectively.

[0116] Regarding vehicle driving conditions, Figure 8 -a shows that the traffic efficiency mode always maintains a higher average speed, which reflects the characteristics of this mode in the scene of non-signalized intersections that pursue efficient traffic. Although the speed of the dynamic adjustment mode is slightly lower than that of the traffic efficiency mode, it can still maintain a higher speed under complex traffic conditions, showing strong adaptability. The safety mode maintains a lower speed because it gives priority to driving safety. Figure 8 -b, the acceleration change rate reflects the degree of aggressiveness of the driving strategy. Experiments show that the safety mode maintains a relatively high acceleration change rate by adjusting the speed more frequently and slightly to cope with emergencies. The acceleration change in the traffic efficiency mode is relatively stable, but the fluctuation is small, showing its stability in optimizing the traffic speed. The dynamic adjustment mode adopts a balanced strategy in acceleration change, which can maintain comfort while increasing speed.

[0117] In terms of vehicle collision, Fig. 9 -a shows that the collision rate of the dynamic adjustment mode is higher in the initial stage, but with the self-adjustment of the driving strategy, the collision rate drops rapidly and tends to stabilize. The safety mode maintains the lowest collision rate from beginning to end, while the collision rate of the traffic efficiency mode is higher in the initial stage, but it also has a significant downward trend after training. Fig. 9 -b In terms of the cumulative number of vehicle collisions, the dynamic adjustment mode quickly reduced the number of collisions through adaptive adjustment strategies, and there were almost no new collisions in the later stages of training. In contrast, the safety mode always kept the cumulative number of collisions to the lowest due to the more conservative driving strategy. Although the traffic efficiency mode had a high cumulative number of collisions, the growth rate of the cumulative number of collisions gradually slowed down as the training progressed, and the performance eventually stabilized.

[0118] Test results and analysis

[0119] This experiment used three different algorithm models (dynamic adjustment, safety, and traffic efficiency) and conducted 1,000 rounds of simulation tests in a non-signaled intersection scenario. The detailed data of the 111th, 333rd, 555th, 777th, and 999th rounds were extracted for comparative analysis. The analysis results are shown in Table 3. The following is a detailed analysis from the three aspects of collision rate, average speed, and intersection travel time.

[0120] The collision rate is an important indicator for measuring vehicle safety. The collision rate of all three strategies is 0.00%, indicating that the three modes can effectively avoid collisions in all rounds. The dynamic adjustment and safety modes effectively avoid the risk of collisions between vehicles through more conservative speed control strategies. The traffic efficiency mode maintains a zero collision rate despite pursuing higher speeds, which shows that its algorithm has achieved a good balance between speed and safety.

[0121] In terms of average speed, the differences between different modes are quite obvious. "Dynamic Adjustment" increased from 41.997km / h in the 111th round to 44.609km / h in the 999th round, showing that this mode can gradually optimize the speed and improve the traffic efficiency of vehicles while maintaining safety. The continuous increase in speed shows that the algorithm is highly adaptable and can flexibly respond to complex traffic conditions. The average speed of "Safety" is relatively low, always maintained between 22.580km / h and 24.227km / h. This shows that in order to minimize the risk of collision, the system sacrifices part of the speed to ensure that there is a sufficient safe distance between vehicles. The average speed of "Traffic Efficiency" is the highest, increasing from 46.159km / h to 47.868km / h, showing its strong optimization ability for traffic efficiency. Despite the high speed, it can still ensure the safe passage of vehicles, indicating that the algorithm can achieve the optimal balance between speed and safety.

[0122] In terms of intersection travel time, the three modes show obvious differences. The travel time of the "dynamic adjustment" model gradually decreases, from 12.858 seconds in the 111th round to 12.105 seconds in the 999th round. As the number of rounds increases, the system gradually optimizes, and by adjusting the acceleration and speed changes, the waiting time of vehicles at the intersection is reduced, improving the overall traffic efficiency. The "safety" travel time is longer, always maintained between 18 and 19 seconds. Because the safety mode prioritizes the safety of the vehicle, the lower speed and cautious acceleration strategy lead to longer travel time. This mode is suitable for scenarios that require extremely high safety, but it has some impact on traffic efficiency. The "traffic efficiency" travel time is the shortest, from 11.698 seconds in the 111th round to 11.281 seconds in the 999th round. Through precise speed control and acceleration optimization, the traffic efficiency mode can ensure that the vehicle passes through the intersection at the fastest speed, minimizing delays. This shows the significant advantages of the algorithm in improving vehicle traffic efficiency.

[0123] Table 4 Comparison of test results

[0124]

[0125] Through the above analysis, we can conclude that the dynamic adjustment mode can improve traffic efficiency while ensuring safety, and the average speed and travel time are balanced; the safety mode maximizes safety but sacrifices travel time; and the traffic efficiency mode performs best in increasing speed and shortening travel time while still maintaining a zero collision rate.

Claims

1. A method for optimizing the passage of intelligent connected vehicles at non-signaled intersections based on a coupling algorithm, characterized in that: The acceleration of the vehicle is adjusted and controlled based on the coupling of deep reinforcement learning algorithm and microscopic traffic flow model, and the maximum acceleration traffic strategy, minimum acceleration traffic strategy and dynamically adjusted acceleration traffic strategy are constructed accordingly; the corresponding traffic strategy is selected according to the actual scenario to control the passage of intelligent connected vehicles at non-signaled intersections.

2. The method for optimizing the passage of intelligent connected vehicles at non-signaled intersections based on a coupling algorithm according to claim 1 is characterized in that: The control equation of the microscopic traffic flow model is: Where: v n is the vehicle speed; Δv n is the speed difference between the front and rear vehicles; Δx n is the distance between the front and rear vehicles; V[Δx n (t)] = V1 + V2 tanh [C1 (Δx n -l c )-C2], V(·) is the optimized speed function identified by measured data; a is the sensitivity coefficient; k is the speed difference coefficient; V1, V2, C1, C2 are the constant terms of the optimized speed function; l c is the length of the car body; tanh(·) is the hyperbolic tangent function.

3. The method for optimizing the passage of intelligent networked vehicles at non-signaled intersections based on a coupling algorithm according to claim 1, characterized in that: The minimum acceleration passing strategy includes the following steps: S11, initializing actual vehicle driving acceleration a drive , the acceleration a generated by the deep learning algorithm DRL , the acceleration a generated by the microscopic traffic flow model TF ; Minimum acceleration a m , maximum acceleration a M ; S12, initializing the status information of the target vehicle i; S13. For each vehicle i, obtain the status information of the target vehicle i; S14, interact with the environment and calculate the corresponding acceleration a i DRL and S15, determine and select the actual driving acceleration: S16. Standardize the actual driving acceleration: S17: Output the actual driving acceleration of vehicle i And use that acceleration to update the state.

4. The method for optimizing the passage of intelligent connected vehicles at non-signaled intersections based on a coupling algorithm according to claim 1, characterized in that: The maximum acceleration passing strategy includes the following steps: S21, initializing actual vehicle driving acceleration a drive , the acceleration a generated by the deep learning algorithm DRL , the acceleration a generated by the microscopic traffic flow model TF ; Minimum acceleration a m , maximum acceleration a M ; S22, initializing the status information of the target vehicle i; S23. For each vehicle i, obtain the status information of the target vehicle i; S24, interact with the environment and calculate the corresponding acceleration a i DRL and S25, determine and select the actual driving acceleration: S26. Standardize the actual driving acceleration: S27: Output the actual driving acceleration of vehicle i And use that acceleration to update the state.

5. The method for optimizing the passage of intelligent connected vehicles at non-signaled intersections based on a coupling algorithm according to claim 1, characterized in that: The adjustment process of the dynamic acceleration strategy is as follows: the two factors of the front and rear vehicle distance Δx and the speed difference Δv are linearly combined to generate a safety index S to measure the safety level under the current driving conditions, as follows: Where Δx is the distance between the front and rear vehicles; Δv is the speed difference between the front and rear vehicles; X s is the safe distance threshold of the front and rear vehicles; V s is the speed difference safety threshold of the front and rear vehicles; S is the safety index, ranging from 0 to 1; When S≤α1, the system adopts a conservative driving strategy and selects the minimum acceleration passing strategy; when S≥α2, the system adopts an aggressive driving strategy and selects the maximum acceleration passing strategy; when α1<S<α2, the linear combination acceleration passing strategy is selected; α1 is the conservative driving strategy threshold; α2 is the aggressive driving strategy threshold, α1 and α2 are 0.3 and 0.6 respectively.

6. The method for optimizing the passage of intelligent connected vehicles at non-signaled intersections based on a coupling algorithm according to claim 5 is characterized in that: The linear combination acceleration passing strategy comprises the following steps: S31, initializing actual vehicle driving acceleration a drive , the acceleration a generated by the deep learning algorithm DRL , the acceleration a generated by the microscopic traffic flow model TF ; Minimum acceleration a m , maximum acceleration a M ; S32, initializing the status information of the target vehicle i; S33, for each vehicle i, obtaining the state information of the target vehicle i; S34, interact with the environment and calculate the corresponding acceleration a i DRL and S35, calculating the actual driving acceleration according to the rules a drive =(1-β)·a DRL +β·a TF , S36. Standardize the actual driving acceleration: S37: Output the actual driving acceleration of vehicle i And use that acceleration to update the state.

Images