A ramp merging decision determination method and device based on autonomous automatic driving, medium and product

Abstract

This application discloses a method, device, medium, and product for determining ramp merging decisions based on autonomous driving, relating to the field of autonomous driving. The method includes: constructing a ramp merging decision model based on a dual deep Q-network; building a simulation environment for ramp merging; training the ramp merging decision model based on the simulation environment; acquiring different forced lane-change trajectory data; performing driving style clustering analysis based on the forced lane-change trajectory data; obtaining forced lane-change decision models for autonomous vehicles with different merging strategies based on the sample data corresponding to the different driving style clustering analysis results and the trained ramp merging decision model; and obtaining the ramp merging decision result based on the current forced lane-change trajectory data and the forced lane-change decision models for autonomous vehicles with different merging strategies. This application can analyze the impact of strategy differences on road traffic efficiency and safety, helping autonomous vehicles to better integrate into mixed traffic environments.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and in particular to a method, device, medium, and product for determining ramp merging decisions based on autonomous driving. Background Technology

[0002] In recent years, with the rapid development of technologies such as artificial intelligence and communication, more and more autonomous vehicles have been deployed on actual roads, and the penetration rate of autonomous vehicles has been continuously increasing. However, due to the differences in vehicle type, brand, or manufacturer, there are differences in the preset driving strategies among autonomous vehicles. Secondly, although autonomous vehicles are not affected by human driving habits, different autonomous driving systems may simulate or refer to different driving habits, which may vary due to the preferences of system designers or market demands. Therefore, different types of autonomous vehicles may exhibit different behaviors when facing the same traffic conditions. For example, when passing through complex intersections with many pedestrians and vehicles, some vehicles may prioritize safety and adopt a conservative driving strategy to slow down; while others may prioritize efficiency and adopt an aggressive driving strategy to avoid stopping and waiting. In addition, the behavior of autonomous vehicles is also affected by external factors such as traffic density and road conditions. In high-density traffic environments, vehicles need to frequently perform maneuvers such as following other vehicles and changing lanes, which may exacerbate the behavioral differences between different autonomous vehicles. The combined effect of these factors further intensifies the diversity of autonomous vehicle behavior in real-world traffic scenarios.

[0003] Lane changing is a fundamental driving behavior. Studies have shown that, influenced by factors such as driver psychology, experience, and personal preferences, vehicles with different driving styles exhibit significant differences in lane changing behavior. These differences are not only reflected in the timing of lane changes but also in speed control, acceleration changes, and interactions with other vehicles during the process. Different lane-changing behaviors not only affect individual efficiency but also significantly impact traffic flow efficiency and safety. Through vehicle interaction, lane changing can trigger deceleration or acceleration in surrounding vehicles. This local disturbance can propagate and amplify through traffic flow, gradually leading to congestion. Especially under high-density traffic conditions, lane changing can trigger a "butterfly effect," further exacerbating traffic congestion. Secondly, lane changing alters vehicle distribution between lanes, potentially leading to uneven lane utilization and affecting overall road capacity. Furthermore, lane changing is closely related to traffic accident risk; improper lane changing can cause rear-end collisions or side collisions, further worsening traffic flow.

[0004] In mixed traffic environments, behavioral differences among autonomous vehicles are becoming increasingly apparent, raising the following questions: How will the diversity of autonomous driving strategies, coupled with external factors such as road conditions and traffic density, impact the overall efficiency of traffic flow? Furthermore, how can we construct autonomous vehicles with different driving strategies to quantitatively analyze their behavioral characteristics, providing a technological foundation for the above questions? In addition, with the continuous improvement of road infrastructure and significant advancements in communication technology, future roads will exhibit a diverse coexistence, including human-driven vehicles, human-machine collaborative vehicles, and autonomous vehicles, all operating in the same space and generating complex and subtle interactions.

[0005] Traditional lane-changing decision-making model research focuses primarily on improving vehicle performance, including reducing the probability of traffic accidents, without fully considering the impact of drivers' personalized needs on lane-changing decision-making strategies. Furthermore, while research on driving style classification and index selection is relatively in-depth, existing studies rarely integrate driving style into vehicle interaction scenarios and lack in-depth discussion of the impact of differences in autonomous driving strategies on overall traffic efficiency in merging scenarios.

[0006] Therefore, in-depth exploration of the differences between autonomous driving strategies and their potential impact on the efficiency and safety of the overall transportation system is not only helpful in understanding the operating mechanism of autonomous driving technology in mixed traffic environments, but also of key significance for optimizing traffic management strategies and enhancing road traffic efficiency and safety. Summary of the Invention

[0007] The purpose of this application is to provide a method, device, medium, and product for determining ramp merging based on autonomous driving, which can improve road traffic efficiency and safety, and help autonomous vehicles better integrate into mixed traffic environments.

[0008] To achieve the above objectives, this application provides the following solution:

[0009] In a first aspect, this application provides a method for determining ramp merging decisions based on autonomous driving, the method comprising:

[0010] A ramp merging decision model is constructed based on a dual-depth Q-network. The state space of the ramp merging decision model includes: the driving state information of the target vehicle and the relative information of vehicles at a set distance during the ramp merging process. The driving state information includes: lateral absolute position x, longitudinal absolute position y, and speed v. The relative information includes: relative position with the target vehicle and vehicle speed. The action space of the ramp merging decision model includes: lane changing and not changing lanes. The reward function of the ramp merging decision model includes a driving efficiency component with a reward value proportional to speed, a safety component that imposes a penalty in the event of a collision, and a merging component that reflects the urgency of merging. Different weight coefficients for the driving efficiency component, the safety component, and the merging component correspond to different merging strategies.

[0011] A simulation environment for ramp merging is established; and the ramp merging decision model is trained based on the simulation environment.

[0012] Acquire different forced lane change trajectory data; and perform driving style clustering analysis based on the forced lane change trajectory data; the driving style clustering analysis results are used to characterize the speed, acceleration, and driving distance distribution in the auxiliary lane for different types of drivers; the forced lane change trajectory data includes: speed, acceleration, headway, driving distance in the auxiliary lane, and number of free lane changes;

[0013] Based on the sample data corresponding to the clustering analysis results of different driving styles, and based on the trained ramp merging decision model, we obtain the forced lane change decision model for autonomous vehicles with different merging strategies.

[0014] Based on the current forced lane change trajectory data, the forced lane change decision model for autonomous vehicles based on different merging strategies is used to obtain the ramp merging decision results.

[0015] Optionally, the reward function R of the ramp merging decision model specifically includes the following formula:

[0016] R=ω1R efficiency +ω2R safety +ω3R interflow ;

[0017] Among them, R efficiency For the driving efficiency section, v rl v represents the current speed of the target vehicle. des R represents the desired speed of the target vehicle. safety For the safety part, "Collision" indicates that the target vehicle collides with other vehicles during the merging process; "otherwise" indicates other situations. R interflow For the merging section, R interflow =-|yydes |·(a cd -1), where y represents the longitudinal coordinate of the lane where the target vehicle is currently located. des The longitudinal coordinate of the target vehicle's expected merging lane is represented by 'a', which is the base and takes the value of 1.5. 'c' is a coefficient used to adjust the penalty magnitude. 'd' represents the current distance the target vehicle is traveling in the auxiliary lane. 'ω1', 'ω2', and 'ω3' are the weighting coefficients for the driving efficiency component, the safety component, and the merging component, respectively.

[0018] Optionally, the simulation environment for setting up ramp merging specifically includes:

[0019] The TraCI interface was called using Python software and combined with the simulation software SUMO to build a simulation environment for ramp merging. The state information obtained from the simulation environment includes: the information of the target vehicle at a certain moment; the target vehicle information includes: the target vehicle's lateral coordinates, longitudinal coordinates, speed in the merging area road network, and the lateral and longitudinal positions and absolute speeds of the target vehicle relative to the target vehicle in the six surrounding positions; the six surrounding positions are the vehicles in front and behind the target vehicle in its current lane, the vehicles in front and behind the target vehicle in the adjacent left lane, and the vehicles in front and behind the target vehicle in its right lane.

[0020] Optionally, different forced lane change trajectory data can be obtained, specifically including:

[0021] Obtain the NGSIM dataset;

[0022] Data cleaning of the NGSIM dataset;

[0023] Based on the cleaned NGSIM dataset, different forced lane change trajectory data were obtained.

[0024] Optionally, different forced lane change trajectory data are acquired, and the process further includes:

[0025] The forced lane change trajectory data are subjected to Z-Score standardization and dimensionality reduction processing in sequence.

[0026] Optionally, driving style clustering analysis is performed based on forced lane change trajectory data, specifically including:

[0027] Based on the forced lane change trajectory data, SPSS software was used to perform driving style clustering analysis.

[0028] Secondly, this application provides a ramp merging decision-making device based on autonomous driving, the ramp merging decision-making device based on autonomous driving comprising:

[0029] A decision model construction module is used to construct a ramp merging decision model based on a dual-depth Q-network. The state space of the ramp merging decision model includes: the driving state information of the target vehicle and the relative information of vehicles at a set distance during the ramp merging process. The driving state information includes: lateral absolute position x, longitudinal absolute position y, and speed v. The relative information includes: relative position with respect to the target vehicle and vehicle speed. The action space of the ramp merging decision model includes: lane changing and not changing lanes. The reward function of the ramp merging decision model includes a driving efficiency component with a reward value proportional to speed, a safety component that imposes a penalty in the event of a collision, and a merging component that reflects the urgency of merging. Different weight coefficients for the driving efficiency component, the safety component, and the merging component correspond to different merging strategies.

[0030] The model training module is used to build a simulation environment for ramp merging; and to train the ramp merging decision model based on the simulation environment.

[0031] The driving style clustering module is used to acquire different forced lane change trajectory data; and to perform driving style clustering analysis based on the forced lane change trajectory data; the driving style clustering analysis results are used to characterize the speed, acceleration, and driving distance distribution in the auxiliary lane for different types of drivers; the forced lane change trajectory data includes: speed, acceleration, headway, driving distance in the auxiliary lane, and number of free lane changes;

[0032] The module for determining the forced lane change decision model for autonomous vehicles with different merging strategies is used to obtain the forced lane change decision model for autonomous vehicles with different merging strategies based on the sample data corresponding to the clustering analysis results of different driving styles and the trained ramp merging decision model.

[0033] The ramp merging decision module is used to obtain the ramp merging decision result based on the current forced lane change trajectory data and the forced lane change decision model of autonomous vehicles with different merging strategies.

[0034] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned method for determining ramp merging decisions based on autonomous driving.

[0035] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for determining ramp merging decisions based on autonomous driving.

[0036] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for determining ramp merging decisions based on autonomous driving.

[0037] According to the specific embodiments provided in this application, this application has the following technical effects:

[0038] This application provides a method, device, medium, and product for determining ramp merging decisions based on autonomous driving. It constructs a ramp merging decision model based on a Double Deep Q-Network (DDQN). The state space consists of the target vehicle's driving state information and the relative information of vehicles at a set distance during ramp merging. The action space consists of lane changing and not changing lanes. The reward function is a combination of driving efficiency (a reward proportional to speed), safety (penalty in case of collision), and rewards for the merging portion reflecting the urgency of merging. Furthermore, it performs driving style clustering analysis based on forced lane-changing trajectory data to obtain the speed, acceleration, and driving distance distribution in auxiliary lanes for different types of drivers. Based on the ramp merging decision model, it constructs forced lane-changing decision models for autonomous vehicles with different merging strategies. This fully considers the impact of personalized driver needs on lane-changing decision strategies and integrates driving style into the vehicle interaction scenario, taking into account the impact of differences in autonomous driving strategies on overall traffic efficiency during merging. This further improves road traffic efficiency and safety, helping autonomous vehicles better integrate into mixed traffic environments. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic flowchart of a ramp merging decision determination method based on autonomous driving in one embodiment of this application;

[0041] Figure 2 This is a schematic diagram of the state-space representation;

[0042] Figure 3 A diagram illustrating the relationship between the coefficient for adjusting the penalty level and the degree of merging;

[0043] Figure 4 A schematic diagram of the training process for the ramp merging decision model;

[0044] Figure 5A schematic diagram illustrating the behavioral characteristics of different drivers during the merging process;

[0045] Figure 6 A schematic diagram of the road structure for the merging zone of a highway.

[0046] Figure 7 A schematic diagram illustrating the change in the reward function of the ramp merging decision model;

[0047] Figure 8 A schematic diagram of vehicle behavior characteristics;

[0048] Figure 9 A schematic diagram illustrating the characteristics of vehicles with different driving styles during the merging process;

[0049] Figure 10 This diagram illustrates the travel distance of the auxiliary lane under different strategies.

[0050] Figure 11 The spacetime velocity diagram at 2000 veh / h;

[0051] Figure 12 The spacetime velocity diagram is shown at 3000 veh / h.

[0052] Figure 13 The spacetime velocity diagram is shown at 4000 veh / h.

[0053] Figure 14 This is a schematic diagram illustrating the principle of a ramp merging decision-making method based on autonomous driving.

[0054] Figure 15 This diagram illustrates the distribution of characteristic parameters of aggressive, normal, and conservative drivers merging onto ramps. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] In one exemplary embodiment, such as Figure 1 As shown, a method for determining ramp merging based on autonomous driving is provided, which includes the following steps S101 to S105. Wherein:

[0058] S101, a ramp merging decision model is constructed based on a dual-depth Q-network;

[0059] like Figure 2 As shown, the state space of the ramp merging decision model includes: the driving state information of the target vehicle and the relative information of vehicles at a set distance during the ramp merging process; the driving state information includes: lateral absolute position x, longitudinal absolute position y and speed v; the relative information includes: relative position (Δx, Δy) with the target vehicle and vehicle speed.

[0060] The action space of the ramp merging decision model includes: lane changing and not changing lanes; where not changing lanes includes maintaining the current speed, accelerating at a speed of 2m / s, and decelerating at a speed of -2m / s.

[0061] To effectively guide the agent to learn efficient and safe lane-changing strategies, the reward function of the ramp merging decision model includes a driving efficiency component with a reward value proportional to speed, a safety component that imposes a penalty in the event of a collision, and a merging component that reflects the urgency of merging. Different weighting coefficients for the driving efficiency component, the safety component, and the merging component correspond to different merging strategies, in order to more accurately simulate aggressive, normal, and conservative driving behaviors, thereby providing more accurate and efficient decision support for lane-changing behavior when vehicles merge onto ramps.

[0062] The driving efficiency component aims to incentivize autonomous vehicles to maximize their speed within a safe range. This is achieved by setting a reward value proportional to speed, encouraging efficient driving. Its reward function R... efficiency for:

[0063]

[0064] v rl v represents the current speed of the target vehicle. des Indicates the desired speed of the target vehicle;

[0065] To ensure the agent's safety during driving, the safety component imposes penalties upon collision to guide the agent away from repeating lane-changing maneuvers in similar situations; its reward function R... safety for:

[0066]

[0067] Here, "collision" indicates that the target vehicle collides with other vehicles during the merging process, and "otherwise" indicates other situations.

[0068] To ensure vehicles merge into the main lane at the appropriate time, the reward function is multiplied by an exponential coefficient, allowing the reward to be adjusted in real-time based on the urgency of the merging action of the agent (target vehicle). When the target vehicle is far from the end of the merging zone, the urgency to merge into the main lane is low, resulting in a smaller penalty. Conversely, as the target vehicle approaches the end of the merging zone, the urgency to merge into the main lane increases, leading to a larger penalty. By dynamically adjusting the reward function according to the target vehicle's position, the agent is guided to effectively explore and utilize driving space during the merging process, completing the merging operation in a timely manner. Furthermore, this dynamic adjustment mechanism allows target vehicles to optimize their path selection while ensuring safety, reducing unnecessary deceleration or waiting time, thereby improving the overall traffic flow efficiency. The reward function R... interflow for:

[0069] R interflow =-|yy des |·(a cd -1);

[0070] Where y represents the longitudinal coordinate of the lane where the target vehicle is currently located. des This represents the longitudinal coordinate of the target vehicle's desired merging lane, where 'a' is the base, set to 1.5, and 'c' is a coefficient used to adjust the penalty. As the travel distance increases, the urgency of merging by the agent also increases, and the penalty for not merging is also greater. Figure 3 As shown; d represents the current distance the target vehicle has traveled in the auxiliary lane;

[0071] Therefore, the reward function R of the ramp merging decision model is:

[0072] R=ω1R efficiency +ω2R safety +ω3R interflow ;

[0073] Here, ω1, ω2, and ω3 are the weighting coefficients for driving efficiency, safety, and merging, respectively, used to balance driving safety and traffic efficiency. When ω1 has a greater weight than ω2 and ω3, the speed reward will guide the vehicle to adopt a more aggressive strategy to achieve higher speeds. When ω2 is adjusted to a higher weight, the vehicle will prioritize driving safety, tending to stay in the current lane and reduce lane-changing conflicts with surrounding vehicles.

[0074] S102, Build a simulation environment for ramp merging; and train the ramp merging decision model based on the simulation environment;

[0075] Setting up a simulation environment for ramp merging specifically includes:

[0076] Using Python software to call the TraCI interface, information about the target vehicle (autonomous vehicle or intelligent agent) at time t in the SUMO (Simulation of Urban Mobility) simulation environment is obtained. This includes the target vehicle's lateral and longitudinal coordinates and speed in the merging area road network, as well as information about vehicles in six surrounding positions, namely, the lateral and longitudinal positions and absolute speeds of the surrounding vehicles relative to the target vehicle. The six vehicles refer to the vehicles in front of and behind the target vehicle in its current lane, and the vehicles in front of and behind the adjacent left and right lanes. If there are no vehicles at the corresponding positions, virtual vehicles and their corresponding state information are set.

[0077] The SUMO simulation environment's TraCI interface connects to Python software for online interaction between real-time road network traffic information and the dual deep Q-network algorithm. To simulate forced lane changes by vehicles in highway merging zones, simulation experiments are conducted using the SUMO simulation environment, including building the road network environment, generating vehicle requirements, and setting training hyperparameters. The road structure and inter-road connections are designed using road network files within the SUMO simulation environment. The final highway merging zone road structure is shown below. Figure 6 As shown, the main road is approximately 600m long, containing two lanes (Lane 0 and Lane 1), and includes an entrance ramp and a 200m-long acceleration lane. The speed limit for each road is set at 30m / s to simulate real highway driving conditions.

[0078] The merging scenario mainly involves two types of vehicles: Human Driving Vehicles (HDVs) are generated simultaneously from both the main road and the entrance ramp, while Autonomous Vehicles (AVs) enter the traffic system only from the entrance ramp. Vehicles on the entrance ramp must merge into the main road before reaching the end of the merging zone to complete the merging task. The car-following behavior of both HDVs and AVs is controlled by SUMO's default Intelligent Driver Model (IDM). The lane-changing behavior of HDVs is controlled by SUMO's default LC2013 model, while the lane-changing behavior of CAVs is controlled by a DDQN-based decision model. The vehicle parameter configurations in the simulation environment are shown in Table 1.

[0079] Table 1

[0080] Parameter Description Value accel acceleration 3.0 m / s2 decel deceleration -4.5 m / s2 length vehicle length 5.0m maxSpeed maximum physical speed 20.0-30.0 m / s carFollowModel car following model IDM sigma degree of imperfect driving 0.5 tau expected minimum time headway 1.0s

[0081] like Figure 4 As shown, the specific training process is as follows:

[0082] The system acquires the relevant state information *s* of the current target vehicle and surrounding vehicles from the SUMO simulation environment of the merging traffic area road network and inputs it into the Q-network. The Q-network calculates the Q-value for each possible action based on the input features, and then selects the optimal action *a* from the action space according to an ε-greedy strategy. If the vehicle receives a command to maintain its current lane or accelerate / decelerate, it uses the car-following model IDM from SUMO to follow the vehicle. If the vehicle receives a lane-changing command, it uses the lane-changing model LC2013 from the SUMO simulation environment to perform the lane-changing operation. After the target vehicle executes its action, it receives a reward value calculated by the set reward function. Simultaneously, the system collects the current vehicle state, the next vehicle state, the executed action, and the corresponding reward value (s, a, r, s'), and stores them in the experience pool.

[0083] During training, a large number of quadruplets are stored in the experience pool. Small batches of data are drawn from the experience pool, and the parameters w of the Q-network are updated using stochastic gradient descent to minimize the loss function. Furthermore, at regular time steps, the parameters w of the Q-network are copied to the parameters w' of the target network to maintain the stability of the target network. Finally, through continuous iterative updates, the Q-network obtains the optimal lane-changing decision strategy.

[0084] After training convergence, the neural network parameters of the ramp merging decision model are saved. Then, when testing the ramp merging decision model, the saved network parameters are loaded to guide the model in making action decisions during merging scenarios. The ramp merging decision model inputs real-time information about surrounding vehicles and the vehicle's own state information (s) into the neural network, and the neural network outputs a decision on whether to change lanes. If the action 'a' corresponding to the maximum Q-value in the network output is a lane change, the agent performs the lane change operation under the default LC2013 model control in the SUMO simulation environment; if it is a car-following or acceleration / deceleration operation, the operation is performed under the support of the IDM model in the SUMO simulation environment. Through continuous interaction between the ramp merging decision model and the SUMO simulation environment, acquiring state information and making action decisions, after running a certain number of simulation steps, the target vehicle gradually merges from the ramp into the highway main road, completing the merging task.

[0085] like Figure 7 The curve showing the reward function value changing over time is used to observe the model's training progress. The horizontal axis represents the number of training epochs, and the vertical axis represents the cumulative reward value per epoch. For example... Figure 7As shown, in the initial training phase, the agent's cumulative reward per round is small. Because the agent is constantly exploring the environment, it has limited effective experience to draw upon, and its decisions are easily penalized. As the number of training rounds increases, the probability of randomly selecting actions decreases, the model network parameters are continuously updated, and the cumulative reward per round shows an upward trend, stabilizing after approximately 100,000 rounds, with the reward value gradually converging. This indicates that the agent has learned a better lane-changing decision-making strategy, achieving an effective balance between exploration and utilization.

[0086] As a specific implementation, in the simulation experiment, each round of simulation runs for 150 time steps, with a time step size of 0.2 seconds. At the beginning of each training round, a Python program randomly generates routing files, constructs environments with various vehicle densities, and randomly generates several vehicles at a certain speed from the starting point of the road segment to simulate real traffic flow. This helps to simulate diverse traffic scenarios, providing sufficient training data for the agent, while avoiding overfitting problems caused by a single scenario. Furthermore, to ensure the stability of the simulation environment, the target vehicle is released into the environment only after the simulation has run for a preset number of warm-up steps. This helps the agent to learn fully under different vehicle densities and traffic conditions. After the target vehicle departs from the ramp at a random initial speed, it gradually learns and obtains the optimal free lane-changing strategy through real-time interaction with surrounding vehicles.

[0087] This embodiment employs an experience replay method for offline training. Before formal training begins, lane-changing decisions are made based on the initialized network parameters, and data is collected to fill the experience pool. When the amount of data in the experience pool meets the minimum batch size for training, the network parameters are updated. The hyperparameters used in the forced lane-changing decision model are shown in Table 2.

[0088] Table 2

[0089]

[0090] To address the problem of mandatory lane-changing decisions for autonomous vehicles in highway merging zones, this application constructs a decision model based on the DDQN algorithm to guide vehicles in lane-changing. Two modules, the algorithm network and the simulation environment, are used for lane-changing decision generation and traffic condition simulation, respectively. The algorithm network consists of four fully connected layers: Dense(21) + Dense(64) + Dense(64) + Dense(4), which generate lane-changing decisions by extracting features from state information. The use of the target network helps the model maintain stability during the decision-making process, thereby improving decision performance. Furthermore, by periodically extracting small batches of samples from the experience replay unit for training, the model can effectively utilize historical experience data, further enhancing learning efficiency and robustness.

[0091] As shown in Table 3, by adjusting the weight system of each reward function in the ramp merging decision model, an autonomous vehicle with diverse driving strategies is generated through simulation to realize the simulation of ramp merging decision.

[0092] Table 3

[0093] driving style type [CDATA[ω1]] [CDATA[ω2]] c aggressive type 3.0 100.0 1 / 50 normal type 2.0 105.0 1 / 65 conservative type 1.0 110.0 1 / 90

[0094] Vehicle speed and acceleration were selected to evaluate the training effect of the lane-changing decision model. To verify the differences in the merging and lane-changing process of vehicles with different driving styles, the same scenario was selected for testing, and the experimental results were analyzed. Figure 8 As shown.

[0095] like Figure 9 As shown, analysis of the agent's driving characteristics during the merging process reveals that the agent initially starts from the entrance ramp, accelerates for a period, and then enters the auxiliary lane. At this point, a vehicle in the adjacent lane on the left maintains a small distance from the agent. To prevent a collision and complete the merging, the agent actively accelerates or decelerates to increase the distance between the vehicles. When the conditions for a safe lane change are met, the agent initiates the lane change operation, successfully merging into the straight lane. It then slowly accelerates to track the speed of the vehicle in front, eventually exiting the merging area and successfully completing the merging task. Aggressive vehicles tend to accelerate when merging into the main line to complete the merging task as quickly as possible, and their acceleration fluctuations are also stronger. In contrast, normal and conservative vehicles prioritize safety, both decelerating and waiting to meet a safe distance before merging into the main line. In conclusion, although vehicles with different driving styles exhibit varying performance during the merging process, they can all dynamically adjust their behavior according to the environment, make reasonable lane-changing decisions, and successfully complete the merging task.

[0096] S103, acquire different forced lane change trajectory data; and perform driving style clustering analysis based on the forced lane change trajectory data; the driving style clustering analysis results are used to characterize the speed, acceleration, and driving distance distribution in the auxiliary lane for different types of drivers; the forced lane change trajectory data includes: speed, acceleration, headway, driving distance in the auxiliary lane, and number of free lane changes;

[0097] Forced lane change trajectory data was extracted based on the NGSIM (Next Generation Simulation) dataset, and physical quantities such as vehicle speed, acceleration, and auxiliary lane travel distance were statistically analyzed to assess the behavioral characteristics of different drivers during the merging process; for example... Figure 5As shown, vehicles with different driving styles exhibit significant differences in the distance traveled in the auxiliary lane. Aggressive driving styles tend to travel a shorter distance in the auxiliary lane before quickly merging into the merging area, while normal and conservative driving styles have a wider range of merging positions. In particular, conservative driving styles tend to travel a longer distance in the auxiliary lane, carefully choosing the merging timing based on real-time traffic conditions, reflecting a high level of emphasis on safety.

[0098] The NGSIM dataset is a high-resolution vehicle trajectory dataset collected and released by the Federal Highway Administration (FLHA) in the United States. It covers vehicle traffic conditions on multiple road segments, including highways (I-80 and US-101) and urban roads. Due to its rich information and high-quality data, the NGSIM dataset has been widely used in fields such as autonomous driving, traffic flow analysis, and urban planning. The sampling interval of this dataset is 100ms, as shown in Table 4.

[0099] Table 4

[0100] Field name Field explanation Unit Vehicle_ID vehicle number — Frame_ID frame number of the vehicle at the current time 0.1s Global_Time global time ms Local_X lateral coordinate of the vehicle in the coordinate system of the collection area ft Local_Y longitudinal coordinate of the vehicle in the coordinate system of the collection area ft Vehicle_Vel instantaneous speed of the vehicle ft / s Vehicle_Acc instantaneous acceleration of the vehicle ft / s 2 ]]> Lane_ID lane number where the vehicle is currently located — Space_Headway vehicle headway ft Time_Headway time headway s Vehicle_Class vehicle type —

[0101] The NGSIM dataset contains some measurement errors, causing displacement, velocity, and acceleration parameters to be affected by noise to varying degrees. If the raw data is used directly for driving style analysis, the errors in the experimental results will gradually accumulate, affecting the accuracy and reliability of the decision-making model. Therefore, data preprocessing to remove outliers is necessary before conducting driving style analysis to ensure data quality.

[0102] Smoothing is a common technique for removing random errors from data. The Savitzky-Golay filter (SG filter) is particularly effective at removing noise while preserving the signal shape and width, thus smoothing the data. Therefore, the SG filter is chosen to remove error information from the trajectory data. To preserve key data features as much as possible while smoothing and denoising, this paper sets the window size and polynomial fitting order of the SG filter to 21 and 3, respectively, to smooth displacement and other information in the trajectory.

[0103] Forced lane changes refer to lane-changing behaviors that drivers must take to meet specific driving needs, such as obstacle avoidance, intersection turning, merging, and diverging. Due to limitations in the detection area of ​​the I-80 road section, some forced lane-changing trajectories may be incomplete. Therefore, the trajectories of merging vehicles entering the main road via lane 7 (entrance ramp) in the US-101 road section were selected for driving style analysis. Simultaneously, to eliminate the influence of outlier data on the results, two types of abnormal trajectories were removed: one type was trajectories that returned to the original lane after changing lanes; the other type was incomplete trajectories caused by detection area limitations. Through the above screening, a total of 247 trajectory data were finally obtained.

[0104] Finally, for the forced lane change trajectory data, statistical indicators such as speed, acceleration, headway, distance traveled in the auxiliary lane, and number of free lane changes were used as feature parameters to comprehensively characterize driving style. Specific feature parameter information is shown in Table 5.

[0105] Table 5

[0106]

[0107] To address the issue of significant differences in magnitude among various driving style characteristic parameters, this paper applies Z-Score standardization to each parameter. Z-Score standardization is a widely used data standardization method. Its core idea is to transform the original data into a form conforming to a standard normal distribution, thereby eliminating dimensional differences between different features and making the data comparable. The formula is as follows:

[0108]

[0109] In the formula, μ represents the mean and σ represents the standard deviation.

[0110] To reduce information redundancy among the feature parameters, factor analysis was used for dimensionality reduction. The basic principle of factor analysis is to introduce a few unobservable variables (i.e., common factors) to explain the correlation between the original variables, thereby achieving dimensionality reduction. SPSS software was used to perform factor analysis on the feature parameters. To assess whether the data was suitable for factor analysis, the KMO test and Bartlett's test of sphericity were first performed on the standardized feature parameters. A KMO statistic greater than 0.5 and a significance level less than 0.05 indicated a significant correlation between the variables, making factor analysis suitable.

[0111] Taking the factor analysis of the characteristic parameters of forced lane-changing driving style as an example, Table 6 shows that the KMO statistic is 0.602 (>0.5) and the Bartlett test significance is 0.000 (<0.05), indicating that there is a significant correlation between the characteristic parameters, and factor analysis can be performed.

[0112] Table 6

[0113]

[0114] The factor analysis method was set to principal component analysis, and a common factor variance table was generated, as shown in Table 7. The larger the extracted value of each variable in Table 7, the better the common factor explains the variable, and the less information loss. As shown in Table 7, the extracted values ​​of most variables are above 0.7, indicating that the effective information in these variables can be well expressed by the common factor.

[0115] Table 7

[0116]

[0117]

[0118] Total variance represents the contribution of each factor to the explanation of a variable, primarily used to determine the number of common factors. Table 8 shows that the eigenvalues ​​of the first four factors are all greater than 1, and their cumulative variance contribution rate reaches 74.702%, indicating that these four factors can adequately explain the characteristic parameters. Therefore, these four common factors are selected for subsequent analysis of forced lane-changing driving styles.

[0119] Table 8

[0120]

[0121] The rotated component matrix is ​​shown in Table 9. As can be seen from the table, common factor F1 has a large loading on the statistics of lateral velocity, as well as the number and time of lane changes; therefore, F1 is named the lateral factor, which mainly reflects the behavioral characteristics of the vehicle in lateral movement. Common factor F2 is mainly related to the statistics of longitudinal velocity; therefore, it is named the longitudinal factor, which reflects the dynamic characteristics of the vehicle in longitudinal movement. Common factor F3 is more closely related to the mean and standard deviation of the headway; therefore, it is named the car-following factor, which characterizes the behavioral pattern of the vehicle during car-following. Common factor F4 has a large loading coefficient on the standard deviation of velocity and acceleration, as well as the distance traveled in the auxiliary lane. Since the standard deviation reflects the fluctuation of velocity and acceleration, it is named the fluctuation factor, which reveals the stability characteristics of the vehicle's motion state.

[0122] Table 9

[0123] Variable F1 F2 F3 F4 Vy_mean 0.041 0.914 0.220 0.162 Vy_std 0.075 -0.651 0.035 0.632 Vx_mean 0.882 0.345 0.034 0.098 Vx_std 0.687 0.520 -0.001 0.070 Acc_mean 0.063 0.359 0.387 0.110 Acc_std 0.175 0.014 -0.052 0.840 Sh_mean -0.199 0.322 0.788 -0.081 Sh_std 0.075 -0.096 0.888 -0.062 Dist -0.266 0.221 -0.022 0.600 LC_T 0.940 -0.138 -0.063 -0.053 LC_t 0.888 -0.163 -0.006 -0.056

[0124] Based on Table 9, construct the factor score coefficient matrix as shown in Table 10;

[0125] Table 10

[0126]

[0127] Specifically, SPSS software was used for cluster analysis to classify driving styles into three types, with a cluster center count of 3, and driving style analysis was performed on lane-changing trajectories.

[0128] Table 11 shows that in Category 1, the common factor F1 has the largest value, while the common factors F2 and F3 have smaller values, indicating that drivers in this category exhibit the most free lane-changing behavior during merging, maintain a small headway with the vehicle in front, and have the lowest longitudinal speed. In Categories 2 and 3, the fluctuation factor F4 is larger, indicating that both categories exhibit more frequent acceleration and deceleration behavior. In Category 3, the car-following factor F3 is the largest, reflecting that drivers in Category 3 prioritize safety during merging and maintain a larger following distance. Therefore, based on the merging behavior of the three types of drivers, they are respectively named as aggressive, normal, and conservative.

[0129] Table 11

[0130] Category F1 F2 F3 F4 Number 1 1.349 -0.239 -0.201 -0.041 65 2 -0.535 0.107 -0.304 -0.039 154 3 -0.233 -0.029 2.148 0.315 28

[0131] like Figure 15 As shown, this paper illustrates the distribution of aggressive, normal, and conservative drivers' merging behavior parameters on ramps. Forced lane change trajectory data was extracted from the NGSIM dataset, and physical quantities such as vehicle speed, acceleration, and distance traveled in the auxiliary lane were statistically analyzed to examine the behavioral characteristics of different drivers during the merging process. The analysis revealed significant differences in the distance traveled in the auxiliary lane among drivers with different driving styles. Aggressive drivers tend to merge quickly into the merging area after traveling a short distance in the auxiliary lane, while normal and conservative drivers have a wider range of merging positions. Conservative drivers, in particular, tend to travel a longer distance in the auxiliary lane, carefully choosing their merging timing based on real-time traffic conditions, reflecting their high regard for safety.

[0132] S104. Based on the sample data corresponding to the clustering analysis results of different driving styles, and based on the trained ramp merging decision model, a forced lane-changing decision model for autonomous vehicles with different merging strategies is obtained; the differences between different driving strategies can be reflected by different weight coefficients.

[0133] During merging, drivers with different driving styles exhibit significant differences in the distance traveled in the auxiliary lane. By continuously loading several agents into the road network from the ramp, the distances traveled in the auxiliary lane by vehicles with different lane-changing strategies were tested and statistically analyzed. This analysis aimed to examine the differences in forced lane-changing decisions among the models. Specific details are as follows: Figure 10As shown, with the increase of the simulation time step, vehicles with different lane-changing styles exhibit varying distances traveled in the auxiliary lane. Aggressive vehicles have the shortest average distance traveled in the auxiliary lane, followed by normal vehicles, while conservative vehicles have the longest average distance traveled, consistent with the previous analysis. Because aggressive vehicles tend to merge into the merging zone within a short distance, their distance traveled in the auxiliary lane is shorter than other types, and they also have a greater urgency to complete the merging task.

[0134] By statistically analyzing the spatiotemporal distribution of vehicles in the merging zone under different forced lane-changing strategies, the traffic efficiency of vehicles in mixed traffic flow was investigated. To more accurately reflect the impact of ramp traffic flow on main road traffic, the vehicle trajectories of the straight lanes closer to the entrance ramps and auxiliary lanes were selected as the research object. Compared to another straight lane on the highway, this lane is more significantly affected by ramp traffic flow and can better reflect the dynamic characteristics of traffic flow during the merging process. Based on this, spatiotemporal velocity maps under different merging strategies were plotted as follows: Figure 11 to Figure 13 As shown, and taking three traffic flow levels as examples, congestion analysis is conducted. It is evident that as traffic flow increases, congestion in the merging area gradually intensifies, and the impact of different lane-changing strategies on traffic efficiency becomes increasingly apparent. Overall, compared to the other two strategies, vehicles using the aggressive strategy are more prone to traffic congestion during merging, with a slightly lower speed distribution. When the traffic flow reaches 2000 veh / h, all three driving strategies can complete the merging task well without causing congestion. When the traffic flow reaches 3000 veh / h, localized congestion begins to appear at the end of the merging area, leading to a significant decrease in the speed of nearby vehicles. Among them, the conservative traffic flow maintains relatively good traffic conditions, while the other two strategies begin to experience small-scale traffic congestion. When the traffic flow further increases to 4000 veh / h, the congestion area expands, vehicle speeds further decrease, and a stop-and-go traffic wave phenomenon appears. Furthermore, analysis of the congestion onset time for traffic flows with different driving strategies revealed that the aggressive traffic flow experienced congestion first. As time progressed, the congestion gradually worsened and spread upstream, covering the largest area compared to the other two styles. The conservative traffic flow began experiencing congestion at 70 seconds, with vehicle speeds decreasing within the merging area and congestion gradually spreading upstream. Due to a series of moving traffic jams, the merging area experienced a stop-and-go traffic pattern. The normal traffic flow experienced congestion last. This may be because these vehicles, upon merging from the entrance ramp onto the main road, thoroughly assessed the surrounding environment and made timely lane-changing decisions, thus alleviating congestion to some extent in the early stages and preventing the congestion trend from spreading to a larger area. However, as the number of vehicles gradually increased, by 100 seconds, the congestion in the merging area gradually worsened and continued to spread upstream.

[0135] Because aggressive strategies assign higher weights to efficiency and merging penalty sub-rewards in the reward function, aggressive vehicles exhibit higher speed demands and a stronger urgency to merge into the main road during merging. These vehicles tend to accelerate in the merging zone and quickly merge into the main road to maximize their own gains. However, this behavior can disrupt the normal driving of other vehicles, forcing vehicles on the main road to slow down and give way, thus causing fluctuations in traffic flow and localized congestion. Therefore, although aggressive strategies improve efficiency at the individual level, their impact on overall traffic flow requires further optimization and balancing.

[0136] Tables 12 and 13 show a comparison of vehicle travel time and traffic volume under three driving strategies. Travel time refers to the time required for a vehicle to travel from the starting point to the destination, which is an important indicator for measuring operational efficiency.

[0137] Table 12 Travel Time (s)

[0138] Traffic flow (veh / h) aggressive type normal type conservative type 2000 33.095 32.704 33.400 3000 37.117 36.527 35.039 4000 68.869 63.276 67.802

[0139] Table 13 Passage Count (veh)

[0140] Traffic flow (veh / h) aggressive type normal type conservative type 2000 46 46 46 3000 82 86 86 4000 81 92 79

[0141] As shown in Tables 12 and 13, the normal driving strategy has the shortest travel time overall, followed by the conservative strategy, and the aggressive strategy has the longest travel time. The normal strategy has a slightly higher number of lanes than the other two types, indicating that the efficiency of the three driving strategies on ramps decreases in that order. In contrast, normal and conservative vehicles have higher weights on the safety sub-reward in the reward function, and are allowed to travel a certain distance in the auxiliary lane before merging into the main lane, resulting in lower merging penalties. This allows vehicles to choose appropriate gaps for merging, and lane-changing behavior is smoother.

[0142] S105. Based on the current forced lane change trajectory data, the forced lane change decision model for autonomous vehicles with different merging strategies is used to obtain the ramp merging decision result.

[0143] like Figure 14 As shown, this application constructs a merging lane-changing decision model based on DDQN, and then sets the state space, action space, and reward function of the merging lane-changing decision model. The reward function includes three parts: driving efficiency, safety, and merging. Then, based on relevant research, the weight coefficients of the merging lane-changing decision model are set to construct diverse decision strategies. A simulation environment is built, the road network structure and traffic demand are set, and simulation training is performed according to the given model training hyperparameters. Finally, the trained models with different driving strategies are simulated and tested, and the driving characteristic parameters of the models are statistically analyzed. Furthermore, based on this model, the congestion and traffic efficiency of road segments are tested under different vehicle density conditions.

[0144] This application analyzes the differences in various behavioral indicators of vehicles with different driving styles in ramp merging zone scenarios, and extracts the feature indicators that best reflect the driver's personalized style for constructing a forced lane-changing decision model for autonomous vehicles with different merging strategies. The forced lane-changing decision models for autonomous vehicles with different merging strategies show differences in the travel distance in the auxiliary lane. Secondly, it examines the impact of lane-changing strategy differences on road traffic efficiency under different traffic density conditions and scenarios. During merging, the normal driving strategy performs better than the conservative and aggressive strategies, and is less likely to cause traffic congestion under the same conditions. The aggressive strategy, driven by the need for high-speed driving and short-distance merging, affects the traffic stability of the road segment, resulting in more severe congestion. This application can provide a reference for traffic flow optimization and safety management, and also helps autonomous vehicles better integrate into mixed traffic environments.

[0145] Based on the same inventive concept, this application also provides an autonomous driving-based ramp merging decision-making device for implementing the aforementioned autonomous driving-based ramp merging decision-making method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the autonomous driving-based ramp merging decision-making device provided below can be found in the limitations of the autonomous driving-based ramp merging decision-making method described above, and will not be repeated here.

[0146] In one exemplary embodiment, a ramp merging decision-making device based on autonomous driving is provided, comprising:

[0147] A decision model construction module is used to construct a ramp merging decision model based on a dual-depth Q-network. The state space of the ramp merging decision model includes: the driving state information of the target vehicle and the relative information of vehicles at a set distance during the ramp merging process. The driving state information includes: lateral absolute position x, longitudinal absolute position y, and speed v. The relative information includes: relative position with respect to the target vehicle and vehicle speed. The action space of the ramp merging decision model includes: lane changing and not changing lanes. The reward function of the ramp merging decision model includes a driving efficiency component with a reward value proportional to speed, a safety component that imposes a penalty in the event of a collision, and a merging component that reflects the urgency of merging. Different weight coefficients for the driving efficiency component, the safety component, and the merging component correspond to different merging strategies.

[0148] The model training module is used to build a simulation environment for ramp merging; and to train the ramp merging decision model based on the simulation environment.

[0149] The driving style clustering module is used to acquire different forced lane change trajectory data; and to perform driving style clustering analysis based on the forced lane change trajectory data; the driving style clustering analysis results are used to characterize the speed, acceleration, and driving distance distribution in the auxiliary lane for different types of drivers; the forced lane change trajectory data includes: speed, acceleration, headway, driving distance in the auxiliary lane, and number of free lane changes;

[0150] The module for determining the forced lane change decision model for autonomous vehicles with different merging strategies is used to obtain the forced lane change decision model for autonomous vehicles with different merging strategies based on the sample data corresponding to the clustering analysis results of different driving styles and the trained ramp merging decision model.

[0151] The ramp merging decision module is used to obtain the ramp merging decision result based on the current forced lane change trajectory data and the forced lane change decision model of autonomous vehicles with different merging strategies.

[0152] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for determining ramp merging decisions based on autonomous driving.

[0153] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0154] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0155] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0158] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0159] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for determining ramp merging based on autonomous driving, characterized in that, The method for determining ramp merging based on autonomous driving includes: A ramp merging decision model is constructed based on a dual-depth Q-network. The state space of the ramp merging decision model includes: the driving state information of the target vehicle and the relative information of vehicles at a set distance during the ramp merging process. The driving state information includes: lateral absolute position x, longitudinal absolute position y, and speed v. The relative information includes: relative position with the target vehicle and vehicle speed. The action space of the ramp merging decision model includes: lane changing and not changing lanes. The reward function of the ramp merging decision model includes a driving efficiency component with a reward value proportional to speed, a safety component that imposes a penalty in the event of a collision, and a merging component that reflects the urgency of merging. Different weight coefficients for the driving efficiency component, the safety component, and the merging component correspond to different merging strategies. A simulation environment for ramp merging is established; and the ramp merging decision model is trained based on the simulation environment. Acquire different forced lane change trajectory data; and perform driving style clustering analysis based on the forced lane change trajectory data; the driving style clustering analysis results are used to characterize the speed, acceleration, and driving distance distribution in the auxiliary lane for different types of drivers; the forced lane change trajectory data includes: speed, acceleration, headway, driving distance in the auxiliary lane, and number of free lane changes; Based on the sample data corresponding to the clustering analysis results of different driving styles, and based on the trained ramp merging decision model, we obtain the forced lane change decision model for autonomous vehicles with different merging strategies. Based on the current forced lane change trajectory data, the forced lane change decision model for autonomous vehicles based on different merging strategies is used to obtain the ramp merging decision results. The reward function R of the ramp merging decision model specifically includes the following formula: ； in, For the driving efficiency section, , This indicates the current speed of the target vehicle. Indicates the desired speed of the target vehicle. For the safety part, "collision" indicates that the target vehicle collides with other vehicles during the merging process, while "otherwise" indicates other situations. For the merging section, , This represents the longitudinal coordinate of the lane where the target vehicle is currently located. This represents the longitudinal coordinate of the lane the target vehicle intends to merge into. The base is 1.

5. This is a coefficient used to adjust the magnitude of the penalty. This indicates the current distance the target vehicle has traveled in the auxiliary lane. These are the weighting coefficients for the driving efficiency, safety, and merging components, respectively.

2. The method for determining ramp merging based on autonomous driving as described in claim 1, characterized in that, The simulation environment for constructing the ramp merging system specifically includes: The TraCI interface was called using Python software and combined with the simulation software SUMO to build a simulation environment for ramp merging. The state information obtained from the simulation environment includes: the information of the target vehicle at a certain moment; the target vehicle information includes: the target vehicle's lateral coordinates, longitudinal coordinates, speed in the merging area road network, and the lateral and longitudinal positions and absolute speeds of the target vehicle relative to the target vehicle in the six surrounding positions; the six surrounding positions are the vehicles in front and behind the target vehicle in its current lane, the vehicles in front and behind the target vehicle in the adjacent left lane, and the vehicles in front and behind the target vehicle in its right lane.

3. The method for determining ramp merging based on autonomous driving as described in claim 1, characterized in that, Obtain different forced lane change trajectory data, specifically including: Obtain the NGSIM dataset; Data cleaning of the NGSIM dataset; Based on the cleaned NGSIM dataset, different forced lane change trajectory data were obtained.

4. The ramp merging decision determination method based on autonomous driving as described in claim 1, characterized in that, Acquire different forced lane change trajectory data, and then include: The forced lane change trajectory data are subjected to Z-Score standardization and dimensionality reduction processing in sequence.

5. The method for determining ramp merging based on autonomous driving as described in claim 1, characterized in that, And based on the forced lane change trajectory data, driving style clustering analysis is performed, specifically including: Based on the forced lane change trajectory data, SPSS software was used to perform driving style clustering analysis.

6. A ramp merging decision-making device based on autonomous driving, used to implement the ramp merging decision-making method based on autonomous driving as described in any one of claims 1-5, characterized in that, The autonomous driving-based ramp merging decision-making device includes: A decision model construction module is used to construct a ramp merging decision model based on a dual-depth Q-network. The state space of the ramp merging decision model includes: the driving state information of the target vehicle and the relative information of vehicles at a set distance during the ramp merging process. The driving state information includes: lateral absolute position x, longitudinal absolute position y, and speed v. The relative information includes: relative position with the target vehicle and vehicle speed. The action space of the ramp merging decision model includes: lane changing and not changing lanes. The reward function of the ramp merging decision model includes a driving efficiency component with a reward value proportional to speed, a safety component that imposes a penalty in the event of a collision, and a merging component that reflects the urgency of merging. Different weight coefficients for the driving efficiency component, the safety component, and the merging component correspond to different merging strategies. The model training module is used to build a simulation environment for ramp merging; and to train the ramp merging decision model based on the simulation environment. The driving style clustering module is used to acquire different forced lane change trajectory data; and to perform driving style clustering analysis based on the forced lane change trajectory data; the driving style clustering analysis results are used to characterize the speed, acceleration, and driving distance distribution in the auxiliary lane for different types of drivers; the forced lane change trajectory data includes: speed, acceleration, headway, driving distance in the auxiliary lane, and number of free lane changes; The module for determining the forced lane change decision model for autonomous vehicles with different merging strategies is used to obtain the forced lane change decision model for autonomous vehicles with different merging strategies based on the sample data corresponding to the clustering analysis results of different driving styles and the trained ramp merging decision model. The ramp merging decision module is used to obtain the ramp merging decision result based on the current forced lane change trajectory data and the forced lane change decision model of autonomous vehicles with different merging strategies.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the ramp merging decision determination method based on autonomous driving as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the ramp merging decision determination method based on autonomous driving as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the ramp merging decision determination method based on autonomous driving as described in any one of claims 1-5.

Images