A road yielding incentive mechanism design method based on deep reinforcement learning

Abstract

The application provides a road yielding incentive mechanism design method based on deep reinforcement learning, and belongs to the technical field of intelligent road traffic. The method comprises the following steps: collecting full-factor road information of a target road section, obtaining preprocessed road network files, preprocessed traffic files and simulation configuration files through high-precision digital road network modeling, time period collection and preprocessing, performing benchmark simulation to obtain benchmark simulation information; performing control training by using a deep reinforcement learning algorithm, and obtaining a yielding-free incentive optimal model by setting a safety and efficiency double-target reward function; and obtaining an optimal incentive mechanism model by performing simulation training based on a fixed incentive layer and a dynamic incentive layer, and comparing full-dimension data according to the yielding-free incentive optimal model. The application realizes compensation of delay loss borne by yielding vehicles, improves the active yielding willingness, reduces the rush and jam, and realizes accurate yielding determination, fair compensation and road section customization incentive.

Description

Technical Field

[0001] This invention belongs to the field of intelligent road traffic technology, and in particular relates to a design method for a road yielding incentive mechanism based on deep reinforcement learning. Background Technology

[0002] With the continuous growth in the number of motor vehicles, urban interchanges, merging areas of expressway ramps, and intersections without traffic signals have become high-incidence areas for traffic conflicts and accidents. A large number of accidents and congestion are caused by unclear right-of-way rules, rushing, disorderly merging, and lane-changing conflicts. At present, traffic management mainly relies on mandatory means such as traffic lights, signs and markings, and penalties, lacking positive incentives for proactive right-of-way, making it difficult to improve traffic order from the source.

[0003] Traditional fixed yield rules cannot adapt to dynamic traffic flow. Different road segments have huge differences in geometry, lanes, and conflict points, and general strategies cannot achieve "one policy for each road". Reinforcement learning has the advantages of autonomous learning, dynamic optimization, multi-objective balance, and scenario adaptation. It can take road segment maps, traffic flow, and benchmark data as input, train and generate yield incentive mechanisms that are suitable for the road segment, and achieve optimal coordination of safety, efficiency, and fairness.

[0004] The merging and weaving areas of urban expressways are sensitive sections for yielding, where the trajectories of vehicles on the main road and ramps frequently intersect. The traditional "main road priority" system has fairness defects: yielding vehicles bear the losses of delays but receive no compensation, resulting in low willingness to yield and frequent cutting in and overtaking. Existing technologies lack accurate yielding judgment, fair compensation, and customized incentives for road sections. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a road yielding incentive mechanism design method based on deep reinforcement learning. This method solves the problems of existing road management systems where yielding vehicles bear delay losses without compensation, low willingness to yield, frequent instances of cutting in and overtaking, and a lack of accurate yielding judgment, fair compensation, and customized incentives for road sections.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a method for designing a road yielding incentive mechanism based on deep reinforcement learning, comprising the following steps: S1. Collect all elements of road information for the target road segment, obtain road network data through high-precision digital road network modeling, and collect traffic flow information of the target road segment in different time periods to obtain traffic flow data. Through preprocessing, obtain preprocessed road network files, preprocessed traffic flow files, and simulation configuration files. S2. By calling the simulation interface of the urban traffic simulation platform, according to the simulation configuration file, and importing the pre-processed road network file and pre-processed traffic flow file, the baseline simulation information is obtained through baseline simulation. S3. Based on the simulation process including the warm-up period and the control learning period, the deep reinforcement learning algorithm is used to conduct control training on the urban traffic simulation platform. By setting a dual-objective reward function of safety and efficiency and piecewise training parameters, the yield information and the optimal model without yield incentive are obtained. S4. Based on the optimal model without yielding incentives and yielding information, the optimal models for fixed incentives and dynamic incentives are obtained through simulation training based on fixed incentive layers and dynamic incentive layers, respectively. The optimal incentive mechanism model is obtained through full-dimensional data comparison, thus completing the design of the road yielding incentive mechanism.

[0007] The beneficial effects of this invention are as follows: This invention achieves customized yielding incentive mechanisms for each road segment through road segment-specific reinforcement learning training, multi-objective hierarchical rewards, accurate yielding judgment, and dynamic fair compensation. It can synergistically optimize traffic safety and efficiency, significantly improve the willingness to yield, and at the same time, the simulation is highly close to the real scene, the model is stable and controllable, the incentive cost is controllable, and it has the advantages of being standardized and replicable throughout the entire process. It can improve urban road yielding order from the source and reduce the risk of conflict and accidents.

[0008] Further, S1 includes the following steps: S101. Collect all elements of road information for the target road segment to obtain full data including core road basic parameters and key elements of traffic flow. S102. Based on the full data, high-precision digital road network modeling is used to obtain road network data in conjunction with the simulation specifications of the urban traffic simulation platform; S103. By using a preset multi-source data fusion method, the traffic flow information of the target road section is collected in different time periods, and the basic information on yielding at different time periods is recorded to obtain traffic flow data. S104. By correcting geometric topology errors in the road network data, unifying the coordinate system, and supplementing missing road attributes, a preprocessed road network file is obtained. S105. Using the Three Sigma principle, invalid data containing abnormal vehicle speeds and extreme traffic volumes are removed from the traffic flow data to obtain the removed traffic flow data. S106. Using interpolation, the missing vehicle trajectory information in the removed traffic flow data is completed, and based on historical data, the unreasonable vehicle departure times in the removed traffic flow data are corrected to generate a preprocessed traffic flow file containing three types of traffic flow densities: peak, off-peak, and low-peak. S107. Based on the preprocessed road network file and the preprocessed traffic flow file, obtain the simulation configuration file.

[0009] The beneficial effects of the above-mentioned further solutions are as follows: This invention constructs a simulation input file that conforms to real road conditions through multi-dimensional information collection and standardized processing, laying a real and reliable data foundation for subsequent full-process simulation training, ensuring that the simulation results can be directly mapped to real road scenarios, and by adopting the three Sigma principle, invalid data in the traffic flow data is removed, reducing the interference of invalid data on the simulation results and improving the accuracy and reliability of the traffic flow data; by using interpolation, missing vehicle trajectory information is completed, and unreasonable vehicle departure times are corrected, ensuring the continuity and rationality of traffic flow data, and being able to truly reflect the traffic flow density characteristics at different times.

[0010] Furthermore, S2 includes the following steps: S201. By calling the simulation interface of the urban traffic simulation platform, the pre-processed road network file and the pre-processed traffic flow file are imported according to the simulation configuration file to obtain the urban traffic simulation platform with imported data. S202. By not applying any yield incentives or interfering with the autonomous driving behavior of vehicles during the simulation process, multiple rounds of benchmark simulation are conducted using an urban traffic simulation platform with imported data to obtain the results of multiple rounds of benchmark simulation. S203. Statistically analyze and integrate the results of multiple rounds of benchmark simulation to obtain benchmark simulation information that includes safety indicators, efficiency indicators, yielding behavior indicators, and basic cost indicators.

[0011] The beneficial effects of the above-mentioned further solutions are as follows: This invention provides an objective and quantifiable comparison benchmark for subsequent three-layer incentive training through real-world scenario benchmark simulation without intervention or incentives. By calling the interface of the urban traffic simulation platform to import the pre-processed road network and traffic flow files, it achieves seamless connection between the simulation scenario and the pre-processed data, ensuring that the benchmark simulation scenario is highly consistent with the real road scenario and guaranteeing the authenticity of the benchmark simulation results. Through multiple rounds of non-intervention benchmark simulation, the randomness of a single simulation is reduced, the stability of the benchmark simulation results is improved, and the reliability of the benchmark data is ensured.

[0012] Furthermore, step S3 includes the following steps: S301. During the warm-up period of the simulation process, the traffic flow is freely controlled using the urban traffic simulation platform, and the traffic state is initialized. The initial environment is obtained by recording basic parameters including the initial traffic flow density and queue length of the road segment. S302. During the control learning period based on the simulation process, a deep reinforcement learning algorithm is used to uniformly control the driving behavior of all vehicles in the initial environment, and to synchronously update the vehicle status and traffic environment to obtain simulation process data. S303. Based on the simulation process data, using the yielding behavior determination algorithm based on multi-feature fusion, in response to the vehicle deceleration operation, the determination logic is automatically triggered, and combined with multi-dimensional information including vehicle position, relative speed and yielding rules, it is determined whether the current behavior belongs to a valid yielding event. If so, the relevant information of the yielding event is completely recorded in the form of structured data to obtain yielding information; otherwise, yielding information is not recorded. S304. Based on vehicle driving behavior and traffic conditions, a dual-objective reward function for safety and efficiency is set by integrating the safety layer and the efficiency layer. S305. By synchronously segmenting the learning rate and exploration rate, segmented training parameters are designed. Based on the dual-objective reward function of safety and efficiency, simulation process data, and segmented training parameters, four-stage simulation training is conducted to obtain the optimal model without yielding incentives.

[0013] Furthermore, the four-stage simulation training specifically includes: In the first stage, the exploration rate is rapidly reduced from the first threshold to the second threshold. By fully exploring the policy space, the high-quality policies that have been explored are obtained, and different vehicle control schemes are quickly traversed to obtain the simulation model for the first stage. In the second stage, based on the simulation model of the first stage, the exploration rate is gradually reduced from the second threshold to the third threshold to enter the stable learning stage. By optimizing the high-quality strategies that have been explored, the optimized high-quality strategies and the second stage simulation model are obtained. In the third stage, based on the simulation model of the second stage, the exploration rate is slowly reduced from the third threshold to the fourth threshold, entering the convergence and stabilization stage. The optimized high-quality strategy is then finely adjusted to obtain the adjusted high-quality strategy and the simulation model of the third stage. In the fourth stage, based on the simulation model of the third stage, the exploration rate is fixed at the fourth threshold, and the adjusted high-quality strategy is subjected to a verification simulation to obtain the optimal model without yielding incentives.

[0014] The beneficial effects of the above-mentioned further solutions are as follows: This invention uses a deep reinforcement learning algorithm for control training, with safety and efficiency as the optimization objectives only, without applying yield incentives. Through simulation training based on the safety and efficiency layers, the optimal model without yield incentives is obtained, providing an incentive-free benchmark for subsequent quantification of incentive effects and verifying the independent improvement effect of the incentive mechanism. By adjusting the exploration rate in four stages, the orderly advancement of strategy exploration and optimization is achieved. The high exploration rate in the first stage achieves comprehensive coverage of the strategy space, reducing the omission of high-quality strategies. The gradual reduction of the exploration rate in subsequent stages achieves steady optimization, fine adjustment, and reliable verification of high-quality strategies, avoiding model bias caused by insufficient or excessive exploration. By iterating the simulation model at different stages, the stability and adaptability of the optimal model without yield incentives are improved, ensuring that the model can accurately reflect the optimal driving control logic of the vehicle under incentive-free conditions.

[0015] Furthermore, step S4 includes the following steps: S401. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, a simulation training based on a fixed incentive layer is conducted by adding a fixed yielding incentive mechanism to obtain the optimal model with fixed incentives. S402. Based on the total revenue of the yielding vehicle and the total revenue of the yielded vehicle in the yielding event in the yielding information, calculate a traffic fairness index that includes the average absolute interpolation of revenue and the standard deviation of revenue difference. S403. Based on yield loss, real-time traffic conditions, fairness constraints, and conflict risks, an adaptive dynamic yield incentive reward function is designed. Simulation training based on the dynamic incentive layer is conducted according to the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yield incentive. Combined with traffic fairness indicators, the optimal dynamic incentive model is obtained. S404. By comparing the best models for fixed incentives, dynamic incentives, and no yielding incentives across all dimensions, a full-scenario benefit assessment is obtained. S405. Compare the full-scenario benefit assessment of fixed incentives and dynamic incentives, and select the optimal yielding incentive mechanism based on the traffic characteristics, management needs and intelligence level of the target road segment, obtain the optimal incentive mechanism model, and complete the design of the road yielding incentive mechanism.

[0016] Furthermore, step S401 includes the following steps: S4011. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, the simulation environment is obtained. S4012. Based on the natural yielding data in the baseline simulation information, determine the minimum fixed incentive threshold by statistically analyzing the yielding losses of different types of yielding behavior, and obtain the fixed incentive value. S4013. Based on the fixed incentive value and combined with the dual-objective reward function of safety and efficiency, a fixed yielding incentive term is added to obtain the fixed yielding reward function; S4014. Based on the simulation environment and the fixed yielding reward function, perform simulation training based on the fixed incentive layer to obtain the optimal model with fixed incentive.

[0017] Furthermore, the expression for the traffic fairness index is as follows: ; ; in, This represents the average absolute interpolation of returns. This indicates the total number of yielding events within the scene. Indicates the first i The total benefit of vehicles yielding in a yielding event Indicates the first i The total benefit of the vehicle being yielded to in a yielding incident. This represents the standard deviation of the earnings difference.

[0018] The beneficial effects of the above-mentioned further solution are as follows: This invention obtains traffic fairness indicators by calculating the average absolute interpolation of benefits and the standard deviation of benefit differences, realizing the accurate quantification of the fairness of benefits between yielding and receiving vehicles in yielding events. Among them, the average absolute interpolation of benefits can intuitively reflect the average deviation of benefits between the two types of vehicles in a single yielding event, clearly reflecting the fairness of a single yielding behavior. The standard deviation of benefit differences can reflect the dispersion of the benefit deviation between the two types of vehicles in all yielding events, measuring the fairness and stability of the overall yielding scenario. Through the synergistic effect of the two indicators, the fairness characteristics in yielding behavior can be comprehensively captured, making up for the shortcomings of only focusing on safety and efficiency while ignoring fairness. It provides a scientific basis for fairness constraints for the design of dynamic incentive mechanisms, ensuring that dynamic incentives can take into account the fairness of benefits for both parties while improving traffic efficiency and safety, and avoiding some vehicles being unwilling to yield voluntarily due to incentive imbalance.

[0019] Furthermore, S403 includes the following steps: S4031. Based on yielding losses, real-time traffic conditions, fairness constraints, and conflict risks, a dynamic incentive value containing incentive parameters is calculated. S4032. Based on the dynamic incentive value and combined with the dual-objective reward function of safety and efficiency, an adaptive dynamic yielding incentive reward function is obtained by learning the optimal incentive parameters. S4033. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, the simulation environment is obtained. S4034. Based on the adaptive dynamic yielding incentive reward function and the simulation environment, perform simulation training based on the dynamic incentive layer to obtain the trained dynamic incentive simulation model. S4035. Based on the traffic fairness index, perform a global optimization evaluation on the trained dynamic incentive simulation model to obtain the optimal dynamic incentive model.

[0020] Furthermore, the expression for the adaptive dynamic yielding incentive reward function is as follows: ; ; in, This represents the adaptive dynamic yielding incentive reward function. This represents a bi-objective reward function. Indicates the incentive weighting coefficient. Indicates the dynamic stimulus value. Indicates the basic incentive value. This represents the yield loss weighting coefficient. This indicates the actual delay in yielding. This indicates the maximum yield delay for yield type. This represents the traffic condition weighting coefficient. Indicates real-time traffic density on road sections. Indicates the maximum service traffic density of the road segment. This represents the conflict risk weighting coefficient. Indicates the level of conflict risk. This represents the fairness correction coefficient.

[0021] The beneficial effects of the above-mentioned further solutions are as follows: By combining safety and efficiency rewards, yielding losses and fairness constraints, this invention designs an adaptive dynamic yielding incentive reward function, obtains the optimal dynamic incentive model through training, realizes real-time dynamic adjustment of incentive values, and balances the four major objectives of safety, efficiency, cost and fairness. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention.

[0023] Figure 2 This is the overall flowchart of the yielding incentive mechanism in this embodiment.

[0024] Figure 3 This is a spatial distribution diagram of the average velocity in the baseline scene in this embodiment.

[0025] Figure 4This is a kernel density distribution diagram of yielding events in the baseline scenario of this embodiment.

[0026] Figure 5 This is a flowchart of the yield determination process in this embodiment.

[0027] Figure 6 This is a spatial distribution diagram of the average velocity in the unexcited scenario of this embodiment.

[0028] Figure 7 This is the kernel density map of the predicted collision time in the unexcited scenario of this embodiment.

[0029] Figure 8 This is the kernel density map of the expected collision time under the fixed excitation scenario in this embodiment.

[0030] Figure 9 This is a scatter plot of the yielding profit and loss benefits under the fixed incentive scenario in this embodiment.

[0031] Figure 10 This is the kernel density map of the expected collision events in the dynamic excitation scenario of this embodiment.

[0032] Figure 11 This is a scatter plot of the profit and loss benefits of yielding in the dynamic incentive scenario of this embodiment. Detailed Implementation

[0033] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0034] Before describing this embodiment, the following terms will be explained: ITS: Intelligent Transportation System; V2X: Vehicle-to-Everything (V2X) TTC: Time of Collision; SUMO: Urban Traffic Simulation Platform; Principle: Three Sigma principle.

[0035] Example In this embodiment, relying on deep reinforcement learning and microscopic simulation, a four-dimensional optimization system of efficiency, safety, fairness, and incentive cost can be constructed. A dynamic incentive mechanism is generated through road segment-specific training to improve the right-of-way order.

[0036] like Figure 1 As shown, this invention provides a method for designing a road yielding incentive mechanism based on deep reinforcement learning, the implementation of which is as follows: S1. Collect all road information for the target road segment. Obtain road network data through high-precision digital road network modeling. Collect traffic flow information for the target road segment in different time periods to obtain traffic flow data. Through preprocessing, obtain preprocessed road network files, preprocessed traffic flow files, and simulation configuration files. The specific steps are as follows: S101. Collect all elements of road information for the target road segment to obtain full data including core road basic parameters and key elements of traffic flow. S102. Based on the full data, high-precision digital road network modeling is used to obtain road network data in conjunction with the simulation specifications of the urban traffic simulation platform.

[0037] In this embodiment, as Figure 2 As shown, the design of the yield incentive mechanism for the road segment is divided into five processing stages, including data acquisition and preprocessing, baseline simulation output based on SUMO simulation, simulation training based on safety and efficiency layers, simulation training based on fixed incentive layers, and simulation training based on dynamic incentive layers. Through progressive training and comparison, the selection and design of the optimal yield incentive mechanism for the road segment are completed. Data acquisition and preprocessing are performed. Detailed information on road length, width, number of lanes, speed limits, and other aspects is collected to establish the corresponding road network for the road segment. Peak traffic flow, off-peak traffic flow, and normal traffic flow at different times of day are collected through field surveys, drone recordings, and historical data prediction. Preprocessing steps such as segmentation and noise reduction analysis are then performed to obtain three traffic flow files with different traffic densities. Through this process, the system obtains basic simulation environment data, which can be used for benchmark simulations of the system interface SUMO. Specifically: Road network information collection and high-precision digital modeling are carried out. Comprehensive road information collection is conducted for the target road segment, covering core road basic parameters such as road segment length, width, number of lanes, lane function division (dedicated lanes for straight, left turn and right turn), speed limit, turning rules, intersection type (signal-controlled, unsignaled and roundabout), signal timing scheme, channelization design, roadside facility layout (bus stops, entrances and exits and guardrails) and pedestrian crossing location. At the same time, key factors affecting traffic flow, such as traffic control measures around the road segment (such as no-parking and speed limit signs), road slope and sight distance conditions, are also collected. Based on the collected full data, a high-precision digital road network model is completed, forming a road network file (.net.xml) that conforms to the SUMO simulation specifications. Through operations such as topology verification and geometric correction, it is ensured that the geometric features, traffic rules, and control requirements of the simulated road network are completely consistent with those of real roads, providing a realistic road environment for simulation.

[0038] S103. By using a preset multi-source data fusion method, traffic flow information of the target road section is collected in different time periods, and the basic information on yielding at different time periods is recorded to obtain traffic flow data.

[0039] In this embodiment, multi-time period and multi-scenario traffic flow data fusion collection is carried out. Multi-source data fusion methods such as on-site surveys, drone aerial photography recording, floating car data extraction, checkpoint data statistics, and historical traffic data prediction are adopted to collect traffic flow operation information of target road sections in different time periods, focusing on covering four typical time periods: morning peak (7:00-9:00), evening peak (17:00-19:00), off-peak (9:00-17:00, 19:00-21:00), and low-peak (21:00-7:00 the next day), distinguishing different traffic flow density scenarios. The data collected includes core traffic flow parameters such as traffic volume, vehicle speed distribution, vehicle type ratio (small cars, large cars, and buses), headway, travel time, queue length, and following distance at different times. It also records basic information such as the frequency of yielding behavior, yielding type distribution, yielding delay, and yielding conflict rate at different times, providing a realistic basis for the design of subsequent incentive mechanisms and ensuring that the incentive scheme is adapted to traffic demand at all times.

[0040] S104. By correcting geometric topology errors in the road network data, unifying the coordinate system, and supplementing missing road attributes, a preprocessed road network file is obtained. S105. Using the Three Sigma principle, invalid data containing abnormal vehicle speeds and extreme traffic volumes are removed from the traffic flow data to obtain the removed traffic flow data. S106. Using interpolation, the missing vehicle trajectory information in the removed traffic flow data is completed, and based on historical data, the unreasonable vehicle departure times in the removed traffic flow data are corrected to generate a preprocessed traffic flow file containing three types of traffic flow densities: peak, off-peak, and low-peak. S107. Based on the preprocessed road network file and the preprocessed traffic flow file, obtain the simulation configuration file.

[0041] In this embodiment, data standardization preprocessing and quality control are performed. The original road network and traffic flow data are systematically preprocessed, including data segmentation, noise removal, outlier correction, missing value completion, format standardization, and consistency verification. For road network data, geometric topology errors are corrected, coordinate systems are unified, and missing road attributes are added to obtain a preprocessed road network file; for traffic flow data, a... The principle is to remove invalid data such as abnormal vehicle speeds and extreme traffic volumes. Missing vehicle trajectory information is supplemented by interpolation. Unreasonable vehicle departure times are corrected based on historical data. Finally, standardized traffic flow files (.rou.xml) corresponding to three types of traffic flow densities—peak, off-peak, and low-peak—are generated, resulting in preprocessed traffic flow files. At the same time, a SUMO simulation configuration file (.sumocfg) is generated to ensure that the data can be directly connected to the SUMO simulation platform, providing stable and high-quality input for benchmark simulation.

[0042] S2. By calling the simulation interface of the urban traffic simulation platform, according to the simulation configuration file, and importing the preprocessed road network file and preprocessed traffic flow file, benchmark simulation information is obtained through benchmark simulation. The specific steps are as follows: S201. By calling the simulation interface of the urban traffic simulation platform, the pre-processed road network file and the pre-processed traffic flow file are imported according to the simulation configuration file to obtain the urban traffic simulation platform with imported data. S202. By not applying any yield incentives or interfering with the autonomous driving behavior of vehicles during the simulation process, multiple rounds of benchmark simulation are conducted using an urban traffic simulation platform with imported data to obtain the results of multiple rounds of benchmark simulation. S203. Statistically analyze and integrate the results of multiple rounds of benchmark simulation to obtain benchmark simulation information that includes safety indicators, efficiency indicators, yielding behavior indicators, and basic cost indicators.

[0043] In this embodiment, during the baseline simulation output stage based on SUMO simulation, the obtained road network and traffic flow files and other data are used to perform a reality-based simulation output for this road segment by calling SUMO, and multiple rounds of simulation are performed to ensure the removal of randomness and special cases; then, the information from the multiple rounds of baseline simulation is output and summarized to obtain the baseline simulation information, including TTC, collision count, overall delay, and yield information summary of the baseline simulation scenario. The preprocessed road network file and the preprocessed traffic flow file complete the benchmark simulation of the real-world scenario without intervention or incentives, and output the comparison benchmark data to provide an objective and quantifiable comparison benchmark for the subsequent three-layer incentive training. It serves as the benchmark reference for the entire incentive mechanism design. By calling the SUMO simulation interface and importing standardized road network files and traffic flow files, multiple rounds of real-world scenario simulations (no less than 15 rounds for each scenario) are carried out for three types of traffic flow density scenarios: peak, off-peak, and low-peak. During the simulation, no yielding incentives are applied, and the autonomous driving behavior of vehicles is not interfered with. The simulation completely replicates the natural yielding, following, lane changing, and traffic flow of vehicles on real roads. Multiple simulation runs effectively eliminate the randomness and extreme special cases of single simulations, ensuring the stability and representativeness of the baseline data. Simultaneously, a simulation step size of 0.1 seconds is set to ensure that the recording accuracy of vehicle trajectories and yielding behaviors meets the requirements of subsequent analysis, resulting in data such as... Figure 3 The baseline scene average velocity spatial distribution map and Figure 4 The diagram shows the kernel density distribution of yielding events in the baseline scenario. After the benchmark simulation is completed, the results of multiple rounds of benchmark simulation are statistically analyzed and integrated to output core indicators covering four dimensions: safety, efficiency, yielding behavior, and cost. Specifically, these include: Safety indicators: TTC (Time to Collision), PET (Potential Time to Collision), number of collisions, number of conflicts, number of emergency braking incidents, rear-end collision risk rate, side collision risk rate, etc. Efficiency indicators: total road segment delay, average travel time, average vehicle speed, queue length, traffic capacity, intersection saturation, travel time reliability, etc. Yielding behavior indicators include: number of natural yielding events, distribution of yielding behavior types, yielding delays, distribution of yielding locations and times, yielding success rate, yielding conflict rate, and yielding fairness. Basic cost metrics: average collision cost, TTC conflict risk cost, time efficiency cost, yielding incentive cost, etc. The above indicators will serve as benchmarks for subsequent simulation training of the safety and efficiency layer, fixed incentive layer, and dynamic incentive layer. They will be used to quantify the effects of different incentive mechanisms on improving road safety and efficiency, and to provide data support for the selection of incentive schemes.

[0044] S3. Based on the simulation process including the warm-up period and the control learning period, the urban traffic simulation platform is trained using a deep reinforcement learning algorithm. By setting a dual-objective reward function for safety and efficiency and piecewise training parameters, the optimal model for yielding information and without yielding incentives is obtained. The specific steps are as follows: S301. During the warm-up period of the simulation process, the traffic flow is freely controlled using the urban traffic simulation platform, and the traffic state is initialized. The initial environment is obtained by recording basic parameters including the initial traffic flow density and queue length of the road segment. S302. During the control learning period based on the simulation process, a deep reinforcement learning algorithm is used to uniformly control the driving behavior of all vehicles in the initial environment, and to synchronously update the vehicle status and traffic environment to obtain simulation process data.

[0045] In this embodiment, simulation training based on the safety and efficiency layers utilizes a deep reinforcement learning algorithm to control and train the SUMO simulation. This stage includes a warm-up period and a control learning period. During the warm-up period, SUMO provides free control, ensuring global and stable traffic flow coverage when entering the control learning period. Then, the control and learning period begins, where all vehicles in the scenario are controlled by the model. When a vehicle decelerates, a yielding decision algorithm is initiated. If a yielding action is successfully determined, the relevant information is recorded; otherwise, it is not recorded. Furthermore, during this stage, rewards and penalties are given based on the vehicle's behavior using a reward function. These rewards and penalties are fed back every 10 steps, allowing the model to adjust its strategy and control behavior in real time, improving learning effectiveness. In this stage, the simulation needs to be continuously trained. The exploration rate and learning rate are designed in segments. In the first stage, the exploration rate ranges from the first threshold of 1 to the second threshold of 0.5, which is the rapid exploration stage in the early stage. In the second stage, the exploration rate decreases from the second threshold of 0.5 to the third threshold of 0.1, which is the stable learning stage in the middle stage. In the third stage, the exploration rate slowly decreases from the third threshold of 0.1, which is the stable convergence stage in the later stage. The learning rate is designed in segments corresponding to the exploration rate. This design can help the algorithm find a better model as much as possible, thus laying the groundwork and providing a reference for the subsequent yielding incentive. In the simulation training of the safety and efficiency layers, the system only incentivizes the safety and efficiency components and does not incentivize yielding. In order to obtain an optimal model without yielding incentive, and thus reflect the improvement of yielding incentive on both safety and efficiency, the effect of yielding incentive can be well quantified.

[0046] In this embodiment, a deep reinforcement learning algorithm is used to train the control of the SUMO simulation. With safety and efficiency as the optimization objectives, no yielding incentive is applied, resulting in the optimal control model without yielding incentive. This provides an incentive-free benchmark for subsequent quantification of incentive effects and verifies the independent improvement effect of the incentive mechanism. The simulation process is divided into two stages: a warm-up period and a control learning period. During the warm-up period, SUMO freely controls the traffic flow for no less than 25% of the total simulation time to ensure stable global traffic flow coverage and saturated road network operation, avoiding interference from unstable traffic flow in the initial stage of simulation on model training. At the same time, traffic state initialization is completed during the warm-up period, and basic parameters such as initial traffic flow density and queue length of road segments are recorded to provide a stable initial environment for the control learning period and obtain the initial environment. During the control learning period, the driving behavior of all vehicles is uniformly controlled by a deep reinforcement learning model. The deep reinforcement learning model takes the real-time position, speed, acceleration of vehicles, status of surrounding vehicles and traffic conditions in the initial environment as input, and outputs decision commands such as acceleration, deceleration and lane changing of vehicles to achieve global optimization control of traffic flow. The vehicle status and traffic environment are updated synchronously in each simulation step to obtain simulation process data.

[0047] S303. Based on the simulation process data, using the yielding behavior determination algorithm based on multi-feature fusion, in response to the vehicle deceleration operation, the determination logic is automatically triggered, and combined with multi-dimensional information including vehicle position, relative speed and yielding rules, it is determined whether the current behavior belongs to a valid yielding event. If so, the relevant information of the yielding event is completely recorded in the form of structured data to obtain yielding information; otherwise, yielding information is not recorded.

[0048] In this embodiment, as Figure 5 As shown, intelligent judgment and full structured recording of yielding behavior are performed. Through the built-in yielding behavior judgment algorithm based on multi-feature fusion, the judgment logic is automatically triggered when the vehicle decelerates (deceleration ≥ 2m / s², or speed deceleration magnitude greater than 4m / s). Combining multi-dimensional information such as vehicle position, relative speed, and yielding rules, the judgment is determined whether the behavior is a valid yielding event. If the yielding is deemed valid, the relevant information of the yielding event is recorded in a complete form of structured data, including the yielding vehicle ID, the vehicle being yielded to ID, the yielding location, the yielding duration, the yielding type, the vehicle speed before and after the yielding, the yielding delay, the benefit of the vehicle being yielded to, and the conflict risk level, etc. If the yielding criteria are not met, no record will be made to ensure the accuracy of yielding data statistics and provide a precise basis for subsequent incentive calculations. Specifically, the yield determination process involves real-time detection of the movement status of each vehicle. When a vehicle decelerates, it is checked whether the deceleration is due to following another vehicle too closely to yield. If not, it is listed as a candidate event. The process also involves detecting and recording neighboring vehicles around the candidate vehicle. When a neighboring vehicle becomes the vehicle in front of the candidate vehicle, or when a vehicle merges from the side, the event is confirmed as a yield event. Finally, when the candidate vehicle returns to its average speed, the yield event is considered to have ended, and a series of raw data are extracted in a structured manner: the starting and ending speeds of the yielding vehicle, the starting and ending speeds of the vehicle being yielded to, the yield delay loss, the benefit of being yielded to, the yield time, location, type, conflict level, fairness, and other data.

[0049] S304. Based on vehicle driving behavior and traffic conditions, a dual-objective reward function for safety and efficiency is set by integrating the safety layer and the efficiency layer.

[0050] In this embodiment, a dual-objective reward function for safety and efficiency is constructed based on vehicle driving behavior and traffic conditions. Indicators such as vehicle collision risk, trip delay, and speed stability are incorporated into the reward. The expression for the dual-objective reward function is shown below: ; in, This represents a bi-objective reward function. This represents a safety bonus, calculated based on metrics such as TTC and collision risk. Higher collision risk results in a higher penalty value. This represents an efficiency bonus, calculated based on indicators such as vehicle delays, travel time, and vehicle speed. The greater the delay, the lower the bonus value. and All represent weighting coefficients, which can be adjusted according to road segment management needs. The default is that they are derived based on actual economic gains and losses. , Furthermore, a penalty value of 2000 is provided for each collision event to greatly reduce collision behavior. A reward or penalty signal is sent back every 10 steps of simulation to guide the model to adjust the control strategy in real time, improve the model's learning efficiency, and avoid the model getting stuck in local optima.

[0051] S305. By synchronously segmenting the learning rate and exploration rate, segmented training parameters are designed. Based on the dual-objective reward function of safety and efficiency, simulation process data, and segmented training parameters, four-stage simulation training is conducted to obtain the optimal model without yielding incentives.

[0052] In this embodiment, the training parameters are designed in a segmented manner to adapt to the needs of different training stages of the model and ensure that the model converges to the optimal state quickly: Phase 1: The exploration rate is rapidly reduced from the first threshold of 1.0 to the second threshold of 0.5, which encourages the initial simulation model to fully explore the policy space and quickly traverse different vehicle control schemes, thus avoiding premature convergence of the initial simulation model and obtaining the first-stage simulation model. Second stage: The exploration rate gradually decreases from the second threshold of 0.5 to the third threshold of 0.1. The simulation model of the first stage enters the stable learning stage, focusing on optimizing the high-quality strategies that have been explored, improving the control accuracy of the simulation model of the first stage, and obtaining the simulation model of the second stage. In the third stage, the exploration rate slowly decreases from the third threshold of 0.1 to the fourth threshold of 0.02. The simulation model in the second stage enters the convergence and stabilization stage. The strategy is further refined to ensure that the simulation model in the second stage converges to the global optimum, thus obtaining the simulation model in the third stage. Phase 4: With the exploration rate fixed at the fourth threshold of 0.02, conduct verification simulations to verify the stability and generalization ability of the simulation model from Phase 3, and obtain the optimal model without yielding incentives; The learning rate and exploration rate are configured synchronously in segments. The initial learning rate is set to 0.001, decreasing synchronously with the exploration rate. Based on a dual-objective reward function for safety and efficiency, simulation process data, and segmented training parameters, four stages of simulation training are performed to obtain the following results: Figure 6 The spatial distribution of average velocity in the unexcited scenario is shown below. Figure 7 The kernel density map of the expected collision time in the unexcited scenario shown is used to output the optimal model of safety and efficiency layer after completing four-stage simulation training. That is, the optimal model without yielding incentive and the corresponding simulation index, which provides an unexcited benchmark for the design of subsequent incentive mechanisms.

[0053] S4. Based on the optimal model without yielding incentives and yielding information, through simulation training based on fixed incentive layers and dynamic incentive layers, the optimal models for fixed incentives and dynamic incentives are obtained respectively. Then, through comparison of data across all dimensions, the optimal incentive mechanism model is obtained, completing the design of the road yielding incentive mechanism. The specific steps are as follows: S401. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, a fixed yielding incentive mechanism is added. Simulation training based on a fixed incentive layer is then performed to obtain the optimal model with fixed incentives. The specific steps are as follows: S4011. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, the simulation environment is obtained. S4012. Based on the natural yielding data in the baseline simulation information, determine the minimum fixed incentive threshold by statistically analyzing the yielding losses of different types of yielding behavior, and obtain the fixed incentive value. S4013. Based on the fixed incentive value and combined with the dual-objective reward function of safety and efficiency, a fixed yielding incentive term is added to obtain the fixed yielding reward function; S4014. Based on the simulation environment and the fixed yielding reward function, perform simulation training based on the fixed incentive layer to obtain the optimal model with fixed incentive.

[0054] In this embodiment, during the simulation training of the fixed incentive layer, information on yielding under natural conditions is obtained through simulation of a baseline scenario. The minimum fixed incentive value is obtained through the yielding event processing of the baseline scenario. The fixed incentive value is a fixed incentive value given only according to the yielding type of the yielding behavior. The basic configuration of the simulation training in this stage is the same as that of the safety and efficiency layer, but the reward function adds a fixed incentive for yielding behavior. When a vehicle is determined to be yielding, a yielding incentive value is given according to the current yielding type of the vehicle, reducing the benefit loss value caused by the yielding vehicle. After this stage of simulation training, the optimal model of the yielding fixed incentive layer and the relevant data of the model simulation are still obtained. By comparing and analyzing the simulation data of the model with the optimal model of the safety and efficiency layer, the improvement effect of the fixed yielding incentive on safety and efficiency and the incentive cost can be obtained, thereby obtaining a benefit evaluation for the whole scenario. A fixed incentive value is set based on yielding information in the benchmark simulation. The yielding losses of different types of yielding behavior (including yielding delays and increased travel time) are statistically analyzed to determine the minimum fixed incentive threshold, ensuring that the incentive value can cover the average direct loss of yielding vehicles. On this basis, differentiated fixed incentive values ​​are allocated according to yielding type, importance of yielding scenario, and conflict risk level to ensure that the incentive value can fully compensate for the benefit loss of yielding vehicles, while avoiding excessive incentive costs and achieving a balance between incentive effect and cost. Standardized simulation training control is implemented, fully utilizing the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the safety and efficiency layers to ensure fairness and scientific rigor in the comparison, based on the dual-objective reward function. By adding a fixed yield incentive, we obtain a fixed incentive reward function, the expression of which is shown below: ; ; in, This represents a fixed incentive reward function. This represents a bi-objective reward function. This represents the incentive weighting coefficient, which defaults to 1. This represents a fixed incentive value, which is assigned according to the vehicle yielding type. This represents the average yielding delay time for yielding vehicles, obtained based on yielding delay samples from a baseline scenario. The value of a driver's time is calculated based on the average income of employed persons in the study area. After conducting multi-dimensional effect comparisons and full-scenario benefit evaluations, and completing the simulation training, the results are as follows: Figure 8 The kernel density map of predicted collision times under the fixed excitation scenario is shown below. Figure 9The scatter plot of yielding benefit under the fixed incentive scenario is shown. The optimal model of fixed incentive and full simulation data are output and compared with the optimal model of safety and efficiency layers in multiple dimensions. The improvement effect of fixed yielding incentive on safety indicators (number of collisions, TTC and number of conflicts), efficiency indicators (total delay and average travel time), incentive cost and yielding behavior indicators (yielding rate, yielding delay and fairness) is quantified. At the same time, the benefit evaluation is carried out for three types of traffic density scenarios: peak, off-peak and low-peak, to analyze the applicability and effectiveness of fixed incentive under different traffic loads. A full-scenario benefit evaluation report is generated, focusing on the advantages and limitations of fixed incentive, and providing a reference for subsequent dynamic incentive design.

[0055] S402. Based on the total revenue of the yielding vehicle and the total revenue of the yielded vehicle in the yielding event in the yielding information, calculate the traffic fairness index, which includes the average absolute interpolation of revenue and the standard deviation of revenue difference.

[0056] In this embodiment, a traffic fairness index is designed for the global optimization evaluation, specifically including: the average absolute interpolation of revenue and the standard deviation of the revenue difference, as shown in the following expression: ; ; in, This represents the average absolute interpolation of returns. This indicates the total number of yielding events within the scene. Indicates the first i The total benefit of vehicles yielding in a yielding event Indicates the first i The total benefit of the vehicle being yielded to in a yielding incident. This represents the standard deviation of the revenue difference; where total vehicle revenue is the sum of efficiency revenue and yielding incentive revenue, and the time loss is quantified by economic value to ensure consistency in the evaluation dimensions. (Standard deviation of revenue difference) The smaller the value, the smaller the fluctuation in the revenue gap between vehicles in each yielding event, and the more stable the fairness performance.

[0057] S403. Based on yielding loss, real-time traffic conditions, fairness constraints, and conflict risks, an adaptive dynamic yielding incentive reward function is designed. Then, based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, simulation training is performed using the dynamic incentive layer. Combined with traffic fairness indicators, the optimal dynamic incentive model is obtained. The specific steps are as follows: S4031. Based on yielding losses, real-time traffic conditions, fairness constraints, and conflict risks, a dynamic incentive value containing incentive parameters is calculated. S4032. Based on the dynamic incentive value and combined with the dual-objective reward function of safety and efficiency, an adaptive dynamic yielding incentive reward function is obtained by learning the optimal incentive parameters. S4033. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, the simulation environment is obtained. S4034. Based on the adaptive dynamic yielding incentive reward function and the simulation environment, perform simulation training based on the dynamic incentive layer to obtain the trained dynamic incentive simulation model. S4035. Based on the traffic fairness index, perform a global optimization evaluation on the trained dynamic incentive simulation model to obtain the optimal dynamic incentive model.

[0058] In this embodiment, during the simulation training of the dynamic incentive layer, the reward function of the dynamic yielding incentive is designed by combining the safety and efficiency reward values ​​of the safety and efficiency layer with the yielding loss and fairness. Through multiple rounds of simulation training in this stage, the dynamic incentive parameters of the optimal model of the dynamic yielding incentive are extracted, and the model is compared and analyzed with the safety and efficiency layer to obtain the improvement effect of the dynamic yielding incentive on safety and efficiency and the incentive cost. The full-scenario benefit evaluation of the dynamic incentive layer is obtained, and the real-time dynamic adjustment of the incentive value is realized to balance the four major objectives of safety, efficiency, cost and fairness. The dynamic incentive mechanism is specifically a multi-factor adaptive dynamic incentive model. Its core logic is as follows: instead of fixing the incentive value, it performs real-time calculations based on four core factors: yielding loss, real-time traffic conditions, fairness constraints, and conflict risk. This achieves adaptive optimization where the greater the yielding loss, the higher the incentive; the more congested the traffic, the higher the incentive; the higher the conflict risk, the higher the incentive; and the worse the fairness, the lower the incentive adjustment. The expression for calculating the dynamic incentive value is shown below: ; in, Indicates the dynamic stimulus value. Indicates the basic incentive value. This represents the yield loss weighting coefficient. This indicates the actual delay in yielding. This indicates the maximum yield delay for yield type. This represents the traffic condition weighting coefficient. Indicates real-time traffic density on road sections. Indicates the maximum service traffic density of the road segment. This represents the conflict risk weighting coefficient. Indicates the level of conflict risk. This represents the fairness correction factor; Basic incentive value It is a fixed value, defined by the fixed incentive layer, and corresponds to the benchmark incentive value of this yielding type in the fixed incentive layer, which is the basis for dynamic incentives; Actual delay of yielding The extra travel time of yielding vehicles is obtained through real-time calculation, in seconds. Maximum yield delay for yield type This is a fixed value, calibrated by benchmark simulation. The maximum yield delay for this yield type in the benchmark simulation is used for normalization. Real-time traffic density on road sections The data is obtained through real-time calculation, specifically the current traffic density of the target road segment. Maximum service traffic density of the road segment It is a fixed value, determined by the road network design, specifically the maximum density of the corresponding road segment at service level E, used for normalization; Conflict risk level Obtained through real-time calculation, with a range of... The conflict risk is calculated based on TTC and PET; the higher the risk, the larger the value. Fairness correction coefficient Obtained through real-time calculation, with a range of This is used to balance the losses and gains of the yielding vehicle and the vehicle being yielded to; Yield loss weighting coefficient This is an adjustable coefficient, with a range of [value missing]. This is used to control the degree of impact of yielding loss on the incentive value; Traffic condition weighting coefficient This is an adjustable coefficient, with a range of [value missing]. Used to control the degree of impact of traffic congestion on the incentive value. Conflict risk weighting coefficient This is an adjustable coefficient, with a range of [value missing]. This is used to control the degree of impact of conflict risk on incentive values; The dynamic incentive value is then fused with the bi-objective reward function to obtain the adaptive dynamic yielding incentive reward function, as shown in the following expression: ; in, This represents the adaptive dynamic yielding incentive reward function. This represents a bi-objective reward function. Indicates the incentive weighting coefficient. This represents the dynamic incentive value; ensuring fairness in the comparison, it automatically learns the optimal dynamic incentive parameters by maximizing the total reward. , as well as This achieves overall optimization of the incentive mechanism; Simulation training and core parameter extraction are performed, using the same simulation configuration, training rounds, and parameter segmentation rules as the previous stages. Multiple rounds of simulation training are conducted until the relative change in the moving average reward over the most recent 50 episodes is <5% and the coefficient of variation (CV) is <0.10, at which point convergence is considered achieved, and training ends. During training, the model calculates the dynamic incentive value for each yielding action in real time, transmits the reward signal, and continuously optimizes the dynamic incentive parameters using traffic fairness indicators. , , as well as (etc.), to achieve adaptive adjustment of the excitation value; After extracting core parameters and optimizing the model, and completing the training, the following results were obtained: Figure 10 The kernel density map of predicted collision events in the dynamically excited scenario is shown below. Figure 11 The dynamic incentive scenario shown is presented as a scatter plot of profit and loss benefits. The core incentive parameters of the optimal dynamic incentive model are extracted, and the corresponding simulation indicators are output to provide data support for effect comparison. Through parameter sensitivity analysis, the impact of each parameter on the incentive effect is verified, and the parameter values ​​are optimized to ensure the stability and generalization ability of the model.

[0059] S404. By comparing the best models for fixed incentives, dynamic incentives, and no yielding incentives across all dimensions, a full-scenario benefit assessment is obtained. S405. Compare the full-scenario benefit assessment of fixed incentives and dynamic incentives, and select the optimal yielding incentive mechanism based on the traffic characteristics, management needs and intelligence level of the target road segment, obtain the optimal incentive mechanism model, and complete the design of the road yielding incentive mechanism.

[0060] In this embodiment, the three-layer model is compared and the optimal incentive mechanism is selected; Comprehensive multi-dimensional comparative analysis: The dynamic incentive layer model is compared with the safety and efficiency layer and fixed incentive layer models in all dimensions to quantify the comprehensive effects of dynamic incentives on safety improvement, efficiency optimization, cost control (total incentive cost and unit incentive benefit), fairness (average absolute interpolation of vehicle incentive benefits and standard deviation of benefit difference), and yielding behavior, and complete the benefit evaluation of the whole scenario. Comparison and selection of incentive mechanisms: The overall benefits of fixed incentives and dynamic incentives are compared, and the advantages and disadvantages of the two incentive mechanisms in different traffic scenarios are analyzed. Finally, based on the traffic characteristics, management needs and intelligence level of the road segment, the optimal yielding incentive mechanism is selected as the final application model for the target road segment, thus obtaining the optimal incentive mechanism model. The entire process design of the yielding incentive mechanism is completed, forming a technical solution that can be directly implemented.

[0061] In this embodiment, the entire process of designing a yield incentive mechanism for a road segment is completed. Through a five-layer progressive design of data collection, benchmark simulation, no-incentive training, fixed-incentive training, and dynamic-incentive training, a complete yield incentive mechanism design system is constructed. Among them, the dynamic incentive mechanism realizes the real-time dynamic adjustment of incentive values ​​through a multi-factor adaptive model. While ensuring the traffic efficiency of the road segment, it significantly improves the traffic safety of the road segment, guides vehicles to actively and properly yield, reduces traffic conflicts and accident risks, and achieves optimal control of incentive costs and a balance of fairness among vehicles across the entire road segment. This provides an intelligent and feasible technical solution for yield management of different types of road segments such as urban roads, highways, and expressways, and also provides a new technical path for vehicle collaborative management in intelligent transportation systems (ITS). A closed loop is formed through data, simulation, training, comparison, and selection. The three-layer model is fully trained: a baseline model without incentives to determine the safety and efficiency baseline under no incentives; a fixed incentive model to provide simple and easy-to-implement incentive schemes; and a dynamic incentive model to provide adaptive and high-precision incentive schemes. A comprehensive comparison and benefit assessment is conducted, quantifying the comparison across five dimensions: safety, efficiency, yielding rate, fairness, and incentive costs, to select the optimal solution. Outputs directly usable results, segment-specific incentive rules (fixed or dynamic), incentive parameters (incentive value, weight, and triggering conditions), and execution standards adaptable to all times of peak, off-peak, or low-peak periods; The sign design closed loop was completed, achieving three major goals: accurate right-of-way determination, fair compensation, and customized incentives for road sections.

[0062] In this embodiment, the optimal incentive mechanism model is directly geared towards intelligent right-of-way management solutions implemented on urban roads, and its core function is: Compensation for lost time and benefits is achieved by providing fair and precise financial compensation to vehicles that cause delays by giving way. Enhancing the willingness to yield by using positive incentives instead of traditional mandatory control can reduce behaviors such as cutting in line, merging into other lanes, and disorderly traffic flow. Customized management and control for each road segment, generating exclusive incentive rules for each target road segment to adapt to different traffic flows, road structures and management needs; Collaborative optimization of safety and efficiency improves traffic flow while reducing the risk of collisions or conflicts and minimizing overall delays; Fairness and cost control are achieved by balancing the benefits of the giver and the receiver, controlling the total incentive investment, and avoiding incentive imbalance or waste. The optimal incentive mechanism model is not an algorithm confined to the laboratory, but a practical tool that can be directly deployed, has quantifiable benefits, and can be iterated over the long term. It can be directly implemented in traffic management by connecting the optimal incentive mechanism model to V2X, roadside perception, vehicle terminals, and traffic platforms. Through the yielding behavior judgment algorithm based on multi-feature fusion in the optimal incentive mechanism model, yielding behavior can be identified in real time, and incentives (points, toll tickets, and small subsidies, etc.) can be issued according to the corresponding incentive reward function in the optimal incentive mechanism model. This supplements incentives to traditional traffic lights, signs and markings, and traffic police enforcement, shifting from passive punishment to active incentives. It will continue to improve traffic order and safety, significantly increase the rate of proactive yielding at merging areas, intersections, and ramps, reduce accidents such as cutting in, running in line, and minor collisions, reduce the number of conflicts and collision risks, and improve the overall traffic efficiency of road sections. Support traffic management decisions, provide traffic management departments with road segment-level data on yielding pain points, incentive costs, and revenue returns, support precise governance with one road policy, and avoid one-size-fits-all rules; Empowering intelligent connected vehicles and autonomous driving, providing autonomous vehicles with yielding decision-making basis and incentive rules, improving collaborative traffic capabilities, and perfecting the yielding interaction logic of vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V); Standardized processes can be developed and quickly replicated across various urban roads, expressways, and highway ramps nationwide. Data can be accumulated over time, incentive parameters can be continuously optimized, and a dynamically evolving traffic governance system can be formed.

Claims

1. A method for designing a road yielding incentive mechanism based on deep reinforcement learning, characterized in that, Includes the following steps: S1. Collect all elements of road information for the target road segment, obtain road network data through high-precision digital road network modeling, and collect traffic flow information of the target road segment in different time periods to obtain traffic flow data. Through preprocessing, obtain preprocessed road network files, preprocessed traffic flow files, and simulation configuration files. S2. By calling the simulation interface of the urban traffic simulation platform, according to the simulation configuration file, and importing the pre-processed road network file and pre-processed traffic flow file, the baseline simulation information is obtained through baseline simulation. S3. Based on the simulation process including the warm-up period and the control learning period, the deep reinforcement learning algorithm is used to conduct control training on the urban traffic simulation platform. By setting a dual-objective reward function of safety and efficiency and piecewise training parameters, the yield information and the optimal model without yield incentive are obtained. S4. Based on the optimal model without yielding incentives and yielding information, the optimal models for fixed incentives and dynamic incentives are obtained through simulation training based on fixed incentive layers and dynamic incentive layers, respectively. The optimal incentive mechanism model is obtained through full-dimensional data comparison, thus completing the design of the road yielding incentive mechanism.

2. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 1, characterized in that, S1 includes the following steps: S101. Collect all elements of road information for the target road segment to obtain full data including core road basic parameters and key elements of traffic flow. S102. Based on the full data, high-precision digital road network modeling is used to obtain road network data in conjunction with the simulation specifications of the urban traffic simulation platform; S103. By using a preset multi-source data fusion method, the traffic flow information of the target road section is collected in different time periods, and the basic information on yielding at different time periods is recorded to obtain traffic flow data. S104. By correcting geometric topology errors in the road network data, unifying the coordinate system, and supplementing missing road attributes, a preprocessed road network file is obtained. S105. Using the Three Sigma principle, invalid data containing abnormal vehicle speeds and extreme traffic volumes are removed from the traffic flow data to obtain the removed traffic flow data. S106. Using interpolation, the missing vehicle trajectory information in the removed traffic flow data is completed, and based on historical data, the unreasonable vehicle departure times in the removed traffic flow data are corrected to generate a preprocessed traffic flow file containing three types of traffic flow densities: peak, off-peak, and low-peak. S107. Based on the preprocessed road network file and the preprocessed traffic flow file, obtain the simulation configuration file.

3. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 1, characterized in that, S2 includes the following steps: S201. By calling the simulation interface of the urban traffic simulation platform, the pre-processed road network file and the pre-processed traffic flow file are imported according to the simulation configuration file to obtain the urban traffic simulation platform with imported data. S202. By not applying any yield incentives or interfering with the autonomous driving behavior of vehicles during the simulation process, multiple rounds of benchmark simulation are conducted using an urban traffic simulation platform with imported data to obtain the results of multiple rounds of benchmark simulation. S203. Statistically analyze and integrate the results of multiple rounds of benchmark simulation to obtain benchmark simulation information that includes safety indicators, efficiency indicators, yielding behavior indicators, and basic cost indicators.

4. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 1, characterized in that, S3 includes the following steps: S301. During the warm-up period of the simulation process, the traffic flow is freely controlled using the urban traffic simulation platform, and the traffic state is initialized. The initial environment is obtained by recording basic parameters including the initial traffic flow density and queue length of the road segment. S302. During the control learning period based on the simulation process, a deep reinforcement learning algorithm is used to uniformly control the driving behavior of all vehicles in the initial environment, and to synchronously update the vehicle status and traffic environment to obtain simulation process data. S303. Based on the simulation process data, using the yielding behavior determination algorithm based on multi-feature fusion, in response to the vehicle deceleration operation, the determination logic is automatically triggered, and combined with multi-dimensional information including vehicle position, relative speed and yielding rules, it is determined whether the current behavior belongs to a valid yielding event. If so, the relevant information of the yielding event is completely recorded in the form of structured data to obtain yielding information; otherwise, yielding information is not recorded. S304. Based on vehicle driving behavior and traffic conditions, a dual-objective reward function for safety and efficiency is set by integrating the safety layer and the efficiency layer. S305. By synchronously segmenting the learning rate and exploration rate, segmented training parameters are designed. Based on the dual-objective reward function of safety and efficiency, simulation process data, and segmented training parameters, four-stage simulation training is conducted to obtain the optimal model without yielding incentives.

5. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 4, characterized in that, The four-stage simulation training is specifically as follows: In the first stage, the exploration rate is rapidly reduced from the first threshold to the second threshold. By fully exploring the policy space, the high-quality policies that have been explored are obtained, and different vehicle control schemes are quickly traversed to obtain the simulation model for the first stage. In the second stage, based on the simulation model of the first stage, the exploration rate is gradually reduced from the second threshold to the third threshold to enter the stable learning stage. By optimizing the high-quality strategies that have been explored, the optimized high-quality strategies and the second stage simulation model are obtained. In the third stage, based on the simulation model of the second stage, the exploration rate is slowly reduced from the third threshold to the fourth threshold, entering the convergence and stabilization stage. The optimized high-quality strategy is then finely adjusted to obtain the adjusted high-quality strategy and the simulation model of the third stage. In the fourth stage, based on the simulation model of the third stage, the exploration rate is fixed at the fourth threshold, and the adjusted high-quality strategy is subjected to a verification simulation to obtain the optimal model without yielding incentives.

6. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 1, characterized in that, S4 includes the following steps: S401. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, a simulation training based on a fixed incentive layer is conducted by adding a fixed yielding incentive mechanism to obtain the optimal model with fixed incentives. S402. Based on the total revenue of the yielding vehicle and the total revenue of the yielded vehicle in the yielding event in the yielding information, calculate a traffic fairness index that includes the average absolute interpolation of revenue and the standard deviation of revenue difference. S403. Based on yield loss, real-time traffic conditions, fairness constraints, and conflict risks, an adaptive dynamic yield incentive reward function is designed. Simulation training based on the dynamic incentive layer is conducted according to the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yield incentive. Combined with traffic fairness indicators, the optimal dynamic incentive model is obtained. S404. By comparing the best models for fixed incentives, dynamic incentives, and no yielding incentives across all dimensions, a full-scenario benefit assessment is obtained. S405. Compare the full-scenario benefit assessment of fixed incentives and dynamic incentives, and select the optimal yielding incentive mechanism based on the traffic characteristics, management needs and intelligence level of the target road segment, obtain the optimal incentive mechanism model, and complete the design of the road yielding incentive mechanism.

7. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 6, characterized in that, S401 includes the following steps: S4011. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, the simulation environment is obtained. S4012. Based on the natural yielding data in the baseline simulation information, determine the minimum fixed incentive threshold by statistically analyzing the yielding losses of different types of yielding behavior, and obtain the fixed incentive value. S4013. Based on the fixed incentive value and combined with the dual-objective reward function of safety and efficiency, a fixed yielding incentive term is added to obtain the fixed yielding reward function; S4014. Based on the simulation environment and the fixed yielding reward function, perform simulation training based on the fixed incentive layer to obtain the optimal model with fixed incentive.

8. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 6, characterized in that, The expression for the traffic fairness index is as follows: in, This represents the average absolute interpolation of returns. This indicates the total number of yielding events within the scene. Indicates the first i The total benefit of vehicles yielding in a yielding event Indicates the first i The total benefit of the vehicle being yielded to in a yielding incident. This represents the standard deviation of the earnings difference.

9. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 6, characterized in that, S403 includes the following steps: S4031. Based on yielding losses, real-time traffic conditions, fairness constraints, and conflict risks, a dynamic incentive value containing incentive parameters is calculated. S4032. Based on the dynamic incentive value and combined with the dual-objective reward function of safety and efficiency, an adaptive dynamic yielding incentive reward function is obtained by learning the optimal incentive parameters. S4033. Based on the simulation environment, training rounds, parameter segmentation rules, and reward function weights of the optimal model without yielding incentives, the simulation environment is obtained. S4034. Based on the adaptive dynamic yielding incentive reward function and the simulation environment, perform simulation training based on the dynamic incentive layer to obtain the trained dynamic incentive simulation model. S4035. Based on the traffic fairness index, perform a global optimization evaluation on the trained dynamic incentive simulation model to obtain the optimal dynamic incentive model.

10. The method for designing a road yielding incentive mechanism based on deep reinforcement learning according to claim 9, characterized in that, The expression for the adaptive dynamic yielding incentive reward function is as follows: in, This represents the adaptive dynamic yielding incentive reward function. This represents a bi-objective reward function. Indicates the incentive weighting coefficient. Indicates the dynamic stimulus value. Indicates the basic incentive value. This represents the yield loss weighting coefficient. This indicates the actual delay in yielding. This indicates the maximum yield delay for yield type. This represents the traffic condition weighting coefficient. Indicates real-time traffic density on road sections. Indicates the maximum service traffic density of the road segment. This represents the conflict risk weighting coefficient. Indicates the level of conflict risk. This represents the fairness correction coefficient.

Images