Traffic optimization method based on three-layer Stackelberg game

Abstract

The invention belongs to the technical field of traffic optimization, and particularly relates to a traffic optimization method based on a three-layer Stackelberg game, and the method comprises the steps: obtaining mixed traffic data; according to the mixed traffic data, constructing a three-layer Stackelberg game model; an equilibrium solution of the three-layer game model is solved through a deep reinforcement learning algorithm, an SAC algorithm is introduced into an upper layer to generate a global constraint strategy, and a DDPG algorithm is adopted in a lower layer to optimize cooperative driving and safe obstacle avoidance driving behaviors of the intelligent vehicle; according to the global constraint strategy and the intelligent vehicle driving behavior optimization result, the traffic efficiency, safety and energy consumption and carbon emission indexes of the mixed traffic flow are optimized, the traffic state is continuously updated through behavior feedback data of a middle layer and a lower layer, and strategy optimization is carried out in combination with SAC and DDPG models. Therefore, the problems of insufficient dynamic adaptability, low efficiency of main body interaction processing, limitation of a game model and the like in the prior art are solved.

Description

Technical Field

[0001] This invention belongs to the field of traffic optimization technology, specifically relating to a traffic optimization method based on a three-layer Stackelberg game. Background Technology

[0002] With the advancement of intelligent connected vehicles, urban traffic is gradually presenting a new "hybrid transportation" model where drivers and autonomous vehicles coexist. In this scenario, the key issue is achieving efficient and safe traffic control. Existing traffic control methods are mostly based on static rules or centralized strategies, facing challenges such as high dynamism, multiple agents, resource constraints, and decision coupling. They struggle to efficiently handle the interactive behaviors between different types of traffic agents, and most of these methods ignore the differences in policy responses among different types of traffic participants, lacking dynamic evolution mechanisms and failing to adapt to continuous changes in the environment and tasks. If local responses lag, overall control becomes unstable, easily leading to local optima or policy conflicts. Although some research has introduced game theory, applying Stackelberg game theory to multi-agent collaborative control, traffic scheduling, and energy allocation, traditional two-layer Stackelberg game models mostly employ static strategies, lacking deep reinforcement learning assistance and failing to achieve a "learning while interacting" mechanism. Furthermore, the coupling feedback between multiple followers is not fully considered, often only achieving local optima or approximate solutions.

[0003] Therefore, it is necessary to construct a model that can accurately depict the strategic interactions between traffic control centers, drivers, and intelligent vehicles. This model should be combined with the powerful pattern recognition capabilities of deep reinforcement learning in areas such as traffic flow prediction, signal control, and route planning to form an efficient and real-time hybrid traffic control optimization strategy. This strategy can address the complex scheduling needs in mixed traffic environments and improve the multi-level collaborative control and adaptive decision-making capabilities of traffic in complex mixed traffic environments. Summary of the Invention

[0004] This application provides a traffic optimization method based on a three-layer Stackelberg game to address the problems of insufficient dynamic adaptability, inefficient subject interaction processing, and limitations of the game model in existing technologies.

[0005] The first aspect of this application provides a traffic optimization method based on a three-layer Stackelberg game, comprising the following steps: acquiring mixed traffic data, including traffic flow density, congestion index, average vehicle speed, traffic light status, and road network traffic status, collected through traffic perception via cameras, RSUs, and V2X terminals; constructing a three-layer Stackelberg game model based on the mixed traffic data, wherein the upper layer is the traffic control center decision layer, the middle layer is the driver-driven vehicle following layer, and the lower layer is the intelligent vehicle response layer; solving the equilibrium solution of the three-layer Stackelberg game model using a deep reinforcement learning algorithm, wherein the upper layer introduces the SAC algorithm to generate traffic signal periodic allocation and global constraint strategies for road priority planning, and the lower layer uses the DDPG algorithm to optimize intelligent vehicle cooperative driving and safe obstacle avoidance driving behavior; optimizing the traffic efficiency, safety, and energy consumption and carbon emission indicators of the mixed traffic flow based on the global constraint strategies and the optimization results of intelligent vehicle driving behavior, continuously updating the traffic status through behavioral feedback data from the middle and lower layers, and combining the SAC and DDPG models for strategy optimization.

[0006] Preferably, a three-layer Stackelberg game model is constructed, including: setting the upper-layer traffic control center as the leader, responsible for global traffic state information perception and global constraint strategy generation, wherein the global traffic state information includes traffic flow distribution, congested areas, and accident alarms, which are acquired in real time through cameras, geomagnetic sensors, and V2X networks; setting the middle-layer driver's vehicle as the first follower, which passively reacts to environmental signals and the behavior of other vehicles based on individual preferences, experience, and reaction speed, and whose behavior is jointly determined by traffic signals, road environment, and driving habits; and setting the lower-layer intelligent vehicle as the second follower, which has perception, prediction, and decision-making capabilities, and optimizes autonomous driving behavior based on the influence of the upper-layer strategy and the middle-layer behavior.

[0007] Preferably, the upper-level traffic control center introduces the SAC algorithm to generate a global constraint strategy, including: constructing an input state space of road conditions, traffic density, accident information, and environmental indicators, as well as an action space of signal timing, speed limit instructions, and lane control; based on the input state space and the action space, the optimization objective of the SAC algorithm is defined as maximizing the weighted sum of comprehensive benefits and policy entropy, wherein the comprehensive benefits include improved traffic efficiency, reduced energy consumption, and avoidance of safety risks; and outputting a global constraint strategy through iterative training of the policy network and the dual-Q network.

[0008] Preferably, the lower layer uses the DDPG algorithm to optimize the driving behavior of the intelligent vehicle, including: constructing a state space containing the intelligent vehicle's own speed, the positions of surrounding vehicles, the L1 layer global policy, and the F1 layer behavior prediction, with acceleration, steering angle, and path selection as the continuous action space; based on the state space and the action space, through an Actor-Critic dual network structure, the Actor network outputs a deterministic driving policy, the Critic network evaluates the value of the policy, and the network parameters are iteratively updated based on the experience replay pool to optimize the cooperative driving and safe obstacle avoidance behavior.

[0009] Preferably, the SAC algorithm formula is:

[0010]

[0011]

[0012]

[0013]

[0014] in, Optimal strategy; strategy Maximize the operation; The current point in time; Total number of time steps; For mathematical expectation; At time step The state; At time step The actions taken at that time; For strategy The probability distribution of the induced state-action pairs; For instant rewards; For temperature parameters; For strategy In a given state Entropy below; For policy networks; For strategy In a given state The action produced below Follows a normal distribution; For parameters A definite value function; For action-value function; It is the natural logarithm; For parameters; For strategy Regarding parameters The objective function; Data distribution in the experience playback buffer; The optimal control strategy for leader L1; The objective function for the leader; The optimal response for follower F1; This is the optimal response for follower F2.

[0015] Preferably, the DDPG algorithm formula is:

[0016]

[0017]

[0018]

[0019]

[0020]

[0021] in, Optimal strategy; strategy Maximize the operation; The current point in time; Total number of time steps; For state distribution Expectations; Discount factor; These are indicators of average vehicle speed and intersection traffic efficiency. At time step The state; At time step The actions taken at that time; For action Strategy According to the status produce; Let be the loss function of the network; For instant rewards; For action-value function; For parameters; The target value; For target strategy; For network parameters The gradient; For action; For the second-level follower F2, this is the policy variable; The optimal strategy for the first-level leader; The objective function for the first-level leader; As a variable; In order to be in Find the value of the variable that makes the function reach its maximum value. The optimal response for follower F1.

[0022] The second aspect of this application provides a traffic optimization system based on a three-layer Stackelberg game, comprising: an acquisition module for acquiring mixed traffic data, including traffic flow density, congestion index, average vehicle speed, traffic light status, and road network traffic status, collected through traffic perception via cameras, RSUs, and V2X terminals; a construction module for constructing a three-layer Stackelberg game model based on the mixed traffic data, wherein the upper layer is a traffic control center decision layer, the middle layer is a driver-following layer, and the lower layer is an intelligent vehicle response layer; a generation module for solving the equilibrium solution of the three-layer Stackelberg game model using a deep reinforcement learning algorithm, wherein the upper layer introduces the SAC algorithm to generate global constraint strategies for traffic signal periodic allocation and road priority planning, and the lower layer uses the DDPG algorithm to optimize intelligent vehicle cooperative driving and safe obstacle avoidance driving behavior; and an optimization module for optimizing the traffic efficiency, safety, and energy consumption and carbon emission indicators of the mixed traffic flow based on the global constraint strategies and the optimization results of intelligent vehicle driving behavior, continuously updating the traffic status through behavioral feedback data from the middle and lower layers, and performing strategy optimization by combining the SAC and DDPG models.

[0023] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the traffic optimization method based on a three-layer Stackelberg game as described in the above embodiments.

[0024] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the traffic optimization method based on a three-layer Stackelberg game as described in the above embodiments.

[0025] A fifth aspect of this application provides a computer program product, including a computer program or instructions, for implementing the traffic optimization method based on a three-layer Stackelberg game as described in the above embodiments.

[0026] Therefore, this application has the following beneficial effects: This application's embodiments collect mixed traffic data, construct a three-layer Stackelberg game model, and use deep reinforcement learning algorithms to solve for equilibrium solutions. Continuous optimization is then performed by combining global constraint strategies and intelligent vehicle driving behavior optimization results to achieve hierarchical precise control and dynamic coordination of mixed traffic flows. The upper layer introduces the SAC algorithm to generate global constraint strategies, while the lower layer uses the DDPG algorithm to optimize intelligent vehicle driving behavior. Traffic status is updated by integrating behavioral feedback data from the middle and lower layers, improving multi-level collaborative control performance and adaptive decision-making capabilities in complex mixed traffic environments. Based on the combination of hierarchical game theory and deep reinforcement learning, this approach overcomes the bottlenecks of traditional static rules or centralized strategies in multi-agent interaction processing and dynamic adaptability, improving the efficiency, safety, and energy consumption and carbon emission optimization effects of mixed traffic flows. It can adapt to complex scenarios where drivers and intelligent vehicles coexist. Therefore, it solves the problems of insufficient dynamic adaptability, inefficient subject interaction processing, and limitations of game models in existing technologies.

[0027] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0028] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a traffic optimization method based on a three-layer Stackelberg game provided according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the mutual constraints and control relationships between different levels of a Stackelberg game model provided according to an embodiment of this application; Figure 3 This is a schematic diagram of the hierarchical definition framework of a three-layer Stackelberg game model provided according to an embodiment of this application; Figure 4 This is a schematic diagram of the upper-level control framework of a three-layer Stackelberg game model according to an embodiment of this application; Figure 5 This is a schematic diagram of the control framework of the lower layer of a three-layer Stackelberg game model according to an embodiment of this application; Figure 6 This is a schematic diagram illustrating the control optimization process of a three-layer Stackelberg game model according to an embodiment of this application; Figure 7 This is a schematic diagram of the traffic optimization system based on a three-layer Stackelberg game according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] The following describes a traffic optimization method based on a three-layer Stackelberg game, according to embodiments of this application, with reference to the accompanying drawings. Addressing the issue of insufficient dynamic adaptability mentioned in the background section, this application provides a traffic optimization method based on a three-layer Stackelberg game. In this method, mixed traffic data is collected, a three-layer Stackelberg game model is constructed, and a deep reinforcement learning algorithm is used to solve for the equilibrium solution. Continuous optimization is performed by combining global constraint strategies and intelligent vehicle driving behavior optimization results, enabling hierarchical precise control and dynamic coordination of mixed traffic flows. The upper layer introduces the SAC algorithm to generate global constraint strategies, while the lower layer uses the DDPG algorithm to optimize intelligent vehicle driving behavior. Traffic status is updated by integrating behavioral feedback data from the middle and lower layers, improving the multi-level collaborative control performance and adaptive decision-making capabilities in complex mixed traffic environments. Based on the combination of hierarchical game theory and deep reinforcement learning, this method overcomes the bottlenecks of traditional static rules or centralized strategies in multi-agent interaction processing and dynamic adaptability, improving the efficiency, safety, and energy consumption and carbon emission optimization effects of mixed traffic flows. It can adapt to complex scenarios where drivers and intelligent vehicles coexist. This solves the problems of insufficient dynamic adaptability, inefficient subject interaction processing, and limitations of game theory models in existing technologies.

[0031] Specifically, Figure 1 This is a flowchart illustrating the traffic optimization method based on a three-layer Stackelberg game provided in an embodiment of this application.

[0032] like Figure 1 As shown, this hybrid traffic optimization method based on a three-layer Stackelberg deep game includes the following steps: In step S101, mixed traffic data is acquired, including traffic density, congestion index, average vehicle speed, traffic light status, and road network traffic status, which are collected through traffic perception data from cameras, RSUs, and V2X terminals.

[0033] It is understood that the embodiments of this application comprehensively capture various traffic state information such as traffic flow density, congestion index, average vehicle speed, and traffic light status in the road network. This avoids the information loss caused by the limited coverage and insufficient data dimensions of a single perception method. It provides data support for the accurate construction of state spaces at each level of the three-layer Stackelberg game model, the generation of global constraint strategies and intelligent vehicle driving behavior optimization schemes, and enhances the perception and modeling capabilities of complex dynamic scenarios such as traffic congestion evolution, vehicle interaction behavior, and signal timing effects.

[0034] In step S102, a three-layer Stackelberg game model is constructed based on the mixed traffic data, wherein the upper layer is the traffic control center decision layer, the middle layer is the driver-driven vehicle following layer, and the lower layer is the intelligent vehicle response layer.

[0035] The Stackelberg game model formula is as follows:

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043] in, The rate of change of state; It is a dynamic state vector; For the control input of leader L1; The response strategy for the follower F1; The response strategy for follower F2; It is a relational function; This is the initial time. This is the initial state; To maximize the objective function; For the leader's goals; For differential variables; It is a state vector; The end point of the time interval; For immediate benefits to leaders; For the final assessment of the leader's status; The value of the variable to be maximized by the objective function; For F1 followers; For drivers to receive immediate benefits; For driver trip outcome evaluation; For F2 follower targets; For the immediate benefits of smart cars; To assess the task completion rate of intelligent vehicles.

[0044] It is understood that the embodiments of this application construct a three-layer Stackelberg game model with the traffic control center as the leader, the driver's vehicle as the first follower, and the intelligent vehicle as the second follower. This avoids the instability of regulation caused by traditional static strategies or two-layer games due to ignoring the differences in the responses of multiple agents and lacking dynamic evolution mechanisms. It provides a hierarchical decision framework for SAC and DDPG deep reinforcement learning algorithms, supports the generation of global constraint policies and the optimization of intelligent vehicle driving behavior, and enhances the ability to accurately model the interactive behavior of multiple agents in mixed traffic environments as well as the overall collaborative control performance and adaptive decision-making ability.

[0045] For example, such as Figure 2As shown, in a morning rush hour scenario on an east-west expressway in a certain city, the 12-kilometer-long section includes 6 entrance / exit ramps and 4 sets of intelligent traffic signals. The average daily traffic flow during the morning rush hour (7:00-9:00) reaches 18,000 vehicles, of which approximately 30% (5,400) are intelligent connected vehicles, and 70% (12,600) are traditionally manually driven vehicles. The Level 1 Leader (L1, the traffic control center) collects real-time road network data via V2X and updates global constraint strategies every 5 minutes: the maximum speed limit on the main road is dynamically adjusted from 60-80 km / h, the ramp traffic flow control threshold does not exceed 1,500 vehicles per hour, and the green light duration of the signal timing scheme is adaptive from 30-60 seconds. Without this model, the average vehicle speed during the morning rush hour is only 28 km / h, and the congestion time exceeds 90 minutes. After introducing the model, the main road traffic efficiency improves by 23% in the first hour. The constraint relationship between L1 and the second-layer follower F1 (traditional vehicle drivers) is reflected in the following ways: L1's speed limit command directly regulates the driving speed of F1, increasing driver compliance with speed limit commands from 65% to 82%, reducing acceleration and deceleration frequency by 37%, shortening the average travel time of traditional vehicles during a weekday morning rush hour from 42 minutes to 29 minutes, and reducing the overall idling time of the road segment by 2800 hours; at the same time, F1's driving behavior has a feedback effect on L1's strategy adjustment. For example, if traditional vehicles frequently change lanes between 7:45 and 8:00, L1 temporarily extends the green light duration during this period to match traffic demand. The constraint relationship between L1 and the third-layer follower F2 (intelligent vehicles) is reflected in the following ways: L1 delineates the driving area for intelligent vehicle platoons, with 50 intelligent vehicles forming a cooperative platoon, achieving an average speed of 56 km / h (21 km / h higher than F1), and receiving L1's energy consumption optimization targets (reducing energy consumption per 100 kilometers from 15 kWh to 12 kWh) through V2X; F2, on the other hand, provides feedback on platoon traffic efficiency data to assist L1 in fine-tuning signal timing. The mutual constraints between F1 and F2 are reflected in the following distance of intelligent vehicles, which maintains a safe following distance of 1.2-1.8 seconds, avoiding the irregular lane-changing peaks of F1 vehicles. The standard deviation of the vehicle spacing is reduced from 3.2 meters to 0.8 meters. Meanwhile, F1 vehicles also reduce interference with the intelligent vehicle queue during the queuing period (e.g., 8:15-8:30), reducing the risk of rear-end collisions by 54%. The three-layer dynamic interaction forms a closed loop. L1 adjusts its strategy every 15 minutes based on feedback from F1 and F2, improving the overall traffic capacity during the morning rush hour by 40%, stabilizing the average vehicle speed above 45 km / h, reducing the congestion index from 1.8 to 1.2, and reducing carbon emissions by approximately 1.2 tons per day. This verifies the value of layered game theory decision-making in the precise control of mixed traffic.

[0046] In this embodiment, a three-layer Stackelberg game model is constructed, including: setting the upper-layer traffic control center as the leader, responsible for global traffic state information perception and global constraint strategy generation, the global traffic state information including traffic flow distribution, congested areas, and accident alarms, which are acquired in real time through cameras, geomagnetic sensors, and V2X networks; setting the middle-layer driver's vehicle as the first follower, which passively reacts to environmental signals and other vehicle behaviors based on individual preferences, experience, and reaction speed, and whose behavior is jointly determined by traffic signals, road environment, and driving habits; setting the lower-layer intelligent vehicle as the second follower, which has perception, prediction, and decision-making capabilities, and optimizes autonomous driving behavior based on the influence of the upper-layer strategy and the middle-layer behavior.

[0047] It is understood that the embodiments of this application clarify the positioning and role boundaries of each level in mixed traffic by setting the roles and functions of the upper, middle and lower layers. This avoids problems such as chaotic strategy execution or delayed response caused by unclear responsibilities and unclear interaction logic. It provides a clear architectural foundation for the decision-making interaction of each level in the three-layer Stackelberg game model, so that the global strategy of the upper layer can be effectively transmitted to the middle and lower layers, and the behavioral feedback of the middle and lower layers can be accurately applied to the optimization of the upper layer strategy. This supports the deep reinforcement learning algorithm to play a targeted role at each level and enhances the orderliness and efficiency of the hierarchical control of mixed traffic.

[0048] For example, such as Figure 3As shown, during the morning rush hour at a crossroads in the city center, which connects a commercial area and a residential area, the traffic flow during the morning rush hour (7:30-9:00) reaches 3,000 vehicles per hour, of which traditional manually driven vehicles account for 70% (2,100 vehicles) and intelligent vehicles account for 30% (900 vehicles). As the traffic control center at the top level, it obtains real-time global traffic status through 8 cameras, 4 geomagnetic sensors, and V2X network around the intersection: traffic flow distribution shows dense east-west straight traffic (15 vehicles per lane / 100 meters), congestion concentrated in the north-to-south left-turn lane (queue length 80 meters), and a minor collision occurred eastward at 7:45. Based on this, a global constraint strategy is generated every 5 minutes—the east-west green light duration is extended to 60 seconds (originally 45 seconds), one more north-to-south left-turn lane is temporarily added, and the speed limit in the accident section is 30 km / h; the middle level, the first follower, traditional vehicle drivers, influenced by traffic signals and their own driving habits, 35% of drivers will rush to cross when the green light is about to end (an increase of 10% compared to usual), with following distance fluctuating between 3-15 meters (novice drivers often less than 5 meters). With the speed limit instruction at the first level, 65% of drivers kept their speed below 40 km / h (competition rate was only 50%). The second-level intelligent vehicle, through V2X perception of the upper-level strategy and middle-level behavior, autonomously optimized its driving behavior—maintaining a following distance of 8-12 meters (avoiding the risk of close following by traditional vehicles), reducing lane changes during peak hours when traditional vehicles were overtaking (7:50-8:05), and forming a coordinated queue of 10 intelligent vehicles with an average speed of 45 km / h (15 km / h higher than traditional vehicles). Through V2X feedback, the queue's traffic efficiency (800 vehicles per hour) and energy consumption data (energy consumption per 100 km decreased from 18 kWh to 15 kWh) were collected. The traffic control center combined the feedback from both types of vehicles to dynamically adjust the signal timing, improving intersection traffic efficiency by 30% and reducing rear-end collisions from 3 per hour to 1.

[0049] In step S103, the equilibrium solution of the three-layer Stackelberg game model is solved by deep reinforcement learning algorithm. The upper layer introduces the SAC algorithm to generate traffic signal periodic allocation and global constraint strategy for road priority planning. The lower layer uses the DDPG algorithm to optimize intelligent vehicle cooperative driving and safe obstacle avoidance driving behavior.

[0050] The global constraint strategy is generated by the upper-level traffic control center based on the SAC algorithm. It includes traffic signal cycle allocation, road priority planning, signal timing, speed limit instructions, lane control, and temporary restricted area delineation. It is used to globally constrain the behavior of the middle-level drivers and the lower-level intelligent vehicles.

[0051] It is understood that the embodiments of this application generate global constraint strategies through the upper-level traffic control center, covering traffic signal cycle allocation, road priority planning, signal timing, speed limit instructions, and lane control. This avoids the problems of chaotic traffic behavior and unbalanced resource allocation caused by the lack of unified global guidance, provides norms and basis for the behavior of mid-level drivers and lower-level intelligent vehicles, supports the optimization of mixed traffic flow in terms of traffic efficiency, safety, energy consumption and carbon emissions, and enhances the ability to coordinate and control the overall traffic status.

[0052] For example, in traffic management on main roads during morning and evening rush hours in a certain city, the global constraint strategy demonstrated significant overall optimization effects. Taking the city's core north-south arterial road as an example, this section is 8 kilometers long, including 5 intersections and 3 pedestrian crossings. During the morning rush hour (7:30-9:00), the hourly traffic volume reaches 3,200 vehicles, and during the evening rush hour (17:30-19:00), it reaches 3,500 vehicles, with intelligent connected vehicles accounting for approximately 25%. The traffic control center, as the leader, updates the global constraint strategy every 3 minutes through 20 sets of high-definition cameras, 12 geomagnetic sensors, and a V2X network along the route. During the morning rush hour, the strategy includes dynamic speed limits (50-60 km / h on main roads and 30-40 km / h on auxiliary roads), intersection signal timing schemes (25-second green light for east-west traffic and 55-second green light for north-south traffic), temporary control of left-turn lanes (left turns prohibited from 7:50 to 8:20), and pedestrian crossing signal linkage (activated 10 seconds after the main road green light). Data shows that without this strategy, the average travel time during the morning rush hour was 47 minutes, and the queue length at intersections reached 85 meters; in the first week after implementation, the travel time was reduced to 31 minutes, and the queue length was reduced to 42 meters. For mid-level conventional vehicles, the strategy pushes information simultaneously through traffic guidance screens and navigation apps, increasing driver compliance with speed limit instructions from 61% to 79% and reducing cutting in line by 43%. Data from a certain period during the evening rush hour shows that the average waiting time for conventional vehicles at intersections decreased from 92 seconds to 54 seconds, and idling fuel consumption decreased by approximately 0.3L / vehicle. For the underlying intelligent vehicles, the strategy provides an open real-time traffic API interface, enabling them to predict right-of-way based on signal timing. In actual operation, the throughput of intelligent vehicles traveling through green wave zones reached 89%, 37 percentage points higher than traditional vehicles, and the standard deviation of following distance was controlled within 0.6 meters, which is more stable than the 2.8 meters of manual driving. The dynamic adjustment mechanism of the global constraint strategy (fine-tuning the timing every 15 minutes based on traffic flow) improved the overall traffic efficiency of this road segment during morning and evening peak hours by 41%, reduced the congestion index from 1.7 to 1.1, and reduced carbon emissions by approximately 0.9 tons per day, verifying its global coordination value for mixed traffic flow.

[0053] In this embodiment, the upper-level traffic control center introduces the SAC algorithm to generate a global constraint strategy, including: constructing an input state space of road conditions, traffic density, accident information, and environmental indicators, as well as an action space of signal timing, speed limit instructions, and lane control; based on the input state space and action space, the optimization objective of the SAC algorithm is defined as maximizing the weighted sum of comprehensive benefits and policy entropy, where comprehensive benefits include improved traffic efficiency, reduced energy consumption, and avoidance of safety risks; and outputting a global constraint strategy through iterative training of the policy network and the dual-Q network.

[0054] Among them, the dual-Q network iteration refers to the process in the SAC algorithm of evaluating the value of actions through two Q networks, alternately updating parameters to reduce overestimation bias, and assisting the policy network in iterative training to output a better global constraint policy.

[0055] It is understood that the embodiments of this application employ a dual-Q network iteration in the SAC algorithm, allowing the two Q networks to evaluate the value of actions and alternately update parameters, thereby avoiding the overestimation of value that is prone to occur in the training of a single Q network. This provides a more accurate value evaluation basis for the network to generate global constraint strategies, supports the upper-level traffic control center to make better decisions in complex traffic conditions, enhances the reliability and effectiveness of global constraint strategies, and improves the optimization effect of mixed traffic.

[0056] In this embodiment, the lower layer uses the DDPG algorithm to optimize the driving behavior of the intelligent vehicle, including: constructing a state space containing the intelligent vehicle's own speed, the positions of surrounding vehicles, the L1 layer global policy, and the F1 layer behavior prediction, with acceleration, steering angle, and path selection as the continuous action space; based on the state space and action space, through an Actor-Critic dual network structure, the Actor network outputs a deterministic driving policy, the Critic network evaluates the value of the policy, and the network parameters are iteratively updated based on the experience replay pool to optimize the cooperative driving and safe obstacle avoidance behavior.

[0057] Among them, deterministic driving strategy refers to a determinate driving behavior scheme based on the state space of the intelligent vehicle itself, such as its own speed and the positions of surrounding vehicles, with acceleration, steering angle and path selection as continuous actions.

[0058] It should be noted that strategy value refers to the assessment result of the benefits that the intelligent vehicle can bring in its current state.

[0059] It is understood that the embodiments of this application, through the explicit driving behavior scheme output by the ctor network, determine continuous actions such as acceleration and steering angle based on the intelligent vehicle's own and surrounding conditions. This avoids the problems of unstable driving behavior and poor coordination caused by ambiguous or random actions, providing precise behavioral guidance for intelligent vehicles to achieve cooperative driving and safe obstacle avoidance. It supports the lower-level intelligent vehicles to efficiently respond to the influence of upper-level strategies and mid-level behaviors in mixed traffic environments. Through the evaluation results of the benefits of deterministic driving strategies by the Critic network, the value of the strategy in terms of traffic efficiency and safety is reflected. This avoids deviations in the direction of intelligent vehicle behavior optimization due to the lack of effective evaluation of the strategy, provides a basis for the iterative update of the Actor network, helps intelligent vehicles to continuously optimize driving strategies, and enhances the rationality and adaptability of lower-level intelligent vehicles in dynamic traffic environments.

[0060] For example, during the morning rush hour on a 10-kilometer stretch of expressway, a convoy of 50 intelligent vehicles needs to cope with dense traffic and frequent acceleration and deceleration. The deterministic driving strategy output by the Actor network is based on real-time data collected on its own speed (60-80 km / h), distance to vehicles in front and behind (50-80 meters), lane position, etc. It specifies the acceleration (-2 m / s² to 1.5 m / s²), steering angle (-3° to 3°), and lane keeping instructions every 0.5 seconds. For example, when a vehicle decelerates 100 meters ahead, the strategy will immediately generate a deceleration instruction of 0.8 m / s² and maintain the lane, avoiding the problem of intelligent vehicles braking suddenly or deviating from the lane under traditional random strategies. This keeps the convoy's following error within ±3 meters, an improvement of 62.5% compared to ±8 meters without a clear strategy. Meanwhile, the Critic network evaluates the value of the strategy in real time, combining indicators such as traffic efficiency (the ratio of current road speed to the speed limit), safety (the rate of achieving safe following distances), and energy consumption (energy consumption per unit distance). When the strategy enables the intelligent vehicle to maintain a safe following distance of 50 meters at a speed of 70 km / h, the evaluated strategy value is 85 points (out of 100). However, if the strategy reduces the following distance to 30 meters, the value drops to 52 points. This quantitative evaluation provides a basis for the Actor network iteration—when the value is below 60 points, the network automatically adjusts the acceleration parameters to reduce rapid acceleration operations. Actual operation data shows that after applying this mechanism, the average traffic speed of the intelligent vehicle platoon increased by 18%, from 55 km / h to 65 km / h, the risk of rear-end collisions decreased by 70%, and the energy consumption per 100 kilometers of a single vehicle decreased by 12%. This fully demonstrates the synergistic effect of deterministic driving strategies in providing precise behavioral guidance for intelligent vehicles, ensuring the value of the strategy, and continuously optimizing the strategy, effectively improving the driving stability and overall efficiency of intelligent vehicles in mixed traffic environments.

[0061] In this embodiment of the application, the SAC algorithm formula is as follows:

[0062]

[0063]

[0064]

[0065]

[0066] in, Optimal strategy; strategy Maximize the operation; The current point in time; Total number of time steps; For mathematical expectation; At time step The state; At time step The actions taken at that time; For strategy The probability distribution of the induced state-action pairs; For instant rewards; For temperature parameters; For strategy In a given state Entropy below; For policy networks; For strategy In a given state The action produced below Follows a normal distribution; For parameters A definite value function; For action-value function; It is the natural logarithm; For parameters; For strategy Regarding parameters The objective function; Data distribution in the experience playback buffer; The optimal control strategy for leader L1; The objective function for the leader; The optimal response for follower F1; This is the optimal response for follower F2.

[0067] It is understood that the embodiments of this application introduce the SAC algorithm into the upper-level traffic control center to construct an input state space containing road conditions, traffic density, accident information, and environmental indicators, as well as an action space for signal timing, speed limit instructions, and lane control. The optimization objective is to maximize the weighted sum of comprehensive benefits and policy entropy. The global constraint policy is output through iterative training of the policy network and the dual-Q network. This avoids the problems of policy rigidity and slow convergence caused by the inability of traditional algorithms to handle high-dimensional, continuous, and nonlinear traffic conditions. It provides algorithmic support for the upper layer to generate dynamic and efficient global constraint policies, supports the optimization of mixed traffic flow in terms of traffic efficiency, safety, energy consumption, and carbon emissions, and enhances the upper layer's ability to explore strategies and optimize globally in complex traffic environments.

[0068] For example, such as Figure 4 As shown, taking a crossroads during the morning and evening rush hours as an example, this intersection frequently experiences congestion due to the mixing of drivers' vehicles and intelligent vehicles. Traditional fixed signal timing strategies are unable to dynamically adapt to changes in traffic flow, resulting in low traffic efficiency and frequent minor accidents. To address this, the SAC algorithm is applied to optimize the decision-making layer of the traffic control center at this intersection. The input state space includes multi-dimensional data collected through cameras, RSUs, and V2X terminals: east-west and north-south traffic flow density (such as the real-time number of vehicles per lane), average vehicle speed (average 20 km / h during morning and evening rush hours), congestion index (0-10 levels, often reaching level 8 during peak hours), current traffic light status, and accident alarm information within 500 meters (such as a minor accident occurring at 7:50 AM on the east side). The action space includes: traffic signal cycle allocation (adjusted from a fixed 90 seconds to a dynamic range of 40-120 seconds), green light duration ratio for each direction (40% for northbound during morning rush hour), temporary speed limit instructions (30 km / h speed limit on main roads during congestion), and temporary adjustment of left-turn lanes (converting a straight lane into a left-turn lane during peak hours). The core optimization of the SAC algorithm aims to maximize the weighted sum of comprehensive benefits and policy entropy. The policy network generates dynamic policies (e.g., extending the green light to 50 seconds when the northbound traffic density reaches 15 vehicles per lane between 7:30 and 8:30). The dual-Q network evaluates the policy value (e.g., evaluating the traffic efficiency gain from extending the northbound green light during this period). The algorithm integrates acceleration and deceleration feedback from mid-level drivers (e.g., adjusting the green light flashing duration during peak hours when traditional vehicles are overtaking) and obstacle avoidance data from lower-level intelligent vehicles (e.g., intelligent vehicle queue throughput) through an experience replay pool, iteratively updating the network parameters. Output and feedback: The traffic control center outputs signal timing and speed limit instructions every 5 minutes, which improves traffic efficiency by 35% during peak hours, reduces the accident rate by 60%, and reduces energy consumption by 20%, fully demonstrating the advantages of the SAC algorithm in balancing global optimization and policy adaptability in dynamic mixed traffic scenarios.

[0069] In this embodiment of the application, the DDPG algorithm formula is as follows:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] in, Optimal strategy; strategy Maximize the operation; The current point in time; Total number of time steps; For state distribution Expectations; Discount factor; These are indicators of average vehicle speed and intersection traffic efficiency. At time step The state; At time step The actions taken at that time; For action Strategy According to the status produce; Let be the loss function of the network; For instant rewards; For action-value function; For parameters; The target value; For target strategy; For network parameters The gradient; For action; For the second-level follower F2, this is the policy variable; The optimal strategy for the first-level leader; The objective function for the first-level leader; As a variable; In order to be in Find the value of the variable that makes the function reach its maximum value. The optimal response for follower F1.

[0076] It is understood that the embodiments of this application employ the DDPG algorithm in the lower-level intelligent vehicle to construct a state space that includes the intelligent vehicle's own speed, the positions of surrounding vehicles, the upper-level global strategy, and the middle-level behavior prediction. Acceleration, steering angle, and path selection are used as continuous action spaces. With the help of the Actor-Critic dual network structure, the Actor network outputs a deterministic driving strategy, and the Critic network evaluates the value of the strategy. Based on the experience replay pool, the network parameters are iteratively updated to optimize cooperative driving and safe obstacle avoidance behavior. This avoids problems such as improper following distance, slow obstacle avoidance response, and inefficient path selection caused by the lack of a precise dynamic adjustment mechanism in the intelligent vehicle's continuous action decision-making. It provides real-time and accurate driving decision support for the intelligent vehicle to respond to the upper-level strategy and deal with the vehicle behavior of the middle-level driver in complex mixed traffic environments. It supports the optimization of mixed traffic flow in terms of traffic efficiency, safety, energy consumption, and carbon emissions, and enhances the autonomous decision-making accuracy and local cooperative capability of the intelligent vehicle in dynamic interaction scenarios.

[0077] For example, such as Figure 5As shown, during the morning and evening rush hours in the city, at the intersection of main roads and auxiliary roads, a large number of drivers (middle level F1) and intelligent vehicles (lower level F2) frequently mix, leading to problems such as intelligent vehicles following too closely and failing to avoid obstacles in time due to their inability to accurately respond to sudden lane changes and emergency braking by F1 vehicles. Traditional intelligent vehicle control algorithms cannot efficiently handle continuous action spaces and dynamic game interactions, resulting in low traffic efficiency and safety hazards. Therefore, the DDPG algorithm is applied to optimize the driving behavior of intelligent vehicles on this road section: the state space of intelligent vehicles includes their own real-time speed (e.g., 30-60km / h) and the distance to vehicles in front and behind (e.g., 5-20 meters). The algorithm considers the positions and historical driving trajectories of surrounding F1 vehicles (used to predict lane change intentions), speed limit commands (e.g., 40 km / h) and lane priority planning issued by the upper-level traffic control center (L1) via the SAC algorithm, and sets the action space as continuous acceleration (-3 m / s² to 2 m / s²), steering angle (-15° to 15°), and path selection (e.g., whether to switch to an auxiliary lane). The DDPG algorithm operates through an Actor-Critic dual-network structure. The Actor network outputs a deterministic driving strategy based on the current state. For example, when it detects that the F1 vehicle in front suddenly decelerates (speed drops from 40 km / h to 20 km / h), it immediately outputs... The braking command, with an acceleration of -2 m / s², is evaluated by the Critic network based on historical data from an experience replay pool (such as braking effects and following safety in similar scenarios). If it detects that excessive braking might cause a rear-end collision, it sends an adjustment signal to the Actor network, optimizing the acceleration to -1.5 m / s² and fine-tuning the steering angle by 5° to maintain a safe distance from vehicles in adjacent lanes. During actual operation in the morning rush hour (7:00-9:00), when an F1 vehicle suddenly changes lanes continuously from the main road to the auxiliary road, the intelligent vehicle responds quickly using the DDPG algorithm: first predicting the lane-changing intention based on the F1 vehicle's trajectory, then the Actor network... The network outputs a steering angle of -10° and an acceleration of -1m / s², which avoids collisions and prevents subsequent vehicles from being affected by sudden braking. When the upper-level L1 issues an instruction to prioritize the auxiliary road, the intelligent vehicle switches to the auxiliary road through path selection actions while ensuring safety. The average travel time is reduced by 15%. According to statistics, the number of collisions between intelligent vehicles and F1 vehicles on this section of the road has decreased by 70%, and the proportion of following distances maintained within a safe range (8-12 meters) has increased to 90%. This fully demonstrates the advantages of the DDPG algorithm in providing accurate and real-time driving decisions for intelligent vehicles in handling continuous actions and dynamic game scenarios, and effectively supports the optimization of safety and traffic efficiency in mixed traffic flow.

[0078] In step S104, based on the global constraint strategy and the optimization results of intelligent vehicle driving behavior, the traffic efficiency, safety, energy consumption and carbon emissions of mixed traffic flow are optimized. The traffic status is continuously updated through the behavioral feedback data of the middle and lower layers, and the strategy is optimized by combining the SAC and DDPG models.

[0079] It is understood that the embodiments of this application optimize the traffic efficiency, safety, energy consumption and carbon emissions of mixed traffic flow through global constraint strategies and intelligent vehicle driving behavior optimization results. At the same time, it continuously updates the traffic status with the help of mid-level and lower-level behavioral feedback data, and optimizes the strategy by combining SAC and DDPG models. It covers dynamic traffic flow control, vehicle conflict avoidance, energy consumption monitoring and other aspects, avoiding the problem of optimization effect decay and insufficient adaptability caused by the disconnect between strategy and actual traffic status. It provides a complete closed-loop mechanism for mixed traffic from decision-making to execution to feedback iteration, supports the continuous improvement of mixed traffic flow in multi-dimensional indicators, and enhances the ability to accurately adapt to and optimize the dynamic traffic environment in the long term.

[0080] The traffic optimization method based on a three-layer Stackelberg game proposed in this application collects mixed traffic data, constructs a three-layer Stackelberg game model, uses deep reinforcement learning algorithms to solve for equilibrium solutions, and continuously optimizes the model by combining global constraint strategies and intelligent vehicle driving behavior optimization results. This enables hierarchical precise control and dynamic coordination of mixed traffic flows. The upper layer introduces the SAC algorithm to generate global constraint strategies, while the lower layer uses the DDPG algorithm to optimize intelligent vehicle driving behavior. Traffic status is updated by integrating behavioral feedback data from the middle and lower layers, improving the multi-level collaborative control performance and adaptive decision-making capabilities in complex mixed traffic environments. Based on the combination of hierarchical game theory and deep reinforcement learning, this method overcomes the bottlenecks of traditional static rules or centralized strategies in multi-agent interaction processing and dynamic adaptability, improving the efficiency, safety, and energy consumption and carbon emission optimization effects of mixed traffic flows. It can adapt to complex scenarios where drivers and intelligent vehicles coexist. Therefore, it solves the problems of insufficient dynamic adaptability, inefficient subject interaction processing, and limitations of game models in existing technologies.

[0081] The following will illustrate the traffic optimization method based on a three-layer Stackelberg game through a specific embodiment, such as... Figure 6 As shown, it includes: At a major intersection in the heart of a city, connecting commercial and residential areas, traffic flow is extremely heavy during morning and evening rush hours. The intersection comprises approximately 30% intelligent vehicles and 70% driver-driven vehicles, resulting in long-standing problems of low traffic efficiency and high accident rates. To improve this situation, a comprehensive traffic perception system was first constructed by deploying eight high-definition cameras (sampling frequency 10Hz), four roadside units (RSU) (communication range 500 meters), twelve geomagnetic sensors, and V2X terminals mounted on intelligent vehicles around the intersection. Traffic data collected continuously for one week yielded mixed traffic data, including: during the morning rush hour (7:30-9:00), the eastbound straight-ahead traffic density reached 25 vehicles / 100 meters, the westbound left-turning traffic density was 18 vehicles / 100 meters, and the average north-south speed was only 18 km / h; during the evening rush hour (17:30-19:30), the average queue length for southbound straight-ahead vehicles reached 80 meters, and northbound left-turning vehicles waited an average of 2 traffic light cycles; the congestion index reached 9.2 (out of 10) at the peak of the morning rush hour and 8.8 at the peak of the evening rush hour; the traffic lights initially had fixed timings, with 45 seconds of green light and 55 seconds of red light for east-west traffic, and 3 seconds of green light for north-south traffic. A 5-second red light and a 65-second red light result in an average of 35 eastbound vehicles per hour being stuck at intersections during the morning rush hour due to insufficient green light time. The intelligent vehicle's real-time location accuracy is controlled within ±0.5 meters, and its onboard sensors can upload real-time acceleration (range -3.5m / s² to 2.8m / s²) and energy consumption data (average 8.2kWh per 100km). Historical lane-change data from drivers shows approximately 20 irregular lane changes per hour during the morning rush hour, 12 of which are consecutive lane changes, highly likely to cause minor collisions. Furthermore, the average driver reaction time to sudden situations is 0.9 seconds, increasing to 1.3 seconds in rainy weather. This data, after being fused and processed by edge computing nodes, forms a traffic state vector containing 280 dimensions of features, providing accurate input for subsequent model construction and algorithm optimization, ensuring that the model fully reflects the complexity of real-world traffic scenarios.

[0082] Based on the collected data, a three-layer Stackelberg game model was constructed. The upper-level traffic control center, as the leader, integrates data from cameras, RSUs, and geomagnetic sensors to monitor the global traffic status in real time. For example, if a minor collision occurs eastbound at 8:10 AM during the morning rush hour, the congestion index for that direction rises from 7.1 to 8.5 within 3 minutes and quickly spreads to 3 intersections within a 1-kilometer radius, resulting in a 40% decrease in regional traffic efficiency. Its core function is to generate dynamic global constraint strategies, including traffic signal cycle allocation (adjustment range 40-120 seconds), green light percentage for each direction (20%-60%), temporary speed limit instructions (20-60 km / h), and the number of left-turn lanes (1-2 lanes). Mid-level drivers, acting as the first follower, have an average reaction delay of 0.7 seconds to traffic lights, according to historical data. As the green light is about to end, about 35% of drivers will choose to accelerate and rush through, with an average speed 15% faster than normal traffic. Influenced by driving habits, their following distance fluctuates greatly (3-15 meters), with novice drivers often following less than 5 meters, increasing the risk of rear-end collisions. Vehicles at this level cannot actively participate in strategy negotiation and can only react passively based on the strategies of the upper level and the behavior of surrounding vehicles. Their behavioral data (such as lane change frequency and braking intensity) are uploaded to the upper level in real time through the vehicle terminal. The lower-level intelligent vehicle, acting as a second follower, receives real-time behavioral data from the upper-level strategies and middle-level vehicles via V2X terminals. Its onboard LiDAR (detection range of 200 meters) can accurately perceive the dynamics of vehicles within a 30-meter radius, with a ranging accuracy of ±0.1 meters and an emergency obstacle avoidance response capability within 0.1 seconds. This layer can autonomously optimize driving behavior based on upper-level strategies and middle-level behavioral predictions, with a decision-making cycle of 0.2 seconds, ensuring rapid response to changes in the traffic environment. A clear interaction relationship is formed among the three layers: the upper layer constrains the behavior of the middle and lower layers through signal timing and lane planning. For example, when the upper layer extends the green light for eastbound traffic to 60 seconds, the passage rhythm of vehicles in the middle and lower layers will be accelerated accordingly. The behavior of the middle layer, such as rushing or sudden braking, will directly affect the obstacle avoidance decisions of the lower layer intelligent vehicles. For example, when a middle layer vehicle suddenly brakes, the lower layer intelligent vehicle needs to make a braking or lane change reaction within 0.3 seconds. The cooperative driving data of the lower layer (such as average following distance and passage speed) is fed back to the upper layer in real time to adjust the global strategy, forming a closed-loop mechanism of "decision-execution-feedback".

[0083] A deep reinforcement learning algorithm is used to solve the equilibrium solution of the three-layer model. The upper-level traffic control center introduces the SAC algorithm, constructing an input state space that includes traffic flow density, average vehicle speed, congestion index, accident alarm signals, and environmental carbon emission data (CO2 emissions per unit hour) in the east, west, south, and north directions. The carbon emission data is estimated using energy consumption information uploaded by intelligent vehicles and the average fuel consumption of traditional vehicles (originally 120 kg / h). The action space includes traffic signal cycles (dynamically adjusted from 40 to 120 seconds), green light percentages for each direction (20%-60%), temporary speed limit instructions (20-60 km / h), and temporary left-turn lane addition strategies. The optimization objective of the SAC algorithm is set to maximize the weighted sum of comprehensive benefits (traffic efficiency improvement weight 0.4, energy consumption reduction weight 0.3, and safety risk avoidance weight 0.3) and policy entropy (temperature parameter α=0.2). It outputs a dynamic global constraint policy through iterative training using a policy network and a dual-Q network (5000 training cycles, 1000 state-action samples per cycle). For example, when the eastbound traffic density exceeds 20 vehicles / 100 meters during the morning rush hour, the SAC algorithm extends the eastbound green light duration from the initial 45 seconds to 60 seconds, while simultaneously issuing a 40 km / h speed limit instruction for westbound vehicles to reduce the interference of westbound traffic on eastbound traffic. When rainy road conditions are detected, the algorithm automatically shortens the green light flashing time in all directions (from 3 seconds to 2 seconds) and lowers the speed limit instruction to 30 km / h to accommodate increased driver reaction delays. The lower-level intelligent vehicle employs the DDPG algorithm. Its state space includes its own speed (20-50 km / h), the real-time positions of the four vehicles in front, behind, to the left, and to the right (accuracy ±0.3 meters), the speed limit command generated by the upper-level SAC algorithm, and the lane change probability prediction of the middle-level driver vehicle (based on an LSTM model trained on historical data, with an accuracy of 82%). Its action space consists of continuous acceleration (-4 m / s² to 3 m / s²), steering angle (-20° to 20°), and path selection (main road / auxiliary road switching). Through an Actor-Critic dual-network structure (the Actor network uses a 3-layer fully connected neural network with 256 hidden nodes, and the Critic network uses a dual-network structure to reduce overestimation), based on an experience replay pool (capacity 10...), it utilizes... 6 The algorithm iteratively updates parameters (based on a data set) to optimize driving behavior. For example, when it detects that the vehicle in front suddenly brakes at -3m / s², the DDPG algorithm will output an acceleration of -2.5m / s² within 0.1 seconds, combined with a 5° steering angle, to maintain a safe following distance of 10 meters. When it predicts that the vehicle in the right lane has a 70% probability of changing lanes, the algorithm will reduce the vehicle speed by 5km / h in advance and fine-tune the steering angle to 3° to leave enough space for avoidance.

[0084] Based on the upper-level global constraint strategy and the lower-level intelligent vehicle driving behavior optimization results, the mixed traffic flow indicators at this intersection have been significantly optimized. In terms of traffic efficiency, the average travel time during the morning peak has been shortened from 12 minutes to 7 minutes, with the most significant reduction in travel time for eastbound vehicles, from 15 minutes to 6 minutes; the number of vehicles passing through the intersection per hour has increased from 580 to 820; the queue length for southbound straight-through vehicles during the evening peak has decreased from 80 meters to 30 meters; and the average waiting time for traffic lights has decreased from 2 to 1 cycle; the average vehicle speed in each direction has significantly improved, with the north-south direction increasing from 18 km / h to 32 km / h and the east-west direction increasing from 22 km / h to 38 km / h. In terms of safety, the number of minor vehicle collisions decreased from 15 per week to 3, with accidents caused by drivers cutting in front decreasing from 10 to 1. The safe distance (8-15 meters) maintained between intelligent vehicles and drivers increased from 65% to 92%, and this percentage remained above 85% even in rainy weather. The number of emergency braking incidents decreased by 60%, and the braking intensity was significantly reduced, with the average acceleration increasing from -3.2 m / s² to -2.1 m / s², reducing the impact on following vehicles. Regarding energy consumption and carbon emissions, the average energy consumption per 100 kilometers for intelligent vehicles decreased from 8.2 kWh to 6.5 kWh, achieving an energy saving rate of 20.7%. Hourly CO2 emissions around intersections decreased from 120 kg to 85 kg, with the reduction in carbon emissions from traditional fuel vehicles mainly due to the reduced idling time resulting from improved traffic efficiency (from an average of 8 minutes / vehicle to 3 minutes / vehicle). Meanwhile, the traffic state matrix is ​​updated every 5 minutes using real-time behavioral data from mid-level drivers (e.g., reducing the frequency of overtaking from 20 times / hour to 8 times / hour, and increasing the average following distance from 6.2 meters to 9.5 meters) and driving trajectories and energy consumption data from lower-level intelligent vehicles. Strategy optimization is performed by combining SAC and DDPG models. For example, when overtaking behavior in the mid-level increases by 15% during a certain period, the SAC algorithm shortens the green light flashing time for that direction (from 3 seconds to 2 seconds), while the DDPG algorithm adjusts the following distance threshold for intelligent vehicles (from 10 meters to 12 meters). When it is detected that the efficiency of intelligent vehicles on auxiliary roads is 15% higher than that on main roads, the SAC algorithm temporarily increases the green light percentage on auxiliary roads from 25% to 40%, guiding some intelligent vehicles to switch to auxiliary roads and relieving pressure on main roads. This dynamic adaptive optimization loop ensures that the entire traffic system maintains a high-efficiency, safe, and low-consumption operation in a complex and ever-changing mixed traffic environment, fully validating the effectiveness of the mixed traffic optimization method based on a three-layer Stackelberg deep game.

[0085] In summary, this invention collects comprehensive traffic data (cameras, RSUs, geomagnetic sensors, etc.) from multiple sources of traffic perception to construct a high-dimensional dynamic traffic state vector, providing accurate input for a three-layer game theory model and comprehensively reflecting the real-time characteristics and interaction patterns of mixed traffic flows. The three-layer Stackelberg game theory model clarifies the hierarchical relationship and strategy interaction mechanism between the traffic control center, drivers / vehicles, and intelligent vehicles, achieving precise coupling between global control and local response, thus solving the problems of decision coupling and response lag in traditional control. The SAC and DDPG algorithms are optimized collaboratively. The upper layer generates dynamic global constraint strategies through SAC, balancing traffic efficiency, safety, and energy consumption optimization objectives, and outputting optimal control schemes such as signal timing and speed limit commands. The lower layer optimizes the continuous action decisions of intelligent vehicles through DDPG, achieving cooperative driving and safe obstacle avoidance, and improving the adaptive capability of intelligent vehicles to complex traffic scenarios. The closed-loop optimization mechanism dynamically updates the traffic state based on the behavioral feedback of the middle and lower layers, driving the continuous iteration of the SAC and DDPG models to ensure real-time adaptation of the strategy to the actual traffic environment. By integrating multi-level strategy collaboration with deep reinforcement learning, the efficiency, safety, and low-carbon characteristics of mixed transportation are significantly improved, providing an adaptive decision-making solution for dynamic traffic management in complex scenarios and supporting the efficient and intelligent upgrade of intelligent transportation.

[0086] Next, referring to the accompanying drawings, a traffic optimization system based on a three-layer Stackelberg game, according to an embodiment of this application, is described.

[0087] Figure 7 This is a block diagram of a traffic optimization system based on a three-layer Stackelberg game according to an embodiment of this application.

[0088] like Figure 7 As shown, the hybrid traffic optimization system 10 based on a three-layer Stackelberg game includes: an acquisition module 100, a construction module 200, a generation module 300, and an optimization module 400.

[0089] The system comprises the following modules: Acquisition module 100, which acquires mixed traffic data, including road network traffic conditions such as traffic density, congestion index, average vehicle speed, and traffic light status, collected through traffic perception via cameras, RSUs, and V2X terminals; Construction module 200, which constructs a three-layer Stackelberg game model based on the mixed traffic data, where the upper layer is the traffic control center decision layer, the middle layer is the driver-following layer, and the lower layer is the intelligent vehicle response layer; Generation module 300, which solves the equilibrium solution of the three-layer Stackelberg game model using deep reinforcement learning algorithms, with the upper layer incorporating the SAC algorithm to generate global constraint strategies such as traffic signal periodic allocation and road priority planning, and the lower layer employing the DDPG algorithm to optimize driving behaviors such as intelligent vehicle cooperative driving and obstacle avoidance; and Optimization module 400, which optimizes indicators such as traffic efficiency, safety, and energy consumption and carbon emissions of mixed traffic flow based on the global constraint strategies and the optimization results of intelligent vehicle driving behavior, continuously updating traffic conditions through behavioral feedback data from the middle and lower layers, and combining the SAC and DDPG models for strategy optimization.

[0090] It should be noted that the foregoing explanation of the traffic optimization method based on the three-layer Stackelberg game also applies to the traffic optimization system based on the three-layer Stackelberg game in this embodiment, and will not be repeated here.

[0091] According to the traffic optimization system based on a three-layer Stackelberg game proposed in this application, this embodiment collects mixed traffic data, constructs a three-layer Stackelberg game model, uses deep reinforcement learning algorithms to solve for equilibrium solutions, and continuously optimizes the system by combining global constraint strategies and intelligent vehicle driving behavior optimization results to achieve hierarchical precise control and dynamic coordination of mixed traffic flow. The upper layer introduces the SAC algorithm to generate global constraint strategies, while the lower layer uses the DDPG algorithm to optimize intelligent vehicle driving behavior. Traffic status is updated by integrating behavioral feedback data from the middle and lower layers, improving the multi-level collaborative control performance and adaptive decision-making capabilities in complex mixed traffic environments. Based on the combination of hierarchical game theory and deep reinforcement learning, this system overcomes the bottlenecks of traditional static rules or centralized strategies in multi-agent interaction processing and dynamic adaptability, improving the efficiency, safety, and energy consumption and carbon emission optimization effects of mixed traffic flow. It can adapt to complex scenarios where drivers and intelligent vehicles coexist. Therefore, it solves the problems of insufficient dynamic adaptability, inefficient subject interaction processing, and limitations of game models in existing technologies.

[0092] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0093] When the processor 802 executes the program, it implements the traffic optimization method based on the three-layer Stackelberg game provided in the above embodiments.

[0094] Furthermore, electronic devices also include: Communication interface 808 is used for communication between memory 801 and processor 802.

[0095] The memory 801 is used to store computer programs that can run on the processor 802.

[0096] The memory 801 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage device.

[0097] If the memory 801, processor 802, and communication interface 808 are implemented independently, then the communication interface 808, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0098] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 808 are integrated on a single chip, then the memory 801, processor 802, and communication interface 808 can communicate with each other through an internal interface.

[0099] The processor 802 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0100] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the traffic optimization method based on a three-layer Stackelberg game as described above.

[0101] Furthermore, this application also provides a computer program product, including a computer program or instructions, which, when executed, implement the aforementioned traffic optimization method based on a three-layer Stackelberg game.

[0102] In the description of this specification, the references to "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0104] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0105] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0106] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0107] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A traffic optimization method based on a three-layer Stackelberg game, characterized in that, include: Acquire mixed traffic data, including traffic density, congestion index, average vehicle speed, traffic light status, and road network traffic status, through traffic perception data collected by cameras, RSUs, and V2X terminals; Based on the mixed traffic data, a three-layer Stackelberg game model is constructed, wherein the upper layer is the traffic control center decision layer, the middle layer is the driver driving vehicle following layer, and the lower layer is the intelligent vehicle response layer. The equilibrium solution of the three-layer Stackelberg game model is solved by deep reinforcement learning algorithm. The upper layer introduces SAC algorithm to generate traffic signal periodic allocation and global constraint strategy for road priority planning. The lower layer uses DDPG algorithm to optimize intelligent vehicle cooperative driving and safe obstacle avoidance driving behavior. Based on the global constraint strategy and the optimization results of intelligent vehicle driving behavior, the traffic efficiency, safety and energy consumption and carbon emission indicators of mixed traffic flow are optimized. The traffic status is continuously updated through the behavioral feedback data of the middle and lower layers, and the strategy is optimized by combining the SAC and DDPG models.

2. The traffic optimization method based on a three-layer Stackelberg game as described in claim 1, characterized in that, Construct a three-layer Stackelberg game model, including: The upper-level traffic control center is set as the leader, responsible for the perception of global traffic status information and the generation of global constraint strategies. The global traffic status information includes traffic flow distribution, congested areas, and accident alarms, which are acquired in real time through cameras, geomagnetic sensors, and V2X networks. The mid-level driver is defined as the first follower, who passively reacts to environmental signals and the behavior of other vehicles based on individual preferences, experience and reaction speed. Their behavior is jointly determined by traffic signals, road conditions and driving habits. The lower-level intelligent vehicle is designated as the second follower, possessing perception, prediction, and decision-making capabilities, and optimizes autonomous driving behavior based on the influence of upper-level strategies and middle-level behaviors.

3. The traffic optimization method based on a three-layer Stackelberg game as described in claim 1, characterized in that, The upper-level traffic control center introduces the SAC algorithm to generate global constraint strategies, including: Construct an input state space for road conditions, traffic density, accident information, and environmental indicators, as well as an action space for signal timing, speed limit instructions, and lane control; Based on the input state space and the action space, the optimization objective of the SAC algorithm is defined as maximizing the weighted sum of the comprehensive benefit and the policy entropy. The comprehensive benefit includes improved traffic efficiency, reduced energy consumption, and avoidance of security risks. Through iterative training of the policy network and the dual-Q network, a global constraint policy is output.

4. The traffic optimization method based on a three-layer Stackelberg game as described in claim 1, characterized in that, The lower layer uses the DDPG algorithm to optimize the driving behavior of intelligent vehicles, including: Construct a state space that includes the speed of the intelligent vehicle itself, the position of surrounding vehicles, L1 layer global strategy and F1 layer behavior prediction, with acceleration, steering angle and path selection as the continuous action space; Based on the state space and the action space, an Actor-Critic dual-network structure is used. The Actor network outputs a deterministic driving strategy, and the Critic network evaluates the value of the strategy. The network parameters are iteratively updated based on the experience replay pool to optimize cooperative driving and safe obstacle avoidance behaviors.

5. The traffic optimization method based on a three-layer Stackelberg game as described in claim 1, characterized in that, The SAC algorithm formula is as follows: ； ； ； ； ； in, This is the optimal strategy; For strategy Maximize the operation; This refers to the current point in time. This represents the total number of time steps. For mathematical expectation; In time step The state; In time step The actions taken at that time; For strategy The probability distribution of the induced state-action pairs; For instant rewards; For temperature parameters; For strategy In a given state Entropy below; For policy networks; For strategy In a given state The action produced below Follows a normal distribution; For parameters A definite value function; For action-value function; It is the natural logarithm; For parameters; For strategy Regarding parameters The objective function; Data distribution in the experience playback buffer; The optimal control strategy for leader L1; The objective function for the leader; The optimal response for follower F1; This is the optimal response for follower F2.

6. The traffic optimization method based on a three-layer Stackelberg game as described in claim 1, characterized in that, The DDPG algorithm formula is as follows: ； ； ； ； ； ； in, Optimal strategy; strategy Maximize the operation; The current point in time; Total number of time steps; For state distribution Expectations; Discount factor; These are indicators of average vehicle speed and intersection traffic efficiency. At time step The state; At time step The actions taken at that time; For action Strategy According to the status produce; Let be the loss function of the network; For instant rewards; For action-value function; For parameters; The target value; For target strategy; For network parameters The gradient; For action; For the second-level follower F2, this is the policy variable; The optimal strategy for the first-level leader; The objective function for the first-level leader; As a variable; In order to be in Find the value of the variable that makes the function reach its maximum value. The optimal response for follower F1.

7. A traffic optimization system based on a three-layer Stackelberg game, characterized in that, include: The acquisition module is used to acquire mixed traffic data, including traffic density, congestion index, average vehicle speed, traffic light status, and road network traffic status, which are collected through traffic perception data from cameras, RSUs, and V2X terminals. The construction module is used to construct a three-layer Stackelberg game model based on the mixed traffic data, wherein the upper layer is the traffic control center decision layer, the middle layer is the driver driving vehicle following layer, and the lower layer is the intelligent vehicle response layer. The generation module is used to solve the equilibrium solution of the three-layer Stackelberg game model through deep reinforcement learning algorithm. The upper layer introduces the SAC algorithm to generate traffic signal periodic allocation and global constraint strategy for road priority planning. The lower layer uses the DDPG algorithm to optimize intelligent vehicle cooperative driving and safe obstacle avoidance driving behavior. The optimization module is used to optimize the traffic efficiency, safety, and energy consumption and carbon emission indicators of mixed traffic flow based on the global constraint strategy and the optimization results of intelligent vehicle driving behavior. It continuously updates the traffic status through behavioral feedback data from the middle and lower layers and performs strategy optimization by combining SAC and DDPG models.

8. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the traffic optimization method based on a three-layer Stackelberg game as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When a computer program or instruction is executed, it implements the traffic optimization method based on a three-layer Stackelberg game as described in any one of claims 1-6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When a computer program or instruction is executed, it implements the traffic optimization method based on a three-layer Stackelberg game as described in any one of claims 1-6.

Images