A multi-scale adaptive cooperative traffic signal control method

Abstract

The application relates to a multi-scale adaptive cooperative traffic signal control method, and relates to the technical field of traffic signal control.The application is aimed at the problem that the existing red light time is too long, leading to low traffic efficiency.The application first utilizes an SAC module and combines real-time traffic conditions to obtain the green light time of each phase, thereby realizing real-time control of the signal light.Then, the application utilizes a DQN module and combines the intention (the intention includes straight driving, left turning and right turning, in the application, only left turning and straight driving involve route changing) of the vehicle in each entrance to judge whether the vehicle is to change the route, wherein the route changing includes that the vehicle with the straight driving intention changes the route to right turn, makes a U-turn, and then right turns to enter the opposite straight driving lane; the vehicle with the left turning intention changes the route to right turn, makes a U-turn, and then straight drives to enter the opposite straight driving lane.Through the technical scheme of the application, the green light time of the signal light can be dynamically adjusted, and the traffic efficiency is greatly improved through the route changing operation.

Description

Technical Field

[0001] This application relates to the field of traffic signal control technology, specifically a multi-scale adaptive cooperative traffic signal control method. Background Technology

[0002] With the continuous growth of urban motor vehicle ownership, intersection traffic congestion has become a significant factor restricting road network efficiency. In existing traffic signal control systems, signal timing schemes are mostly pre-set based on historical traffic data or simply adjusted according to local traffic flow, generally resulting in unreasonable red light phase durations and excessively long red light times. It is precisely because of these excessively long red light durations that vehicles wait for extended periods, leading to low traffic efficiency. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-scale adaptive cooperative traffic signal control method to address the problem of excessively long red light times leading to low traffic efficiency.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0005] A multi-scale adaptive cooperative traffic signal control method includes the following steps:

[0006] The system acquires real-time traffic conditions at intersections and inputs these conditions into the intelligent agent to enable traffic signal control.

[0007] The intelligent agent includes a SAC module and a DQN module. The SAC module obtains the green light duration of the traffic lights for each approach lane based on the real-time traffic conditions at the intersection. The DQN module is used to determine whether vehicles that do not intend to turn right at each approach lane of the intersection need to change lanes. If they do not change lanes, they will proceed according to their original intention. If they do change lanes, they will enter the corresponding approach lane after changing lanes.

[0008] The rerouting specifically includes:

[0009] If the vehicle intended to turn left, after changing lanes, the vehicle will enter the right turn lane, and after turning right, making a U-turn, and going straight, it will enter the lane in the original left-turn direction.

[0010] If the vehicle intends to go straight, after changing lanes, the vehicle will enter the right-turn lane, and after turning right-turning-making a U-turn, it will enter the lane in the original direction of its straight-going intention.

[0011] Furthermore, the SAC module specifically performs the following steps:

[0012] Step 1: At each decision moment Obtain the real-time traffic conditions at the intersection and use them as a state representation. , represented as:

[0013] ,

[0014] in, One-hot encoding of the phase of the currently active signal. The number of signal phases. The normalized waiting time ratio for each entrance lane. Number each entrance lane. The maximum queuing occupancy rate for each entrance lane. This represents the proportion of vehicles in each approach lane to the total number of vehicles at the intersection. One-hot encoding for the highest collision risk among all signal phases;

[0015] Step 2: Represent the state Inputting the pre-trained SAC network yields a two-dimensional continuous action vector as output. , represented as:

[0016] ,

[0017] in, The duration of the green light for the current phase. For signal phase;

[0018] Step 3: Based on the two-dimensional continuous action vector Traffic signal control is achieved by adjusting the timing parameters of the signal controller. These timing parameters include phase selection and green light duration. Phase selection is expressed as follows:

[0019] ,

[0020] in, The number of feasible green light phases. To select the phase number,

[0021] The duration of a green light is expressed as follows:

[0022] ,

[0023] in, and These are the minimum and maximum green light durations.

[0024] Furthermore, the DQN module specifically performs the following steps:

[0025] Step 21: At each decision time t, obtain the real-time traffic conditions at the intersection and use them as the system state. , represented as:

[0026] ,

[0027] in, The proportion of heavy-duty trucks entering the road; The remaining capacity of the route after the diversion;

[0028] Step 22: Set system status Input the pre-trained DQN network into the action space Select the redirection action to output. , represented as:

[0029] ,

[0030] ,

[0031] ,

[0032] in, and These are the minimum and maximum speeds for the road segment. To save time, For the minimum acceptable time savings, For space occupancy rate, For vehicles Distance from the parking line For vehicle location threshold, The minimum remaining capacity required for the route. For vehicles The total length of the road currently being traveled.

[0033] Furthermore, the time savings Represented as:

[0034] ,

[0035] in, For vehicles Based on the estimated travel time for the current route, For vehicles Estimated travel time for the rerouted route.

[0036] Furthermore, the minimum speed of the aforementioned road section The maximum speed of the road segment is a fixed value. For dynamic values, where,

[0037] If the vehicle is not an autonomous vehicle, the maximum speed of the road segment is... Represented as:

[0038] ,

[0039] ,

[0040] in, and The congestion threshold , , These represent vehicle speeds under smooth traffic, moderate congestion, and congested conditions, respectively. For lane Normalized queue occupancy rate The number of stopped vehicles in the lane. For lane Length,

[0041] If the vehicle is an autonomous vehicle, the maximum speed on the road segment is... Represented as:

[0042] ,

[0043] in, The saturation threshold, For the rated free-flow speed, This is the minimum permissible speed.

[0044] Furthermore, the reward function of the intelligent agent Represented as:

[0045] ,

[0046] in, This is an extreme queuing penalty item. For the overall queuing reward items, For the smallest parking reward item, This is a penalty item for accident risk.

[0047] Furthermore, the extreme queuing penalty item Represented as:

[0048] ,

[0049] ,

[0050] ,

[0051] in, For the Sigmoid function, For the maximum queuing occupancy rate in each direction, It belongs to the import channel The set consisting of all lanes, For import channels Lane-level queuing occupancy.

[0052] Furthermore, the overall queuing reward item Represented as:

[0053] ,

[0054] in, The average occupancy rate in each direction.

[0055] Furthermore, the minimum parking reward item Represented as:

[0056] ,

[0057] ,

[0058] in, for Number of parking spaces at any given time for The total number of vehicles at any given time For vehicles Instantaneous velocity.

[0059] Furthermore, the aforementioned accident risk penalty item Represented as:

[0060] ,

[0061] ,

[0062] in, This is an ordered sequence of accident risk values ​​for all intersection approach lanes. For the number of import lanes,

[0063] This is a function for sorting in descending order. To return the given In sequence Functions with ordinal numbers, A well-known function that takes the index corresponding to the maximum value. The VaR value is the intersection scale. The VaR value is the inlet gauge dimension.

[0064] The beneficial effects of this invention are:

[0065] This application first utilizes the SAC module, combined with real-time traffic conditions, to obtain the green light duration for each phase, achieving real-time control of the traffic lights. Then, this application uses the DQN module, combined with the intentions of vehicles in each approach lane (intentions include going straight, turning left, and turning right; in this application, only left turns and going straight involve lane changing), to determine whether a vehicle should change lanes. Lane changing includes vehicles intending to go straight changing to a right turn, then making a U-turn, and then turning right again to enter the oncoming straight lane. Vehicles intending to turn left change to a right turn, then making a U-turn, and then going straight to enter the oncoming straight lane. Through this technical solution, the green light duration of the traffic lights can be dynamically adjusted, and through lane changing operations, traffic efficiency is greatly improved. Attached Figure Description

[0066] Figure 1 This is the overall flowchart of the intelligent agent in this application; Figure 2 This is a schematic diagram of the iterative convergence process of the intelligent agent in this application; Figure 3 This is a schematic diagram comparing the intelligent agent of this application with the benchmark method in terms of safety, efficiency, and fuel consumption. Figure 4 This is a schematic diagram comparing the efficiency and fuel consumption of the intelligent agent of this application and the benchmark method at intersection 1. Figure 5 This is a schematic diagram comparing the efficiency and fuel consumption of the intelligent agent of this application and the benchmark method at intersection 2. Figure 6 This is a schematic diagram comparing the TTAC, VaR, and acceleration variance distribution of the intelligent agent of this application and the benchmark method in intersection 1. Figure 7 This is a schematic diagram comparing the TTAC, VaR, and acceleration variance distributions of the intelligent agent of this application and the benchmark method in intersection 2. Detailed Implementation

[0067] It should be noted that, where there is no conflict, the various embodiments disclosed in this application can be combined with each other.

[0068] Specific Implementation Method 1: The multi-scale adaptive cooperative traffic signal control method described in this implementation method includes:

[0069] To collaboratively optimize traffic efficiency and safety, this application proposes a Multi-Scale Adaptive Cooperative Traffic Signal Control (MSA-CoTSC) framework. Its core idea is to establish a hierarchical cooperative architecture, enabling different control agents to operate at complementary scales. Specifically: the Soft Actor-Critic (SAC) agent governs the macro-level traffic signal phase and timing, ensuring balanced traffic flow distribution at each approach lane and providing a stable cooperative environment for lower-level vehicle control; the Deep Q-Network (DQN) agent implements meso-level traffic intervention measures (i.e., dynamic speed limits and path planning) to optimize vehicle operation under a given signal scheme; furthermore, the presence of connected autonomous vehicles (CAVs) enables micro-level platoon control through car-following and platooning mechanisms, further smoothing traffic flow and improving vehicle-level safety. These three layers of control operate synchronously under the same environment and reward mechanism, forming a rich and integrated cooperative control paradigm that integrates traffic signal control, traffic intervention measures, and vehicle-level collaboration.

[0070] Adaptive traffic signal control aims to dynamically adjust signal phase and timing based on real-time traffic conditions. Traditional timing control or inductive control schemes lack multi-objective optimization capabilities under dynamic demands, while reinforcement learning (RL)-based methods provide a data-driven adaptive decision-making paradigm. This application constructs the traffic signal control problem as a Markov decision process and optimizes it using the soft actor-commentator (SAC) algorithm. This design enables the controller to clearly balance traffic efficiency and safety through state representation and reward mechanisms. The SAC algorithm proposed in this application has three core advantages in adaptive traffic signal control: (1) it can directly optimize the green light duration, which is impossible with purely discrete methods; (2) the alternative safety indicators (SSMs) embedded in the state and reward can guide the strategy to avoid high-risk (i.e., high collision risk) phase selection; and (3) the entropy regularization mechanism improves the stability of the strategy under fluctuating and uncertain traffic demands. This section will describe the state representation, action space, control objectives, and learning mechanism of the SAC-based adaptive traffic signal control method.

[0071] First, the Soft Actor-Critic (SAC) algorithm requires real-time traffic conditions at signalized intersections as a state representation. Traffic conditions directly reflect the efficiency and stability of traffic flow. In this application, the SAC state representation includes five aspects: signal phase, waiting time, queue length, capacity, and collision risk. Based on the algorithmic characteristics of SAC, this state can be constructed by concatenating discrete and continuous indicators, as expressed below: ;

[0072] in, One-hot encoding of the phase of the currently active signal. Indicates the number of signal phases; Normalized waiting time ratio of each entrance lane ( (corresponding to the four entrances: east, south, west, and north). This represents the maximum queuing occupancy rate for each approach lane (i.e., the ratio of queue length to lane length). The proportion of vehicles in each approach lane to the total number of vehicles at the intersection; The one-hot encoding for the highest collision risk among all signal phases (calculation method explained below). For ease of understanding, the following is... and Example explanation: Suppose a signalized intersection has four phases, corresponding to the left-turn and through traffic flows of the east, south, west, and north approach lanes; when the phase corresponding to the east approach lane is activated, The unique thermal encoding is [1,0,0,0]; similarly, if the west entrance has the highest collision risk, then... The one-hot encoding is [0,0,1,0].

[0073] Secondly, another core aspect is the action space of SAC. The SAC agent outputs a continuous two-dimensional action vector. Its expression is as follows: ;

[0074] in, Corresponding to the green light duration in seconds. Corresponding phase index.

[0075] These actions are mapped to actual traffic signal control parameters, namely green light duration and phase. The formula for calculating green light duration is as follows: ;

[0076] in, =20s =50s is the actual constraint condition set according to traffic regulations.

[0077] Meanwhile, phase selection can be obtained through mapping in the following way: ;

[0078] in, The number of feasible green light phases is defined. This design allows SAC to simultaneously optimize continuous green light duration and select discrete phases, offering greater flexibility compared to discrete action methods such as Deep Q-Networks (DQN).

[0079] Furthermore, the control objective is to maximize the expected long-term reward within the maximum entropy reinforcement learning (RL) framework, i.e.:

[0080] ;

[0081] in, A comprehensive reward that integrates efficiency and security; Discount factor; To regulate the entropy coefficient of exploring and utilizing trade-off relationships; Representation strategy; Represents policy entropy.

[0082] SAC employs an actor-critic architecture with a stochastic policy and two critics. When applied to traffic signal control, SAC effectively balances efficiency and safety by leveraging the maximum entropy objective and continuous-discrete coupling states; it also supports exploring dynamic phase activation schemes, thereby circumventing the rigid limitations of traffic signals. Furthermore, the dual Q-network suppresses overestimation problems, while the objective Q-network possesses stable training characteristics. These characteristics enable the algorithm to accurately evaluate complex traffic efficiency and safety scenarios and reduce control decision errors. The actor network outputs Gaussian parameters. and (in (representing the set of learnable parameters of the policy network), actions are obtained through reparameterization techniques:

[0083] ;

[0084] in, For pre-action; State; This refers to actual actions.

[0085] To ensure unbiased gradient estimation, the expression for log probability correction is as follows: ;

[0086] in, For the i-th action; It is a small constant used to avoid numerical singularities.

[0087] Critics estimate soft Q value and The target values ​​are as follows:

[0088] ;

[0089] in, For the first A target Q network, Its parameter set; For the next state From the current strategy The next action is obtained from the mid-sample. It should be noted that SAC and Deep Q-Network (DQN) agents share the same comprehensive reward defined in Section 2.2. This ensures consistency between continuous and discrete control paradigms.

[0090] The core of the Q-network is to make the Q-value approach the target value. In traffic signal control, the Q-value quantifies the combined efficiency and safety benefits of the current phase duration and phase switching. By fitting this loss to an accurate benefit evaluation model, a basis for strategic decision-making can be provided (Haarnoja et al., 2018). The expression for the critic loss is as follows:

[0091] ;

[0092] The optimization objective of the policy network is to maximize the Q-value, which is used to determine the signal action. This objective term guides the policy towards high-return actions through the Q-value, while controlling entropy to ensure the policy's exploratory nature. Its expression is as follows:

[0093] ;

[0094] Traffic signal control needs to balance exploration and utilization, ensuring that while exploring new actions, the strategy does not deviate from the optimization direction of efficiency and safety. This requires dynamic adjustment of the entropy coefficient, the expression of which is as follows:

[0095] ;

[0096] in, Let be the target entropy.

[0097] While adaptive signal control can achieve phase timing optimization at the macro level, more targeted interventions are still needed under specific traffic conditions to alleviate local congestion and reduce the risk of sudden accidents. In this application, such traffic intervention is achieved through the coordinated use of dynamic speed limits and route planning, both acting at the meso level. Its purpose is to reallocate traffic demand, balance lane utilization, and reduce the probability of congestion spread or secondary traffic conflicts. This control problem is constructed as a discrete action decision process and solved using a deep Q-network (DQN) algorithm.

[0098] Decision step size The system state at time is denoted as This status reflects an aggregated index covering the efficiency and safety of the entire road segment, and its expression is as follows: ;

[0099] in, The percentage of heavy-duty trucks (HDVs) on the relevant import lanes; Describe the remaining capacity of alternative routes. This phase-based design enables the DQN controller to make discrete decisions based on local congestion, traffic composition heterogeneity, and available diversion capacity, thereby complementing the continuous phase and timing optimization performed by the SAC controller.

[0100] The action space of the DQN controller integrates two types of measures: dynamic speed limiting and route planning. Formally, the action set is defined as: ;

[0101] in, The speed limit range for the specified road section, expressed in m / s; Indicates the first Alternative routes are provided. These actions enable the controller to stabilize traffic flow by reducing vehicle speed and suppressing shock wave propagation, and to alleviate congestion by reallocating vehicles to routes with lower saturation. In this application, route planning specifically refers to replacing a left turn with a combination of "right turn + U-turn + straight ahead".

[0102] Dynamic speed limits are determined based on lane-specific queuing rates. For a given lane... When its normalized queuing occupancy rate is At that time, the maximum permissible speed is set as follows: ;

[0103] in, and This is the congestion threshold, expressed in m / s. , , These correspond to vehicle speeds under free-flowing, moderately congested, and congested conditions, respectively, in m / s. This tiered strategy ensures that the vehicle adjusts its speed according to the current congestion level, thereby reducing speed variance and accident risk. For a smoother control effect, a continuous mapping method can also be used, the expression of which is as follows: ;

[0104] in, This is the saturation threshold; The rated free-flow speed is expressed in m / s. The minimum permissible speed is expressed in m / s.

[0105] Route planning decisions are based on estimated travel time and remaining route capacity. For considering the selection of the [missing information]... Vehicles on the alternative routes The time savings are calculated as follows: ;

[0106] in, The estimated travel time of vehicle i on the current route is in seconds. Let be the estimated travel time for vehicle i on the m-th alternative route, in seconds. A detour is triggered when the following conditions are met:

[0107] ;

[0108] in, The minimum acceptable time saving, expressed in seconds; The minimum remaining capacity required for alternative routes, expressed in veh (veh). To reflect the heterogeneity of driver behavior, a probabilistic detour strategy can also be used, calculated as follows:

[0109] ;

[0110] in, For the Sigmoid function; Control the slope of the response curve; Based on vehicle type compliance, this feature is used to reflect the difference in detour tendency between manually driven vehicles (HDVs) and connected autonomous vehicles (CAVs). The indicator function is defined as follows: when the remaining capacity of the m-th alternative route exceeds the minimum remaining capacity at time t. The value is 1 if it is true, and 0 otherwise. It represents the remaining available capacity of the m-th alternative route at time t.

[0111] The DQN agent learns a value function to evaluate the long-term benefits of dynamic speed limiting and route planning interventions given a given state. The action value function is defined as:

[0112] ;

[0113] in, For time step In the state of being Next action The instant rewards received; Take future time steps for discount factors The power of the time difference with the current time step t; and , .

[0114] The optimal Q-function satisfies the Bellman equation (Wang et al., 2022), and its expression is as follows:

[0115] ;

[0116] In practical applications, DQN agents utilize... A neural network approximating the action value function with parameters The network is trained using temporal difference learning, with the goal of minimizing the difference between the current Q-value estimate and the target value derived from the Bellman equation (Jiangetal., 2025a). Accordingly, the loss function is defined as:

[0117] ;

[0118] Among them, the target value The calculation is as follows: ;

[0119] in, This represents the parameters of the target network, which are updated periodically to stabilize the learning process.

[0120] The synergy between SAC-based adaptive traffic signal control and DQN-based traffic interventions ensures cross-scale complementarity. While the SAC agent dominates macro-level optimization through phase and timing adjustments, the DQN agent provides meso-level corrective measures, selectively employing dynamic speed limits and route planning to alleviate local bottlenecks that cannot be addressed by signal adjustments alone. This dual-agent design reduces traffic heterogeneity, stabilizes traffic flow, and enhances system robustness.

[0121] In this application, connected autonomous vehicles (CAVs) are modeled using Cooperative Adaptive Cruise Control (CACC) in the SUMO simulation software (Lopeze et al., 2018). Specifically, CACC enables vehicles to form and maintain a convoy by utilizing vehicle-to-vehicle (V2V) communication and an adaptive following strategy. This application limits the communication range to 100 meters, ensuring that each vehicle can only exchange information with neighboring convoy members within this distance. To reflect different penetration rates of CAVs, the market penetration rate (MPR) is set to 0%, 20%, 40%, 60%, 80%, and 100%, respectively. On the other hand, heavy-duty trucks (HDVs) are simulated using an Intelligent Driver Model (IDM). To eliminate interference from other factors, the following model parameters for CAVs and HDVs are set to be the same. For specific parameter settings, please refer to the research of Fu et al. (2025).

[0122] The introduction of Cooperative Adaptive Cruise Control (CACC) lays the foundation for cooperative traffic management. At the micro level, connected autonomous vehicles (CAVs) can reduce inter-vehicle spacing and improve fleet stability; at the macro level, their predictable and cooperative response characteristics enable integration with higher-level control strategies, such as adaptive traffic signal control, dynamic speed limits, and route planning. The synergy between fleet-level control and system-level traffic interventions promises to improve traffic efficiency and safety in emerging mixed traffic flows. Therefore, this application selects CAV-driven fleet control as another traffic control method.

[0123] The design of the reward mechanism is crucial, aiming to ensure that reinforcement learning agents (RLagents) converge to a policy that simultaneously improves efficiency and safety. In this application, the reward function is designed as a composite function integrating multiple performance metrics. Specifically, each metric is normalized through a smooth mapping to ensure numerical stability and avoid the dominance of extreme values. At each decision step... The total reward is defined as the sum of the four components: (1) extreme queuing penalty. (2) Overall queuing reward items (3) Minimum parking bonus (4) Accident risk penalty items Reward function Defined as:

[0124] ;

[0125] For each length of lane Its queuing occupancy rate is estimated to be: ;

[0126] in, This represents the number of stopped vehicles in the lane; the constant 5 corresponds to the average vehicle length (in meters), a value referenced from the SUMO simulation software (Lopez et al., 2018). For each approach lane... Lane-level queuing occupancy The aggregation result is ,in For the intersection approach lane Maximum queue occupancy rate; Indicates it belongs to the import channel The set consists of all lanes. Therefore, the maximum queuing occupancy and average queuing occupancy in each direction are derived as follows: ;

[0127] ;

[0128] Based on the above definitions, the four reward components can be further derived. The extreme queuing penalty term is used to suppress excessive queuing accumulation at any entrance lane, and its expression is: ;

[0129] To promote operational balance, an overall queuing reward term is introduced, the expression of which is: ;

[0130] To reduce unnecessary parking and improve traffic flow, parking incentives are defined as follows: ;

[0131] ;

[0132] In this application, the accident risk of each approach lane is classified to support the construction of the state and reward function. To capture the stochastic characteristics of rare and severe traffic conflict events, this application adopts the non-stationary generalized extreme value (GEV) distribution. The cumulative distribution function expression of the GEV distribution is (Coles, 2001):

[0133] ;

[0134] in Indicates position parameters, Indicate scale parameters, Indicates shape parameters, This application uses extreme conflict samples to identify traffic conflict events through Time-to-Avoid (TTAC). For the derivation and verification of TTAC, please refer to the research of Fu et al. (2024) and Fu et al. (2025).

[0135] In the context of nonstationary processes, parameters and All are related to exogenous covariates Related. Following the method of Fu et al. (2025), this application selects the acceleration variance of signalized intersections as a covariate. Therefore, the three parameters of the GEV distribution are modeled as follows: ;

[0136] in, For the first The covariates about The coefficient, and ; For the first The covariates about The coefficient, and ; , and This is the random intercept term that varies with different locations; This indicates the signage for different locations. It is important to note that, to ensure... A logarithmic join function is required. This function maps the linear prediction term from the real domain to the positive domain through an exponential transformation, thus ensuring that the scaling parameter is strictly positive and improving the numerical stability of the parameter estimation process. Furthermore, the shape parameter... The main reason why it is not usually set as non-stationary is that it is difficult to accurately model it with covariates, and existing research and theory support treating it as a fixed value that is not affected by observed factors (Fu and Sayed, 2022a; Fu and Sayed, 2022b; Fu and Sayed, 2023).

[0137] The key indicator focused on in risk assessment is the quantile function, namely the Value at Risk (VaR), which corresponds to the distribution of the Generalized Extreme Value (GEV). Quantiles (Gencay and Selçuk, 2004; Fuetal., 2025). A non-stationary GEV distribution can be represented as:

[0138] ;

[0139] The probability density function of the nonstationary GEV distribution can be written as:

[0140] ;

[0141] Furthermore, Value at Risk (VaR) can be derived from the cumulative distribution function. It is deduced that:

[0142] ;

[0143] Finally, according to the definition of quantiles (Beirlan et al., 2004), the GEV distribution... Quantiles can be calculated, and their expression is: ;

[0144] This framework can estimate the extreme quantiles of accident risk indicators under different traffic conditions. By incorporating intersection-specific covariates into a non-stationary GEV model, the resulting Value at Risk (VaR) provides a systematic basis for comparing and classifying the relative safety levels of approach lanes at different intersections. The shape parameter in the generalized extreme value distribution is known. For quantiles, For the first Location parameters of the generalized extreme value distribution in each block group (known). For the first The scaling parameter of the generalized extreme value distribution in each block group (known). To further explain, Y is a vector of covariates. It is the first The specific values ​​of the covariate vectors in each block group.

[0145] However, using a non-stationary GEV model for globally signalized intersections to estimate the accident risk of each approach lane may introduce biases. The accident risk classification for each approach lane is based on the aforementioned non-stationary GEV model: specifically, the Value at Risk (VaR) is calculated using the covariates corresponding to each approach lane, thus obtaining the risk classification result. It is assumed that the extreme values ​​of the Time to Avoid Collision (TTAC) for each approach lane follow a certain pattern. Distribution, among which , representing the There are several entrance lanes. Therefore, the non-stationary GEV model parameters for each entrance lane are modeled as follows:

[0146] ;

[0147] Therefore, the actual accident risk of each approach lane is defined as follows:

[0148] ;

[0149] In this case, the estimated accident risk for each approach lane can be expressed as:

[0150] ;

[0151] Based on the above assumptions, the real-world accident risk classification and the estimated accident risk classification of each approach lane can be considered equivalent. The accident risk classification of each approach lane is expressed as follows: ;

[0152] in, This is an ordered sequence of accident risk values ​​for all intersection approach lanes.

[0153] Based on the risk of accidents, this reward can be calculated as follows: ;

[0154] in The function returns the position of the accident risk value of the current green light phase in the sorted sequence.

[0155] Based on formula (41), it can be determined whether the currently active approach lane (in the green light phase) corresponds to the highest accident risk. Ideally, the green light signal should be allocated to the approach lane with the highest Value at Risk (VaR), i.e. However, in real-world scenarios, the currently active inlet lane may not be the same as the inlet lane with the highest risk. This discrepancy is quantified by the normalized level distance between the current risk level position and the highest risk level position, scaled to the [0,1] interval. Finally, the Sigmoid function... The value is smoothed to ensure differentiability, and the result is mapped to a bounded reward. Intuitively, the closer the current green light phase is to serving the highest-risk entry lane, the higher the reward; conversely, the greater the deviation from the highest-risk entry lane, the smaller the reward. This makes the selection of each phase tend to activate the highest-risk entry lane.

[0156] The training process designed in this application aims to enable the proposed MSA-CoTSC framework to gradually learn effective traffic control strategies through repeated interaction with a SUMO-based traffic simulation environment. At the beginning of each episode, the traffic state is reset, and agents are deployed without prior knowledge, ensuring that experience is accumulated from scratch and continuously optimized in consecutive episodes. At each step of the episode, the agent observes the current traffic situation, selects the appropriate control action, and receives a scalar reward reflecting traffic efficiency and safety. All control strategies employ a unified reward function, providing a consistent optimization objective and enabling different agents to collaboratively achieve balanced performance.

[0157] This application employs a 200-episode training process at a single intersection, serving as a controlled and dynamically dynamic learning environment. The vehicle composition remains consistent across all episodes, ensuring that any performance improvements are attributable solely to the agents' adaptive strategies, rather than changes in traffic composition. To expose the agents to diverse operating conditions and enhance their robustness, traffic flow and demand patterns are adjusted for each episode. Furthermore, a balance between exploration and exploitation is incorporated into the training process, ensuring that the agents can both discover alternative strategies and continuously optimize the optimal strategy.

[0158] Although this application employs a single-intersection training setup, three key measures were implemented to enhance robustness and ensure the learned policy exhibits generalization ability across different traffic environments. First, diverse demand scenarios and randomized initialization of traffic flow are incorporated into the training process, preventing the agent from memorizing specific traffic states and instead encouraging it to adapt to different conditions. Second, the reward function is constructed based on core traffic performance indicators, rather than features specific to a particular location. Third, the policy is periodically evaluated in unseen demand scenarios, retaining only policies that demonstrate stable performance. These measures reduce the risk of overfitting and enhance the transferability of the trained model. Through this iterative cycle of "interaction, reward-driven learning, and cross-scenario evaluation," the agent gradually converges to a policy that simultaneously improves operational efficiency and safety.

[0159] To ensure the reproducibility of the proposed MSA-CoTSC framework, this section systematically lists all the key hyperparameters used in the experiments. These hyperparameters are categorized according to their functional roles within the framework, specifically including: general hyperparameters shared by the entire system, SAC-specific hyperparameters for traffic signal phase and duration control, DQN-specific hyperparameters for traffic intervention, and training process parameters for iterative processing. Specific values ​​and their respective functions are summarized in Table 1.

[0160] Table 1 Hyperparameter Configuration

[0161]

[0162] This application takes two signalized intersections as research objects and evaluates the objective performance of the MSA-CoTSC framework under different market penetration rates (MPR) of connected autonomous vehicles (CAVs) (i.e., 0%, 20%, 40%, 60%, 80%, and 100%). The evaluation is conducted in two phases: In the first phase, the basic control effect of the proposed framework is verified by benchmarking it against two representative adaptive traffic signal control methods (i.e., DQN and SAC) that do not incorporate multi-strategy cooperation; In the second phase, the performance advantages of the proposed framework are further verified through two sets of comparative experiments: (1) comparison between the benchmark SAC signal control algorithm and the fixed multi-strategy cooperative control variant (SAC-F); (2) comparison between the benchmark SAC signal control algorithm and the multi-strategy cooperative control variant (SAC-PPO) based on the PPO algorithm. The evaluation focuses on three key dimensions, namely efficiency, safety, and fuel consumption, which are measured by mean waiting time, collision avoidance time (TTAC), and mean fuel consumption, respectively. Before conducting the performance evaluation, this section will summarize the data sources and report the training results of the proposed framework.

[0163] The real-world trajectory data used in this application was extracted from two signal-controlled intersections in Harbin, China (see...). Figure 1 Vehicle trajectories were acquired using high-resolution aerial video captured by drones, providing detailed spatiotemporal information on individual vehicle movement under heterogeneous traffic conditions. Based on the extracted trajectory data, a microscopic traffic simulation environment was built in SUMO, and the behavioral parameters of the two intersections were calibrated using a Bayesian optimization framework. Calibration parameters included desired acceleration (ACC), desired deceleration (DEC), maximum speed (MaxSpeed), desired headway (Tau), and minimum safe distance (MinGap). This calibration process ensured a high degree of consistency between the simulated driving behavior and the vehicle dynamics observed in real-world scenarios. Detailed data acquisition procedures, descriptive statistical results, and complete calibration methods can be found in the research of Fu et al. (2025).

[0164] This application extracted the 25th and 75th percentiles of the observed vehicle speed distribution at each intersection and used them as reference boundaries for dynamic speed limit control within the MSA-CoTSC framework. These percentile-based speed limit values ​​accurately reflect the upper and lower limits of operating vehicle speeds observed in urban traffic. The summary statistical results of this distribution are detailed in Table 2.

[0165] Table 2. Statistical results of vehicle speed data (m / s) at intersection 1 and intersection 2.

[0166]

[0167] The training process for this application was conducted on a desktop computer configured with an Intel® Core™ i5-13490F (13th generation, 2.50GHz) processor, 32GB of memory, and an NVIDIA GeForce RTX 4060 Ti (16GB VRAM) graphics card. Figure 2 The convergence process of the proposed framework during training is demonstrated. Due to the inherent exploratory behavior of reinforcement learning (RL), the total reward fluctuates significantly in the early stages of training; as training progresses, the average reward shows a stable, monotonically increasing trend, indicating that the policy gradually learns to balance traffic efficiency and safety. After approximately 150 rounds, both the total reward and the average reward reach convergence, with the average reward stabilizing at around 2850, indicating that the agent has achieved stable control performance. The observed convergence pattern verifies the stability of the embedded SAC and DQN modules in the framework, ensuring that the learned policy can reliably capture potential traffic dynamics characteristics, rather than fluctuating around suboptimal behavior.

[0168] During training, the SAC and DQN modules share the same reward function to ensure consistency across different learning paradigms. This reward function jointly reflects efficiency and safety metrics, ensuring that improvements in one dimension do not come at the expense of another. To avoid overfitting and improve generalization, traffic demand distribution characteristics and calibration parameters are randomly varied in each round, introducing randomness into traffic flow intensity, vehicle composition, and signal timing conditions. This randomization exposes the agent to diverse operating scenarios, enabling it to learn policies robust to environmental uncertainties. Notably, the proposed framework only requires training at a single four-entry intersection—because both state representation and action space are defined in a "structure-independent" manner. Therefore, after training, the model can be directly transferred to other intersections with similar topologies without retraining, significantly reducing computational costs while maintaining adaptability.

[0169] Average waiting time reflects the operational efficiency of a traffic system. Below are the waiting time results for various control methods at two intersections under different market penetration rates (MPR) of connected autonomous vehicles (CAVs). A comparison of all methods is shown below. Figure 3 As shown. In the comparison matrix of "Latency, Time to Avoid Collision (TTAC), and Fuel Consumption" for each MPR, the upper left area indicates whether there are significant differences when comparing different control methods pairwise; the lower right area indicates whether the control method on the left side of the horizontal axis is superior to the control method on the right side of the vertical axis in terms of latency, TTAC, or fuel consumption. For clarity, two examples are given below: Figure 3In the waiting time comparison graph of (a), when MPR is 0%, the cell at the intersection of DQN and MSA-CoTSC in the upper left corner of the matrix is ​​light yellow, indicating that there is a significant difference in the waiting time distribution of the two control methods; while the cell at the intersection of DQN and MSA-CoTSC in the matrix is ​​light blue, indicating that MSA-CoTSC is more efficient than DQN, that is, it has better performance.

[0170] At intersection 1, the average waiting time of DQN initially increases and then decreases with the increase in market penetration rate (MPR) of connected autonomous vehicles (CAVs): the initial value is 45.8s when MPR is 0%, reaching a peak of 56.1s at both 20% and 40%, and then decreasing to 47.0s at 60%, 41.7s at 80%, and 38.7s at 100%, respectively. The average waiting time of SAC generally decreases: it is 58.9s when MPR is 0%, rises slightly to 59.2s at 20%, and then gradually decreases to 54.2s at 40%, 50.8s at 60%, 48.9s at 80%, and 42.6s at 100%. The average waiting time for SAC-F showed a continuous downward trend: from 34.8s at 0% to 31.9s at 20%, 30.8s at 40%, and 24.2s at 60%. It rebounded slightly to 24.7s at 80%, and finally dropped to 22.2s at 100%. In contrast, the average waiting time for SAC-PPO was relatively stable, with only minor fluctuations: 23.4s at 0%, rising to 29.6s and 29.4s at 20% and 40% respectively, before dropping to 25.1s at 60%, 21.6s at 80%, and 21.0s at 100%. Among all control methods, MSA-CoTSC consistently exhibits the lowest average waiting time with minimal fluctuations, specifically: 19.3s at 0%, 17.8s at 20%, 18.1s at 40%, 17.6s at 60%, 13.6s at 80%, and 14.7s at 100%.

[0171] At intersection 2, the average waiting time for DQN remained high across most connected autonomous vehicle (CAV) market penetration rates (MPR): the initial value was 65.3s at 0% MPR, and remained relatively stable between 20% and 80% (66.1s, 64.7s, 63.9s, and 64.1s respectively), before sharply dropping to 39.9s at 100%. The average waiting time for SAC fluctuated within a moderate range: 41.1s at 0%, decreasing to 39.1s and 39.3s at 20% and 40% respectively, rising to 44.2s at 60%, then slightly decreasing to 42.1s at 80%, and finally reaching 41.7s at 100%. The average waiting time of SAC-F was relatively low: 10.4s at 0%, rising to 12.7s and 12.2s at 20% and 40% respectively, decreasing slightly to 11.5s at 60%, and rising again to 15.5s and 17.0s at 80% and 100% respectively. The average waiting time of SAC-PPO remained consistently low and stable, with the following values: 8.7s at 0%, 11.7s at 20%, 8.9s at 40%, 10.7s at 60%, 10.8s at 80%, and 11.2s at 100%. Ultimately, MSA-CoTSC consistently outperformed all other control methods, exhibiting the lowest average waiting time across all MPR levels: 6.0s at 0%, 6.6s at 20%, 7.1s at 40%, 7.9s at 60%, 7.1s at 80%, and 6.5s at 100%.

[0172] At both intersections, MSA-CoTSC consistently demonstrated superior performance in reducing waiting times compared to all benchmark algorithms. At intersection 1, the average waiting time of MSA-CoTSC ranged from 13.6 to 19.3 seconds, representing a reduction of approximately 60%-70% compared to DQN (38.7-56.1 seconds) and approximately 55%-65% compared to SAC (42.6-59.2 seconds). Even compared to the enhanced variant, MSA-CoTSC achieved significant improvements: a 35%-45% reduction in average waiting time compared to SAC-F (22.2-34.8 seconds) and a 30%-35% reduction compared to SAC-PPO (21.0-29.6 seconds). At intersection 2, the advantages of MSA-CoTSC are more significant: its average waiting time consistently remains within the range of 6.0-7.9 seconds, a reduction of nearly 85%-90% compared to DQN (39.9-66.1 seconds), and a reduction of 75%-85% compared to SAC (39.1-44.2 seconds); compared to SAC-F (10.4-17.0 seconds), the waiting time is further reduced by 40%-55%; and compared to SAC-PPO (8.7-11.7 seconds), it still achieves a further performance improvement of 25%-35%. Collision Avoidance Time (TTAC) is a quantitative indicator of traffic conflict. The TTAC values ​​of each control method at the two intersections under different connected automated vehicle (CAV) market penetration rates (MPR) are summarized below, and the comparison results are as follows. Figure 3 As shown.

[0173] At intersection 1, the average collision avoidance time (TTAC) of DQN generally shows a downward trend with the increase in market penetration rate (MPR) of connected autonomous vehicles (CAVs): 3.6s at 0% MPR, decreasing to 3.3s at 20%, stabilizing at 3.1s at 40% and 60%, remaining at 3.1s at 80%, and slightly increasing to 3.3s at 100%. The average TTAC of SAC also shows a gradual downward trend: 3.7s at 0%, 3.5s at 20%, 3.4s at 40%, 3.3s at 60%, 3.2s at 80%, and slightly rebounding to 3.3s at 100%. The average TTAC of SAC-F shows a continuous downward trend, with values ​​of 3.0s, 2.8s, 2.7s, 2.5s, 2.4s, and 2.5s at all MPR levels. Similarly, the average TTAC of SAC-PPO shows a downward trend: 3.3s at 0%, decreasing to 3.1s at 20%, 2.9s at 40%, 2.6s at 60%, 2.5s at 80%, and slightly rebounding to 2.7s at 100%. The average TTAC of MSA-CoTSC gradually decreases: 3.4s at 0%, decreasing to 3.2s at 20%, 2.9s at 40%, 2.7s at 60%, 2.5s at 80%, and slightly rebounding to 2.8s at 100%.

[0174] At intersection 2, the average collision avoidance time (TTAC) of DQN remained relatively stable overall: 3.5s at 0% MPR, 3.4s at 20%, 3.3s within the 40%-80% MPR range, and slightly increased to 3.4s at 100%. The average TTAC of SAC again showed a gentle downward trend: 2.9s at 0%, 2.8s for 20%-40% MPR, 2.7s at 60%, and stable at 2.6s for both 80% and 100% MPR. The average TTAC of SAC-F showed a similar downward trajectory: 3.4s at 0%, 3.2s at 20%, 3.0s at 40%, 2.7s at 60%, and 2.6s for both 80% and 100% MPR. The average TTAC of SAC-PPO also showed a gradual downward trend: 3.4s at 0%, 3.2s at 20%, 2.9s at 40%, 2.7s at 60%, 2.6s at 80%, and 2.7s at 100%. Finally, the average TTAC of MSA-CoTSC showed a continuous downward trend: 3.3s at 0%, 3.1s at 20%, 2.9s at 40%, 2.6s at 60%, 2.5s at 80%, and a slight increase to 2.6s at 100%.

[0175] Average fuel consumption is a key indicator of the operating cost of a transportation system. The relevant results for two intersections under different market penetration rates (MPR) of connected autonomous vehicles (CAVs) are as follows: Figure 3As shown in the figure, at intersection 1, the average fuel consumption of DQN generally shows a downward trend: 102.9g at 0% MPR, rising to 111.9g at 20%, and then gradually decreasing to 109.3g at 40%, 96.1g at 60%, 88.1g at 80%, and 79.2g at 100%. The average fuel consumption of SAC shows a more stable downward trajectory: 113.7g at 0%, then steadily decreasing to 112.6g at 20%, 105.5g at 40%, 99.6g at 60%, 93.6g at 80%, and 82.4g at 100%. While the absolute value of average fuel consumption for SAC-F remains consistently high, it shows a decreasing trend with increasing MPR: from 140.4g at 0% to 135.7g at 20%, 125.7g at 40%, 114.1g at 60%, 105.3g at 80%, and 86.3g at 100%. Similarly, the average fuel consumption for SAC-PPO is relatively high, specifically: 143.6g at 0%, 141.6g at 20%, 137.4g at 40%, 127.8g at 60%, 112.3g at 80%, and 88.5g at 100%. MSA-CoTSC outperforms SAC-F and SAC-PPO, but its average fuel consumption is still higher than DQN and SAC, with the value gradually decreasing from 138.0g at 0% to 132.1g at 20%, 127.7g at 40%, 120.4g at 60%, 104.3g at 80%, and 83.8g at 100%.

[0176] At intersection 2, the average fuel consumption of DQN shows a significant downward trend: 95.1g at 0%, 92.8g at 20%, 93.1g at 40%, 87.0g at 60%, 78.8g at 80%, and 73.7g at 100%. The average fuel consumption of SAC fluctuates within a narrow range: 92.6g at 0%, decreasing to 89.3g at 20%, 88.2g at 40%, rising slightly to 91.3g at 60%, then dropping again to 85.7g at 80%, and finally reaching 81.9g at 100%. SAC-F has a higher average fuel consumption figure, which shows a gradual downward trend: 105.1g at 0%, slightly increasing to 107.0g at 20%, 104.4g at 40%, 100.2g at 60%, 95.7g at 80%, and 87.6g at 100%. SAC-PPO performs slightly better, with the following values: 109.9g at 0%, 106.7g at 20%, 102.4g at 40%, 98.3g at 60%, 88.0g at 80%, and 79.5g at 100%. However, the MSA-CoTSC had the highest overall fuel consumption at this intersection, with the value gradually decreasing from 118.7g at 0% to 117.5g at 20%, 111.9g at 40%, 108.0g at 60%, 95.2g at 80%, and 78.9g at 100%.

[0177] In terms of fuel consumption, MSA-CoTSC outperforms all adaptive traffic signal control methods without multi-strategy coordination (i.e., DQN and SAC). This result can be attributed to the integration of path planning functionality into MSA-CoTSC, which occasionally extends vehicle travel distance, thus increasing fuel consumption. Nevertheless, the overall increase in fuel consumption remains moderate. Specifically, at all MPR levels at both intersections, MSA-CoTSC's average fuel consumption is approximately 10%-15% higher than SAC and approximately 15%-20% higher than DQN, but still lower or slightly higher than SAC-F and SAC-PPO. Considering the significant improvement in traffic efficiency achieved by MSA-CoTSC, this moderate increase in fuel consumption can be considered acceptable in the trade-off between operating costs and efficiency. On the other hand, as MPR increases, the average fuel consumption of all control methods tends to decrease. Clearly, connected autonomous vehicles (CAVs) can reduce fuel consumption at intersections, consistent with the findings of Almannaa et al. (2019), Ma et al. (2021), and Zohdy and Rakha (2016).

[0178] The results section presents detailed evidence regarding traffic efficiency, safety levels, and fuel consumption under different control methods. However, it is crucial to emphasize that these results should not be interpreted in isolation. In real-world traffic systems, improvements in one dimension often come at the expense of another (Dogan et al., 2011), creating an inherent trade-off between traffic efficiency, safety levels, and fuel consumption. Therefore, this section will move beyond descriptive results and critically analyze the interactions among these three dimensions, focusing on how the MSA-CoTSC framework balances traffic efficiency and fuel consumption while improving safety levels across different connected automated vehicle (CAV) market penetration rates (MPR). Furthermore, this section will verify the correctness of the collision risk ranking method applicable to each approach lane of a signalized intersection.

[0179] A comparative evaluation of the proposed MSA-CoTSC framework demonstrates that it achieves a significant improvement in the balance between efficiency and fuel consumption in novel mixed traffic flows. Figure 4 As shown, at all CAV market penetration rates (MPR) levels at intersection 1, MSA-CoTSC consistently exhibits the lowest median and dispersion of waiting times, demonstrating its superior efficiency in reducing waiting times. At MPR levels of 0% and 20%, its waiting time reduction effect is particularly significant compared to SAC and DQN—indicating that even in scenarios without cooperating vehicles, multi-strategy collaborative mechanisms can effectively improve traffic efficiency. With increasing MPR, waiting times under MSA-CoTSC control continuously decrease and stabilize at extremely low levels, reflecting a further enhancement of collaborative efficiency after the increase in CAV connectivity. In contrast, MSA-CoTSC's fuel consumption is slightly higher than the baseline method, but this difference gradually narrows at medium-to-high MPR levels (60%-100%). This trend reflects the improved collaborative efficiency resulting from higher connectivity: unnecessary acceleration and deceleration behaviors are reduced, thus narrowing the fuel consumption gap with the baseline method (Gao et al., 2024). In addition, studies have confirmed that CAV fleet control can effectively reduce fuel consumption (Qin et al., 2018; Vahidi and Sciarretta, 2018). Figure 4 The trade-off between efficiency and fuel economy is clearly presented—MSA-CoTSC minimizes waiting time at the cost of a slight increase in fuel consumption. However, the fluctuations in fuel consumption remain within acceptable limits (especially at high MPR levels), indicating that the efficiency improvement of MSA-CoTSC does not come at the expense of excessive fuel consumption.

[0180] Figure 5The patterns observed are highly similar to those at intersection 1: across all connected autonomous vehicle (CAV) market penetration (MPR) levels, MSA-CoTSC consistently exhibits the lowest median waiting time and the most concentrated distribution. At low MPR levels (0%-40%), waiting times under MSA-CoTSC control are significantly reduced compared to all benchmark methods—indicating that even without large-scale connectivity support, this framework still possesses the ability to optimize traffic signal timing and coordination. As MPR increases, waiting times remain at extremely low levels with gradually decreasing variance, reflecting further enhancements in the operational stability and coordination efficiency among CAVs. Regarding fuel consumption, MSA-CoTSC's values ​​are still slightly higher than the benchmark methods, but this gap gradually narrows as MPR increases, consistent with the trend at intersection 1. However, at high MPR levels, the gap convergence effect at intersection 2 is not as good as at intersection 1, which can be attributed to its denser traffic flow conditions and more complex geometry (Ding et al., 2022). Similarly, at high MPR levels, the waiting time reduction at intersection 2 is also more significant than that at intersection 1. Furthermore, the inverse relationship between waiting time and fuel consumption further confirms the inherent trade-off between efficiency and fuel consumption. (Comparison) Figure 4 and Figure 5 As can be seen, MSA-CoTSC exhibits consistent control performance and good generalization under different geometric structures and traffic flow conditions. The proposed framework significantly improves traffic efficiency while maintaining acceptable energy consumption at all MPR levels. To quantitatively verify these findings, this paper conducts a detailed statistical comparison of DQN, SAC, SAC-F, SAC-PPO, and MSA-CoTSC. At intersection 1, the average waiting time of MSA-CoTSC at 0% MPR is 19.3s, a reduction of 57.8% and 67.2% compared to DQN and SAC, respectively; and a further reduction of 44.5% and 17.6% compared to the fixed coordination method (SAC-F) and the PPO-based coordination method (SAC-PPO), respectively. When the MPR increases to 100%, the waiting time of MSA-CoTSC drops to 14.7s, significantly lower than SAC-F (22.2s) and SAC-PPO (21.0s). This advantage also applies to intersection 2: at full penetration (100% MPR), the waiting time for MSA-CoTSC is reduced to 6.5 s, representing performance improvements of 61.8% and 42.0% compared to SAC-F (17.0 s) and SAC-PPO (11.2 s), respectively. Furthermore, compared to recent research by Cai et al. (2025), Huang et al. (2021), and Lu et al. (2025), the proposed framework demonstrates significantly superior waiting time reduction capabilities, indicating its strong competitiveness among existing adaptive traffic signal control methods.

[0181] Corresponding fuel consumption analysis confirms that these efficiency improvements do not come at the cost of excessive fuel consumption. For example, at a 40% MPR level at intersection 1, the MSA-CoTSC's fuel consumption is approximately 21% higher than the SAC, but comparable to the SAC-F, and significantly lower than the SAC-PPO by 7%. At high MPR levels (≥80%), the fuel consumption gap between the MSA-CoTSC and SAC narrows to approximately 2%-12%, while still remaining 5%-7% lower than the SAC-PPO. Statistical tests confirm the statistical significance of the reduced waiting time (p<0.001, Hedges'g>0.5) and the marginal impact on fuel consumption (p>0.05 at MPR≥80%). This further demonstrates that the proposed framework achieves a statistically robust and practically meaningful efficiency-energy balance.

[0182] These comparative results reveal three key insights. First, while both SAC-F and SAC-PPO demonstrate that multi-policy coordination improves efficiency compared to single-control baseline methods, their adaptability remains limited by rigid coordination or over-constrained optimization. In contrast, the dynamic coordination mechanism of MSA-CoTSC allows for flexible decision-making across control levels, maintaining excellent performance even under varying MPR and traffic flow conditions. Second, the fusion of SAC and DQN in MSA-CoTSC exhibits structural complementarity: SAC's continuous action space and entropy regularization strategy ensure smooth and stable control of traffic signal timing (Haarnoja et al., 2018), while DQN's discrete decision structure efficiently handles path selection and velocity coordination among multiple agents (Li et al., 2024a; Zhang et al., 2025b). This modular decoupling improves training efficiency and reduces policy interference between control levels. Third, although alternative PPO-based collaborative frameworks have theoretical advantages in continuous policy optimization (Chu et al., 2024), their excessive gradient pruning and limited sample reuse often lead to slow convergence and weak adaptability in non-stationary multi-agent environments. In contrast, the DQN algorithm, with its value-based hetero-policy mechanism and stable Q-learning updates (Wang et al., 2024), provides a more responsive and computationally efficient collaborative hierarchy, which can be more effectively adapted to the continuous signal control dynamics of SAC. Although the proposed MSA-CoTSC can reduce... Figure 6 and Figure 7The TTAC (Time to Avoid Collision) central tendency indicator shown is not globally optimal across all intersections and MPR (market penetration rate of connected autonomous vehicles). However, safety assessment cannot be achieved through simple comparison of sample means alone. Multiple studies have shown that point estimates based solely on the mean cannot fully reflect safety conditions (Guo et al., 2020; Wang et al., 2018; Zheng et al., 2025). Therefore, a distribution-oriented and risk-oriented perspective is needed, employing extreme value theory (EVT) to model the tail behavior of TTAC, thereby extrapolating observed traffic conflicts to potential collision risks.

[0183] First, the analysis of the distribution characteristics of TTAC (time to avoid collision) reveals systematic differences between MSA-CoTSC and the baseline method in terms of distribution pattern, dispersion, and tail behavior (Songchitruksa and Tarko, 2006). Figure 6 and Figure 7 As shown. For example, at 0% MPR (market penetration rate of connected autonomous vehicles) at intersection 1, the average TTAC of MSA-CoTSC is 3.17s, and the median is 3.54s (standard deviation = 0.84), indicating a slightly right-skewed distribution—a few extremely high TTAC observations lengthen the upper tail of the distribution. When the MPR increases from 20% to 80%, the average TTAC drops to the 2.48-2.07s range, while the median and percentile shift. This pattern indicates that the overall empirical distribution of TTAC is not a simple translation, but rather a structural reshaping, with the upper tail of the distribution significantly compressed as the MPR increases. In contrast, the baseline method exhibits different skewness characteristics and variability patterns within the same MPR range. Overall, these statistical results show that MSA-CoTSC can systematically redistribute the probability mass in the TTAC distribution, reducing the probability of extreme conflict events by compressing the extreme upper tail and stabilizing the central region. Intersection 2 also exhibits the same pattern, as shown... Figure 7 As shown.

[0184] To further evaluate the safety performance of different control methods under different MPR (market penetration rate of connected automated vehicles), this application constructs non-stationary generalized extreme value (GEV) models for DQN, SAC, SAC-F, SAC-PPO, and the proposed MSA-CoTSC. For two signalized intersections, models are constructed at six MPR levels (0%, 20%, 40%, 60%, 80%, and 100%). Acceleration variance is used as a covariate in each model, capturing the dynamic characteristics of traffic flow and its potential impact on extreme conflict events. The time block length is set to 1 minute to capture short-term collision risks (Fu et al., 2025). All models are implemented in Python, and the parameters of the GEV distribution are estimated using maximum likelihood estimation. Independent models are fitted for the two intersections to account for differences in their geometric layout and traffic composition. The parameter estimation results for each control method under different MPRs are summarized in Tables 3 and 4. All parameter estimates were statistically significant, which confirms that the non-stationary GEV model has good robustness and explanatory power in capturing the relationship between acceleration variance and the tail behavior of the TTAC (time to avoid collision) distribution.

[0185] Table 3. Parameter estimation results of the non-stationary generalized extreme value (GEV) model at intersection 1.

[0186]

[0187] Table 4. Parameter estimation results of the nonstationary generalized extremum (GEV) model at intersection 2.

[0188]

[0189] To verify the reliability and predictive ability of the constructed non-stationary generalized extreme value (GEV) model, this paper conducts an accuracy verification test on extreme event predictions. Specifically, based on... Figure 6 and Figure 7The TTAC (Time to Avoid Collision) distribution characteristics shown are illustrated, with four TTAC thresholds (5.0s, 5.5s, 6.0s, and 6.5s) selected—these thresholds correspond to relatively large TTAC values, representing potential extreme near-collision scenarios. This validation also includes sensitivity analysis to examine the model's predictive ability for extreme conflict events at different severity thresholds. The validation process follows the methods proposed by Fu et al. (2025) and Kamel et al. (2023): for each threshold, the prediction accuracy is defined as the ratio of the number of correctly classified time blocks (with or without extreme events) to the total number of time blocks. For example, if the model correctly identifies 50 time blocks with extreme events out of 60 time blocks, the prediction accuracy is 50 / 60 = 83.3%. The validation results are summarized in Table 5, showing that all models exhibit strong predictive ability for extreme conflict events: the prediction accuracy exceeds 80% in all scenarios, and reaches 100% in most configurations (combination of control method + MPR level). These findings confirm that the non-stationary GEV model can accurately capture the distributed tail behavior of TTAC and reliably predict extreme conflict events (i.e., collision risk) under different traffic control strategies and MPR (market penetration rate of connected autonomous vehicles).

[0190] Table 5. Accuracy of Extreme Event Prediction for Non-Stationary Generalized Extrema (GEV) Models

[0191]

[0192] Based on the estimated non-stationary generalized extreme value (GEV) model, this paper calculates the Value at Risk (VaR) to characterize the upper tail behavior of the collision risk distribution. Figure 6 and Figure 7The estimated VaR values ​​for intersections 1 and 2 are presented, showing that MSA-CoTSC maintains a significant safety advantage over benchmark methods across all connected autonomous vehicle (CAV) market penetration (MPR) levels. Compared to single-policy benchmark methods such as SAC and DQN, MSA-CoTSC consistently achieves significantly lower VaR values—typically reducing top-tail risk by approximately 30%-60% across all MPR levels at both intersections. This result demonstrates that even with relatively mild mean differences among methods, the proposed collaborative framework effectively suppresses extreme conflict events. In addition to single-policy benchmark methods, MSA-CoTSC also outperforms fixed or alternative collaborative schemes such as SAC-F and SAC-PPO: its VaR value is reduced by an average of 10%-25% in most MPR scenarios, and these reductions are statistically significant (p≪0.001). This improvement highlights the advantages of adaptive multi-control collaboration in mitigating rare but severe conflict events. In summary, MSA-CoTSC not only improves the overall safety level, but also compresses the extreme upper tail of the TTAC (time to avoid collision) distribution, indicating that the intensity of collision risk has been substantially reduced.

[0193] To reconcile the unevenness of mean performance with the continuous improvement in tail behavior, it is necessary to focus on how MSA-CoTSC alters the shape of the TTAC (Time to Avoid Collisions) distribution (Hasain and Ahmed, 2024). A control strategy that reduces the probability of maximal TTAC values ​​while allowing a slight increase in variability in the central region may result in a mean TTAC that is not globally minimum, but can significantly reduce the Value at Risk (VaR). In other words, MSA-CoTSC appears to trade a slight increase in dispersion in the central region for significant tail compression in some scenarios. This strategy shifts probabilistic quality away from catastrophic high TTAC observations (thus reducing VaR), even if the median or mean statistics do not always reach an absolute minimum. This explanation is supported by a comparison of paired effect sizes (Hedges'g) and Tukey's post-hoc test results. The VaR comparison showed a large effect size and a highly significant p-value (p≪0.001), indicating that the extreme collision risk was indeed redistributed from the dangerous tail end—this result validates the core safety advantage of MSA-CoTSC in "non-mean optimal but tail risk optimal".

[0194] The observed negative correlation between acceleration variance and Value at Risk (VaR) can be interpreted from both traffic dynamics and statistical perspectives (Saifuzzaman and Zheng, 2014; Liu and Khattak, 2016). Higher acceleration variance in cooperative traffic environments does not necessarily indicate aggressive or unsafe driving behavior; rather, it typically reflects more frequent fine adjustments in acceleration and deceleration as vehicles utilize available gaps and coordinate within the platoon to maintain smooth traffic flow. Under MSA-CoTSC control, the system significantly reduces waiting times, creating smoother traffic conditions and enabling both connected and autonomous vehicles (CAVs) and human-driven vehicles (HDVs) to perform continuous, small-scale corrective actions, rather than occasional large-scale emergency responses. In this scenario, acceleration variance increases due to more proactive short-term control; however, thanks to earlier vehicle intervention and platoon cooperation suppressing the shock wave amplification effect, the probability of extremely high-value events in the upper tail of the TTAC (Time to Avoid Collision) distribution is significantly reduced. Therefore, the higher acceleration variance is accompanied by compression of the upper tail of the danger zone, ultimately leading to a decrease in VaR. Notably, the negative correlation between acceleration variance and collision risk VaR observed in this application contrasts with earlier research results under fixed-timing signal control—in fixed-timing control, higher acceleration variance is generally associated with greater traffic instability and higher collision risk (Fu et al., 2025; Sinha et al., 2020). In fixed-timing signal control, higher acceleration variance typically reflects the driver's non-cooperative response to the rigid signal phase, and is therefore associated with increased traffic instability and collision risk. In contrast, under adaptive traffic signal control, acceleration variance is given a different meaning: it reflects more frequent but smaller-amplitude vehicle adjustments triggered by real-time adaptive signal adjustments. These adjustments smooth traffic flow transitions and dissipate local disturbances before they escalate into traffic conflicts. Therefore, the higher acceleration variance under adaptive control becomes an indicator of responsiveness rather than instability, which also explains its negative correlation with VaR.

[0195] Even though the average TTAC (Time to Avoid Collision) is not globally optimal across all scenarios, MSA-CoTSC can still significantly reduce the risk of rear-end collisions. The core safety advantage of this strategy lies in its distribution effect—by reshaping the TTAC distribution, it reduces the frequency and severity of the most dangerous events. This effect is clearly reflected in the VaR (Value at Risk) estimate and... Figure 6 and Figure 7The highly significant pairwise comparison results show that the observed negative correlation between acceleration variance and VaR further indicates that traffic state response under MSA-CoTSC control is faster and more coordinated: the proactive micro-adjustments are not a manifestation of instability, but rather a key factor contributing to a significant improvement in overall safety.

[0196] In this application, the safety consideration for adaptive traffic signal control lies in capturing the collision risk of each approach lane at an intersection and accurately locating the approach lane with the highest collision risk. The Generalized Extreme Value (GEV) model can capture the temporal evolution characteristics of collision risk, but its ability to characterize spatial collision risk has not yet been proven. However, the framework proposed in this application only needs to achieve the ranking of collision risks for each approach lane; therefore, the correctness of the ranking results for each approach lane needs to be mathematically proven. Consider a certain intersection, its approach lanes... The collision risk follows a generalized extreme value (GEV) distribution, which includes location parameters. Scale parameters and common shape parameters (Coles, 2001).

[0197] For confidence level Define the intersection approach lane of quantile (i.e., Value at Risk, VaR) is .

[0198] make , making Corresponding to .

[0199] Assuming all position parameters and scale parameters If all are bounded, then the following conclusion holds:

[0200] like Then the estimated value at risk satisfies And there are:

[0201] ;

[0202] Therefore, if Then it exists This makes it possible for all All have ;like The above difference simplifies to .

[0203] like Then the estimated value at risk satisfies And there are:

[0204] ;

[0205] Therefore, if Then it exists This makes it possible for all All have ;like Then the above difference equals .

[0206] like Then we have: (Finite upper endpoint).

[0207] Therefore, if Then it exists This makes it possible for all All have .

[0208] In summary, when the shape parameters of each approach lane at the intersection... When fixed: For In this case, the dominant factor determining the ranking of high quantiles is the scale parameter. If the dimensional parameters of each inlet are equal, then the sorting is based on the position parameters. Decision; when At that time, the sorting is based on a finite number of upper endpoints. Decide.

[0209] This proof can be obtained by directly comparing quantile functions. The monotonicity and asymptotic ordering property of the generalized extremum (GEV) quantile function are derived based on the theoretical results of Beirlant et al. (2006).

[0210] when At that time, there were: ;when When taking the limit, we get: .

[0211] Scene The difference between the estimated value-at-risk expressions for the two import lanes yields:

[0212] .like Then when When, the first term diverges to positive infinity ( Therefore, a threshold exists. This makes it possible for all All have ;like Then the divergent term is eliminated, and the difference simplifies to ;like If , then the converse inequality holds.

[0213] Scene At this point, the estimated value at risk difference between the two entrances is:

[0214] ,because ,like Then the difference diverges to positive infinity. ), making the threshold Existence; if Then the difference is equal to .

[0215] Scene At this time, Sometimes, ,therefore .

[0216] Therefore, asymptotic ordering is determined by these finite endpoints. If one endpoint is greater than another, then according to the definition of a limit, there exists a threshold. The ranking remains fixed thereafter. Therefore, the collision risk ranking conclusion holds true for all scenarios.

[0217] This application proposes a hybrid traffic flow signal cooperative control framework (MSA-CoTSC), integrating continuous signal optimization, discrete traffic intervention, and vehicle-level fleet cooperation technologies to jointly improve the operational efficiency and safety of emerging hybrid traffic flows. Methodologically, the framework employs a hybrid dual-agent reinforcement learning (RL) architecture: a soft Actor-Critic (SAC) agent outputs continuous traffic signal control commands, while a deep Q-network (DQN) agent executes discrete meso-level traffic interventions (i.e., dynamic speed limits and route planning). The framework embeds a non-stationary generalized extreme value (GEV) model, estimates the approach lane-level risk value (VaR) based on collision avoidance time (TTAC) sequences, and integrates VaR into a unified reward function to guide policy learning towards risk-aware decision-making. The training process utilizes an experience replay buffer, a DQN target network, SAC entropy regularization, and stochastic demand scenarios to enhance the model's stability and transferability.

[0218] Based on extensive SUMO simulation experiments at two typical signalized intersections and six CAV market penetration rates (MPR, 0%-100%), the main empirical results are summarized as follows: (1) At all MPR levels and in both intersection scenarios, MSA-CoTSC significantly reduced the average waiting time, with delays typically only half or less than those of the single adaptive traffic signal control (SAC and DQN) baseline methods. Statistical tests (Tukey post-hoc tests and effect size analysis) confirmed that these improvements have high statistical significance and practical application value; (2) Although the mean collision avoidance time (TTAC) of MSA-CoTSC is not optimal under all conditions, the embedded value at risk (VaR) optimization objective achieves significant compression of the hazard tail. Compared with the baseline method, MSA-CoTSC systematically reduces the incidence of high-risk events, that is, the number of near-collision events is significantly reduced; (3) The SAC-DQN hybrid agent interaction mechanism and CAV fleet collaborative control increase the short-term micro-adjustment behavior of vehicles (i.e., the acceleration variance increases), but effectively suppress extreme braking behavior and near-collision events; (4) After integrating path planning and dynamic speed limit functions, fuel consumption increases slightly (about 10%-20% higher than single adaptive traffic signal control in low MPR scenarios). The key is that as the CAV penetration rate increases (≥60%), this fuel consumption cost gradually decreases - thanks to the reduction of redundant acceleration and deceleration behavior by CAV collaborative control. Therefore, in high penetration scenarios, the trade-off between efficiency, safety and energy consumption is significantly optimized; (5) The strategy trained at a single intersection has good generalization ability at similar intersections. The random demand scenario setting, the structure-independent state design (i.e., the use of relative indicators), and the decoupling design of SAC and DQN during the training process all help the model achieve stable convergence and improve cross-scenario transferability. The results of this application demonstrate that the proposed MSA-CoTSC framework can balance multiple objectives and clearly present the trade-off between efficiency, security, and energy consumption, providing a scientific basis for decision-making.

[0219] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solution of the present invention and should not be used to limit the scope of protection. Any modifications made in accordance with the claims and specification of the present invention that are only partial should still fall within the protection scope of the present invention.

Claims

1. A multi-scale adaptive cooperative traffic signal control method, characterized in that... Includes the following steps: The system acquires real-time traffic conditions at intersections and inputs these conditions into the intelligent agent to enable traffic signal control. The intelligent agent includes a SAC module and a DQN module. The SAC module obtains the green light duration of the traffic lights for each approach lane based on the real-time traffic conditions at the intersection. The DQN module is used to determine whether vehicles that do not intend to turn right at each approach lane of the intersection need to change lanes. If they do not change lanes, they will proceed according to their original intention. If they do change lanes, they will enter the corresponding approach lane after changing lanes. The rerouting specifically includes: If the vehicle intended to turn left, after changing lanes, the vehicle will enter the right turn lane, and after turning right, making a U-turn, and going straight, it will enter the lane in the original left-turn direction. If the vehicle intends to go straight, after changing lanes, the vehicle will enter the right-turn lane, and after turning right-turning-making a U-turn, it will enter the lane in the original direction of its straight-going intention.

2. The multi-scale adaptive cooperative traffic signal control method according to claim 1, characterized in that... The SAC module specifically performs the following steps: Step 1: At each decision moment Obtain the real-time traffic conditions at the intersection and use them as a state representation. , is represented as: ， in, One-hot encoding of the phase of the currently active signal. The number of signal phases. The normalized waiting time ratio for each entrance lane. Number each entrance lane. The maximum queuing occupancy rate for each entrance lane. This represents the proportion of vehicles in each approach lane to the total number of vehicles at the intersection. One-hot encoding for the highest collision risk among all signal phases; Step 2: Represent the state Inputting the pre-trained SAC network yields a two-dimensional continuous action vector as output. , is represented as: ， in, The duration of the green light for the current phase. For signal phase; Step 3: Based on the two-dimensional continuous action vector Traffic signal control is achieved by adjusting the timing parameters of the signal controller. These timing parameters include phase selection and green light duration. Phase selection is expressed as follows: ， in, The number of feasible green light phases. To select the phase number, The duration of a green light is expressed as follows: ， in, and These are the minimum and maximum green light durations.

3. The multi-scale adaptive cooperative traffic signal control method according to claim 2, characterized in that... The DQN module specifically performs the following steps: Step 21: At each decision time t, obtain the real-time traffic conditions at the intersection and use them as the system state. , is represented as: ， in, The proportion of heavy-duty trucks entering the road; The remaining capacity of the route after the diversion; Step 22: Set system status Input the pre-trained DQN network into the action space Select the redirection action to output. , is represented as: ， ， ， in, and These are the minimum and maximum speeds for the road segment. To save time, For the minimum acceptable time savings, For space occupancy rate, For vehicles Distance from the parking line For vehicle location threshold, The minimum remaining capacity required for the route. For vehicles The total length of the road currently being traveled.

4. The multi-scale adaptive cooperative traffic signal control method according to claim 3, characterized in that... The time savings Represented as: ， in, For vehicles Based on the estimated travel time for the current route, For vehicles Estimated travel time for the rerouted route.

5. The multi-scale adaptive cooperative traffic signal control method according to claim 4, characterized in that... Minimum speed on the road section The maximum speed of the road segment is a fixed value. For dynamic values, where, If the vehicle is not an autonomous vehicle, the maximum speed on the road segment is... Represented as: ， ， in, and The congestion threshold , , These represent vehicle speeds under smooth traffic, moderate congestion, and congested conditions, respectively. For lane Normalized queue occupancy rate The number of stopped vehicles in the lane. For lane Length, If the vehicle is an autonomous vehicle, the maximum speed on the road segment is... Represented as: ， in, The saturation threshold, For the rated free-flow speed, This is the minimum permissible speed.

6. The multi-scale adaptive cooperative traffic signal control method according to claim 5, characterized in that... The reward function of the agent Represented as: ， in, This is an extreme queuing penalty item. For the overall queuing reward items, For the smallest parking reward item, This is a penalty item for accident risk.

7. The multi-scale adaptive cooperative traffic signal control method according to claim 6, characterized in that... The extreme queuing penalty item Represented as: ， ， ， in, For the Sigmoid function, For the maximum queuing occupancy rate in each direction, It belongs to the import channel The set consisting of all lanes, For import channels Lane-level queuing occupancy.

8. The multi-scale adaptive cooperative traffic signal control method according to claim 7, characterized in that... The overall queuing reward items Represented as: ， in, The average occupancy rate in each direction.

9. A multi-scale adaptive cooperative traffic signal control method according to claim 8, characterized in that... The minimum parking reward item Represented as: ， ， in, for Number of parking spaces at any given time for The total number of vehicles at any given time For vehicles Instantaneous velocity.

10. A multi-scale adaptive cooperative traffic signal control method according to claim 9, characterized in that... The aforementioned accident risk penalty items Represented as: ， ， in, This is an ordered sequence of accident risk values ​​for all intersection approach lanes. For the number of import lanes, This is a function for sorting in descending order. To return the given In sequence Functions with ordinal numbers, A well-known function that takes the index corresponding to the maximum value. The VaR value is the intersection scale. The VaR value is for the inlet gauge.

Images