Traffic signal lamp adaptive timing method and system based on surface control

Abstract

The present application relates to the technical field of traffic signal control, in particular to a traffic signal adaptive timing method and system based on area control, comprising: acquiring road network topology information of multiple intersections in a target area controlled region, and constructing a state tensor; constructing a signal timing decision model, taking the state tensor as input, the signal timing parameters of each intersection as decision actions, and defining a reward function that fuses local release efficiency and overall operation indicators; taking local release efficiency as the first optimization target, and performing first-stage training on the signal timing decision model; taking the fusion result of overall operation indicators and local release efficiency as the second optimization target, and performing second-stage training on the signal timing decision model; collecting the state tensor in real time and inputting it into the signal timing decision model, and outputting the signal timing scheme of each intersection. The purpose is to realize global resource allocation at the regional level and improve signal timing decision efficiency.

Description

Technical Field

[0001] This invention relates to the field of traffic signal control technology, specifically to a method and system for adaptive timing of traffic signals based on area control. Background Technology

[0002] Traffic signal control is a core means of alleviating urban traffic congestion and ensuring the efficiency of road network traffic. Existing traffic signal timing methods are mainly divided into two categories: fixed timing and adaptive timing. Fixed timing methods pre-set the duration of traffic lights for each phase based on historical traffic data, maintaining basic traffic order in scenarios with stable traffic flow patterns. However, actual traffic flow exhibits significant time-varying and random characteristics, with dynamic changes in traffic volume in different directions at different phases. Fixed timing schemes struggle to match real-time traffic demands, easily leading to wasted intersection capacity or exacerbated local congestion. To address the limitations of fixed timing, adaptive timing methods have emerged. Early adaptive methods used multiple pre-set timing schemes to address traffic characteristics at different times, but their response granularity remained relatively coarse, making it difficult to adapt to traffic fluctuations at the minute or even second level. In recent years, deep learning-based adaptive methods have significantly improved the traffic efficiency of single-point intersections by detecting intersection traffic volume in real time and dynamically adjusting signal timing.

[0003] However, existing adaptive timing methods primarily focus on single-point control, making independent decisions based solely on the local traffic conditions of a single intersection. Strong coupling exists between adjacent intersections in a traffic network; the flow rate at an upstream intersection directly affects the arrival flow at a downstream intersection, and the queuing status at a downstream intersection is also transmitted back to the upstream. Because point-control methods lack consideration for the inter-intersection correlation, they often fail to control traffic when local traffic pressure is excessive. For example, when the traffic demand at an intersection far exceeds its capacity, adjusting the timing only within that intersection cannot alleviate congestion; coordinated control of the input flow from upstream intersections is necessary to address the pressure at its root. Therefore, optimizing traffic signal control is essentially a regional-level global resource allocation problem, requiring the balance and maximization of overall regional capacity while ensuring the local flow efficiency of each intersection. Summary of the Invention

[0004] To achieve global resource allocation at the regional level and improve the efficiency of traffic light timing decisions, this invention provides a method and system for adaptive traffic light timing based on area control. The specific technical solution adopted is as follows:

[0005] The first aspect of the present invention provides a method for adaptive timing of traffic lights based on area control, the method comprising:

[0006] Obtain road network topology information of multiple intersections in the target area control region, and construct a state tensor representing the overall traffic situation of the target area control region based on the road network topology information and traffic detection data of each intersection in continuous signal cycles;

[0007] A signal timing decision model is constructed, with the state tensor as input and the signal timing parameters of each phase at each intersection as decision actions, and a reward function that integrates local release efficiency and overall operation indicators is defined.

[0008] The signal timing decision model is trained in the first stage with the local release efficiency as the first optimization objective.

[0009] Using the fusion result of the overall operation index and the local release efficiency as the second optimization objective, the signal timing decision model is trained in the second stage.

[0010] The state tensor of the current cycle is collected in real time and input into the signal timing decision model after the first and second training phases have been trained and converged, and the signal timing scheme of each intersection in the next cycle is output.

[0011] Furthermore, the construction of the state tensor representing the overall traffic situation of the target area includes:

[0012] The traffic detection data includes periodic traffic state data and phase-level release characteristic data. The periodic traffic state data includes the total release volume and entrance queuing characteristics. The phase-level release characteristic data includes the release volume of each phase, the idle time at the end of each phase, and the utilization rate of each phase.

[0013] The periodic traffic state data and phase-level release feature data of the same intersection within the same cycle are concatenated in a preset order to form the feature vector corresponding to the intersection.

[0014] Based on the road network topology information, the feature vectors of multiple intersections in multiple cycles are combined into a state tensor, and the dimension of the state tensor corresponds to the number of intersections, the number of cycles, and the dimension of the feature vectors.

[0015] Furthermore, the definition of the reward function that integrates local release efficiency and overall operational metrics includes:

[0016] Define a basic reward item, which consists of a release revenue item based on the release volume of each phase, an idle release penalty item based on the idle release duration at the end of each phase, and a timing smoothing item based on the timing change of adjacent cycles.

[0017] Calculate the basic reward items for each intersection in the target area control region, and statistically average the basic reward items for each intersection. Use the average value as the local release efficiency.

[0018] Define the overall operational metrics, including at least one of regional traffic volume, average travel time, and average waiting time;

[0019] The reward function is obtained by weighting and fusing the local release efficiency with the overall operation indicators.

[0020] Furthermore, the first phase of training includes:

[0021] Input the current cycle's state tensor into the signal timing decision model, and output the signal timing parameters for each phase at each intersection;

[0022] Calculate the first reward value corresponding to the current decision action based on the local release efficiency in the reward function;

[0023] The parameters of the signal timing decision model are adjusted with the goal of maximizing the first reward value.

[0024] Furthermore, the second phase of training includes:

[0025] Input the current cycle's state tensor into the signal timing decision model, and output the signal timing parameters for each phase at each intersection;

[0026] The second reward value corresponding to the current decision action is calculated based on the fusion result of the overall operation index and the local release efficiency in the reward function.

[0027] With the goal of maximizing the second reward value, the parameters of the signal timing decision model are further adjusted.

[0028] Furthermore, the signal timing decision model includes a local feature extraction network and a global feature fusion network;

[0029] The local feature extraction network is used to perform self-attention operations on the features of each intersection and its neighboring intersections in the state tensor, and output the local fusion features corresponding to each intersection.

[0030] The global feature fusion network is used to stitch together the local fusion features corresponding to each intersection, and to perform self-attention operation on the stitched features to output global features that represent the global traffic situation in the region. The global features are used to calculate the overall operation index.

[0031] Furthermore, in the second phase of training:

[0032] The signal timing decision model also includes multiple parallel abstract index prediction sub-networks. Each of the abstract index prediction sub-networks is used to predict the corresponding abstract index based on the local fusion features. The abstract index includes average travel time, average delay time, and average waiting time.

[0033] The signal timing decision model also includes a mutual attention fusion module, which is used to perform feature interaction on the outputs of multiple abstract index prediction sub-networks and output the fused abstract index prediction result.

[0034] The second reward value is calculated based on the fusion result of the predicted result of the fused abstract index and the result of the local release efficiency.

[0035] Furthermore, the training process of the signal timing decision model also includes:

[0036] Construct an adversarial perturbation generator, which takes the state tensor as input and outputs a perturbation vector representing unobserved confounding factors;

[0037] The perturbation vector is applied to the state tensor to generate state samples affected by the perturbation;

[0038] In the second phase of training, the signal timing decision model and the adversarial perturbation generator are alternately optimized, including:

[0039] The optimization objective of the adversarial perturbation generator is to generate a perturbation vector that minimizes the cumulative reward value of the signal timing decision model under the action of the perturbation vector;

[0040] The optimization objective of the signal timing decision model is to maximize the cumulative reward value under the influence of the disturbance vector.

[0041] Furthermore, the training process of the signal timing decision model also includes:

[0042] Define a robust reward function and configure it to introduce a perturbation sensitivity penalty term based on the reward function;

[0043] Construct an adversarial discriminant to distinguish whether the input state is affected by a disturbance;

[0044] The features output by the feature extraction part of the signal timing decision model are input into the adversarial discriminator. The parameters of the feature extraction part are updated with the goal of minimizing the adversarial loss between the feature extraction part and the adversarial discriminator.

[0045] Under the influence of the perturbation vector generated by the adversarial perturbation generator, the signal timing decision model is optimized based on the robust reward function.

[0046] The second aspect of the present invention provides a traffic signal adaptive timing system based on area control, used to execute the traffic signal adaptive timing method based on area control described in the first aspect of the present invention, the system comprising:

[0047] The state tensor construction module is configured to acquire road network topology information of multiple intersections in the target area control region, and construct a state tensor representing the overall traffic situation of the target area control region based on the road network topology information and traffic detection data of each intersection in continuous signal cycles.

[0048] The model building module is configured to build a signal timing decision model, taking the state tensor as input, the signal timing parameters of each phase at each intersection as decision actions, and defining a reward function that integrates local release efficiency and overall operation indicators.

[0049] The first-stage training module is configured to perform the first-stage training of the signal timing decision model with the local release efficiency as the first optimization objective.

[0050] The second-stage training module is configured to use the fusion result of the overall operating indicators and the local release efficiency as the second optimization objective to perform the second-stage training on the signal timing decision model.

[0051] The timing output module is configured to collect the state tensor of the current cycle in real time and input it into the signal timing decision model after the first training stage and the second training stage, and output the signal timing scheme of each intersection in the next cycle.

[0052] The present invention has the following beneficial effects:

[0053] The adaptive traffic signal timing method based on area control provided by this invention constructs a reward function that integrates local release efficiency and overall operational indicators. It employs a two-stage training strategy: first, training with local release efficiency as the optimization objective; then, training with the fusion of overall operational indicators and local release efficiency as the optimization objective. This allows the signal timing decision model to not only fully learn the basic release patterns of each intersection but also further master regional collaborative optimization capabilities. This improves the overall traffic operation level of the area control zone while ensuring local traffic efficiency at each intersection. The method uniformly represents the topology and real-time traffic situation of the area control zone as a state tensor, which serves as the model input. This ensures that the decision-making process fully considers the correlation between intersections, realizing a shift from isolated single-point control to regional global collaborative control. It solves the problems of local overload and uneven resource allocation caused by neglecting the coupling relationship between intersections in existing technologies, enhancing the adaptability and balance of the timing strategy in complex traffic environments. Attached Figure Description

[0054] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0055] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present invention for a method of adaptive timing of traffic lights based on area control.

[0056] Figure 2 This is a structural block diagram of a traffic signal adaptive timing system based on surface control provided in an exemplary embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0058] The present invention will be further described below with reference to the embodiments shown in the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0059] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0060] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0061] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0062] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0063] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0064] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0065] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0066] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0067] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0068] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0069] Example 1

[0070] Figure 1 This is a schematic flowchart of an exemplary embodiment of the present invention for a traffic light adaptive timing method based on area control. The method can be executed on a server (e.g., a cloud service platform or a locally deployed server).

[0071] Specifically, refer to Figure 1 The adaptive timing method for traffic lights based on area control includes:

[0072] Step 100: Obtain road network topology information for multiple intersections within the target area under control. Based on this information and traffic detection data from each intersection over consecutive signal cycles, construct a state tensor representing the overall traffic situation of the target area under control. Specifically, the road network topology information includes the connectivity between intersections within the target area under control, lane attributes, phase settings, etc., which can be extracted from traffic geographic information systems or road network planning data. Traffic detection data is acquired through sensors deployed at each intersection, collecting data such as lane traffic flow, occupancy rate, queue length, and vehicle transit time. At the end of each signal cycle, the periodic-level traffic state data and phase-level release feature data collected from each intersection are aggregated to form a feature vector for that intersection. Based on the road network topology information, the feature vectors from multiple intersections over consecutive cycles are combined into a three-dimensional state tensor. The three dimensions correspond to the number of intersections, the number of cycles, and the feature vector dimension, respectively, thus characterizing the traffic situation of the area under control in both time and space dimensions.

[0073] Step 200: Construct a signal timing decision model, using the state tensor as input and the signal timing parameters of each phase at each intersection as decision actions, and define a reward function that integrates local traffic release efficiency and overall operation indicators; wherein, the signal timing parameters are the absolute values ​​of the green light duration of each phase at each intersection, or the adjustment amount of the green light duration of each phase relative to the previous cycle; the timing parameters output by the model must meet preset constraints: the minimum green light duration of a single phase is not less than 12 seconds, the maximum green light duration is not more than 99 seconds, the total cycle duration is the sum of the durations of each phase, and fluctuates within a preset range;

[0074] In step 200, the signal timing decision model is constructed based on a deep reinforcement learning framework, and the nearest-end policy optimization algorithm (PPO) is selected for policy optimization. This model takes the state tensor generated in step 100 as input and outputs the signal timing parameters for each phase at each intersection, i.e., the green light duration or the adjustment amount of the green light duration for each phase. The adjustment amount must meet preset minimum and maximum duration constraints. To evaluate the quality of the decision action, a reward function that integrates local release efficiency and overall operational indicators is defined. Local release efficiency is calculated based on the basic reward items for each intersection. The basic reward items include a release benefit based on the release volume of each phase, an idle penalty based on the idle duration at the end of each phase, and a timing smoothing item based on the timing changes of adjacent cycles. Local release efficiency refers to the statistical average of the independent release efficiency of each intersection within the control area, reflecting only the throughput capacity of each intersection and not considering global collaborative indicators such as overall travel time and queue overflow. The local release efficiency is obtained by statistically averaging the basic reward items of each intersection within the control area. Overall operational indicators include at least one of regional traffic volume, average travel time, and average waiting time, used to measure the overall traffic operation level of the region.

[0075] Step 300: Using the local traffic release efficiency as the first optimization objective, perform the first stage training of the signal timing decision model; input the state tensor of the current cycle into the signal timing decision model, and the model outputs the signal timing parameters for each phase of each intersection; calculate the first reward value corresponding to the current decision action based on the local traffic release efficiency in the reward function; adjust the parameters of the signal timing decision model with the goal of maximizing the first reward value. This stage aims to enable the model to first master the basic traffic release logic of each intersection and learn to improve traffic efficiency at the local level.

[0076] Step 400: Using the fusion result of the overall operational indicators and local traffic release efficiency as the second optimization objective, the signal timing decision model undergoes a second stage of training. Based on the convergence of the first stage training, the state tensor of the current cycle is input into the signal timing decision model, and the model outputs the signal timing parameters for each phase at each intersection. At this point, the second reward value corresponding to the current decision action is calculated based on the fusion result of the overall operational indicators and local traffic release efficiency in the reward function. With the goal of maximizing the second reward value, the parameters of the signal timing decision model are further adjusted. By introducing the overall operational indicators, the model gradually takes into account regional coordination during the learning process, optimizing the timing of each intersection to improve overall traffic capacity and achieving a balance between local efficiency and global equilibrium.

[0077] Step 500: Real-time acquisition of the state tensor for the current period and input into the signal timing decision model after training and convergence in the first and second training phases, outputting the signal timing scheme for each intersection in the next period. Specifically, after completing the training and deploying the model, real-time acquisition of traffic detection data for the current period, construction of the state tensor for the current period in the manner of step 100, inputting it into the signal timing decision model after training and convergence in the first and second phases, the model outputs the signal timing scheme for each phase of each intersection in the next period, and sends it to the corresponding traffic signal controller for execution. This process is repeated to achieve real-time adaptive and cooperative control of traffic signals in the area under control.

[0078] As described above, the adaptive traffic signal timing method based on area control provided by this invention constructs a reward function that integrates local release efficiency and overall operational indicators. It employs a two-stage training strategy: first, training with local release efficiency as the optimization objective; then, training with the fusion of overall operational indicators and local release efficiency as the optimization objective. This allows the signal timing decision model to not only fully learn the basic release patterns of each intersection but also further master regional collaborative optimization capabilities. Thus, while ensuring local traffic efficiency at each intersection, it improves the overall traffic operation level of the area control zone. This method uniformly represents the topology and real-time traffic situation of the area control zone as a state tensor, which serves as the model input. This ensures that the decision-making process fully considers the correlation between intersections, realizing a shift from isolated single-point control to regional global collaborative control. It solves the problems of local overload and uneven resource allocation caused by neglecting the coupling relationship between intersections in existing technologies, enhancing the adaptability and balance of the timing strategy in complex traffic environments.

[0079] Example 2

[0080] Based on the above embodiment 1, as an optional implementation, the construction of the state tensor representing the overall traffic situation of the target area includes:

[0081] Step 110: The traffic detection data includes periodic traffic state data and phase-level release characteristic data. The periodic traffic state data includes the total release volume and entrance queuing characteristics of the period. The phase-level release characteristic data includes the release volume of each phase, the idle time at the end of each phase, and the utilization rate of each phase. A phase refers to the combination of traffic light states corresponding to a group of traffic flows that simultaneously obtain the right of way within a signal cycle. For example, in a crossroads, the signal state corresponding to east-west straight-ahead vehicles receiving a green light is one phase. The phase scheme for each intersection, i.e., the number of phases and the traffic flow direction corresponding to each phase, is predetermined by the intersection lane attributes and traffic organization method in the road network topology information and is used as known input in the implementation of the method of this invention. Preferably, each signal cycle is used as the basic statistical unit. Within each cycle, the system continuously records the traffic information at the lane level in 1-second increments and aggregates the data at the end of the cycle to generate periodic and phase-level characteristic data.

[0082] Phase-level release characteristic data are statistically analyzed separately for each controllable phase:

[0083] The number of vehicles allowed in each phase refers to the total number of vehicles that pass through the stop line during the green light period of that phase; the idle time at the end of each phase refers to the time difference between the last time a vehicle passes through during the green light period of that phase and the end of the green light period. If no vehicles are allowed in a phase, the idle time is equal to the green light duration of that phase; the utilization rate of each phase is defined as the number of vehicles allowed in that phase divided by the green light duration of that phase, reflecting the traffic efficiency per unit of green light time.

[0084] Periodic traffic status data includes the total number of vehicles allowed in a period, which is the sum of the number of vehicles allowed in all phases, and the entrance queuing characteristics. These characteristics are obtained by counting the cumulative number of seconds that queuing occurs in the entrance lanes within each period. Specifically, the number of vehicles waiting in the entrance lanes is detected per second. If the number is greater than zero, it is accumulated to one second. The cumulative value at the end of the period is the number of seconds that queuing occurs at the entrance.

[0085] Step 120: Concatenate the periodic-level traffic state data and phase-level release feature data of the same intersection within the same cycle in a preset order to form the feature vector corresponding to the intersection; specifically, for each intersection in each cycle, combine the various feature data obtained in Step 110 into a one-dimensional feature vector according to a fixed concatenation order. Assume that each intersection within the area control zone is equipped with... If there are several controllable phases, then the phase-level release characteristic data includes... The duration of each phase in the dimension, The release amount of each phase of the dimension, The duration of the final idling at the end of each phase of the dimension and The utilization rate of each phase of the dimension, in total Dimensionality. Periodic traffic state data includes one dimension of total traffic volume per period and one dimension of entrance queuing features, for a total of two dimensions. Therefore, the feature vector dimension for each intersection in each period is [dimension missing]. In this embodiment, The value is 9, so the feature vector dimension is 38;

[0086] Based on this, the preset sequence splicing refers to a pre-set and fixed feature splicing order; in this embodiment, the 38-dimensional feature vectors corresponding to each intersection in each cycle are spliced ​​in the following order: Dimensions 1-9: duration of each phase .

[0087] Dimensions 10-18: Release quantity for each phase Dimensions 19-27: End-of-phase idling duration Dimensions 28-36: Phase utilization 37th dimension: Total release volume per cycle 38th Dimension: Number of seconds the queue appears at the entrance In this model, phase numbers 1 to 9 correspond to predefined phase settings in the road network topology information, such as phase 1 for east-west straight traffic, phase 2 for east-west left turns, and phase 3 for north-south straight traffic. If the actual number of phases at an intersection is less than 9, the feature corresponding to the missing phase is set to zero. It should be noted that this splicing order remains unchanged during model training and deployment to ensure the consistency of the input data.

[0088] Step 130: Based on the road network topology information, combine the feature vectors of multiple intersections in multiple cycles into a state tensor. The dimension of the state tensor corresponds to the number of intersections, the number of cycles, and the dimension of the feature vectors.

[0089] Specifically, road network topology information describes the spatial connections between intersections within the controlled area, such as adjacent intersections and upstream / downstream relationships. Based on this topology information, the total number of controlled intersections is determined, denoted as . To depict the dynamic changes of the control area over time, the most recent one was selected. A series of consecutive signal cycles are used as the time window. The feature vectors corresponding to each intersection in each cycle are arranged according to the intersection index and cycle index to form a three-dimensional tensor. ,in The feature vector dimension is 38, and the elements in this state tensor are... Indicates the first The intersection at the first The first cycle The state tensor contains both the temporal changes in traffic conditions at each intersection and, through its spatial dimension, the implicit relationships between intersections, providing a comprehensive and structured representation of the regional traffic situation for subsequent signal timing decision models.

[0090] As described above, this embodiment concatenates multi-dimensional periodic and phase-level traffic information from each intersection within the area of ​​control into a feature vector in a fixed order, and then combines it into a three-dimensional state tensor based on road network topology information. This method considers the spatiotemporal characteristics of traffic data, and the state tensor can simultaneously reflect the evolution of each intersection in the time series and the spatial coupling relationship between intersections, providing rich and standardized input features for subsequent reinforcement learning models. This supports the model in learning regional collaborative control strategies more accurately and improves the adaptability and reliability of the area control method in complex traffic scenarios.

[0091] Example 3

[0092] Based on the above embodiments 1 and 2, as an optional implementation, the definition of the reward function that integrates local release efficiency and overall operational indicators includes:

[0093] Step 210: Define the basic reward item, which consists of a release revenue item based on the release amount of each phase, an idle release penalty item based on the idle release duration at the end of each phase, and a timing smoothing item based on the timing change of adjacent cycles.

[0094] Specifically, the basic reward item uses a single signal cycle as the decision step, in the first... At the end of each cycle, an instant reward is calculated based on the traffic data collected during that cycle. ;

[0095] Meanwhile, to avoid deviations caused by different period lengths, each component is divided by the period length. Normalization, basic reward The expression is:

[0096]

[0097] In the formula, This is the reward scaling factor, used to adjust the magnitude of the reward value; For period The total number of vehicles released within the time limit, which is the sum of the number of vehicles released in all phases; For the first The duration of each cycle, in seconds; The weighting coefficient for the penalty term for idling; For period The total duration of the final idle period of the effective phase within the inner phase; These are the weighting coefficients for the timing smoothing term; This refers to the timing variation between adjacent cycles. It should be noted that the phase-level release characteristic data, idle release duration statistics, and timing parameter optimization only apply to valid release phases. The set of valid release phases... Exclude safe phases such as all-red phases, yellow transition phases, and pedestrian clearing phases;

[0098] It should be noted that the weighting coefficients in the formula and It is not a purely dimensionless constant, but rather one that can be determined through experimental optimization in practical applications. Its value implicitly transforms the dimensions of the idling penalty term and the timing smoothing term into those comparable to the traffic efficiency term. Specifically, and The value of makes and Numerically with They are all of the same magnitude, thus ensuring a reasonable weighting of each item in the reward function. Final reward value. It is a dimensionless comprehensive score used to compare the merits of different strategies, rather than a quantity with strict physical meaning;

[0099] In the above formula, the release revenue item This value reflects the traffic efficiency per unit time; a higher value indicates higher efficiency. (Idle vehicle penalty item) Used to suppress the phenomenon of wasted green light, i.e., the green light being wasted due to no vehicles passing through at the end of the phase; timing smoothing term. Used to penalize drastic changes in timing between adjacent cycles in order to maintain the stability of signal control;

[0100] Total release volume Release quantity of each phase The summation is obtained and expressed as:

[0101]

[0102] In the formula, This represents the total number of controllable phases at the intersection. Indicates period The Middle The number of vehicles released per phase;

[0103] Total idle time Defined as the sum of the idle durations at the end of all valid phases, expressed as:

[0104]

[0105] In the formula, To effectively release the phase set, safe phases such as all-red and transitional clear phases are usually excluded to avoid false penalties for necessary safe phases; For period The Middle The end-of-phase release duration of the effective phase, if the last release within that phase occurs in the [missing information]. If the time is seconds, then the idle time is... , This indicates the duration of the green light for that phase. If the phase is never given permission, then... It equals the green light duration for that phase;

[0106] Timing variation Defined as the current period phase duration vector Phase duration vector of the previous cycle The L1 distance is expressed as:

[0107]

[0108] In the formula, For period The Middle The green light duration for a phase, in seconds; this penalty is used to encourage a smooth transition between adjacent time periods, avoiding traffic flow oscillations caused by frequent and large adjustments.

[0109] The effective phase refers to the phase that actually performs the function of releasing vehicles during the signal cycle, typically excluding safe phases that do not generate vehicle release, such as all-red phases, pedestrian-only phases, and cleared transition phases. The specific set of effective phases can be pre-set according to the phase scheme of each intersection;

[0110] It should be noted that different phase schemes may be used at intersections within the controlled area. For example, some intersections may have four phases: east-west straight, east-west left turn, north-south straight, and north-south left turn; some intersections may have two phases: east-west and north-south. For example, for a four-phase intersection, all four release phases are valid phases. For phase schemes that include a full-red clearing time, the full-red phase is not included in the set of valid release phases. This set remains unchanged during training and deployment to ensure the accuracy of the reward function calculation. This invention pre-obtains the number of phases and the corresponding traffic flow direction for each intersection through intersection lane attributes and phase setting data in the road network topology information. When constructing the state tensor, for intersections with insufficient phases, zero-padding or truncation is used to unify to the maximum number of phases; for intersections with a large number of phases, only the main phases are used for calculation. In this embodiment, each intersection is preset with 9 controllable phases. Intersections with fewer than 9 phases are filled with zero values ​​for the corresponding features to ensure consistent model input dimensions.

[0111] Step 220: Calculate the basic reward items for each intersection in the target area control region, and calculate the average of the basic reward items for each intersection. Use the average value as the local release efficiency.

[0112] Specifically, let the total number of controlled intersections within the controlled area be... For each intersection Calculate its basic reward for the period according to the method described in step 210. ,Right now:

[0113]

[0114] In the formula, For the first The intersection in the cycle Basic rewards; For the intersection In the cycle Total release volume The total duration of the idle discharge at the end of its effective phase. The timing variation of adjacent cycles is calculated; the basic bonus at each intersection is arithmetically averaged to obtain the local release efficiency. :

[0115]

[0116] in, The total number of controlled intersections within the area controlled by the traffic control system; local traffic release efficiency. It reflects the average level of local traffic efficiency at each intersection within the controlled area and is a component of subsequent integration rewards.

[0117] Step 230: Define the overall operational indicators, including at least one of regional traffic volume, average travel time, and average waiting time; specifically, the overall operational indicators are used to characterize the overall macroscopic traffic operation status of the controlled area. This embodiment uses a combination of three indicators: regional traffic volume, average travel time, and average waiting time. Regional traffic volume... Defined as period The sum of the total traffic volume allowed at all intersections within the inner control area, i.e., the total traffic volume allowed at all intersections. The cumulative value. Average travel time. This refers to the average time required for a vehicle to complete one full journey within the controlled area, which can be obtained through floating car data or simulation statistics. Average waiting time. This refers to the average time vehicles wait at a red light at an intersection. The above indicators can be weighted and combined to form an overall reward. :

[0118]

[0119] In the formula, These are the weighting coefficients for each indicator, which can be adjusted according to management objectives; For the first The average queue length of the entrance lanes at each intersection within the controlled area within a given period; the higher the overall operational index, the smoother the traffic flow in the area.

[0120] It should be noted that the average travel time and average waiting time There is a certain time lag in the statistical analysis of macroeconomic indicators. In this invention, Specifically defined as in the first The average travel time of vehicles leaving the area control zone within a given cycle. These vehicles' journeys may span multiple signal cycles, but their travel time reflects the combined effect of the signal control strategies within those cycles. In reinforcement learning training, delayed rewards can be implemented through a discount factor. Effective processing methods, such as qualification traces, enable the model to learn the correct causal relationships from delayed feedback.

[0121] Furthermore, during the simulation training phase, the system can record vehicle IDs and entry / exit times, instantly calculating the travel time of a vehicle upon departure and attributing it to the actions performed in each cycle the vehicle has experienced, ensuring that the reward signal is aligned with the timing of the actions. In the real-world deployment phase, if precise travel time cannot be obtained in real time, a proxy metric calculated in real time, such as the regional average speed, can be used as an approximation, or a moving average of historical statistical data can be used as the reward.

[0122] Step 240: Weight and fuse the local release efficiency with the overall operation index to obtain the reward function; specifically, the local release efficiency obtained in step 220... Overall operating indicators obtained in step 230 Weighted fusion is performed to form the final reward function. :

[0123]

[0124] In the formula, The fusion coefficient, ranging from [0,1], is used to balance the importance of local efficiency and global optimization. This reward function retains the incentive for the independent release efficiency of each intersection while introducing constraints on the overall operational level of the area, guiding the signal timing decision model to improve the overall traffic performance of the controlled area while pursuing local capacity. It should be noted that in some embodiments, the fusion coefficient... It can be dynamically adjusted according to the congestion level of the target area, with a selectable range of [0.3, 0.7]. The larger value is taken during off-peak hours and the smaller value is taken during peak hours.

[0125] As described above, this embodiment defines basic reward items including traffic release benefits, idle time penalty, and timing smoothing. Based on this, a weighted fusion reward function of local traffic release efficiency and overall operational indicators is constructed, enabling the signal timing decision model to simultaneously optimize single-point traffic efficiency and regional coordination levels. This method overcomes the deficiency of point-control methods in ignoring inter-intersection correlations. By introducing overall operational indicators, it guides the model to learn global resource allocation strategies, improving the balance and capacity of traffic flow in the controlled area. Simultaneously, the idle time penalty and smoothing items in the basic rewards help reduce idle green light periods and timing jitter.

[0126] Example 4

[0127] Based on the above embodiments 1, 2, and 3, as an optional implementation, the first stage of training includes:

[0128] Step 310: Input the state tensor of the current cycle into the signal timing decision model and output the signal timing parameters of each phase at each intersection; specifically, the signal timing decision model is constructed based on a deep reinforcement learning framework and optimizes the policy by selecting the near-end policy optimization algorithm (PPO).

[0129] In each training iteration, the state tensor corresponding to the current period is sampled from the experience replay pool or obtained online. This tensor is constructed in the manner described in Example 2, and includes the area control region. The nearest intersection Multi-dimensional traffic characteristics of each cycle. The state tensor is input into the signal timing decision model. After forward propagation, the model outputs the signal timing parameters for each phase at each intersection. Let the total number of intersections be... Each intersection contains If there are multiple controllable phases, the model output will be one. A dimensional matrix, where elements Indicates the first The first intersection The timing parameters are the green light duration for each phase within the current decision cycle, or the adjustment amount relative to the duration of that phase in the previous cycle. It should be noted that the output timing parameters must meet preset constraints, such as a minimum green light duration of 12 seconds and a maximum of 99 seconds, to ensure the safety and rationality of the control.

[0130] Step 320: Calculate the first reward value corresponding to the current decision action based on the local release efficiency in the reward function; specifically, after the model outputs the timing parameters of each phase at each intersection, the system executes the timing scheme and runs a complete signal cycle. At the end of the cycle, traffic detection data of each intersection during the cycle is collected according to the method described in Example 2, and the local release efficiency is calculated according to step 220 in Example 3. The local release efficiency is defined as the statistical average of the basic reward items at each intersection within the area under control; this local release efficiency is the first reward value used in the first phase of training, denoted as... The reward value focuses on the traffic efficiency of each intersection itself, without introducing overall regional indicators. The purpose is to let the model first learn how to effectively allow vehicles to pass through individual intersections.

[0131] Step 330: Adjust the parameters of the signal timing decision model with the goal of maximizing the first reward value. Specifically, a policy gradient method in reinforcement learning, such as the PPO algorithm, is used to update the model parameters with the goal of maximizing the expected cumulative reward. For the current decision cycle, the timing parameters output by the model are compared with the actual local release efficiency. The strategy loss function is calculated. The PPO algorithm ensures training stability by pruning the strategy update magnitude. In each update, the network parameters of the signal timing decision model are adjusted using the gradient ascent method, so that the model tends to output timing actions that achieve higher local release efficiency under similar conditions. This training phase continues until the model's local release efficiency on the validation set stabilizes or reaches the preset convergence condition. After the first phase of training, the model has mastered the basic release rules of each intersection and can reasonably allocate green light duration according to local traffic conditions.

[0132] As mentioned above, this stage employs local reward-driven model training, reducing the learning difficulty and avoiding training instability caused by prematurely introducing complex global metrics. By focusing on detailed learning first, the model can quickly grasp the basic release logic, providing initial parameters for collaborative optimization after introducing overall operational metrics in the second stage, thereby improving the convergence speed of the overall training process and the performance of the final strategy.

[0133] Example 5

[0134] Based on the above embodiments 1, 2, 3, and 4, as an optional implementation, the second stage of training includes:

[0135] Step 410: Input the state tensor of the current period into the signal timing decision model, and output the signal timing parameters for each phase of each intersection; specifically, after completing the first stage of training and the model parameters have initially converged, the second stage of training begins. This stage is still based on the same signal timing decision model, and the model structure remains unchanged. In each training iteration, the state tensor corresponding to the current period is sampled from the experience replay pool or obtained online. This tensor is constructed as described in Example 2 and includes the area control region. The nearest intersection Multi-dimensional traffic characteristics of each cycle. The state tensor is input into the signal timing decision model. After forward propagation, the model outputs the signal timing parameters for each phase at each intersection. Let the total number of intersections be... Each intersection contains If there are multiple controllable phases, the model output will be one. A dimensional matrix, where elements Indicates the first The first intersection The timing parameters are the green light duration for each phase within the current decision cycle, or the adjustment amount relative to the duration of that phase in the previous cycle. The output timing parameters must meet preset constraints, such as a minimum green light duration of no less than 12 seconds and a maximum of no more than 99 seconds, to ensure the safety and rationality of the control.

[0136] Step 420: Calculate the second reward value corresponding to the current decision action based on the fusion result of the overall operation index and the local release efficiency in the reward function; specifically, after the model outputs the timing parameters of each phase at each intersection, the system executes the timing scheme and runs a complete signal cycle. At the end of the cycle, traffic detection data of each intersection during the cycle is collected according to the method described in Example 2, and the following two types of indicators are calculated: one is the local release efficiency. The first is the statistical average of the basic reward items for each intersection within the controlled area, calculated according to step 220 in Example 3; the second is the overall operational indicators, including regional traffic volume. Average travel time Average delay time and average waiting time Among them, average delay time This refers to the additional time loss incurred by the vehicle during its actual journey due to factors such as signal control; the above overall operational indicators can be calculated using floating car data, simulation statistics, or traffic detector data. The second reward value used in the second phase of training. The weighted fusion result of partial release efficiency and overall operational indicators is calculated using the following formula:

[0137]

[0138] In the formula, This is the reward value for the first phase of training, also known as the local release efficiency. ; These are the weighting coefficients for each item, which can be configured according to management objectives to balance the importance of local efficiency and global optimization. This fusion reward retains the incentive for local release efficiency in the first stage, introduces constraints on the overall operation level of the region, and guides the model to improve the overall traffic performance of the controlled area while ensuring the basic traffic capacity of each intersection.

[0139] Step 430: Continue adjusting the parameters of the signal timing decision model with the goal of maximizing the second reward value. Specifically, use the same reinforcement learning algorithm as in the first stage, such as the PPO algorithm, to update the model parameters with the goal of maximizing the expected cumulative reward. For the current decision period, adjust the timing parameters based on the model output and the actual collected second reward value. The strategy loss function is calculated. The PPO algorithm ensures training stability by pruning the policy update magnitude. In each update, the network parameters of the signal timing decision model are further adjusted using the gradient ascent method, so that the model tends to output timing actions that obtain higher second reward values ​​under similar conditions. This training phase continues until the overall performance index of the model on the validation set tends to stabilize or reaches the preset convergence condition. Through the second stage of training, the model, based on mastering the local release rules, further learns regional collaborative optimization strategies, enabling it to dynamically adjust the timing of each intersection according to the overall traffic situation in the region, achieving a balance between local efficiency and global equilibrium.

[0140] As described above, this embodiment introduces a reward function that integrates overall operational indicators in the second stage of training. This allows the signal timing decision model to further learn regional collaborative optimization strategies based on the local release capability inherited from the first stage. This method overcomes the deficiency of point control methods in ignoring the correlation between intersections, guiding the model to improve the overall traffic capacity and traffic balance of the area control zone while ensuring the basic traffic efficiency of each intersection. This enhances the adaptability and robustness of the timing strategy in complex traffic environments.

[0141] Example 6

[0142] Based on the above embodiments 1, 2, 3, 4 and 5, as an optional implementation, the signal timing decision model includes a local feature extraction network and a global feature fusion network;

[0143] First, a local feature extraction network is used to perform self-attention operations on the features of each intersection and its neighboring intersections in the state tensor, outputting the local fusion features corresponding to each intersection. It should be noted that the neighboring intersections refer to the intersections directly connected to the target intersection, i.e., first-order neighbors. For example, in a road network topology, if an intersection... Intersection If there is a direct road connection, then belong Neighborhood intersection collection Each target intersection's neighborhood set contains itself, that is... The neighborhood range remains fixed during model training and deployment to ensure consistent input dimensions for the local feature extraction network. In scenarios with large road networks or requiring modeling of broader spatial relationships, a second-order neighborhood (i.e., neighbors of neighbors) can be used as an extension; however, this embodiment preferably uses a first-order neighborhood.

[0144] Specifically, the state tensor is For each target intersection Determine its neighboring intersection set based on road network topology information. The set includes intersections The state tensor contains the state itself and its directly connected adjacent intersections. Extract the nearest intersections within this set from the state tensor. The characteristics of each cycle form a shape as The local tensor. Reshape this local tensor into a sequence, i.e., a sequence of length . Each element is 3D eigenvectors.

[0145] The sequence is input into a self-attention module, which calculates the attention weights between elements within the sequence to perform weighted aggregation of features and capture local spatiotemporal dependencies. The self-attention module can employ a multi-head attention mechanism. After the self-attention operation, the output sequence is pooled or features from specific locations are extracted to obtain the intersection. Corresponding local fusion feature vector ,in This represents the dimension of the local fusion features. Repeating the above process for all intersections yields the set of local fusion features. .

[0146] Furthermore, the global feature fusion network is used to concatenate the local fused features corresponding to each intersection, and performs self-attention operations on the concatenated features to output global features representing the overall traffic situation in the region. Specifically, the above... The local fusion feature vectors are concatenated into a matrix according to the intersection order. That is, the local fusion features of each intersection are treated as a single row. This matrix is ​​then input into the global self-attention module for processing. The relationships between intersections are modeled. By calculating the attention weights between intersections and fusing information from different intersections, an updated global feature matrix is ​​obtained. Subsequently, regarding Global pooling yields a fixed-dimensional global feature vector. This vector integrates collaborative information from all intersections within the controlled area. This global feature is used to subsequently calculate overall operational metrics, such as regional traffic volume and average travel time.

[0147] As described above, this embodiment achieves multi-level feature extraction of regional traffic conditions by constructing a signal timing decision model composed of a local feature extraction network and a global feature fusion network. The local feature extraction network utilizes a self-attention mechanism to fuse spatiotemporal information from each intersection and its neighborhood, effectively characterizing the evolution of local traffic flow. The global feature fusion network models the overall regional collaborative relationship through self-attention operations between intersections, enabling the model to optimize timing strategies from a global perspective. This divide-and-conquer re-fusion network structure reduces the complexity of directly processing high-dimensional state tensors while fully preserving the spatial correlation in traffic data, providing high-quality feature input for reinforcement learning training, thereby improving the accuracy of timing decisions and the effectiveness of regional collaborative control.

[0148] Example 7

[0149] Based on the above embodiments 1, 2, 3, 4, 5, and 6, as an optional implementation method, in the second stage of training:

[0150] The signal timing decision model also includes multiple parallel abstract index prediction sub-networks. Each of the abstract index prediction sub-networks is used to predict the corresponding abstract index based on the local fusion features. The abstract index includes average travel time, average delay time, and average waiting time.

[0151] Specifically, let the local fused features of each intersection output by the local feature extraction network be a set. ,in For the first The local fusion feature vectors of each intersection are used. To predict the overall operational indicators of the controlled area, three parallel abstract indicator prediction subnetworks are constructed, corresponding to average travel time, average delay time, and average waiting time, respectively. Each subnetwork first performs self-attention operations on the input local fusion features to capture the correlation between intersections. Specifically, the local fusion feature vectors of each intersection are used. The local fusion feature vectors of each intersection are concatenated into a matrix. The input is fed into the self-attention module of each sub-network. The self-attention module updates the feature matrix by calculating the attention weights between intersections, thus obtaining the feature representation unique to each sub-network. ,in These correspond to three abstract metrics. Subsequently, [the following is discussed]... Global pooling is performed to obtain the initial prediction representation of this sub-network. .

[0152] Furthermore, the signal timing decision model also includes a mutual attention fusion module, which is used to perform feature interaction on the outputs of multiple abstract index prediction sub-networks and output the fused abstract index prediction result; specifically, the mutual attention fusion module receives the initial prediction representations of the three sub-networks. As input, information fusion is achieved through a cross-attention mechanism. For each metric... Using its own representation as the query and the representations of the other two metrics as the key and value, attention is calculated separately to obtain the fused representation of the metrics.

[0153] For example, for the subnetwork corresponding to the average travel time, As a query and As keys and values, a fused representation is obtained through attention weighting. Similarly, the average delay time and average waiting time were calculated separately. and This process ensures that each indicator's representation incorporates information from other indicators, enhancing the synergy between them. Finally, the fused representations are input into their respective output layers to obtain the final abstract indicator prediction, namely, the average travel time prediction. Average delay time prediction and average waiting time prediction .

[0154] Furthermore, the second reward value is calculated based on the fusion result of the fused abstract index prediction and the fusion result of the local release efficiency. In the second phase of training, at the end of each cycle, the actual overall operational index value is calculated based on the actual collected traffic data and compared with the above predicted value to form a supervision signal for training each sub-network. Simultaneously, the fused abstract index prediction result and the local release efficiency are compared... A weighted summation is performed to obtain the second reward value. The specific summation method is the same as the weighted summation method in Example 5.

[0155] As described above, this embodiment introduces multiple parallel abstract indicator prediction sub-networks and mutual attention fusion modules, enabling the signal timing decision model to learn the feature representations of different macro-indicators and enhance the synergy between indicators through cross-indicator information interaction. This structure improves the prediction accuracy of overall operational indicators, allowing the reward function to more comprehensively reflect the regional traffic operation level, thereby guiding the strategy to optimize local efficiency while synergistically optimizing multi-dimensional global objectives.

[0156] Example 8

[0157] Based on the above embodiments 1, 2, 3, 4, 5, 6, and 7, as an optional implementation, the training process of the signal timing decision model further includes:

[0158] Construct an adversarial perturbation generator, which takes the state tensor as input and outputs a perturbation vector representing unobserved confounding factors;

[0159] The adversarial perturbation generator is specifically a learnable neural network module whose input is the state tensor of the current cycle. This tensor contains the area under control. The nearest intersection The multi-dimensional traffic characteristics of each cycle; the disturbance generator outputs a disturbance vector by performing a series of nonlinear transformations on the state tensor. ,in The dimension of the perturbation vector can be set according to actual needs.

[0160] This disturbance vector is used to simulate confounding factors that cannot be directly observed, such as weather intensity, the impact of sudden accidents, and heterogeneity in driver behavior. Although these factors are not captured by sensors, they can have a significant impact on traffic state evolution and timing effectiveness.

[0161] The perturbation vector is then applied to the state tensor to generate state samples affected by the perturbation; the perturbation vector Needs to be related to the original state tensor The data is fused to generate state samples affected by the disturbance. Simultaneously, to preserve the physical meaning of the data, it is necessary to ensure that the fused state samples remain within a reasonable feature space range, and the generated state samples... This represents the regional traffic situation under the influence of unobserved confounding factors, and is used to test the robustness of the model in subsequent adversarial training.

[0162] In some embodiments, before the perturbation vector is applied to the state tensor, the z-score of each dimension of the original state tensor needs to be normalized to obtain the normalized state tensor. The perturbation vector is expanded into a perturbation tensor of the same dimension as the normalized state tensor through a learnable mapping network. ,Will and By adding each element and then restoring it to the original feature dimension range through denormalization, the perturbed state sample is obtained. The above process can ensure that the perturbation has a balanced impact on the features of each dimension, which is consistent with the influence of unobserved confounding factors in real-world scenarios.

[0163] Furthermore, in the second stage of training, the signal timing decision model and the adversarial perturbation generator are alternately optimized. This second stage of training is based on the multi-stage training described in Example 5, at which point the model already possesses basic release capability and preliminary collaborative capability. The specific process of alternating optimization is as follows:

[0164] In each training iteration, the parameters of the signal timing decision model are first fixed, and the adversarial perturbation generator is optimized. The optimization objective of the adversarial perturbation generator is to generate a perturbation vector that minimizes the cumulative reward value of the signal timing decision model under the action of the perturbation vector. Specifically, the generated perturbation vector is applied to the state tensor to obtain perturbed state samples. The input signal is used to set the timing decision model, which outputs timing parameters and executes the calculation to determine the cumulative reward value under the disturbance environment. ,in As a discount factor, For the first The reward value for the period. The adversarial perturbation generator adjusts its parameters using gradient descent to minimize this cumulative reward value, i.e., to find the worst perturbation.

[0165] The optimization objective of the adversarial perturbation generator is to generate a perturbation vector that minimizes the cumulative reward value of the signal timing decision model under the action of the perturbation vector. The signal timing decision model is optimized by fixing the parameters of the adversarial perturbation generator. In this case, the optimization objective of the model is to maximize the cumulative reward value under the action of the perturbation vector. That is, the generated perturbation vector is applied to the state tensor to obtain the perturbed sample, which is then input into the model. The model outputs timing parameters, the cumulative reward is calculated, and the model parameters are updated using the policy gradient method so that it can still obtain a high cumulative reward under perturbation conditions. Through alternating optimization, the signal timing decision model is forced to learn a robust policy that is insensitive to unobserved confounding factors.

[0166] The optimization objective of the signal timing decision model is to maximize the cumulative reward value under the influence of the perturbation vector. This alternating optimization process continues in the second stage of training until the model performance converges or reaches the preset number of training epochs. By introducing an adversarial perturbation generator, the model actively exposes itself to various potential perturbations during the training phase, thereby better enabling it to cope with uncertainties in real-world environments during actual deployment.

[0167] Finally, by constructing an adversarial perturbation generator and alternately optimizing it with the signal timing decision model, the model actively learns robustness to unobserved confounding factors during the training phase. This method simulates perturbation factors that are not directly perceptible in reality, enabling the policy network to remain efficient under perturbations and improving the reliability and generalization ability of the timing strategy in actual deployment.

[0168] Example 9

[0169] Based on the above embodiments 1, 2, 3, 4, 5, 6, 7, and 8, as an optional implementation, the training process of the signal timing decision model further includes:

[0170] Define a robust reward function, which is based on the reward function that integrates local release efficiency and overall operation indicators as described in Example 3, and introduces a disturbance sensitivity penalty term;

[0171] Let the fusion reward obtained in Example 3 be:

[0172]

[0173] In the formula, For the first Local release efficiency per cycle For the first Overall operating indicators for each cycle The fusion coefficient is used to balance the importance of local efficiency and global optimization. To improve the policy's robustness to unobserved confounding factors, two penalties are added to the fusion reward: the volatility of the reward value under multiple perturbation samplings and the cumulative reward in the worst case. Specifically, a robust reward function... Represented as:

[0174]

[0175] In the formula, and This is a balancing coefficient used to adjust the relative importance of the two penalties. This indicates the fusion reward under multiple perturbation samplings. The variance is used to measure the stability of the strategy, and the subscript is... This indicates that the variance depends on the disturbance variable. ; The cumulative reward in the worst-case scenario is approximated by sampling using an adversarial perturbation generator, and is defined as follows: That is, the minimum cumulative reward among all possible perturbations, where As a discount factor, For the first The fusion reward for each cycle, This represents an adversarial perturbation generator. This robust reward function guides the strategy to actively avoid significant performance fluctuations caused by perturbations while pursuing high efficiency.

[0176] Construct an adversarial discriminant to distinguish whether the input state is affected by a disturbance; adversarial discriminant It is a binary classification neural network whose input is a state tensor or a representation of the state tensor after feature extraction, and whose output is a scalar representing the probability that the input state is perturbed. During training, the original state tensor is... As an unperturbed positive sample, the perturbed state sample generated in Example 8 is used. As perturbated negative samples, the discriminator is trained to accurately distinguish between the two. The discriminator's loss function uses cross-entropy loss, expressed as:

[0177]

[0178] In the formula, This represents the loss function of the discriminator; Represents the original state sample Expectations; Represents the tensor of the perturbed state Expectations; This indicates that the discriminator is sensitive to the original state. The output is the probability of no disturbance. This indicates that the discriminator responds to the disturbed state. The output is the probability of no disturbance. The loss is a function of the natural logarithm; by minimizing this loss, the discriminator learns to identify whether the state is affected by unobserved confounding factors.

[0179] The features output from the feature extraction part of the signal timing decision model are input into the adversarial discriminator. The parameters of the feature extraction part are updated with the goal of minimizing the adversarial loss between the feature extraction part and the adversarial discriminator. The feature extraction part is the front end of the local feature extraction network and the global feature fusion network described in Example 6, and its output is a hidden feature vector representing the regional traffic situation. The original state and the disturbed state are input into the feature extraction part respectively to obtain the corresponding features. and These features are then fed into an adversarial discriminator. However, the goal of the feature extraction part is to prevent the discriminator from accurately distinguishing the feature sources; that is, to make the discriminator's output probabilities for the two types of features as close as possible. Therefore, the adversarial loss of the feature extraction part is defined as:

[0180]

[0181] In the formula, This represents the adversarial loss in the feature extraction part; Indicates from the original state Extracted features; Indicates a state of disturbance Extracted features; This indicates that the discriminator recognizes the features. The output is the probability of no disturbance. This indicates that the discriminator recognizes the features. The output is the probability of being undisturbed; by minimizing this adversarial loss, the feature extraction part is forced to learn a robust representation that is insensitive to perturbations, making the discriminator unable to determine whether the state is perturbed based on the features; by minimizing The feature extraction part is forced to learn a robust representation that is insensitive to perturbations, making it impossible for the discriminator to determine whether the state is disturbed based on the features.

[0182] Finally, under the influence of the perturbation vector generated by the adversarial perturbation generator, the signal timing decision model is optimized based on the robust reward function. Specifically, in the second stage of training, the robust reward function and feature invariance learning are introduced simultaneously, combined with the alternating optimization process of Example 8. In each training iteration, a perturbation vector is first generated using the current perturbation generator and applied to the state tensor to obtain the perturbed sample; then, the robust reward value under the perturbed sample is calculated. The signal timing decision model includes a feature extraction component and a subsequent policy network, updating parameters to maximize the cumulative robust reward. Simultaneously, the feature extraction component participates in adversarial training against the discriminator, updating parameters to minimize... Through this joint optimization, the model not only learns to maintain high rewards under perturbations, but also learns to extract features that are insensitive to perturbations, thereby further improving the robustness of the policy. This process continues until the model converges.

[0183] It should be noted that the first stage of training employs a distributed single-intersection optimization strategy. The local feature extraction network of the signal timing decision model is split into independent branches corresponding to each intersection. Each intersection's own basic reward term serves as the independent optimization objective, and the network parameters of each branch are updated separately. This stage does not perform global feature fusion or introduce regional overall indicators; it only allows the model to learn the basic release rules of a single intersection, ensuring that each intersection possesses independent and efficient release capabilities. The second stage of training freezes the low-level parameters of the local feature extraction network trained in the first stage, only fine-tuning the upper-level parameters, and enables the global feature fusion network. The model's regional collaborative optimization capability is trained with the fusion result of local release efficiency and global operational indicators as the optimization objective.

[0184] It should also be noted that the signal timing decision model includes both a local feature extraction network and a global feature fusion network in its definition, and both training phases use the same structure. In the first training phase, the parameters of the global feature fusion network are frozen, and only the output of the local feature extraction network is used for decision-making and reward calculation during forward propagation, thus achieving the effect of not performing global feature fusion. At this time, although the global feature fusion network exists in the model, its parameters are not updated, and its output is not used.

[0185] After the first phase of training is completed, the second phase of training begins. At this stage, the parameters of the global feature fusion network are unfrozen and fed into the model. The output of the local feature extraction network serves as the input to the global feature fusion network, enabling forward propagation and parameter updates for the complete model. The parameters of the local feature extraction network are inherited from the first phase of training, while the parameters of the global feature fusion network are trained from random initialization. This parameter freezing and unfreezing mechanism seamlessly connects the two training phases, ensuring the consistency of the model structure and the reproducibility of the training process.

[0186] As described above, this embodiment implements a dual defense mechanism against unobserved confounding factors by introducing a robust reward function and an adversarial discriminator. The variance penalty and worst-case penalty in the robust reward function directly constrain the performance stability of the strategy under perturbations, while the adversarial discriminator guides the feature extraction part to learn invariant representations, fundamentally reducing the impact of perturbations on decision-making. This mechanism significantly enhances the reliability and generalization ability of the signal timing decision model in real-world complex traffic environments, enabling it to effectively cope with interference from unpredictable factors such as weather and emergencies, ensuring the continuous and efficient operation of traffic signal control in area control zones.

[0187] In some embodiments, taking a city core area containing six consecutive intersections as an example, i.e., regional coordinated traffic signal control, the complete implementation process of the above-mentioned adaptive timing method for traffic lights based on area control is explained.

[0188] All six intersections are crossroads or subsets thereof, with a clear road network topology, uniform spacing between adjacent intersections, and traffic flow characteristics exhibiting typical morning and evening peak hours and off-peak fluctuations.

[0189] First, data acquisition and state tensor construction are performed. Geomagnetic coils and video detectors are deployed at each intersection to collect real-time data on traffic flow, occupancy, queue length, and other data for each lane. The signal cycle is used as the basic decision unit, with each cycle lasting [duration missing]. The average duration is 120 seconds.

[0190] Within each cycle, the system continuously records data in 1-second increments, and at the end of the cycle, aggregates the data to generate a 38-dimensional feature vector for each intersection. The phase-level features include the green light duration for nine phases. Release volume Ending idle time and utilization rate Periodic-level characteristics include total release volume per period. And the number of seconds it takes to appear in the queue at the entrance. This refers to the cumulative number of seconds that vehicles are waiting in the entrance lane within a given period.

[0191] All the above features are concatenated in a preset order to form the feature vector corresponding to each intersection in each cycle; based on the road network topology information, the nearest... In one cycle, the feature vectors of the six intersections are combined into a state tensor. , of which elements Indicates the first The intersection at the first The first cycle 3D eigenvalues;

[0192] Secondly, a signal timing decision model is constructed, which includes a local feature extraction network and a global feature fusion network. For each target intersection... Based on the road network topology, a set of neighboring intersections is determined, including the intersection itself and its adjacent intersections. For example, the neighborhood of intersection 1 is {1,2}, and the neighborhood of intersection 2 is {1,2,3}. Features of the most recent 10 cycles for all intersections within the neighborhood are extracted from the state tensor to form a local tensor. This local tensor is then input into a self-attention module to capture spatiotemporal dependencies and output a locally fused feature vector. The local fusion features of the six intersections are stitched together into a matrix. Input the global self-attention module to obtain the updated global feature matrix. Then, global pooling is used to obtain the global feature vector. In addition, the model includes three parallel abstract index prediction subnetworks, each taking local fusion features as input and predicting average travel time through self-attention and mutual attention fusion. Average delay time and average waiting time ;

[0193] The mutual attention fusion module uses the output of one sub-network as the query and the other two as keys to achieve information exchange, ultimately outputting the fused metric prediction value. A reward function is defined, with the basic reward term occurring within a certain period. Calculate at the end:

[0194]

[0195] in, This is the scaling factor. , Effective phase set Excluding all-red phases; ;

[0196] The local release efficiency is calculated as the average of the base bonuses for the six intersections:

[0197]

[0198] Overall operational indicators are determined by regional traffic volume. Average travel time Average delay time and average waiting time Composed of various elements;

[0199] Based on this, multi-stage training is conducted. The first stage focuses on optimizing local traffic release efficiency, using the PPO algorithm to train the model. In each training round, the current cycle's state tensor is input into the model, and the model outputs the green light duration adjustment for each phase at each intersection. After execution, the calculation is performed. As a reward, the model parameters are updated; training continues for 2000 epochs until the local release efficiency converges. The second stage introduces overall performance metrics and adversarial training. An adversarial perturbation generator is constructed. Its input is the state tensor Output perturbation vector The perturbation vector is expanded into a perturbation tensor of the same dimension as the state tensor through a mapping network, and then added to the original tensor to generate perturbed state samples. In the second phase of training, the model and generator are optimized alternately: with the model parameters fixed, the generator updates its parameters to minimize the cumulative fusion reward of the model under perturbation; with the generator fixed, the model updates its parameters to maximize the cumulative robust reward.

[0200] At the same time, construct an adversarial discriminator. The first stage of training uses the locally fused features output by the feature extraction part as input to distinguish whether the features come from the original state or the perturbed state. The feature extraction part is trained adversarially with the discriminator to make the extracted features insensitive to perturbations. The second stage of training lasts for 3000 rounds.

[0201] The third stage is adversarial fine-tuning, where the perturbation generator parameters are fixed, and the model is further optimized under the worst perturbation environment. That is, the perturbation corresponding to the minimum cumulative reward generated by the generator is applied to the state, and the model updates its parameters with the goal of maximizing robust rewards. The training is carried out for 1000 rounds until convergence. It should be noted that the core training process of this invention is a two-stage training, where the second stage of training may further include adversarial training sub-steps and robust fine-tuning sub-steps. The above sub-steps are all optional implementations of the second stage of training and do not deviate from the core architecture of the two-stage training of this invention.

[0202] Finally, the converged signal timing decision model was deployed to the area control system at the six intersections. Before the end of each signal cycle, the system collected real-time traffic data from each intersection for the current cycle and constructed a state tensor in the same manner. The model takes an input signal and outputs the green light duration for each phase at each intersection in the next cycle. For example, the model outputs that at intersection 1, the east-west straight-ahead phase will increase by 3 seconds, and the north-south left-turn phase will decrease by 2 seconds, etc. The duration of each phase must meet the constraint of 12-99 seconds. The output timing scheme is then sent to the traffic signal controllers at each intersection for execution, realizing coordinated adaptive control of regional traffic signals.

[0203] As described above, the present invention has the following beneficial effects:

[0204] A two-stage training strategy is adopted. The first stage aims at local traffic release efficiency, enabling the model to quickly master the basic traffic release rules of each intersection. The second stage aims at the fusion of local and global results, enabling the model to learn regional collaborative optimization capabilities. This solves the problems of slow convergence and easy local efficiency decline in existing single-stage training methods, while balancing the fairness of local intersection traffic release and regional global traffic efficiency.

[0205] A reward function that integrates local traffic release efficiency and overall operational indicators is constructed, taking into account both the traffic efficiency of individual intersections and indicators such as travel time and queue length of the entire region. This solves the problem of unfair timing caused by existing methods that focus on the overall situation while neglecting local aspects.

[0206] By adopting a local-global two-level attention network structure, the spatiotemporal coupling relationship between intersections is captured, realizing the transformation from single-point isolated control to regional global collaborative control, and solving the local overload problem caused by the neglect of inter-intersection correlation in existing point control methods;

[0207] By training the model against adversarial perturbation generators and adversarial discriminators, the model learns robust features that are insensitive to unobserved confounding factors. Combined with a robust reward function, this improves the model's generalization ability and anti-interference capability in real and complex traffic scenarios.

[0208] Example 10

[0209] It should be understood that the aforementioned embodiments of the adaptive timing method for traffic lights based on area control can also be similarly applied to the following adaptive timing system for traffic lights based on area control for similar extensions. For simplicity, it has not been described in detail.

[0210] Figure 2 This is an exemplary embodiment of the present invention providing a traffic light adaptive timing system based on area control. (Refer to...) Figure 2 The system includes:

[0211] The state tensor construction module is configured to acquire road network topology information of multiple intersections in the target area control region, and construct a state tensor representing the overall traffic situation of the target area control region based on the road network topology information and traffic detection data of each intersection in continuous signal cycles.

[0212] The model building module is configured to build a signal timing decision model, taking the state tensor as input, the signal timing parameters of each phase at each intersection as decision actions, and defining a reward function that integrates local release efficiency and overall operation indicators.

[0213] The first-stage training module is configured to perform the first-stage training of the signal timing decision model with the local release efficiency as the first optimization objective.

[0214] The second-stage training module is configured to use the fusion result of the overall operating indicators and the local release efficiency as the second optimization objective to perform the second-stage training on the signal timing decision model.

[0215] The timing output module is configured to collect the state tensor of the current cycle in real time and input it into the signal timing decision model after the first training stage and the second training stage, and output the signal timing scheme of each intersection in the next cycle.

[0216] Example 11

[0217] In addition, embodiments of the present invention also provide an electronic device, including:

[0218] At least one processor; and,

[0219] A memory communicatively connected to the at least one processor; wherein,

[0220] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the area-controlled adaptive timing method for traffic lights as described in the first aspect of the present invention.

[0221] Figure 3 This is a schematic diagram of the structure of an application embodiment of the electronic device of the present invention. Below, refer to... Figure 3 This describes an electronic device according to embodiments of the present invention. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0222] like Figure 3As shown, the electronic device includes one or more processors and memory. The processor may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the area-based adaptive timing method for traffic lights based on various embodiments of the present invention described above, and / or other desired functions.

[0223] In one example, the electronic device may further include input and output devices, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). Furthermore, the input device may include, for example, a keyboard, a mouse, etc. The output device can output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0224] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0225] In addition to the methods and devices described above, embodiments of the present invention may also be computer program products, wherein a computer-readable storage medium stores a program implementing a method for adaptive timing of traffic lights based on area control, and the program implementing the method for adaptive timing of traffic lights based on area control is executed by a processor to implement the steps of the method for adaptive timing of traffic lights based on area control as described in various embodiments of the present invention.

[0226] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0227] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the area-controlled adaptive timing method for traffic lights according to various embodiments of the present invention described in the foregoing portion of this specification.

[0228] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0229] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0230] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details of the invention described above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the specific details described above.

[0231] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0232] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0233] The methods and apparatus of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0234] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of the present invention.

[0235] The above description of aspects of the invention is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features of the invention herein.

[0236] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms described herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

[0237] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0238] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for adaptive timing of traffic lights based on area control, characterized in that, The method includes: Obtain road network topology information of multiple intersections in the target area control region, and construct a state tensor representing the overall traffic situation of the target area control region based on the road network topology information and traffic detection data of each intersection in continuous signal cycles; A signal timing decision model is constructed, with the state tensor as input and the signal timing parameters of each phase at each intersection as decision actions, and a reward function that integrates local release efficiency and overall operation indicators is defined. The signal timing decision model is trained in the first stage with the local release efficiency as the first optimization objective. Using the fusion result of the overall operation index and the local release efficiency as the second optimization objective, the signal timing decision model is trained in the second stage. The state tensor of the current cycle is collected in real time and input into the signal timing decision model after the first and second training phases have been trained and converged, and the signal timing scheme of each intersection in the next cycle is output.

2. The adaptive timing method for traffic lights based on area control as described in claim 1, characterized in that, The construction of the state tensor representing the overall traffic situation of the target area includes: The traffic detection data includes periodic traffic state data and phase-level release characteristic data. The periodic traffic state data includes the total release volume and entrance queuing characteristics. The phase-level release characteristic data includes the release volume of each phase, the idle time at the end of each phase, and the utilization rate of each phase. The periodic traffic state data and phase-level release feature data of the same intersection within the same cycle are concatenated in a preset order to form the feature vector corresponding to the intersection. Based on the road network topology information, the feature vectors of multiple intersections in multiple cycles are combined into a state tensor, and the dimension of the state tensor corresponds to the number of intersections, the number of cycles, and the dimension of the feature vectors.

3. The adaptive timing method for traffic lights based on area control as described in claim 2, characterized in that, The defined reward function, which integrates local release efficiency and overall operational metrics, includes: Define a basic reward item, which consists of a release revenue item based on the release volume of each phase, an idle release penalty item based on the idle release duration at the end of each phase, and a timing smoothing item based on the timing change of adjacent cycles. Calculate the basic reward items for each intersection in the target area control region, and statistically average the basic reward items for each intersection. Use the average value as the local release efficiency. Define the overall operational metrics, including at least one of regional traffic volume, average travel time, and average waiting time; The reward function is obtained by weighting and fusing the local release efficiency with the overall operation indicators.

4. The adaptive timing method for traffic lights based on area control as described in claim 1, characterized in that, The first phase of training includes: Input the current cycle's state tensor into the signal timing decision model, and output the signal timing parameters for each phase at each intersection; Calculate the first reward value corresponding to the current decision action based on the local release efficiency in the reward function; The parameters of the signal timing decision model are adjusted with the goal of maximizing the first reward value.

5. The adaptive timing method for traffic lights based on area control as described in claim 1, characterized in that, The second phase of training includes: Input the current cycle's state tensor into the signal timing decision model, and output the signal timing parameters for each phase at each intersection; The second reward value corresponding to the current decision action is calculated based on the fusion result of the overall operation index and the local release efficiency in the reward function. With the goal of maximizing the second reward value, the parameters of the signal timing decision model are further adjusted.

6. The adaptive timing method for traffic lights based on area control as described in claim 1, characterized in that, The signal timing decision model includes a local feature extraction network and a global feature fusion network; The local feature extraction network is used to perform self-attention operations on the features of each intersection and its neighboring intersections in the state tensor, and output the local fusion features corresponding to each intersection. The global feature fusion network is used to stitch together the local fusion features corresponding to each intersection, and to perform self-attention operation on the stitched features to output global features that represent the global traffic situation in the region. The global features are used to calculate the overall operation index.

7. The adaptive timing method for traffic lights based on area control as described in claim 6, characterized in that, In the second phase of training: The signal timing decision model also includes multiple parallel abstract index prediction sub-networks. Each of the abstract index prediction sub-networks is used to predict the corresponding abstract index based on the local fusion features. The abstract index includes average travel time, average delay time, and average waiting time. The signal timing decision model also includes a mutual attention fusion module, which is used to perform feature interaction on the outputs of multiple abstract index prediction sub-networks and output the fused abstract index prediction result. The second reward value is calculated based on the fusion result of the predicted result of the fused abstract index and the result of the local release efficiency.

8. The adaptive timing method for traffic lights based on area control as described in any one of claims 1 to 7, characterized in that, The training process of the signal timing decision model also includes: Construct an adversarial perturbation generator, which takes the state tensor as input and outputs a perturbation vector representing unobserved confounding factors; The perturbation vector is applied to the state tensor to generate state samples affected by the perturbation; In the second phase of training, the signal timing decision model and the adversarial perturbation generator are alternately optimized, including: The optimization objective of the adversarial perturbation generator is to generate a perturbation vector that minimizes the cumulative reward value of the signal timing decision model under the action of the perturbation vector; The optimization objective of the signal timing decision model is to maximize the cumulative reward value under the influence of the disturbance vector.

9. The adaptive timing method for traffic lights based on area control as described in claim 8, characterized in that, The training process of the signal timing decision model also includes: Define a robust reward function and configure it to introduce a perturbation sensitivity penalty term based on the reward function; Construct an adversarial discriminant to distinguish whether the input state is affected by a disturbance; The features output by the feature extraction part of the signal timing decision model are input into the adversarial discriminator. The parameters of the feature extraction part are updated with the goal of minimizing the adversarial loss between the feature extraction part and the adversarial discriminator. Under the influence of the perturbation vector generated by the adversarial perturbation generator, the signal timing decision model is optimized based on the robust reward function.

10. A traffic signal adaptive timing system based on area control, characterized in that, The system is used to execute the area-controlled adaptive timing method for traffic lights according to any one of claims 1 to 9, the system comprising: The state tensor construction module is configured to acquire road network topology information of multiple intersections in the target area control region, and construct a state tensor representing the overall traffic situation of the target area control region based on the road network topology information and traffic detection data of each intersection in continuous signal cycles. The model building module is configured to build a signal timing decision model, taking the state tensor as input, the signal timing parameters of each phase at each intersection as decision actions, and defining a reward function that integrates local release efficiency and overall operation indicators. The first-stage training module is configured to perform the first-stage training of the signal timing decision model with the local release efficiency as the first optimization objective. The second-stage training module is configured to use the fusion result of the overall operating indicators and the local release efficiency as the second optimization objective to perform the second-stage training on the signal timing decision model. The timing output module is configured to collect the state tensor of the current cycle in real time and input it into the signal timing decision model after the first training stage and the second training stage, and output the signal timing scheme of each intersection in the next cycle.

Images