Traffic signal control method and system based on multi-source event and double-ring phase cooperation

Abstract

The application discloses a traffic signal control method and system based on multi-source events and double-ring phase coordination, relates to the technical field of traffic signal control, constructs a road intersection mathematical model, sets a double-ring phase structure, and collects multi-source sensing data in real time; based on the multi-source sensing data, a deep Q network model is used to model an intersection traffic signal control problem, a state space, an action space and a reward function are defined, and the double-ring phase structure is decomposed for cooperative control by multiple agents; the reward function is constructed based on vehicle delay time difference, waiting time difference, parking frequency difference and queue length difference; the reward fed back by the environment to each agent is used to control each agent to perform an action, whether phase retention or phase switching is determined, an optimal phase scheme of signal control is obtained, time stamp driving, multi-event coupling analysis and double-ring phase structure are fused, and deep reinforcement learning algorithm is combined to realize intersection signal timing optimization.

Description

Technical Field

[0001] This invention relates to the field of traffic signal control technology, and in particular to a traffic signal control method and system based on multi-source events and dual-loop phase coordination. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] With accelerated urbanization and rapid growth in motor vehicle ownership, urban road intersections have become major bottlenecks for traffic congestion. Traditional signal control methods, such as timed control and sensor-based control, while playing a vital role in early traffic management, are no longer sufficient to meet the complex demands of modern urban traffic. Timed control, with its fixed cycles and green light ratios, cannot adapt to the dynamic fluctuations in traffic flow, often resulting in wasted green light time or queue overflows, especially during morning and evening rush hours. While sensor-based control can adjust signals in real time through detectors, its response is lagging and it relies on high-precision detection equipment, making it unstable in mixed traffic flow scenarios.

[0004] In recent years, intelligent traffic signal control technology has gradually developed, with reinforcement learning-based methods demonstrating significant advantages. Existing reinforcement learning techniques applied to traffic signal control mainly fall into two categories: one employs dynamic phase sequence strategies, which flexibly adjust phase order to adapt to traffic demands, but suffers from strong randomness in phase switching and violates driver behavior habits; the other employs fixed phase sequence strategies, which, while ensuring orderly phase switching, lose the flexibility inherent in reinforcement learning for handling sudden traffic flows. Furthermore, existing research largely focuses on theoretical verification at single-point intersections, lacking compatibility design with actual signal control equipment, making direct application in engineering practice difficult.

[0005] Traditional methods treat the dual-loop phases as a whole for control, failing to handle the temporal coupling between phases. Secondly, the state-space design of reinforcement learning models does not consider safety constraints during phase switching, easily generating control commands for dangerous, conflicting phases. Existing methods also suffer from time-varying adaptability deficiencies: firstly, they fail to consider diurnal traffic flow differences, leading to significant wasted green light time during off-peak hours with fixed strategies; secondly, they lack awareness of seasonal and event-driven traffic changes, making them unable to respond to emergencies such as severe weather or large-scale events. Although some methods attempt to adjust timing through time-segmentation, they still struggle to effectively capture minute-level traffic fluctuations and the real-time impact of emergencies. Furthermore, most methods lack adaptive mechanisms for different traffic levels, resulting in a significant decline in control performance under medium-to-high traffic conditions. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a traffic signal control method and system based on multi-source events and dual-loop phase coordination, which integrates timestamp-driven, multi-event coupling analysis and dual-loop phase structure, and combines deep reinforcement learning (improved DQN) algorithm to achieve intersection signal timing optimization.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, the present invention provides a traffic signal control method based on multi-source events and dual-loop phase coordination, comprising:

[0009] Construct a mathematical model of the road intersection and set up a double-loop phase structure to collect multi-source sensing data in real time;

[0010] Based on the multi-source sensing data, a deep Q-network model is used to model the traffic signal control problem at intersections, defining the state space, action space, and reward function. The dual-loop phase structure is decomposed and controlled collaboratively by multiple agents. The reward function is constructed based on the vehicle delay time difference, waiting time difference, number of stops difference, and queue length difference.

[0011] Based on the rewards provided by the environment to each agent, the agent executes actions, determines whether to maintain the phase or switch the phase, and obtains the optimal phase scheme for signal control.

[0012] A further technical solution is that the multi-source sensing data includes lidar data, video monitoring data, meteorological sensor data, and manually reported events. The lidar data includes vehicle presence status, real-time vehicle speed, headway, and traffic flow. The video monitoring data includes lane-level queue length, vehicle type identification, and traffic density. The meteorological sensor data includes rainfall intensity, visibility, and road surface condition. The manually reported events are pre-reported event information.

[0013] A further technical solution involves fusing traffic state, time state, and event state into a state space. Traffic state and event state are obtained based on multi-source sensing data, and combined with time state to obtain a state vector.

[0014] In a further technical solution, the action space is defined as a binary decision combination of multiple agents, including a first agent, a second agent, and a third agent. The first and second agents decide whether the preceding phase in the dual-loop phase structure should be switched after the minimum green light time is completed, and the third agent decides whether the following phase in the dual-loop phase structure should be cut off simultaneously after the minimum green light time ends.

[0015] A further technical solution involves setting safety constraints during phase switching, including the minimum green light time for the preceding phase, the transition time between yellow and full red lights, and the forced synchronization cutoff of the following phase.

[0016] In a further technical solution, the reward function is expressed as:

[0017]

[0018] in, Indicates the reward value. Indicates the attenuation coefficient. Indicates the weight of vehicle delays. This indicates the average vehicle delay result of the previous training round. This indicates the average vehicle delay result for the current training round. Indicates the weight of waiting time. This represents the average waiting time result from the previous training round. This indicates the average waiting time for the current training round. Indicates the weight of the number of parking times. This represents the average number of stops in the previous training round. This represents the average number of stops in the current training round. Indicates the weight of queue length. This represents the average queue length result from the previous training round. This represents the average queue length for the current training round.

[0019] A further technical solution is that the deep Q-network model dynamically controls the shortest green light time, expressed as:

[0020]

[0021] in, express The shortest green light time under the timestamp Based on the minimum green light time, This is the morning peak adjustment coefficient. The current time in hours. This is the event intensity adjustment coefficient. For event feature vectors Norm.

[0022] Secondly, the present invention provides a traffic signal control system based on multi-source events and dual-loop phase coordination, comprising:

[0023] The real-time data acquisition module is configured to: construct a mathematical model of the road intersection and set up a double-ring phase structure, and collect multi-source sensing data in real time;

[0024] The reinforcement learning model construction module is configured to: model the intersection traffic signal control problem using a deep Q-network model based on the multi-source perception data, define the state space, action space and reward function, and decompose the double-loop phase structure for collaborative control by multiple agents; the reward function is constructed based on the vehicle delay time difference, waiting time difference, number of stops difference and queue length difference;

[0025] The optimization decision module is configured to: control the actions of each agent based on the rewards fed back to each agent by the environment, determine whether to maintain the phase or switch the phase, and obtain the optimal phase scheme for signal control.

[0026] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the traffic signal control method based on multi-source events and dual-loop phase coordination as described in the first aspect.

[0027] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the traffic signal control method based on multi-source events and dual-loop phase coordination as described in the first aspect.

[0028] The above one or more technical solutions have the following beneficial effects:

[0029] This invention comprehensively considers multiple requirements in intersection signal control, such as phase timing constraints and dynamic changes in traffic flow, providing a complete control scheme from state perception to decision output. A hierarchical decision-making mechanism ensures the safety of phase switching, and the self-learning capability of deep reinforcement learning adapts to different traffic conditions, meeting multiple control objectives such as reducing delays, minimizing queues, and improving traffic efficiency. Furthermore, by responding to sudden traffic flows in real time, it avoids the congestion exacerbation caused by fixed timing in traditional methods, providing a more efficient solution for urban intersection signal control.

[0030] This invention decomposes the double-ring phase structure into intelligent agents and achieves phase safety constraints through the collaborative control of three intelligent agents, thus solving the problem that traditional methods cannot handle the double-ring phase coupling relationship.

[0031] This invention improves the deep Q-network algorithm by constructing a hybrid state space that integrates Discrete Traffic State Encoding (DTSE) and phase combination. A reward function is constructed using multiple weighted indicators such as vehicle delay and waiting time to achieve precise mapping between traffic state and signal control. A dual-network architecture of main network and target network is used for model training and optimization. By introducing a spatiotemporal joint encoder, the timestamp t is decomposed into a periodic feature vector, which is then concatenated with multi-source event features and DTSE spatial features to form a high-dimensional hybrid state vector, enabling comprehensive perception and representation of traffic state.

[0032] This invention establishes a virtual intersection model using the SUMO simulation platform, trains and verifies it for various traffic scenarios, and achieves adaptive optimization of signal timing under a dual-loop phase structure. Attached Figure Description

[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0034] Figure 1 This is a diagram of the improved deep Q-network structure based on timestamps in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the double-ring phase structure according to an embodiment of the present invention;

[0036] Figure 3 This is a traffic state space diagram according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of intelligent agent control according to an embodiment of the present invention. Detailed Implementation

[0038] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0040] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0041] Example 1

[0042] like Figure 1 As shown, this embodiment discloses a traffic signal control method based on multi-source events and dual-ring phase coordination. The method includes the following steps:

[0043] S1: Construct a mathematical model of the road intersection and set up a double-loop phase structure to collect multi-source sensing data in real time;

[0044] The Negative-Electronic-Electronic-Aspect (NEMA) phase structure is a widely used phase organization method in traffic signal control. It divides the intersection phases into two independent control loops, each containing a set of conflicting traffic flows. This structure allows non-conflicting phases in different loops to operate simultaneously and achieves synchronization and coordination between phases through barriers. Its core features include a combination of protecting left-turn phases and allowing right turns.

[0045] In this embodiment, information such as intersection type (cross, T, X, Y, etc.), intersection lane information (number of lanes and direction of each approach lane), traffic flow path information, intersection traffic volume information (actual survey or proposed), and traffic light phase information (each phase setting and duration) are obtained. Based on the above information, a mathematical model of the road intersection is constructed in the SUMO simulation software.

[0046] Construct a mathematical model of the road intersection and set up a double-loop phase structure, with the left loop containing... , , , Four phases, the right loop contains , , , Four phases. Specifically, a six-lane intersection model is built in SUMO, and a double-loop phase structure is configured, such as... Figure 2 As shown, left loop phase Turn left at the east entrance. For the eastbound entrance, proceed straight ahead. Turn left at the west entrance. For westbound straight-through; right-hand loop phase Turn left at the south entrance. For the south entrance, proceed straight ahead. Turn left at the north entrance. It is a straight-through route from the north entrance.

[0047] A multi-source event perception system is deployed to comprehensively collect hardware and software data on traffic conditions and emergencies at intersections, enabling real-time acquisition of multi-source perception data. The system includes a LiDAR vehicle detection subsystem, a video queue length detection subsystem, a meteorological event sensing subsystem, and a manually reported event interface. The multi-source perception data includes LiDAR data, video monitoring data, meteorological sensor data, and manually reported events.

[0048] Furthermore, the lidar vehicle detection subsystem uses lidar point cloud detectors installed at each entrance lane of the intersection to detect vehicle position, speed, and traffic flow in real time; the video queue length detection subsystem uses video queue length detectors to monitor queue length and traffic density in each lane; the meteorological event sensing subsystem uses meteorological sensors to detect weather conditions such as rain, snow, and fog; and the manual event reporting interface receives pre-reported special event information such as large-scale events (e.g., concerts, sporting events) and road construction.

[0049] Furthermore, the LiDAR data includes vehicle presence status, real-time vehicle speed, headway, and traffic flow; video monitoring data includes lane-level queue length, vehicle type identification, and traffic density; meteorological sensor data includes rainfall intensity, visibility, and road surface condition; and manually reported events include the time, scale, and impact range of large-scale events; as well as road construction periods and the number of closed lanes. All of the aforementioned LiDAR data, video monitoring data, meteorological sensor data, and manually reported events are features extracted from the raw data (LiDAR point cloud, video stream, sensor data, and document reports).

[0050] Configure a multi-source event sensing system in the SUMO simulation platform, and deploy and integrate a lidar point cloud detector to collect lidar data. This provides vehicle-level precise motion state information for reinforcement learning agents. This indicates the vehicle's real-time coordinates within the lane. Indicates the instantaneous speed of the vehicle. Indicates the vehicle's heading angle. Represents UNIX timestamps, accurate to milliseconds; the video queue length detector collects video monitoring data. Used to quantify the degree of congestion at intersections. This indicates the number of vehicles queuing in the lane. This indicates the distance from the end of the queue to the stop line. Indicates lane traffic density, Indicates lane number; meteorological sensors collect meteorological sensor data. , Indicates rainfall intensity. Indicates snow depth. Indicates visibility. Indicates the road surface is slippery, represented by a 0-1 normalized value; the manual reporting event interface collects manually reported events. , This indicates the event type code (such as concert, road construction, sports event, etc.). Indicates the start and end event stamps. The event impact intensity level is indicated by a quantitative classification index for the degree of impact of various sudden or planned events (such as large-scale events, traffic accidents, and road construction) on the traffic flow at intersections. The range is 1-5. The higher the level, the greater the disturbance to the traffic flow, and the more significant the adaptive adjustment of the signal control strategy needs to be.

[0051] The system is based on To periodically and synchronously collect the aforementioned multi-source sensing data and perform time alignment and feature fusion, lidar data, video monitoring data, meteorological sensor data, and manually reported events are concatenated according to predefined rules (such as DTSE's 10×10 grid) and dimensions (6-dimensional time and 20-dimensional event) to form a fixed-dimensional vector that comprehensively reflects the current intersection status. That is, to generate a multimodal traffic-event state sequence (hybrid state features) under a unified spatiotemporal reference.

[0052] S2: Based on the multi-source sensing data, a deep Q-network model is used to model the traffic signal control problem at the intersection, defining the state space, action space and reward function, and decomposing the double-loop phase structure for collaborative control by multiple intelligent agents;

[0053] (1) State space

[0054] The state space is defined as a 226-dimensional vector. The first 100 dimensions represent the traffic states in Discrete Traffic State Encoding (DTSE) (10×10 grid, each grid containing vehicle presence status (whether a vehicle exists in a grid, represented by 0 / 1) and normalized speed values ​​(the ratio of vehicle speed in that grid to the road speed limit)). The time dimension is 6 (time-state encoding), the multi-source events (multi-source sensing data) are 20-dimensional, and the last 100 dimensions are a historical state cache, represented as follows:

[0055]

[0056] in, Represents timestamp state, Indicates traffic status. Indicates time status. This represents the event feature vector (event state). For example... Figure 3 As shown, (a) is an example diagram of traffic vehicles, (b) shows the vehicle's state, and (c) shows the normalized speed value.

[0057] Furthermore, set time status coding. Represented as:

[0058]

[0059] in, , , These represent the current timestamp. The corresponding hour, weekday, and month.

[0060] Discrete Traffic State Coding (DTSE) is a coding method that discretizes continuous traffic flow information into gridded feature vectors to obtain traffic state. By dividing the intersection approach lanes into 10×10 grid cells, each cell records the vehicle's state (Boolean value) and normalized speed value (actual speed / speed limit), forming a multidimensional state representation that includes spatial distribution and motion features. It is suitable for constructing input features for reinforcement learning algorithms.

[0061] Furthermore, set up event detection functions for any event. Its detection function Defined as:

[0062]

[0063] in, The current timestamp (UNIX seconds); ) indicates an event The valid time period (half-open interval, including the start time but excluding the end time); For the event Start timestamp For the event End timestamp.

[0064] For events that occur multiple times For example, in the case of periodic construction, the detection function is expanded as follows:

[0065]

[0066] in, For the event The total number of occurrences; For the event The time period of the nth occurrence; It represents the union of multiple time periods.

[0067] Generate event feature vectors by combining all event types. ( (corresponding to 10 types of events)

[0068]

[0069] in, For the event One-hot encoded vector; This indicates a bitwise OR operation, used to combine the codes of multiple concurrent events.

[0070] Table 1. One-hot encoding vectors for each event

[0071]

[0072] (2) Action space

[0073] The NEMA dual-loop phase structure is decomposed into intelligent agents, and phase safety constraints are achieved through the collaborative control of three intelligent agents. The first and second intelligent agents decide whether the preceding phase in the dual-loop phase structure should be switched after the minimum green light time is completed, and the third intelligent agent decides whether the following phase in the dual-loop phase structure should be cut off simultaneously after the minimum green light time ends. This solves the problem that traditional methods cannot handle the coupling relationship of dual-loop phases.

[0074] The action space is defined as a combination of binary decisions (phase switching / holding) by three agents. The first and second agents determine the two preceding concurrent phases (i.e., , as well as , Whether to switch, for example Figure 4 that is Should we change it? , Should we change it? , Should we change it? , Should we change it? Then, a third agent is set to determine whether the phase ending at the end of the minimum green light time in the subsequent phases should switch. 1 means do not execute, 0 means execute. That is, the role of the third agent is to simultaneously cut off the subsequent phases (the left ring in the diagram is...). and The right ring is and The green light is given by the first and second agents. The third agent is executed only after the first and second agents have finished. In other words, from the perspective of agent output, when the first agent is 1 and the second agent is 1, the third agent is forced to be 0. The execution of the first and second agents begins after the third agent has finished. In other words, from the perspective of agent output, when the third agent is 1, the first agent is 0 and the second agent is forced to be 0.

[0075] Furthermore, such as Figure 4As shown, the first intelligent agent controls the front phase ( / Whether to switch depends on the duration of the front phase green light. When the minimum green light time + yellow light time + all-red time are combined, a decision is made on whether to switch; the second agent controls the preceding phase ( / Whether to switch; the third agent controls the rear phase ( and / and Synchronous cut-off means simultaneously cutting off the green light of the subsequent phase.

[0076] In addition, safety constraints are set during phase switching. The timing rules that must be followed in the dual-loop phase control include: (1) the minimum green light time of the preceding phase (usually 15 seconds, which varies depending on time and events); (2) the transition time between yellow and all-red (a total of 5 seconds); and (3) the forced synchronization cut-off of the following phase. The above constraints are implemented through dual-loop phase structure constraints and binary decision-making by three agents to ensure that conflicting phases do not simultaneously obtain the right of way.

[0077] (3) Reward function

[0078] Based on the importance of four indicators—average vehicle delay time, average waiting time, average number of stops, and average queue length—influencing vehicle traffic efficiency, this embodiment initially assigns weights of 4:3:2:1, and the multi-objective reward function is expressed as follows:

[0079]

[0080] in, Indicates the reward value; Indicates the attenuation coefficient. This parameter is used to control the degree of decay of the previous round's results and can be adjusted according to the quality of the training results. Indicates the weight of vehicle delays; This indicates the average vehicle delay result of the previous training round; This indicates the average vehicle delay result for the current training round; Indicates the weight of waiting time; This represents the average waiting time result from the previous training round; This indicates the average waiting time for the current training round; Indicates the weight of the number of parking sessions; This represents the average number of stops in the previous training round; This represents the average number of stops in the current training round; Indicates the weight of queue length; This represents the average queue length result from the previous training round; This represents the average queue length for the current training round.

[0081] The multi-objective reward function set by this invention includes four weighted indicators: vehicle delay time difference, waiting time difference, number of stops difference, and queue length difference.

[0082] S3: Based on the rewards fed back to each agent by the environment, control the actions of each agent, determine whether to maintain the phase or switch the phase, and obtain the optimal phase scheme for signal control.

[0083] In this embodiment, the Deep Q-Network (DQN) includes a main network and a target network. The main network includes an input layer, a fully connected hidden layer, and an output layer. The target network is completely identical to the main network; that is, the input data of both the main network and the target network are state vectors. The two networks have the same structure, outputting three Q-values ​​corresponding to three agents, but their network parameters are different. The parameters are periodically (every 100 steps apart) synchronized from the main network to stabilize training. The core improvements of the deep Q-network include the experience replay mechanism and the target network. The former breaks sample correlation by storing and randomly sampling historical transition data, while the latter uses a network with independent parameters to calculate the target Q-value to stabilize the training process.

[0084] The input layer includes a static feature extraction branch and a temporal feature extraction branch. The static feature extraction branch is a fully connected layer with two sets of parallel neurons (the number of neurons in each set is adjusted according to actual training, initially set to 16), used to process traffic states and event states. The temporal feature extraction branch is a Long Short-Term Memory (LSTM) network with 32 neurons, used to process temporal states within the time state. The fully connected hidden layer uses multiple layers of neurons (in this embodiment, the initial number of layers is set to 2, with 64 neurons in each layer) to achieve deep feature fusion and abstraction. The output layer includes three output nodes, corresponding to the Q-value outputs of the three agents (Q-ValueAction1 / 2 / 3).

[0085] S301: The main network is trained based on historical multi-source sensing data, with a learning rate of 0.0004, a discount factor of 0.95, and an experience replay buffer capacity of 3200. The target network is slowly and synchronously updated. Network training uses experience data generated in a simulation environment (SUMO). The agent continuously explores the intersections in the SUMO simulation, recording the interaction data (current state) at each step. Actions taken The rewards received Next state This experience is stored as an experience replay buffer. During training, the model randomly samples a batch of such experience data from the buffer and updates the parameters of the main network by calculating the temporal difference error.

[0086] S302: Dynamically control the shortest green light time.

[0087] The shortest green light time is dynamically adjusted based on the timestamp and event variables. Taking the morning rush hour (7:30 AM) as an example, it is represented as follows:

[0088]

[0089] in, express The shortest green light time under the timestamp The minimum green light time is the baseline value under normal circumstances; in this embodiment, it is set to 15 seconds. This is the morning peak adjustment coefficient (to control the extension of the green light during peak hours; in this embodiment, it is set to 0.3). The current time in hours; This is the event intensity adjustment coefficient (the extended green light time corresponding to each unit of event intensity, set to 2 seconds in this embodiment). For event feature vectors Norm.

[0090] S303: Phase decision process.

[0091] Running the simulation in SUMO, traffic conditions are collected as input every 5 seconds, meaning every... Collect current status every second ;

[0092] For each phase group (a phase timing scheme for a certain period), the Q value is calculated using the acquired current state, and is expressed as:

[0093]

[0094] in, For state vectors, For action vectors, These are network parameters.

[0095] intelligent agents according to The strategy selection action, i.e., choosing to switch to the next flow direction or delay the current flow direction for 5 seconds, is initialized in this embodiment. It linearly decays to 0.1.

[0096] S304: Training and Optimization of Deep Q-Network Models

[0097] (1) Define the storage format of the experience playback buffer B as follows:

[0098]

[0099] in, The current state vector is a 200-dimensional real vector that contains the observed features of the environment at the current moment. This represents a three-dimensional action vector, composed of binary decisions made by three agents (e.g., each agent chooses 0 or 1). The immediate reward value is a scalar that reflects the action performed. The reward from the subsequent environmental feedback is calculated using a reward function; Termination flag, 1 indicates the current state The state is terminated (e.g., end of round), 0 indicates not terminated; buffer. Limit the maximum number of stored data points to the capacity; discard old data when the limit is exceeded.

[0100] (2) The minimum sampling batch is , After sampling batch data, the target Q value is first calculated through the target network, and is expressed as:

[0101]

[0102] in, The target Q value; This represents the future reward discount factor, which is set to 0.95 in this example. It is used to balance the importance of current rewards and future rewards. The closer the value is to 1, the more emphasis is placed on long-term returns. The target network's next state , Q-value prediction; Target network parameters; batch size This means that 64 experiences are randomly sampled from the buffer each time for training.

[0103] (3) Main network parameters Updated via gradient descent, the loss function uses the mean squared error, expressed as:

[0104]

[0105] in, Main network state and actions Q-value prediction, with parameters as ; The target Q value calculated in step (2) serves as the supervision signal for the loss function; the loss function measures the mean square error between the main network's predicted value and the target value, guiding the direction of parameter updates.

[0106] (4) Target network parameters Updates will be slow, and the update rules are as follows:

[0107]

[0108] in, To ensure slow parameter updates, this embodiment sets the value to 0.001, whereby the target network only absorbs data from the main network during each update. The change in the proportional parameter, the remainder Maintain your original parameters to achieve a smooth transition.

[0109] (5) Repeat the above steps iteratively.

[0110] S305: Result evaluation and dynamic adjustment of reward function.

[0111] Based on the importance of four indicators affecting vehicle traffic efficiency—average vehicle delay time, average waiting time, average number of stops, and average queue length—the initial weights are divided into a 4:3:2:1 ratio. The traffic state changes according to the actions taken by the agent during the phase decision-making process, thereby calculating the multi-objective reward value:

[0112]

[0113] The reward weights are dynamically adjusted (40% for delays, 30% for waiting, 20% for parking, and 10% for queuing), and the weights are reallocated when the indicator changes by more than 15%.

[0114] Observe the training results and the index values ​​of the four evaluation results. If the model has not converged, observe which index has a relative change rate greater than 0.15. Then, increase the weight of that index by 0.03 in the next training, while decreasing the weights of the other indices by 0.01 respectively.

[0115] Repeat the above steps until the model converges.

[0116] S306: Phase switching control.

[0117] Based on the calculated reward value, the agent makes a decision. For the preceding phase, if the current green light time... The minimum green light time + yellow light time (3 seconds in this embodiment) + all-red time (2 seconds in this embodiment) are used to perform a phase switch when the first or second agent outputs an action equal to 1. Specifically, for example... Figure 2 , Figure 4 As shown, based on a preset state space setting table, when the first agent or the second agent performs action 1, it switches to the corresponding next phase. Figure 2 or Figure 4 China is (First intelligent agent control) (Second-party control).

[0118] For the rear phase, when the third agent outputs 0 actions, the dual-ring rear phase green light is immediately switched.

[0119] After the phase switch, the decision clock is reset and the next decision cycle begins (that is, a new double-loop cycle is restarted).

[0120] S307: Generate phase timing scheme (optimal phase scheme).

[0121] The real-time phase timing generation scheme is represented as follows:

[0122]

[0123] in, It is a set of phase timing schemes; For the first The identifier or number of each phase; For the first The start time of each phase; For the first The end time of each phase; there are a total of 8 phase timing configurations.

[0124] The generated phase timing scheme is sent to the signal controller to execute the signal control scheme.

[0125] For example, period 1: (25s)→ (20s)→ (30s)→ (25s), Period 2: (20s)→ (15s)→ (25s)→ (20s)→ (30s) and then sent to the signal controller to execute the signal control scheme.

[0126] Example 2

[0127] This embodiment discloses a traffic signal control system based on multi-source events and dual-loop phase coordination, including:

[0128] The real-time data acquisition module is configured to: construct a mathematical model of the road intersection and set up a double-ring phase structure, and collect multi-source sensing data in real time;

[0129] The reinforcement learning model construction module is configured to: model the intersection traffic signal control problem using a deep Q-network model based on the multi-source perception data, define the state space, action space and reward function, and decompose the double-loop phase structure for collaborative control by multiple agents; the reward function is constructed based on the vehicle delay time difference, waiting time difference, number of stops difference and queue length difference;

[0130] The optimization decision module is configured to: control the actions of each agent based on the rewards fed back to each agent by the environment, determine whether to maintain the phase or switch the phase, and obtain the optimal phase scheme for signal control.

[0131] Example 3

[0132] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 1.

[0133] Example 4

[0134] The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 1.

[0135] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0136] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0137] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0138] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A traffic signal control method based on multi-source events and dual-loop phase coordination, characterized in that, include: Construct a mathematical model of the road intersection and set up a double-loop phase structure to collect multi-source sensing data in real time; Based on the multi-source sensing data, a deep Q-network model is used to model the traffic signal control problem at intersections, defining the state space, action space, and reward function. The dual-loop phase structure is decomposed and controlled collaboratively by multiple agents. The reward function is constructed based on the vehicle delay time difference, waiting time difference, number of stops difference, and queue length difference. The reward function is expressed as follows: in, Indicates the reward value. Indicates the attenuation coefficient. Indicates the weight of vehicle delays. This indicates the average vehicle delay result of the previous training round. This indicates the average vehicle delay result for the current training round. Indicates the weight of waiting time. This represents the average waiting time result from the previous training round. This indicates the average waiting time for the current training round. Indicates the weight of the number of parking times. This represents the average number of stops in the previous training round. This represents the average number of stops in the current training round. Indicates the weight of queue length. This represents the average queue length result from the previous training round. This represents the average queue length result for the current training round; The deep Q-network model dynamically controls the shortest green light time, expressed as: in, express The shortest green light time under the timestamp Based on the minimum green light time, This is the morning peak adjustment coefficient. The current time in hours. This is the event intensity adjustment coefficient. For event feature vectors Norm; Based on the rewards provided by the environment to each agent, the agent executes actions, determines whether to maintain the phase or switch the phase, and obtains the optimal phase scheme for signal control.

2. The traffic signal control method based on multi-source events and dual-loop phase coordination as described in claim 1, characterized in that, The multi-source sensing data includes lidar data, video monitoring data, meteorological sensor data, and manually reported events. The lidar data includes vehicle presence status, real-time vehicle speed, headway, and traffic flow. The video monitoring data includes lane-level queue length, vehicle type identification, and traffic density. The meteorological sensor data includes rainfall intensity, visibility, and road surface condition. The manually reported events are pre-reported event information.

3. The traffic signal control method based on multi-source events and dual-loop phase coordination as described in claim 2, characterized in that, The state space integrates traffic state, time state, and event state. Traffic state and event state are obtained based on multi-source perception data, and the state vector is obtained by combining the time state.

4. The traffic signal control method based on multi-source events and dual-loop phase coordination as described in claim 1, characterized in that, The action space is defined as a binary decision combination of multiple agents, including a first agent, a second agent, and a third agent. The first and second agents decide whether the preceding phase in the dual-loop phase structure should be switched after the minimum green light time is completed, and the third agent decides whether the following phase in the dual-loop phase structure should be switched off simultaneously after the minimum green light time ends.

5. The traffic signal control method based on multi-source events and dual-loop phase coordination as described in claim 4, characterized in that, The phase switching is subject to safety constraints, including the minimum green light time for the preceding phase, the transition time between yellow and full red lights, and the forced synchronization cutoff for the following phase.

6. A traffic signal control system based on multi-source events and dual-loop phase coordination, characterized in that, include: The real-time data acquisition module is configured to: construct a mathematical model of the road intersection and set up a double-ring phase structure, and collect multi-source sensing data in real time; The reinforcement learning model construction module is configured to: model the intersection traffic signal control problem using a deep Q-network model based on the multi-source perception data, define the state space, action space and reward function, and decompose the double-loop phase structure for collaborative control by multiple agents; the reward function is constructed based on the vehicle delay time difference, waiting time difference, number of stops difference and queue length difference; The reward function is expressed as follows: in, Indicates the reward value. Indicates the attenuation coefficient. Indicates the weight of vehicle delays. This indicates the average vehicle delay result of the previous training round. This indicates the average vehicle delay result for the current training round. Indicates the weight of waiting time. This represents the average waiting time result from the previous training round. This indicates the average waiting time for the current training round. Indicates the weight of the number of parking times. This represents the average number of stops in the previous training round. This represents the average number of stops in the current training round. Indicates the weight of queue length. This represents the average queue length result from the previous training round. This represents the average queue length result for the current training round; The deep Q-network model dynamically controls the shortest green light time, expressed as: in, express The shortest green light time under the timestamp Based on the minimum green light time, This is the morning peak adjustment coefficient. The current time in hours. This is the event intensity adjustment coefficient. For event feature vectors Norm; The optimization decision module is configured to: control the actions of each agent based on the rewards fed back to each agent by the environment, determine whether to maintain the phase or switch the phase, and obtain the optimal phase scheme for signal control.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the traffic signal control method based on multi-source events and dual-loop phase coordination as described in any one of claims 1-5.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the traffic signal control method based on multi-source events and dual-ring phase coordination as described in any one of claims 1-5.

Images