A dynamic traffic guidance and signal control coordination method under traffic accidents

Abstract

The application belongs to the technical field of intelligent transportation, and relates to a dynamic traffic guidance and signal control coordination method under a traffic accident, comprising the following steps: constructing a directed weighted complex road network model, obtaining a real-time operation state of a road network and evaluating the real-time operation state; based on the complex road network model and the operation state, generating a path guidance module by using a double Q learning algorithm; based on a deep deterministic policy gradient algorithm, constructing a signal control agent and generating a signal control module; establishing a bidirectional coordination mechanism between the path guidance module and the signal control module, and executing coordination iteration in a rolling time window mode to form a closed-loop coordination optimization system of path guidance and signal control. The application solves the technical problem that existing traffic guidance and signal control are often managed and optimized as independent systems, lack effective coordination mechanisms and fail to form an integrated solution.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, and in particular relates to a dynamic traffic guidance and signal control coordination method under traffic accidents. Background Technology

[0002] With the acceleration of urbanization and the continuous growth of motor vehicle ownership, urban traffic congestion has become an increasingly prominent problem. Traffic accidents, as sudden events in the transportation system, not only disrupt local traffic flow but also trigger widespread traffic congestion, severely impacting the normal efficiency and safety of urban traffic. Traditional methods of handling traffic accidents, such as manual traffic control and fixed signal timing, often suffer from problems such as delayed response, untimely information transmission, and inflexible control strategies, making them ineffective in dealing with the complex traffic situations caused by accidents.

[0003] In existing technologies, traffic management methods for traffic accidents mainly include traffic guidance and signal control. Traffic guidance typically provides drivers with road condition information and suggested detour routes through variable information signs (VMS), broadcasts, and navigation systems to disperse traffic flow and alleviate pressure on the accident area. However, existing traffic guidance systems often lack real-time performance, dynamism, and intelligence, making it difficult to provide optimal guidance strategies based on the real-time impact of the traffic accident and the dynamic changes in traffic flow. For example, some systems may only provide static detour routes without considering the evolution of real-time road conditions, resulting in poor guidance effectiveness and potentially even shifting congestion to other areas.

[0004] In terms of signal control, traditional signal control systems mostly employ fixed timing or inductive control. Their control logic is relatively simple, making it difficult to quickly and adaptively adjust to drastic fluctuations in traffic flow during sudden events such as traffic accidents. When a traffic accident occurs, the traffic demand and capacity of intersections surrounding the accident area change significantly. If signal timing is not adjusted in a timely and reasonable manner, it will lead to a large backlog of traffic at the intersection, exacerbating congestion and even triggering secondary accidents. Furthermore, traffic guidance and signal control are often managed and optimized as independent systems, lacking an effective coordination mechanism and failing to form an integrated solution, thus limiting the overall improvement of traffic management effectiveness. Therefore, how to achieve dynamic, real-time, and intelligent coordinated optimization of traffic guidance and signal control under traffic accidents is a key issue that urgently needs to be addressed in the field of traffic management. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic traffic guidance and signal control coordination method under traffic accidents, which solves the problems of lack of coordination between path guidance and signal control, delayed guidance response, and poor adaptability of control strategies in existing methods, thereby improving road traffic efficiency and network operation stability in accident scenarios.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is a dynamic traffic guidance and signal control coordination method under traffic accidents, comprising the following steps:

[0007] S1: Construct a directed weighted complex road network model, obtain the real-time operating status of the road network and evaluate it;

[0008] S2: Based on the complex road network model and its operating status constructed by S1, a path guidance module is generated using the dual Q-learning algorithm;

[0009] S3: Based on the deep deterministic policy gradient algorithm, a signal control agent is constructed. The agent perceives the traffic state of the intersection, outputs the green light duration of the next phase as an action, and learns by combining reward signals with vehicle delay and queue length as the core to generate a signal control module.

[0010] S4: Establish a two-way collaborative mechanism between the path guidance module and the signal control module, and perform collaborative iteration in a rolling time window manner to form a closed-loop collaborative optimization system for path guidance and signal control.

[0011] Furthermore, the specific steps of S1 are as follows:

[0012] S101: Construct a directed weighted complex road network graph model using the original road network. ;

[0013] S102: The real-time operating status of the road network is obtained by combining data collected by roadside sensing units with data reported by vehicles and then aggregated to the control center.

[0014] S103: Obtain information on lane travel time, road usage, and queue length; use the BPR function to calculate the travel time impedance value of each road segment; and use this impedance value as the road segment weight to evaluate the road network operation status.

[0015] Furthermore, the specific steps of S2 are as follows:

[0016] S201: Construct a reinforcement learning model for path induction, the model comprising a first value function network configured in parallel. Second value function network And the experience replay pool;

[0017] S202: Obtain and fuse node state information and road segment state information of the road network to construct an environmental state vector representing the real-time operating state of the road network. ;

[0018] S203: Transfer the environmental state vector The input is given to the reinforcement learning model, and then... Strategy, based on probability From all the valid adjacent nodes of the current node, select the next hop node and use that next hop node as action a;

[0019] S204: Execute action a, moving the recommended path from the current node to the next node. The environment generates a comprehensive reward value based on the new road network state after the action is executed. and the new environment state vector ;

[0020] S205: Will be determined by the environment state vector Action a, Overall Reward Value New environment state vector The resulting experience tuples are stored in the experience replay pool. Experience data is periodically sampled from the experience replay pool, and the first value function network is then processed using a dual Q-learning algorithm. Second value function network The parameters are iteratively updated to optimize the path decision-making logic and generate a path guidance module.

[0021] Furthermore, the comprehensive reward value described in S204 Rewards for travel time Path length reward Bandwidth utilization reward Node impedance reward Goal-oriented rewards The weighted sum is shown in equation (1):

[0022] (1)

[0023] in: As a reward for path length, As a reward for travel time, As a reward for bandwidth utilization, As a reward for node impedance, Rewards based on goals For the overall reward value, The weighting coefficient for the path length reward. The weighting coefficient for travel time rewards. The weighting coefficient for bandwidth utilization rewards. The weighting coefficient for node impedance reward. The weighting coefficient for goal-oriented rewards.

[0024] Furthermore, the specific steps of S3 are as follows:

[0025] S301. Construct a signal control agent based on deep deterministic policy gradient (DDPG), wherein the signal control agent includes an actor network for outputting control policies and a critic network for evaluating policy value.

[0026] S302. Construct the Actor state and Critic state. The Actor state is used to observe the vehicle density and queuing situation in the lanes entering at the intersection. Its state space... Defined as shown in equation (2):

[0027] (2)

[0028] in: Let Actor state space be the state space. For the set of real numbers, For the entry lanes of the fork, The number of signal phases at the intersection. It is a binary set;

[0029] S303: Select an action based on the Actor state. The action The duration of the next phase is determined by the following constraints: the duration of the next phase is not less than the minimum green light time and not greater than the maximum green light time.

[0030] S304. Establish a comprehensive reward and punishment function for the signal control algorithm based on DDPG. The comprehensive reward and punishment function Including vehicle delay penalties Penalties for vehicle queue length ;

[0031] S305. Iteratively train the DDPG-based signal control agent, and generate a signal control module by optimizing the parameters of the actor network and the critic network.

[0032] Furthermore, the specific steps of S4 are as follows:

[0033] S401: Establish a collaborative data interface between the path guidance module and the signal control module. The path guidance module transmits the generated detour path set, the time-based traffic forecast values ​​of each node, and the flow direction ratio to the signal control module through this interface. After adjusting the timing scheme based on the received traffic forecast information, the signal control module sends back the predicted delay, saturation change data, and green ratio parameters of each key intersection to the path guidance module.

[0034] S402: Based on a preset rolling time window period, the path guidance scheme and signal timing scheme are dynamically updated through continuous data interaction and strategy iteration between the path guidance module and the signal control module, thereby achieving closed-loop collaborative optimization of path selection and signal control.

[0035] Furthermore, in S402, through continuous data interaction and iteration between the path guidance module and the signal control module, the specific steps are as follows:

[0036] S4021: The route guidance module generates the optimal detour route based on real-time traffic status data and predicts the traffic flow distribution of each detour route within a future rolling time window.

[0037] S4022: The signal control module receives the path traffic prediction results output by the path guidance module, uses a deep deterministic strategy gradient algorithm to optimize the green ratio parameters of key intersections, and generates a new signal timing scheme that adapts to traffic changes.

[0038] S4023: The signal control module will send the optimized delay data and saturation data of each node back to the path guidance module in real time;

[0039] S4024: The route guidance module updates the path impedance value and path selection probability of each road segment based on the received delay and saturation feedback data, and redistributes the traffic flow of detour vehicles.

[0040] S4025: Using a preset rolling time window as the iteration cycle, repeat the process from S4021 to S4024 to achieve real-time coordination between the path selection strategy and the signal timing strategy, ensuring that the system dynamically adapts to changes in traffic flow.

[0041] Furthermore, the node status information mentioned in S402 includes: node computing utilization. Storage utilization rate Current node marker Target node marking and entry node marker ;

[0042] The road segment status information includes: road segment utilization rate. and segment normalization delay .

[0043] Compared with the prior art, the beneficial effects of the present invention include the following:

[0044] 1) This invention uses dynamic traffic guidance and signal control to quickly guide vehicles around the accident area, shorten traffic recovery time, and reduce road congestion duration.

[0045] 2) This invention can reduce the risk of vehicle collisions and conflicts through precise guidance instructions and appropriate signal timing, effectively prevent secondary accidents, and ensure road traffic order.

[0046] 3) The signal control method based on DDPG proposed in this invention is adaptable and scalable when traffic flow changes drastically on key road sections. Compared with adaptive signal control methods, Webster timing method and unified timing control, it can reduce the impact of traffic flow fluctuations in the total travel time of vehicles on key road sections, intersection delay time and queue length, and has more development and utilization value for traffic flow changes. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a traffic guidance logic framework diagram in an embodiment of the present invention;

[0049] Figure 2 This is a flowchart of the road network operation status acquisition process in an embodiment of the present invention;

[0050] Figure 3 This is a flowchart of the road network operation status evaluation in an embodiment of the present invention;

[0051] Figure 4 This is a diagram of the architecture of the deep deterministic strategy gradient traffic signal controller in an embodiment of the present invention;

[0052] Figure 5 This is a diagram of the distributed execution and centralized learning framework in the embodiments of the present invention;

[0053] Figure 6 This is a structural diagram of the two-layer planning model for traffic guidance and signal control in an embodiment of the present invention;

[0054] Figure 7 This is a topology diagram of the Sioux Falls transportation network in an embodiment of the present invention;

[0055] Figure 8 These are SUMO simulation scene diagrams and experimental scene diagrams in the embodiments of the present invention;

[0056] Wherein: (a) is a SUMO simulation scene diagram, and (b) is a schematic diagram of the experimental scene.

[0057] Figure 9This is a diagram showing the initial phase and signal timing of a key intersection in an embodiment of the present invention;

[0058] Wherein: (a) is the original phase diagram and signal timing diagram of intersection 10, and (b) is the original phase diagram and signal timing diagram of intersection 11.

[0059] Figure 10 These are box plots of different travel times in embodiments of the present invention;

[0060] Figure 11 This is a diagram showing the vehicle queuing situation at a key intersection in an embodiment of the present invention;

[0061] Where: (a) is the vehicle queue length recorded at intersection 10 over time, (b) is the vehicle queue length recorded at intersection 11 over time, (c) is the vehicle queue length at intersection 10 after removing the SOTL signal control method results, and (d) is the vehicle queue length at intersection 11 after removing the SOTL signal control method results.

[0062] Figure 12 This is a time delay diagram of key intersections in an embodiment of the present invention;

[0063] Wherein: (a) is the change of vehicle delay at intersection 10 over time, (b) is the change of vehicle delay at intersection 11 over time, (c) is the change of vehicle delay at intersection 10 over time after removing the SOTL signal control method results, and (d) is the change of vehicle delay at intersection 11 over time after removing the SOTL signal control method results.

[0064] Figure 13 This is a flowchart of a dynamic traffic guidance and signal control coordination method under traffic accidents according to the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] A specific embodiment of the present invention provides a dynamic traffic guidance and signal control coordination method under traffic accidents, which specifically includes the following steps:

[0067] S1. Construct a directed weighted complex road network model, obtain the real-time operating status of the road network and evaluate it;

[0068] In a real road network, each road contains one or more lanes, and different lanes have different driving directions. Each lane in each direction has different road attribute characteristics. Due to their different positions and functions in the road network, road segments and intersections play different roles and occupy different positions. The physical attributes of each intersection and road are particularly important. In the specific implementation of this application, in order to build a traffic guidance method framework based on deep reinforcement learning, the original method is used to construct a directed weighted complex road network graph model for the original road network.

[0069] S101: This application uses the original method to construct a directed weighted complex road network graph model of the original road network. Road network model As shown in equation (1):

[0070] (1)

[0071] in: For a directed weighted complex road network model, For the reason Intersection The set of nodes that make up the node For multiple road sections The set of edges formed For directed edges weight The set of weights that make up the weights;

[0072] Road network model Zhongyou Intersection The set of nodes As shown in equation (2):

[0073] (2)

[0074] in: It is by Intersection The set of nodes that make up the node set;

[0075] Road network model The middle consists of multiple road sections The set of edges As shown in equation (3):

[0076] (3)

[0077] in: It consists of multiple road sections The set of edges formed by it;

[0078] Road network model From the middle to the edge weight The weight set As shown in equation (4):

[0079] (4)

[0080] in: It is a directed edge weight The weighted set consists of traffic characteristic data such as road grade, road capacity, and length.

[0081] S102: The real-time operating status of the road network is obtained by combining data collected by roadside sensing units with data reported by vehicles and then aggregated to the control center. The vehicles include non-connected vehicles, connected manually driven vehicles and connected autonomous vehicles.

[0082] In the specific implementation of this application, the acquisition of road network operation status includes two processes: vehicle travel information acquisition and accident duration prediction. Currently, motor vehicles operating normally in cities can be divided into three types: non-connected vehicles (NCVs), connected human-driven vehicles (CHVs), and connected autonomous vehicles (CAVs). Roadside perception units can effectively acquire travel information of these three types of vehicles. Furthermore, connected human-driven vehicles and connected autonomous vehicles can directly upload travel information to the city control center. The city control center can achieve full coverage of travel information for all individual vehicles in traffic. The process of acquiring and transmitting road network operation status is as follows: Figure 2 As shown, Figure 2 The left side represents three types of traffic participants: non-connected vehicles, connected manually driven vehicles, and connected autonomous vehicles. Non-connected vehicles interact with the system through roadside units (RSUs), while the two types of connected vehicles communicate directly via network transmission. All this information eventually converges at the control center and enters a core processing flow. This flow begins with accident data processing and uses information from the accident database to predict the duration of event impact through a traffic accident duration prediction model. This process then reconstructs the road network time information and ultimately evaluates the overall operational status of the road network system.

[0083] S103: Obtain lane passage time, road usage and queue length information, calculate the travel time impedance value of each road segment using the BPR function, and use the impedance value as the road segment weight to evaluate the road network operation status. In the specific implementation of this application, the road segment weight is calculated based on the BPR (Bureau of Public Roads) function set in the HCM (Hypercapability Manual), as shown in Equation (5):

[0084] (5)

[0085] in, For road section The driving time impedance value, in units of ; For road section Ideal driving time impedance value, in units of ; For road section Actual traffic capacity, in units of ; For road section Hourly traffic volume, in units ; This is the first preset parameter in the BPR function, with a value of 0.15. This is the second preset parameter in the BPR function, with a value of 0.14.

[0086] The process for evaluating the operational status of the road network is as follows: Figure 3 As shown, the system first acquires key road network operation status parameters in real time, such as saturation, headway, and queue length. Then, it comprehensively assesses the road congestion level based on these parameters. On this basis, the system dynamically generates a road network impedance function (to quantify the difficulty of road traffic) using information such as congestion level and generates alternative guidance paths.

[0087] S2: Based on the complex road network model and its operating status constructed by S1, a path guidance module is generated using the dual Q-learning algorithm;

[0088] In the specific embodiments of this application, the route guidance module includes alternative guidance route generation and decision guidance route, and the dynamic traffic guidance logic framework is as follows: Figure 1 As shown, the specific execution steps are as follows:

[0089] Step a: Obtain road operation status through roadside monitoring equipment or mobile service providers, or directly obtain specific accident details through traffic accident reporting information;

[0090] Step b: Obtain static traffic information at the time of the traffic accident, such as the specific location of the accident, road grade, weather conditions, etc., and use a machine learning pipeline model to predict the duration of the traffic accident.

[0091] Step c: Derive the impact of the accident on the overall operation of the road network, calculate the road impedance function, and generate a set of alternative traffic guidance routes;

[0092] Step d: Check if there are any traffic accidents among the alternative routes. If so, further analysis of travel time and route utility value is required.

[0093] Step e: Evaluate the equilibrium results of each objective and determine the optimal induction path.

[0094] S201: Construct a reinforcement learning model for path induction, the model comprising a first value function network configured in parallel. Second value function network And the experience replay pool;

[0095] In a specific implementation of this application, Double Deep Q-Network (DDQN) is used for dynamic path guidance. Double Deep Q-Network (DDQN) is designed to address the overestimation problem in DQN (Deep Q-Network, a method combining deep learning and reinforcement learning) when estimating action value by maximizing the operation. DDQN decouples action selection and value assessment, reduces overestimation bias, and enables the vehicle to learn global optimization. The steps of the DDQN algorithm are as follows:

[0096] Step 1a: Randomly initialize two layers, each containing an input layer, a hidden layer, and an output layer. Value function and At the same time, initialize the experience pool;

[0097] Step 2a: Convert the current environment state vector... Input to contain and In a neural network, the action of each action in the current state is obtained. value;

[0098] Step 3a: Select an action, based on the current state of the two... Average value of the function ,use Strategy selection action;

[0099] Step 4a: Perform the action and observe the reward returned by the environment and the next state;

[0100] Step 5a: Store experience by storing the current state, the selected action, the reward obtained, and the next state into the experience pool;

[0101] Step 6a: Experience replay, randomly select a batch of experience from the experience pool, and apply it to... and Conduct training;

[0102] Specifically, for the first one Value function Use the second one Value function To select the optimal action in the next state Then use the first one Value function To assess the value of the action and calculate the primary objective. The value is shown in equation (6):

[0103] (6)

[0104] in: For the primary goal value, For instant rewards, As a discount factor, For the first Value function, For the next state, For the first The parameters of a value function To determine the optimal action for the next state, use the second... Value function The calculation is shown in equation (7):

[0105] (7)

[0106] in: To find the maximum value, For the second Value function, For the first The parameters of a value function For action.

[0107] For the second Value function Use the first one Value function To select the optimal action in the next state Then use the second one Value function To evaluate the value of the action and calculate the second objective The value is shown in equation (8):

[0108] (8)

[0109] S202: Acquire and fuse node state information and road segment state information of the road network to construct an environmental state vector representing the operating state of the road network. ;

[0110] In the path guidance method, the state refers to the state of the path in the road network, which is a high-dimensional state vector containing information about road nodes and road segments. The core idea of ​​the state definition method is to integrate the static topological structure features of the network with dynamic operation information to characterize the overall operation of the road network at each time step. The reinforcement learning agent can comprehensively observe the real-time state of the traffic network through the state.

[0111] In the specific implementation of this application, it is assumed that there are a total of [states] in the network. Let there be n nodes, and denote the set of nodes as . For each node Construct a state vector containing the following five features. As shown in equation (9):

[0112] (9)

[0113] in: For state vectors, Calculate the utilization rate for the nodes. To improve storage utilization, Mark the current node. Mark the target node. Mark the entry node. This is a transpose.

[0114] Node computational utilization It is measured by the ratio of node load to computing power, representing the node's processing capacity, as shown in equation (10):

[0115] (10)

[0116] in: For nodes The current workload being processed For nodes computing power This represents the computational utilization rate of the node.

[0117] The traffic capacity of each node is limited, and storage utilization is... The calculation is shown in equation (11), storage utilization rate Used to characterize the usage of node data storage resources.

[0118] (11)

[0119] in: For the currently visited nodes Total traffic flow For road network nodes Storage capacity, For storage utilization rate.

[0120] Current node marker To represent the state variables of the node where the vehicle is currently located, for the node... The indicator function is defined as shown in equation (12):

[0121] (12)

[0122] in: This indicates the current node position of the vehicle. Mark the current node.

[0123] To guide vehicles toward their destination, i.e., the target, the nodes... Assigning a label means labeling the target node. As shown in (13):

[0124] (13)

[0125] in: The location of the endpoint node is a feature that helps vehicles perceive the target location. Mark the target node.

[0126] Nodes receiving traffic flow input are marked, which is called entry node marking. As shown in (14):

[0127] (14)

[0128] in: This flag represents the set of nodes receiving traffic; it indicates whether a node receives traffic. Mark the entry node.

[0129] In summary, the state space vector representation of each node is shown in equation (15):

[0130] (15)

[0131] In a road network, nodes are connected by road segments. For any two nodes... , If there is a road segment connecting two points, construct the road segment state vector. Set up points and points The bandwidth between the road segments is The bandwidth already used is Road segment utilization rate As shown in equation (16):

[0132] (16)

[0133] in: To improve road segment utilization, For the bandwidth already used, To set points and points Bandwidth between road segments.

[0134] Delays will occur in vehicle traffic on each section of the road. To ensure uniformity across different delay dimensions, this application introduces a normalization factor. , If the maximum delay is the total delay across the entire road network, then the normalized delay for the road segment is... As shown in equation (17):

[0135] (17)

[0136] in: Delays due to road segment normalization This will cause delays for vehicles traveling on each section of the road. This represents the maximum delay across the entire road network.

[0137] If node and nodes If there are no road segments between them, the normalized delay of the road segment is defined as 0, and the state vector of the road segment is: , remember .

[0138] Concatenate all node states and all road segment states in sequence to form a complete environment state vector. If there is on the network If there are nodes, then the dimension of the node state part is... The road segment status dimension is (Road section status according to) (arranged in sequence), the overall state vector is shown in equation (18):

[0139] (18)

[0140] The state vector Convert them into one-dimensional arrays or tensors to serve as input variables for the dual Q network.

[0141] S203: Transfer the environmental state vector The input is given to the reinforcement learning model, and then... Strategy, based on probability From all the valid adjacent nodes of the current node, select the next hop node and use that next hop node as action a;

[0142] In the Double Deep Q-Network (DDQN) path-guided reinforcement learning framework, the main task of the vehicle (agent) is to move step by step from the starting point to the target node based on the observed environmental state, ultimately forming an optimal path. The action is defined as the selection of a route from the current node to its neighboring nodes, and its specific construction and meaning are as follows.

[0143] In a specific embodiment of the present invention, the traffic network is abstracted as a graph. ,in, This represents all nodes in the network. This represents the road segment connections between nodes, defining the action space. Defined as the form represented by equation (19):

[0144] (19)

[0145] in: For the action space, The number of nodes.

[0146] Vehicle selects actions on each road segment This indicates that a node is selected as the next node to be traversed in the path.

[0147] At any given moment, the current node of the vehicle is denoted as . In a transportation network, not every two nodes are directly connected by a road. Therefore, from the node... Starting from a point, only nodes directly connected to it can form a valid set of actions. ,make Under the current road network conditions, vehicles can only originate from the set. Select an action, that is, jump to the current node. A directly connected node .

[0148] Execute action The meaning is: to move the vehicle from the current node. Move to the next node This updates the environment state. Specifically, the action definition has the following semantics:

[0149] (1) If the vehicle is in state Take action below The environment will be updated based on road segment attributes and node status, transitioning to a new state. .

[0150] (2) By continuously selecting legal actions, the vehicle can form an effective path from the starting point to the target node. Each action decision not only affects the update of the current state, but also has a cumulative effect on the overall performance of subsequent paths.

[0151] (3) During the training process, the vehicle uses a dual Q learning network to estimate the value of each legal action and adopts a greedy strategy to balance exploration and exploitation, and gradually learns the optimal jump strategy.

[0152] To achieve a full balance between exploration and utilization, the specific embodiments of this invention employ the following... The strategy selects actions in each state:

[0153] (1) Based on probability Random selection set Any action within it can be used to explore unknown states;

[0154] (2) Based on probability The choice makes The action with the largest value is shown in equation (20):

[0155] (20)

[0156] in: For optimal action, for Value function.

[0157] To enable the vehicle to better utilize its learned experience to optimize its decision-making process, as the training progresses, As the value gradually decays, vehicles gradually shift from extensive exploration in the early stages to strategic utilization, forming a stable path planning strategy.

[0158] S204: Execute action a, moving the recommended path from the current node to the next node. The environment generates a comprehensive reward value based on the new road network state after the action is executed. and the new environment state vector ;

[0159] To guide vehicles in selecting optimal routes in complex traffic network environments, this application designs a comprehensive reward and penalty function that considers time delay, path length, resource utilization, node impedance, and proximity to the target destination. This function encourages vehicles to choose routes that reduce overall travel time and minimize congestion risk, while penalizing illegal or inefficient road segments, specifically including:

[0160] (1) Travel time reward

[0161] Travel time reward Starting from the predicted duration at the time of the accident, the time delays of road segments in the road network are comprehensively considered. and node impedance Impact on overall travel time.

[0162] path Delays on various sections of the road The sum is shown in equation (21):

[0163] (twenty one)

[0164] in: For path Delays on various sections of the road the sum of This is due to time delays on road segments within the road network.

[0165] node impedance sum As shown in equation (22):

[0166] (twenty two)

[0167] in: node impedance the sum of This represents the node impedance.

[0168] Trip Time Bonus As shown in equation (23):

[0169] (twenty three)

[0170] In equation (23), the denominator " "To avoid division by zero, a shorter transit time will result in a higher reward value."

[0171] (2) Path length reward

[0172] path Include Each node and the road segment traversed along the path is used to define a path length reward to encourage vehicles to choose shorter routes. As shown in equation (24):

[0173] (twenty four)

[0174] in: As a reward for path length, The number of nodes.

[0175] (3) Bandwidth utilization bonus

[0176] To reflect the degree of congestion on a road segment, this application introduces a bandwidth utilization bonus. Let the total utilization of all road segments along the path be... and their corresponding total bandwidth is Then bandwidth utilization bonus As shown in equation (25):

[0177] (25)

[0178] in: As a reward for bandwidth utilization, This represents the sum of the utilization rates of all road segments along the route. This represents the total bandwidth corresponding to each segment along the path.

[0179] When there is no congestion on a road segment, the bandwidth utilization rate of the segment is low, and the corresponding reward is higher; conversely, the reward is lower when there is congestion.

[0180] (4) Node impedance bonus

[0181] The reward for this application, which considers the influence of node impedance separately, is shown in equation (26):

[0182] (26)

[0183] in: As a reward for node impedance, node impedance sum.

[0184] (5) Goal-oriented rewards

[0185] To ensure the vehicle moves toward the target node, this application designs a goal-oriented reward function. As shown in equation (27):

[0186] (27)

[0187] in: A goal-oriented reward function.

[0188] Goal-oriented rewards The aim is to guide vehicles to consider target proximity when making decisions and encourage the selection of actions that shorten the target distance.

[0189] (6) Comprehensive reward function

[0190] The above rewards are linearly combined according to their weights to form the final reward function, where the weight of the path length reward is denoted as... The weight of the travel time reward is The weight of bandwidth utilization reward is The weight of the node impedance reward is The weight of goal-oriented rewards is Then the final reward function As shown in equation (28):

[0191] (28)

[0192] In a specific embodiment of the present invention, the values ​​of each weighting coefficient are as follows: =0.15, , .

[0193] S205: Will be determined by the environment state vector Action a, Overall Reward Value New environment state vector The resulting experience tuples are stored in the experience replay pool. Experience data is periodically sampled from the experience replay pool, and the first value function network is then processed using a dual Q-learning algorithm. Second value function network The parameters are iteratively updated to optimize the path decision-making logic and generate a path guidance module.

[0194] S3. Based on the deep deterministic policy gradient algorithm, a signal control agent is constructed. The agent perceives the traffic state of the intersection, outputs the green light duration of the next phase as an action, and learns by combining the reward signal with vehicle delay and queue length as the core to generate a signal control module.

[0195] S301. Construct a signal control agent based on deep deterministic policy gradient (DDPG), wherein the signal control agent includes an actor network for outputting control policies and a critic network for evaluating policy value.

[0196] The signal control method based on Deep Deterministic Policy Gradient (DDPG) aims to achieve a signal cycle that includes the duration of dynamic phases. This cycle is designed to maintain fairness for vehicles entering from all directions, ensuring that waiting vehicles in each entry lane at the intersection have a minimum green light time. By observing the queuing situation in each lane within a signal cycle, the lane with the longest queue is given a green light phase. Subsequently, minimum green light times are given in sequence according to the cycle phases. If there are no vehicles in a certain direction entering a lane, the green light for that lane within the cycle is skipped.

[0197] S302. Construct the Actor state and Critic state. The Actor state is used to observe the vehicle density and queuing situation in the lanes entering at the intersection. Its state space... Defined as shown in equation (29):

[0198] (29)

[0199] in: Let Actor state space be the state space. For the set of real numbers, For the entry lanes of the fork, The number of signal phases at the intersection. It is a binary set.

[0200] The critic's state is the environmental state. With Actor Action The combined state formed after splicing, such as Figure 4 As shown, the input to the strategy function is the local traffic state (environmental state) of the intersection. The input to the action value function is the concatenation of the local state and the local action. FC stands for fully connected layer, and the number 1 indicates the number of neurons in this layer.

[0201] S303: Select an action based on the Actor state. The action The duration of the next phase shall be no less than the minimum green light time and no more than the maximum green light time.

[0202] As shown in equation (30):

[0203] (30)

[0204] in: The duration of the next phase, For the set of integers, Minimum green light time, This is the maximum green light time.

[0205] Specifically, the action space of the DDPG traffic signal control algorithm is such that the agent can only choose the duration of the next green light phase, but cannot choose which phase to enter. The output value of the DDPG signal control algorithm is a real number. The reason is that the phase duration is input into the SUMO traffic simulation software in seconds. Therefore, the output phase duration will be rounded to the nearest integer. In actual applications, the next phase duration is affected by the minimum green light time. and maximum green light time Two hyperparameters are used to limit the quality of travel services for all users.

[0206] S304: Establish a comprehensive reward and punishment function for the signal control algorithm based on DDPG. The comprehensive reward and punishment function Including vehicle delay penalties Penalties for vehicle queue length ;

[0207] Specifically, the training reward for the DDPG-based signal control algorithm is vehicle delay. and queue length The equalization reward function, where time delay It is the difference between the actual travel time affected by the traffic accident and the original free-flow time of the road segment. It is necessary to calculate the sum of the delays of all vehicles traveling between adjacent intersections, as shown in equation (31):

[0208] (31)

[0209] in: This represents the set of all vehicles entering the intersection in all lanes. Penalties for vehicle delays It is a vehicle In time The delay.

[0210] Queue length This refers to the length of the queue of vehicles leaving a lane at a downstream intersection at the time of a traffic accident. If the queue of vehicles leaving the lane exceeds this length... The agent will be given a negative reward, namely a queue length penalty. , As shown in equation (32):

[0211] (32)

[0212] in: It is a vehicle In time Distance from the intersection Penalty for queue length.

[0213] The comprehensive reward and punishment function is shown in equation (33):

[0214] (33)

[0215] in: For comprehensive rewards and punishments.

[0216] S305: Iteratively train the DDPG-based signal control agent, and generate a signal control module by optimizing the parameters of the actor network and the critic network.

[0217] The actor network in the intelligent agent approximates the policy through deep artificial neural networks. ,Strategy Strategy for selecting the duration of the green light for each phase The function consists of two hidden layers, each with... The layer consists of 1 fully connected neurons using batch normalization and ELU activation functions, and the output layer is a single neuron using the hyperbolic tangent activation function.

[0218] The value of commenter networks learning actions in intelligent agents Function, action value function It consists of two hidden layers, each with The layer consists of four fully connected neurons, using batch normalization and ELU activation functions. The output layer is a single neuron using a linear activation function. The action value function uses... of Weighted regularization.

[0219] The deep deterministic gradient training strategy uses multiple executors and learners for parallel computation, thereby accelerating the training process. The training strategy used in this invention is distributed execution and centralized learning, combining and optimizing the executors and learners, as shown in the architecture below. Figure 5 As shown.

[0220] Figure 5 It includes distributed execution and centralized learning architectures for reinforcement learning. Figure 5In this model, the learner and executor play different roles during training. The learner has an environment experience buffer that stores the interaction data between the executor and the environment within each time step, allowing random sampling to avoid data correlation and promote training stability. The cross-network data structure within the learner stores and updates the learned model. The learner uses experience to update the neural network parameters and gradually learns the optimal policy. The learner provides the executor with the trained parameters, enabling the executor to execute the optimal policy. The cross-network data structure in the executor is an instance that runs the simulation, processes state information from the environment, and selects actions. The executor generates a batch of experience data from the environment and passes it to the learner for model training.

[0221] To improve the performance of traffic signal control algorithms, this application optimizes the hyperparameters of each traffic signal controller. By exploring different combinations of hyperparameters, the optimal configuration of the traffic signal controller is achieved. This application uses a parallel computing architecture for simulation and learning, searches for algorithm hyperparameter combinations using a grid search method, and selects the optimal hyperparameters by measuring the travel time index of different parameter combinations.

[0222] Each traffic signal control algorithm has a set of hyperparameters. The combination of hyperparameters affects the learning effect and performance of the algorithm. The experiment generates Cartesian products for multiple hyperparameters, that is, generates all possible combinations of hyperparameters to form a complete set of hyperparameters.

[0223] The hyperparameter combination will be used for a single experiment. The hyperparameters set for each model experiment are as follows: (1) Learning Rate (LR): controls the step size of the network weight update. A smaller learning rate may lead to slower training, while a larger learning rate may lead to unstable training. In this application, 0.0001 and 0.00005 are used for the experiment. (2) Learning Rate Exponent (LRE): the learning rate decay parameter, used to gradually reduce the learning rate during training. In this application, 0.0001, 0.000001, 0.0000001, and 0.00000001 are used for the experiment. (3) Number of Updates: specifies how many Q-network updates are performed in each training cycle. In this application, it is set to 15,000. (4) Batch Size: the size of the experience batch used in each training session. In this application, it is set to 32. (5) Replay Buffer Size: the size of the buffer used to store the agent's experience. In this application, it is set to 15,000. (6) Number of Steps: The maximum number of actions the agent can perform within each time step. This application uses 1 and 2 for experiments. (7) Number of Hidden Layers: The number of hidden layers in the neural network. This application sets it to 3 layers. (8) Target Network Update Frequency: The frequency at which the target network parameters are updated. This application uses 32, 64, and 128 for experiments. (9) Green Minimum Time: Defines the minimum green time in the signal controller. This application uses 5s and 10s for experiments.

[0224] DDPG's learning rate, number of updates, batch size, replay cache size, hidden layer hyperparameters, and number of steps are the same as DQN's. The hyperparameters can take the same range. Additionally, the following hyperparameters are also available:

[0225] (1) Learning rate decay: Experiments were conducted using values ​​of 0.0001, 0.000001, and 0.00000001. (2) Soft update rate: The soft update system of the target network was tested using values ​​of 0.01 and 0.005. (3) Critic learning rate: The learning rate of the Critic network, used to train the value network. Experiments were conducted using values ​​of 0.0005 and 0.0001. (4) Target network update frequency: Experiments were conducted using values ​​of 1, 16, and 64.

[0226] S4, such as Figure 13As shown, a bidirectional collaborative mechanism is established between the path guidance module and the signal control module, and collaborative iteration is performed in a rolling time window manner to form a closed-loop collaborative optimization system for path guidance and signal control.

[0227] In accident-induced disturbance scenarios, relying solely on route guidance to redirect vehicles often leads to secondary congestion at key intersections along detour routes, reducing the overall diversion effect. Therefore, this invention proposes a two-way collaborative mechanism of route guidance and signal control, building upon S2 and S3. Through data sharing, objective function coupling, and synchronous policy updates, it achieves simultaneous optimization of traffic flow at both the path and node levels.

[0228] S401. Establish a collaborative data interface between the path guidance module and the signal control module. The path guidance module transmits the generated detour path set, the time-based traffic forecast values ​​of each node, and the flow direction ratio to the signal control module through this interface. After adjusting the timing scheme according to the received traffic forecast information, the signal control module sends back the predicted delay, saturation change data, and green ratio parameters of each key intersection to the path guidance module.

[0229] S402. Based on a preset rolling time window period, the path guidance scheme and signal timing scheme are dynamically updated through continuous data interaction and strategy iteration between the path guidance module and the signal control module, thereby achieving closed-loop collaborative optimization of path selection and signal control.

[0230] In S402, the continuous data interaction and iteration between the path guidance module and the signal control module are carried out in the following steps:

[0231] S4021: The route guidance module generates the optimal detour route based on real-time traffic status data and predicts the traffic flow distribution of each detour route within a future rolling time window.

[0232] S4022: The signal control module receives the path traffic prediction results output by the path guidance module, uses a deep deterministic strategy gradient algorithm to optimize the green ratio parameters of key intersections, and generates a new signal timing scheme that adapts to traffic changes.

[0233] S4023: The signal control module will send the optimized delay data and saturation data of each node back to the path guidance module in real time;

[0234] S4024: The route guidance module updates the path impedance value and path selection probability of each road segment based on the received delay and saturation feedback data, and redistributes the traffic flow of detour vehicles.

[0235] S4025: Using a preset rolling time window as the iteration cycle, repeat the process from S4021 to S4024 to achieve real-time coordination between the path selection strategy and the signal timing strategy, ensuring that the system dynamically adapts to changes in traffic flow.

[0236] This embodiment adopts a two-layer model collaborative optimization architecture, as follows: Figure 6 As shown, the lower-level model is the traffic guidance method, and the upper-level model is the signal control strategy. This architecture defines the reward function, state vector, action space and output content of each module through mathematical methods, thereby realizing closed-loop collaborative optimization between path guidance and signal control.

[0237] Specifically, the following steps are included:

[0238] Step a: Initial Induction

[0239] The route guidance module generates the optimal detour route based on the current traffic conditions and predicts the traffic flow of each route within the future rolling time window.

[0240] Step b: Signal optimization

[0241] The signal control module receives the traffic prediction output by the path guidance and uses the DDPG algorithm to optimize the green light ratio at key intersections to generate a new timing scheme.

[0242] Step c: Delay Feedback

[0243] The signal control module sends the optimized delays and saturation of each node back to the path guidance module.

[0244] Step d: Secondary induction

[0245] Based on the feedback information, the path guidance module updates the path impedance and selection probability, and redistributes detour vehicles.

[0246] Step e: Update cyclically

[0247] Steps a through d above are iterated continuously using a rolling time window to ensure real-time coordination between path selection and signal timing.

[0248] To verify the effectiveness of the dynamic traffic guidance and signal control coordination method under traffic accidents submitted in this invention, this embodiment selects the following... Figure 7The directed road network shown serves as the experimental scenario. This network contains n nodes (numbered 1 to 24), with arrows indicating the directions vehicles can travel. In the experiment, node 2 is set as the starting point, node 23 as the ending point, and a traffic accident disturbance is introduced between nodes 4 and 5 to simulate a sudden event causing a decrease in road capacity. In this experimental scenario, after an accident occurs, the traffic guidance method of the lower-layer model calculates detour path schemes under the accident disturbance conditions based on the real-time road network status and predicts the traffic flow distribution of each path within a rolling time window. Subsequently, the signal control strategy of the upper-layer model performs signal timing optimization at key intersections on the detour paths and feeds back the delay prediction results to the lower-layer model, thereby updating path impedance and redistributing vehicles. During the accident, the two-layer model continuously iterates through path guidance and signal optimization in a rolling time window until the accident is resolved, thus ensuring a significant improvement in the overall network operating efficiency and stability.

[0249] The simulation experiment utilizes a parallel computing architecture for both simulation and learning tasks. The software used is SUMO (Simulation of Urban Mobility), a traffic micro-simulation software, controlled via Python's Traci interface. The simulation scenario involves key road sections and adjacent upstream and downstream intersections along the guidance path, simulating congestion caused by a sharp increase in traffic flow. To simulate the dynamic changes in traffic flow during traffic accidents, the entry rate of simulated vehicles is calculated and set using a sine wave function. The periodic variation of the sine wave function simulates the changes in traffic flow during accidents, with the peak value corresponding to congested periods and the minimum value corresponding to non-congested periods. To increase data diversity, the offset of the sine wave is randomly adjusted during reinforcement learning model training, generating different traffic flows in different simulation runs to simulate real-world traffic fluctuations. The intervals between simulated vehicles are randomly generated based on an exponential distribution, ensuring the randomness of vehicle arrival times and further enhancing the realism of the simulation.

[0250] like Figure 8 (a) SUMO simulation scene diagram and such Figure 8 (b) The corresponding experimental scenario diagram shows that intersections 10 and 11 are two adjacent signalized intersections on the key road segments of the guidance path. Intersection 10 has six lanes in both directions from south to north and from east to west. Intersection 11 has six lanes in both directions from south to north and from east to west. The original phase diagram and signal timing diagram of intersection 10 are shown below. Figure 9 As shown in (a), the original phase diagram and signal timing diagram of intersection 11 are as follows: Figure 9 As shown in (b), by Figure 9From the signal timing diagram of intersection 10 in (a), it can be observed that there are all-red signals in the signal timing control of phase 3 (east-west straight, left turn, right turn) and phase 4 (east-west left turn). This is because it is considered that the signal at the upstream signal intersection of the accident can be adjusted to all-red to control the traffic flow entering the accident section. The simulation time is set to 3 hours to simulate the dynamic changes of the traffic flow on the key section within 3 hours.

[0251] Under the same experimental simulation scenario, five traffic signal control algorithms are tested, and the distribution of the total vehicle travel time data obtained is as Figure 10 shown. The vertical coordinate in the figure is the travel time (unit: seconds), and the smaller the value, the shorter the time it takes for the vehicle to pass through the key section from entering the lane to exiting the lane.

[0252] Figure 10 The box plot in the figure consists of three parts: "box", "whisker", and "outlier". The "box" shows the first quartile ( )、median、third quartile ( ) of the data distribution; the calculation method of the value of the "whisker" is shown in Equation (34):

[0253] (34)

[0254] Where: is the first quartile, is the third quartile, is the interquartile range.

[0255] In the to range, the data points are regarded as values within the normal range; the "outliers" represent the points that exceed the range defined by the above "whisker" and are shown in the form of individual scatter points. At the upper right of each box plot, the key statistics of the control algorithm are annotated with three data: mean, standard deviation, and median, so that the overall distribution and dispersion degree of each control algorithm in terms of travel time are represented more carefully.

[0256] The horizontal line in the "box" represents the median position, reflecting the "typical level" of the control algorithm in terms of travel time. Figure 10 Among them, the median of the signal control algorithm based on DQN is 62, which is the minimum value of the median of the travel time under all signal control algorithms, indicating that it can provide a shorter travel time than other algorithms in most vehicle passing situations; the upper and lower edges of the "box" are determined by and . The "shorter" the "box", the more concentrated the travel time of the control algorithm, and the "higher" the "box", the more dispersed the travel time distribution of the vehicles. Figure 10It can be clearly seen that the "box" of the SOTL (Self-Organizing Traffic Lights) signal controller is "higher" than other algorithms, indicating that the travel time of the vehicles it controls fluctuates more.

[0257] The length of the "whisker" is reflected in The data span within the range. A longer "must" in the SOTL box plot indicates that there are still considerable differences in travel time outside the "outlier" range; "outlier" indicates that the vehicle travel time is too long in extreme cases. A large number of SOTL outliers and their distribution in the high value range indicate that SOTL has a very undesirable control result in the simulation scenario, causing severe congestion and long waiting times for vehicles on key road sections.

[0258] Overall, SOTL performed the worst, with the average travel time of vehicles under its control being more than twice that of other methods, and exhibiting many significant outliers. Signal control methods based on DQN and DDPG performed slightly better than Uniform (Uniform Delay Model) and Webster (Webster's Delay Minimization Method) signal control methods. The presence of outliers in DQN and Uniform signal control methods indicates that some vehicles had significantly longer travel times than most.

[0259] The experiment reflects the change in queuing conditions at each intersection over time by recording the number of vehicles queuing at the two intersections. Figure 11 (a) is the time-varying vehicle queue length recorded at intersection 10. Figure 11 (b) is the time-varying vehicle queue length recorded at intersection 11. Figure 11 In each subplot, the solid line represents the average queue length of vehicles in each direction at the intersection at the corresponding time. The shaded area near the solid line represents the 95% confidence interval. In each subplot, the queue length of intersections 10 and 11 under various traffic signal control methods clearly shows a "first increase, then decrease" trend over time. This is because at the initial stage of the simulation, vehicles begin to load into the target lane, and traffic flow gradually increases; at the end stage, vehicles have already passed the target lane, and traffic flow is relatively low; in the middle of the simulation, traffic demand reaches its peak, resulting in a significant increase in the number of queued vehicles. The peak shape of the queue length at the intersection illustrates that different signal control algorithms have different capabilities in handling peak traffic on key road sections, and this is a key factor affecting the performance of traffic control algorithms.

[0260] Depend on Figure 11 (a) and Figure 11(b) It can be intuitively observed that SOTL's signal control effect at intersections is consistent with the previous hyperparameter optimization results, also resulting in a relatively long queue length at intersections, far inferior to other compared signal control methods. To more significantly compare the signal control effects of DDPG, DQN, Uniform, and Webster, Figure 11 (c) and Figure 11 (d) shows the queue lengths of vehicles at intersections 10 and 11 after removing the results from the SOTL signal control method. Figure 11 (c) and Figure 11 (d) shows that the DQN-based and DDPG-based signal control methods outperform the Uniform and Webster methods in controlling queuing at intersections during peak traffic hours, and the DQN-based method is superior to the DDPG method in controlling peak traffic flow. The DQN-based method can non-periodically select the next phase during peak traffic periods when traffic accidents occur, thus avoiding invalid green signals during peak hours, resulting in fewer vehicle queues at intersections compared to the DDPG-based signal control algorithm, which is constrained by signal cycles.

[0261] Figure 12 Record the changes in vehicle delays at the intersection over time. Figure 12 (a) and Figure 12 (b) It can be observed that the time delay levels of intersections 10 and 11 under SOTL control are much higher than those of other signal control algorithms, which is consistent with the hyperparameter optimization results. To intuitively and in detail observe the comparison of intersection delay times between other signal control methods, the time delay records of the two intersections were removed and plotted using the SOTL method. Figure 12 (c) and Figure 12 In (d), Figure 12 (c) and Figure 12 In (d), at signalized intersections 10 and 11, the signal control methods based on DQN and DDPG outperform the rule-based Uniform and Webster signal control methods in terms of time delay control during peak traffic demand, indicating that the traffic signal control method based on deep reinforcement learning is more adaptable to sudden traffic situations and complex traffic flow changes. In the initial and final stages of the simulation, it can be seen from... Figure 12 (c) and Figure 12 (d) The shaded area near the solid line of time delay based on the DQN signal control method shows that its 95% confidence interval is relatively wide, indicating that the results fluctuate greatly when the simulation is repeated. This also reveals the mechanism that the DQN-based signal control method is greatly affected by the hyperparameter setting.

[0262] comprehensive Figure 11 and Figure 12The signal control method based on DDPG can be found to be adaptable and scalable when traffic flow changes drastically on key road sections. Compared with the signal control method based on SOTL, the signal control methods based on DQN, Uniform, Webster, and DDPG can mitigate the impact of traffic flow fluctuations in terms of total vehicle travel time, intersection delay time, and queue length on key road sections, and are more valuable for development and utilization of traffic flow changes.

[0263] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0264] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for coordinated dynamic traffic guidance and signal control in the event of a traffic accident, characterized in that, Includes the following steps: S1: Construct a directed weighted complex road network model, obtain the real-time operating status of the road network and evaluate it; S2: Based on the complex road network model and its operating status constructed by S1, a path guidance module is generated using the dual Q-learning algorithm; S3: Based on the deep deterministic policy gradient algorithm, a signal control agent is constructed. The agent perceives the traffic state of the intersection, outputs the green light duration of the next phase as an action, and learns by combining reward signals with vehicle delay and queue length as the core to generate a signal control module. S4: Establish a two-way collaborative mechanism between the path guidance module and the signal control module, and perform collaborative iteration in a rolling time window manner to form a closed-loop collaborative optimization system for path guidance and signal control; The specific steps for S4 are as follows: S401: Establish a collaborative data interface between the path guidance module and the signal control module. The path guidance module transmits the generated detour path set, the time-based traffic forecast values ​​of each node, and the flow direction ratio to the signal control module through this interface. After adjusting the timing scheme based on the received traffic forecast information, the signal control module sends back the predicted delay, saturation change data, and green ratio parameters of each key intersection to the path guidance module. S402: Based on the preset rolling time window period, the path guidance scheme and signal timing scheme are dynamically updated through continuous data interaction and strategy iteration between the path guidance module and the signal control module, so as to realize the closed-loop collaborative optimization of path selection and signal control. In S402, the continuous data interaction and iteration between the path guidance module and the signal control module are carried out in the following steps: S4021: The route guidance module generates the optimal detour route based on real-time traffic status data and predicts the traffic flow distribution of each detour route within a future rolling time window. S4022: The signal control module receives the path traffic prediction results output by the path guidance module, uses a deep deterministic strategy gradient algorithm to optimize the green ratio parameters of key intersections, and generates a new signal timing scheme that adapts to traffic changes. S4023: The signal control module will send the optimized delay data and saturation data of each node back to the path guidance module in real time; S4024: The route guidance module updates the path impedance value and path selection probability of each road segment based on the received delay and saturation feedback data, and redistributes the traffic flow of detour vehicles. S4025: Using a preset rolling time window as the iteration cycle, repeat the process from S4021 to S4024 to achieve real-time coordination between the path selection strategy and the signal timing strategy, ensuring that the system dynamically adapts to changes in traffic flow.

2. The method for coordinated dynamic traffic guidance and signal control under traffic accidents according to claim 1, characterized in that, The specific steps for S1 are as follows: S101: Construct a directed weighted complex road network graph model using the original road network. ; S102: The real-time operating status of the road network is obtained by combining data collected by roadside sensing units with data reported by vehicles and then aggregated to the control center. S103: Obtain information on lane travel time, road usage, and queue length; use the BPR function to calculate the travel time impedance value of each road segment; and use this impedance value as the road segment weight to evaluate the road network operation status.

3. The method for coordinated dynamic traffic guidance and signal control under traffic accidents according to claim 1, characterized in that, The specific steps for S2 are as follows: S201: Construct a reinforcement learning model for path induction, the reinforcement learning model comprising a first value function network configured in parallel. Second value function network And the experience replay pool; S202: Obtain and fuse node state information and road segment state information of the road network to construct an environmental state vector representing the real-time operating state of the road network. ; S203: Transfer the environmental state vector The input is given to the reinforcement learning model, and then... Strategy, based on probability From all the valid adjacent nodes of the current node, select the next hop node and use that next hop node as action a; S204: Execute action a, moving the recommended path from the current node to the next node. The environment generates a comprehensive reward value based on the new road network state after action a is executed. and the new environment state vector ; S205: The environment state vector will be used. Action a, Overall Reward Value New environment state vector The resulting experience tuples are stored in the experience replay pool. Experience data is periodically sampled from the experience replay pool, and the first value function network is then processed using a dual Q-learning algorithm. Second value function network The parameters are iteratively updated to optimize the path decision-making logic and generate a path guidance module.

4. The method for coordinated dynamic traffic guidance and signal control under traffic accidents according to claim 3, characterized in that, The comprehensive reward value described in S204 Rewards for travel time Path length reward Bandwidth utilization reward Node impedance reward Goal-oriented rewards The weighted sum is shown in equation (1): （1） in: As a reward for path length, As a reward for travel time, As a reward for bandwidth utilization, As a reward for node impedance, Rewards based on goals For the overall reward value, The weighting coefficient for path length reward. The weighting coefficient for travel time rewards. The weighting coefficient for bandwidth utilization rewards. The weighting coefficient for node impedance reward. The weighting coefficient for goal-oriented rewards.

5. The method for coordinated dynamic traffic guidance and signal control under traffic accidents according to claim 1, characterized in that, The specific steps for S3 are as follows: S301. Construct a signal control agent based on deep deterministic policy gradient (DDPG), wherein the signal control agent includes an actor network for outputting control policies and a critic network for evaluating policy value. S302. Construct the Actor state and Critic state. The Actor state is used to observe the vehicle density and queuing situation in the lanes entering at the intersection. Its state space... Defined as shown in equation (2): （2） in: Let Actor state space be defined. For the set of real numbers, For the entry lanes of the fork, The number of signal phases at the intersection. It is a binary set; S303: Select an action based on the Actor state. This action The duration of the next phase is determined by the following constraints: the duration of the next phase is not less than the minimum green light time and not greater than the maximum green light time. S304. Establish a comprehensive reward and punishment function for the signal control algorithm based on DDPG. The comprehensive reward and punishment function Including vehicle delay penalties Penalties for vehicle queue length ; S305. Iteratively train the signal control agent based on deep deterministic policy gradient (DDPG), and generate a signal control module by optimizing the parameters of the actor network and the critic network.

6. The method for coordinated dynamic traffic guidance and signal control under traffic accidents according to claim 3, characterized in that, The node status information mentioned in S202 includes: node computing utilization. Storage utilization rate Current node marker Target node marking and entry node marker The road segment status information includes: road segment utilization rate. and segment normalization delay .

Images