A trunk intersection cooperative control method considering robustness

Abstract

The application aims to provide a trunk intersection cooperative control method considering robustness, belonging to the technical field of intelligent traffic control, which comprises the following steps: constructing a multi-agent trunk intersection system, regarding each intersection as an independent agent; performing trunk intersection robustness configuration, introducing a robustness optimization strategy, establishing a traffic flow uncertainty fluctuation model, setting a robustness target and constraint condition; building a robust trunk multi-agent network framework, defining a robust joint traffic state space, action control parameters and a robust reward function; training the agent using a deep reinforcement learning algorithm, constructing a multi-agent deep deterministic policy gradient algorithm considering robustness; and optimizing the control strategy according to real-time traffic data through online learning and adaptive adjustment. The application improves the adaptability and stability of the trunk intersection in complex traffic environment through robustness setting and agent cooperative control, and has good scalability.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic control technology, and in particular to a robust collaborative control method for arterial intersections. Background Technology

[0002] With the acceleration of urbanization, arterial intersections face the challenge of surging and fluctuating traffic flow. Traditional traffic control methods, relying on fixed signal cycles and static traffic flow models, struggle to cope with the complex changes in dynamic traffic environments. During peak hours or in the event of sudden accidents, fixed signal timing schemes are prone to problems such as queue overflow and sudden increases in vehicle delays, severely restricting the efficiency of arterial traffic. Although multi-agent cooperative control methods improve flexibility through distributed decision-making, existing technologies still have significant limitations: they lack robust modeling for uncertainties such as sudden changes in traffic flow and severe weather, making control strategies prone to instability under disturbances; multi-agent cooperative mechanisms rely heavily on local information interaction and do not fully integrate the dynamic correlation characteristics of adjacent intersections, easily falling into local optima and sacrificing the overall coordination of the arterial road; furthermore, traditional reinforcement learning algorithms converge slowly in dynamic scenarios and do not introduce robust constraints, resulting in lag in strategy adjustments when facing sudden events, exacerbating traffic imbalance. For example, while existing patented solutions attempt to dynamically adjust signal timing, they do not incorporate road conditions and environmental perception parameters into real-time decision-making, leading to a significant decrease in control effectiveness under complex disturbances. Therefore, how to construct a trunk intersection control method with robust response, global coordination and online adaptive capabilities in the context of increasing traffic flow uncertainty has become a key challenge to improve urban traffic efficiency. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a robust arterial intersection cooperative control method. By integrating deep reinforcement learning, robust optimization and multi-agent cooperation mechanism, it realizes traffic light cooperative control under dynamic traffic flow, so as to cope with traffic flow fluctuations, emergencies and environmental interference, and improve the traffic efficiency and system stability of arterial intersections.

[0004] To achieve the above objectives, the present invention provides a robust collaborative control method for trunk intersections, comprising the following steps:

[0005] Step S1: Construction of a Multi-Agent Trunk Road Intersection System: Modular configuration, treating each intersection on the traffic artery as an intelligent agent. Each intelligent agent has independent decision-making and information processing capabilities. Each intelligent agent is an independent module with good scalability, allowing for easy addition, deletion, or replacement as the city develops. The system includes a perception module, a decision-making module, an execution module, and a communication module. Intelligent agents exchange information through a communication network to achieve collaborative control. The perception module acquires real-time traffic data and performs data cleaning and feature extraction to provide input for subsequent uncertainty fluctuation modeling.

[0006] Step S2, Robust Configuration of Arterial Intersections: Considering the uncertainties of arterial traffic flow, such as fluctuations in vehicle arrival rates and changes in road conditions, a robust optimization strategy is introduced. A robust modeling strategy based on deep reinforcement learning is proposed, and a negative binomial moving average randomization algorithm is introduced. By dynamically adjusting the model parameters combined with real-time environmental perception information, the robustness of the system is enhanced. A model for modeling the uncertainty fluctuations of arterial traffic flow is proposed. A robust objective function and robust constraints are proposed, setting minimizing vehicle delays and traffic congestion as the robustness objective. Based on real-time perception data, the parameters of the objective function are dynamically adjusted to cope with environmental changes and improve the robustness of the control system through coordinated control of arterial intersection signals.

[0007] Step S3, Robust Multi-Agent Model Configuration: Based on the traffic flow uncertainty fluctuation model, robust objective setting, and robust constraint conditions from Step S2, a robust mainline-intelligent agent-actor-critic (RMIAC) network framework is constructed. This framework integrates the robust joint traffic state space, action control parameters, and robust reward function, enabling each agent to not only autonomously optimize its own decisions but also comprehensively consider the dynamic changes of adjacent intersections to ensure the optimization of global traffic flow. Taking into full account the traffic flow dynamics at intersections and their correlation with adjacent intersections, a definition of robust joint traffic state space, action control parameters, and a robust reward function definition are proposed, incorporating a multi-level feedback mechanism.

[0008] Step S4: Configuration of Cooperative Control Strategy for Arterial Crossings: A deep reinforcement learning algorithm is used to train the agents, proposing a robust agent-based cooperative control algorithm for arterial crossings (CR-MADDPG). This algorithm enables agents to select appropriate control strategies based on the current traffic conditions, allowing each agent to maximize its own interests while considering the impact on the overall traffic flow of the arterial road. Cooperative control of arterial crossings is achieved through information exchange and collaboration among agents. Robustness parameters are introduced to enhance the system's ability to respond to unexpected events (such as traffic accidents and weather changes), and a robustness-considered strategy update formula is proposed. An agent objective function is proposed, incorporating not only traffic flow rewards but also a robustness adjustment factor based on environmental perception, thereby improving the system's adaptability in uncertain environments. A robustness-considered loss function is proposed, considering not only standard error but also a robustness adjustment term, comprehensively considering the different challenges faced by agents in cooperation and competition, ensuring the system can operate stably in uncertain environments.

[0009] Step S5, Online Learning and Adaptive Adjustment: In actual operation, the robust multi-agent deep policy gradient algorithm from Step S4 is used as the initial policy for online learning. Experience is collected based on real-time traffic data, and random batch samples are used to update the network parameters of the agents to fine-tune the online learning strategy. That is, the agents can collect experience based on real-time traffic data, learn online, and randomly select small batches of samples to update network parameters, continuously optimizing the control strategy. At the same time, the agents can adaptively adjust the control parameters according to changes in traffic flow, improving the adaptability and robustness of the control system.

[0010] Beneficial effects: By introducing a robust modeling strategy based on deep reinforcement learning and a negative binomial moving average randomization algorithm, the model parameters are dynamically adjusted in combination with environmental perception information, effectively addressing uncertainties such as fluctuations in vehicle arrival rates, changes in road conditions, and weather conditions. The setting of robust objective functions and constraints, combined with robust reward functions, ensures that the multi-agent arterial intersection system can still operate stably under emergencies such as accidents and extreme weather. Compared with traditional fixed-period methods, traffic delays are significantly reduced.

[0011] Furthermore, the construction of the multi-agent trunk intersection system described in step S1 specifically includes:

[0012] S11. Construction of a single intersection intelligent agent: Each intersection is regarded as an independent intelligent agent with independent decision-making and information processing capabilities. It is responsible for monitoring and managing the traffic conditions of its intersection, including vehicle arrival and departure, lane occupancy, traffic light status, etc. It can make appropriate control decisions based on its own status and the surrounding environment, and exchange information with other intelligent agents to achieve collaborative control between arterial intersections.

[0013] S12. Intelligent Agent Software Structure Configuration: This includes a perception module, a decision-making module, an execution module, and a communication module. The perception module is responsible for collecting traffic data; the decision-making module makes decisions based on the proposed algorithm; the execution module is responsible for controlling traffic facilities such as traffic lights; and the communication module is responsible for exchanging information with other intelligent agents.

[0014] S13. Establishment of Cooperative Control Mechanism: The goal of cooperative control is to achieve optimal traffic conditions at all intersections on the main line while maintaining robustness. A proposed deep learning algorithm is used to train the agent, enabling it to select appropriate cooperative control strategies based on the current traffic conditions.

[0015] S14. System Scalability Configuration: The system adopts a modular configuration, with each agent being an independent module. It has good scalability and can be easily added, deleted, or replaced as the city develops.

[0016] Furthermore, the robust configuration of the trunk intersection described in step S2 specifically includes the following steps:

[0017] S21. Modeling Uncertain Fluctuations in Trunk Traffic Flow: To effectively address the uncertain fluctuations in trunk traffic flow, the impact of real-time environmental factors such as vehicle arrival rate, road conditions, and weather conditions is considered. A robust modeling strategy based on deep reinforcement learning is proposed, and a negative binomial moving average randomization algorithm is introduced. The robustness of the system is enhanced by dynamically adjusting the model parameters in conjunction with real-time environmental perception information; the specific traffic flow modeling formula is as follows:

[0018]

[0019] In the formula: Q m It represents the maximum traffic flow on the main line, k represents the number of vehicles arriving per unit time, and p... k β is the probability of k vehicles arriving per unit time, and β is the maximum traffic flow Q on the main road. m The difference between the number of vehicles arriving per unit time and the number of vehicles arriving per unit time, δ B The vehicle arrival fluctuation parameter has a value range of [0,2], δ W This is a weather condition parameter, with a value range of [0,1], δ CThe parameter represents the road condition variation, with values ​​ranging from [0,1], δ Env The environmental perception information parameter represents the impact of real-time environmental changes, including road conditions and accidents, and its value ranges from [0,1].

[0020] S22. Robustness Objective Setting: To ensure system stability under dynamically changing traffic flow and real-time environmental changes, minimizing vehicle delays and traffic congestion is set as the robustness objective. Based on real-time sensing data, the parameters of the objective function are dynamically adjusted to cope with environmental changes. The updated robustness objective function is as follows:

[0021]

[0022] In the formula: Q represents the uncertain and fluctuating traffic flow. m R is the maximum traffic flow on the main road, R is the red light duration at the intersection, a is the robustness parameter, and ε is the maximum traffic flow on the main road. R,d It is a robust parameter for vehicle delays, with a value range of [0, a], ε R,j It is a robust parameter for traffic congestion, with a value range of [0, a], δ B V is the vehicle arrival fluctuation parameter. Q The average speed on the trunk line under uncertain and fluctuating traffic flow;

[0023] S23. Robustness Constraints: To limit the system's behavior under uncertainties and ensure its stability and security, a red light time R is set. Fluctuations are implemented to avoid traffic chaos caused by frequent, large-scale switching of traffic light cycles.

[0024] Furthermore, the robust multi-agent model configuration proposed in step S3 specifically includes the following steps:

[0025] S31. To further enhance the robustness of the arterial intersection cooperative control system, a robust arterial multi-agent (RMIAC) network framework was configured and built. This framework integrates the robust traffic joint traffic state space, action control parameters, and robust reward function, enabling each agent to not only autonomously optimize its own decisions but also comprehensively consider the dynamic changes of adjacent intersections to ensure the optimization of global traffic flow. The network training adopts a "centralized learning, distributed execution" mode: centralized learning allows global state and reward information to be processed centrally, while distributed execution ensures that each intersection agent can operate independently and respond quickly to emergencies. Specifically, during the testing phase, each intersection agent will output the optimal control action through its independent Actor network to ensure rapid response and effective handling of traffic flow fluctuations and environmental changes.

[0026] S32. Definition of Robust Joint Traffic State Space: To fully consider the traffic flow dynamics at intersections and their correlation with adjacent intersections, a robust joint traffic state space is proposed. This state space not only includes the traffic characteristics of the local intersection but also integrates traffic data and environmental perception information from adjacent intersections. Specifically, the state vector of each intersection consists not only of basic traffic data such as maximum queue length and vehicle arrival rate but also incorporates real-time environmental perception information such as road conditions, sudden accidents, and weather, thus forming a robust joint traffic state space. Its definition is as follows:

[0027]

[0028] In the formula, This represents the maximum queue length of the i-th phase of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control cycle. This represents the arrival flow rate per unit time of the i-th phase of agent m in the previous control cycle; This represents the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control cycle. The ε represents the arrival flow rate per unit time of the i-th phase of the neighboring agent n of agent m during the previous control cycle; a is a robustness parameter, ε R,d It is a robust parameter for vehicle delays, with a value range of [0, a], ε R,j It is a robust parameter for traffic congestion, with a value range of [0, a].

[0029] S33. Robust Action Control Parameter Definition: Regarding action control, a dynamic control parameter definition scheme based on the maximum cycle length is proposed, taking robustness into consideration. Each intersection agent selects the optimal control action (i.e., the green light ratio of each phase) based on the input joint traffic state vector. This control strategy not only optimizes local traffic flow but also considers the impact of adjacent intersections on global traffic flow. Robust action control parameters are proposed in the configuration of control parameters.

[0030]

[0031] In the formula, This represents the ratio of the green light duration to the cycle duration in the i-th phase of agent m; This represents the maximum queue length of the i-th phase of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control cycle. denoted by , representing the arrival flow rate per unit time of the i-th phase of agent m within the previous control cycle; 'a' is a robustness parameter.

[0032] S34. Robust Reward Function Definition: In configuring the reward function, we introduce a multi-layered feedback mechanism, considering both the control effect of the local intersection agent and the state changes of adjacent intersections. This configuration ensures that the system can optimize traffic flow at a single intersection while also comprehensively considering the global traffic situation, avoiding local optima. The specific robust reward function... The definition is as follows:

[0033]

[0034] In the formula, This represents the arrival flow rate of agent m in the i-th phase during the previous control cycle. This represents the average vehicle delay of agent m in the i-th phase during the previous control cycle. This represents the arrival flow rate of agent n in the i-th phase during the previous control cycle. This represents the average vehicle delay of agent n in the i-th phase during the previous control cycle; This represents the maximum queue length of the i-th phase of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control cycle. This represents the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control cycle. The number of vehicles corresponding to the maximum queue length of the i-th phase of the neighboring agent n of agent m in the previous control cycle; a is the robustness parameter; S R It is a robustness objective function; this reward function takes into account the combined effects of vehicle delays, traffic flow, and the system robustness objective, maximizing traffic flow while ensuring that the system can maintain efficient and stable operation in the face of traffic fluctuations and external disturbances.

[0035] Beneficial effects: By adopting a modular multi-agent architecture and a "centralized learning-distributed execution" mode, each intersection agent can make independent decisions while integrating dynamic data from adjacent intersections through a joint traffic state space to achieve global optimization at the trunk line level. The reward function incorporates vehicle delay and traffic flow data from local and adjacent intersections, avoiding trunk line congestion caused by local optimization and improving overall traffic efficiency.

[0036] Furthermore, the robust multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4 specifically includes:

[0037] S41. Propose a robust policy update formula: To achieve stronger robustness, this step proposes a robustness-enhanced policy update formula that optimizes the agent's policy selection while considering traffic flow and environmental disturbances. The update formula not only optimizes the policy using traditional gradient information but also introduces robustness parameters to enhance the system's ability to cope with unexpected events (such as traffic accidents and weather changes). The specific update formula is as follows:

[0038]

[0039] Where: μ i (s i |θ i ) is the state s of agent i. i The strategy is as follows; α is the learning rate; a is the robustness parameter; It concerns the policy network parameters θ i gradient; ∈ i It is adversarial noise, designed to enhance the robustness of agents in decision-making in complex and dynamic environments; through this update method, agents can optimize traffic flow while ensuring the stability and robustness of their strategies in the face of uncertainties.

[0040] S42. Propose a robust agent objective function: To better measure the robustness of the agent's behavior, this step introduces a new objective function. This function, based on the traditional objective function, incorporates robustness parameters to optimize the agent's decision-making process. Specifically, the objective function not only considers traffic flow rewards but also incorporates a robustness adjustment factor based on environmental perception, thereby improving the system's adaptability to uncertain environments. The objective function is defined as follows:

[0041]

[0042] In the formula: J i (θ i ) is the objective function of agent i; For trajectory τ i The expected value; T is the time termination step; γ is the discount factor; Is Take action in a state The reward; S R is the robustness objective function; 'a' is the robustness parameter; this objective function improves the system's stability and anti-interference ability by optimizing long-term rewards and considering robustness.

[0043] S43. Propose a robust loss function: To further improve the learning efficiency of multi-agent systems in non-ideal environments, a robust loss function is proposed. This function, in addition to considering the standard Q-learning error, incorporates a robust adjustment term, comprehensively considering the different challenges faced by agents in cooperation and competition, ensuring the system can operate stably in uncertain environments. The specific loss function is as follows:

[0044]

[0045] in:

[0046] In the formula: L i (φ i ) is the loss function of agent i; It is an experience replay pool; Q i (s,a∣φ i ) is the value function, where φ i These are the network parameters of the value function; y i It is the target value; These are the parameters of the target network; γ is the discount factor; S j ,a j It is a small batch of experience data extracted from the experience pool; 'a' is the robustness parameter. This loss function improves the robustness and stability of the system in complex and dynamic traffic environments by comprehensively considering the cooperation and competition of multiple agents.

[0047] Furthermore, the online learning and adaptive adjustment proposed in step S5 specifically includes the following steps:

[0048] S51. Initialize based on the robust multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4: Initialize the policy network parameters θ of all agents. i and value function network parameters φ i and target network parameters Set to φ i Similarly, this initialization step lays the foundation for the system's learning process and ensures that the agent can adaptively learn in dynamic environments. The specific steps are as follows:

[0049] θ i ,φ i initialization:

[0050] S52. Based on the robust multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4, experience is collected: at each time step t, each agent collects experience according to the current policy. Choose an action The action selection takes into account not only the current state. Robustness parameters were also added to ensure that the agents can cope with fluctuations in traffic flow, and that the experience gained by each agent... Stored in the experience replay pool This provides data support for subsequent learning and parameter updates. The specific steps are as follows:

[0051] Experience collection: Store in experience pool

[0052] S53. Based on the robust multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4, the algorithm is used to refine the experience pool. Randomly select small batches of samples from the empirical replay pool to update network parameters: Randomly sample small batches of empirical data and update the agent's value function network parameters φ based on this experience. i Policy network parameters θ i and target network parameters During the update process, in addition to traditional policy updates and value function optimization, a robust objective function S is also introduced. R As a pre-tuning term, this step ensures that the agent can maintain high-quality decision-making when facing complex dynamic traffic flows. This step continues until the policy network converges, ensuring that the system can adapt to various traffic changes and optimize signal control. The specific steps are as follows:

[0053] Small batch sampling: Update network parameter φ i ,θ i ,

[0054] Beneficial effects: By collecting traffic data and updating network parameters in real time through an online learning mechanism, and combining a robust strategy update formula with adversarial noise injection, it can adapt to sudden changes in traffic flow.

[0055] Compared with existing technologies, this invention achieves significant optimization of trunk intersection signal control through a combination of multi-agent cooperative architecture and robust design. The beneficial effects achieved by this invention are as follows:

[0056] This invention is based on a distributed system construction of modular intelligent agents. Through independent decision-making and information interaction mechanisms, it enhances global coordination capabilities while ensuring rapid response at each intersection. Furthermore, this invention innovates robust configuration by using dynamic traffic flow modeling, an adaptive objective function, and red light time constraints to effectively address uncertainties such as fluctuations in vehicle arrival rates, weather, and road conditions. The robust multi-agent model configured in this invention integrates traffic state data from adjacent intersections with a multi-level feedback mechanism, fusing local and environmental perception information within a joint traffic state space definition to achieve optimal balance of global traffic flow. It further employs deep reinforcement learning algorithms, introducing robust parameters and adversarial noise to enhance the stability of policy updates and the anti-interference capability of the loss function, thereby improving decision robustness in complex environments. Additionally, an online learning mechanism supports real-time data acquisition and network parameter updates, enabling the control strategy to continuously adapt to dynamically changing traffic environments. This significantly enhances the system's adaptability to sudden events and environmental disturbances while improving arterial traffic efficiency. Attached Figure Description

[0057] The accompanying drawings, which form part of this specification, illustrate embodiments of the invention and, together with the specification, serve to explain the principles of the invention.

[0058] The invention will be more clearly understood with reference to the accompanying drawings and the following detailed description, wherein:

[0059] Figure 1 This is a flowchart of a robust trunk intersection cooperative control method provided in an embodiment of the present invention;

[0060] Figure 2 This is a schematic diagram illustrating the construction of a multi-agent trunk intersection system provided in an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of the robust multi-agent model network framework provided in an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of the optimal trajectory of trunk line vehicles under the cooperative control strategy provided in an embodiment of the present invention.

[0063] Figure 5 The collaborative control strategy provided in this embodiment of the invention is shown in the iterative convergence graph. Detailed Implementation

[0064] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, and not limitations thereof. Where there is no conflict, the embodiments and technical features in the embodiments can be combined with each other. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0065] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0066] like Figures 1-5 As shown, this invention provides a robust collaborative control method for trunk intersections, specifically including the following steps:

[0067] Step S1: Construction of a Multi-Agent Arterial Intersection System: Each intersection on the traffic artery is considered as an intelligent agent, with each agent possessing independent decision-making and information processing capabilities. Information is exchanged through a communication network to achieve collaborative control. The intelligent agent includes a perception module, a decision-making module, an execution module, and a communication module. Specifically, the perception module collects real-time data on vehicle arrival rate, lane queue length, and traffic light status at the intersection, performs data cleaning and feature extraction, and provides input for subsequent uncertainty fluctuation modeling. The communication module enables traffic status information exchange between adjacent intelligent agents. The execution module adjusts the traffic light timing, and the decision-making module generates control commands.

[0068] Step S2, Robust Configuration of Arterial Intersections: Considering the uncertainties of arterial traffic flow, such as fluctuations in vehicle arrival rates and changes in road conditions, a robust optimization strategy is introduced. This involves modeling the uncertainty fluctuations of arterial traffic flow, setting robustness objectives, and constructing robustness constraints to improve the robustness of the multi-agent arterial intersection system. Preferably, a traffic flow model is constructed based on vehicle arrival rate fluctuation parameters, road capacity, road condition parameters, and weather condition parameters, where road capacity represents the maximum number of vehicles that can pass through the intersection per unit time. A robust objective function is set to minimize vehicle delay time and lane queue length, and the range of values ​​for vehicle delay robustness parameters and congestion robustness parameters in the objective function is dynamically adjusted. The red light time at the intersection is constrained to fluctuate within a preset range.

[0069] Step S3, Robust Multi-Agent Model Configuration: A robust mainline-intelligent-agent-actor-critic (RMIAC) network framework is built using the traffic flow uncertainty fluctuation model, robust objective setting, and robust constraints from Step S2. A robust joint traffic state space, action control parameters, and robust reward function are defined and integrated. Preferably, the joint traffic state space is defined, including the following parameters for the local intersection and adjacent intersections in the previous control cycle: the ratio of the maximum queue length to the corresponding number of vehicles in each phase, and the ratio of arrival flow rate per unit time to road capacity. A robust joint state vector is generated by combining vehicle delay robust parameters and congestion robust parameters. The action control parameter is defined as the ratio of the green light duration to the cycle duration in each phase, calculated from the joint state vector and robust parameters. A reward function is configured, and a robust objective function is introduced as an adjustment factor, integrating vehicle delay, arrival flow, and queuing characteristics of the local and adjacent intersections.

[0070] Step S4: Configuration of Cooperative Control Strategy for Arterial Crossings: The robust joint traffic state space and action control parameters are used as inputs to a deep reinforcement learning algorithm to design the agent's observation space and action selection strategy. The agent is trained using a deep reinforcement learning algorithm, and a robust multi-agent deep deterministic policy gradient algorithm (CR-MADDPG) is constructed to select an appropriate control strategy based on the current traffic state. This allows each agent to maximize its own interests while considering the impact on the overall traffic flow of the arterial road. Through information exchange and cooperation among the agents, cooperative control of the arterial intersection is achieved, and vehicles proceed according to... The agent travels along the optimal trajectory. Preferably, the policy network parameters of the agent are updated using a centralized learning approach, and control actions are output using a distributed execution approach. Adversarial noise and robust parameters are introduced into the policy update to generate a policy gradient. When defining the objective function, the reward function and the robust objective function are fused, and a robust adjustment term is added to the loss function to define a robust agent objective function, thereby driving the agent to learn a robust policy and achieve global robust optimization. The decision module can be used to execute control commands generated based on the robust multi-agent deep deterministic policy gradient algorithm (CR-MADDPG).

[0071] Step S5, Online Learning and Adaptive Adjustment: The intelligent agent collects experience based on real-time traffic data, learns online, and updates network parameters by randomly sampling batches of samples. It continuously optimizes the control strategy and can adaptively adjust control parameters according to changes in traffic flow, thereby continuously improving the adaptability and robustness of the control system. Preferably, traffic state data is collected in real time and stored in the experience replay pool, and network parameters are updated in batches by randomly sampling samples. The robustness parameters and adversarial noise intensity are dynamically adjusted according to traffic flow fluctuations until the strategy network converges.

[0072] In summary, this invention provides a reliable and practical means for the collaborative control of arterial intersections when facing fluctuating traffic flow by proposing robust configuration of arterial intersections, configuring a robust arterial multi-agent (RMIAC) network framework, and using a robust multi-agent depth-determining policy gradient algorithm.

[0073] Furthermore, the construction of the multi-agent trunk intersection system in step S1 specifically includes:

[0074] Step S11: Construction of a single intersection agent: Each intersection is regarded as an independent agent, and each agent has independent decision-making and information processing capabilities;

[0075] Step S12, Intelligent agent software structure configuration: including perception module, decision module, execution module and communication module; configure the perception module to collect traffic data, configure the decision module to generate control commands based on the robust multi-agent deep determination strategy gradient algorithm, configure the execution module to adjust the traffic light timing, and configure the communication module to transmit the status information of adjacent intersections;

[0076] Step S13: Establishment of cooperative control mechanism: The agent is trained using deep learning algorithms so that it can select appropriate cooperative control strategies based on the current traffic conditions.

[0077] Step S14, System Scalability Configuration: A modular configuration is adopted, with each agent as an independent module, to facilitate expansion.

[0078] Beneficial effects: The modular intelligent agent design supports the rapid addition or replacement of intersection nodes, and adding a new intersection only requires initializing the local intelligent agent network.

[0079] Furthermore, the robust configuration of the trunk intersection in step S2 specifically includes:

[0080] Step S21: Modeling Uncertainty Fluctuations in Mainline Traffic Flow: Considering the impact of vehicle arrival rate, road conditions, weather conditions, and real-time environmental factors, a robust modeling strategy based on deep reinforcement learning is constructed, introducing a negative binomial moving average randomization algorithm. The following formula is used to model the uncertainty fluctuations of trunk traffic flow:

[0081]

[0082] Among them, Q m It represents the maximum traffic flow on the main line; k is the number of vehicles arriving per unit time; p k β is the probability of k vehicles arriving per unit time; β is the maximum traffic flow Q on the main road. m The difference between the number of vehicles arriving per unit time and the number of vehicles arriving per unit time; δ B For vehicle arrival fluctuation parameters; δ W For weather conditions; δ C δ is a parameter for changes in road conditions. Env These are environmental sensing information parameters;

[0083] Step S22: Robustness Objective Setting. Minimizing vehicle delays and traffic congestion is set as the robustness objective. The parameters of the objective function are dynamically adjusted based on real-time sensing data. The robustness objective function is:

[0084]

[0085] Where Q represents the uncertain and fluctuating traffic flow, Q m R is the maximum traffic flow on the main road, R is the red light duration at the intersection, a is the robustness parameter, and ε is the maximum traffic flow on the main road. R,d It is the robustness parameter for vehicle delays, ε R,j It is a robust parameter for traffic congestion, V Q The average speed on the trunk line under uncertain and fluctuating traffic flow;

[0086] Step S23, Robustness Constraints: Set the red light time R in... Fluctuations within a certain range.

[0087] Furthermore, the robust multi-agent model configuration in step S3 specifically includes:

[0088] Step S31: Build a robust trunk multi-agent network framework, update the policy network parameters of the agents in a centralized learning manner, and output control actions through a distributed execution method.

[0089] Step S32: Define the robust joint traffic state space. Defined as:

[0090]

[0091] in, This represents the maximum queue length of the i-th phase of agent m within the previous control period t; This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control period t; Let C represent the arrival flow rate per unit time of the i-th phase of agent m in the previous control period t, and C be the road capacity. This represents the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control period t. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of the neighboring agent n of agent m during the previous control period t. The flow rate per unit time of the i-th phase of the neighboring agent n of agent m during the previous control period t is represented by ε; a is the robustness parameter; R,d It is the robustness parameter for vehicle delays; ε R,j It is a robust parameter for traffic congestion levels;

[0092] Step S33: Define robust motion control parameters. Defined as:

[0093]

[0094] in, denoted by , represents the ratio of the green light duration to the cycle duration in the i-th phase of agent m, where 'a' is a robustness parameter;

[0095] Step S34: Define the robust reward function. Defined as:

[0096]

[0097] in, This represents the arrival flow rate of agent m in the i-th phase during the previous control period t; This represents the average vehicle delay of agent m in the i-th phase during the previous control period t. This represents the arrival flow rate of agent n in the i-th phase during the previous control period t; S represents the average vehicle delay of agent n in the i-th phase during the previous control period t; R It is a robust objective function.

[0098] Beneficial effects: When designing robust motion control parameters, a dynamic control scheme with maximum cycle duration is introduced, combined with the red light time fluctuation constraint in step S23. This effectively avoids frequent large-scale switching of the signal cycle.

[0099] Furthermore, the robust multi-agent deep policy gradient determination algorithm constructed in step S4 specifically includes:

[0100] Step S41: Construct a policy update formula that considers robustness:

[0101]

[0102] Where, μ i (s i |θ i ) is the state s of agent i. i The strategy is as follows; α is the learning rate; a is the robustness parameter. It concerns the policy network parameters θ i gradient; ∈ i It is adversarial noise;

[0103] Step S42: Construct the agent objective function considering robustness:

[0104]

[0105] Among them, J i (θ i ) is the objective function of agent i; For trajectory τ i The expected value; T is the time termination step, and γ is the discount factor; It is in state Take action below The reward; S R It is a robust objective function;

[0106] Step S43: Construct a loss function that considers robustness:

[0107]

[0108] Among them, L i (φ i ) is the loss function of agent i; It's an experience replay pool, Q i (s,a∣φ i ) is the value function; φ i These are the network parameters of the value function; y i It is the target value; These are the parameters of the target network; γ is the discount factor; s j and a j It is a batch of experience data extracted from the experience replay pool.

[0109] Furthermore, the online learning and adaptive adjustment in step S5 specifically include:

[0110] Step S51: Initialize the policy network parameters θ for all agents. i and value function network parameters φ i and target network parameters Set to φ i same;

[0111] Step S52: Each agent, according to the current policy... Select Action The experience of each agent Stored in the experience replay pool middle;

[0112] Step S53, from the experience replay pool Randomly sample small batches of empirical data and update the agent's value function network parameters φ based on this experience. i Policy network parameters θ i and target network parameters Until the policy network converges.

[0113] Example 1

[0114] Figure 1 This is a flowchart illustrating a robust arterial intersection collaborative control method according to Embodiment 1 of the present invention. This flowchart merely shows the logical sequence of the method described in this embodiment; however, in other possible embodiments of the present invention, different methods may be used, provided there are no conflicts. Figure 1 Complete the steps shown or described in the order indicated.

[0115] This embodiment is a typical implementation of the present invention, providing a robust collaborative control method for trunk intersections. This method can be applied to a terminal and can be executed by an electronic terminal. The electronic terminal can be implemented in software and / or hardware and can be integrated into a terminal, such as any smartphone, tablet, or computer device with communication capabilities. Figure 1 As shown, the method in this embodiment specifically includes the following steps:

[0116] Step S1: Construction of a multi-agent trunk intersection system; Each intersection on the trunk line is considered as an agent, and each agent has independent decision-making and information processing capabilities; Agents exchange information through a communication network to achieve collaborative control, as detailed below:

[0117] (11) Construction of a single intersection intelligent agent: Each intersection is regarded as an independent intelligent agent with independent decision-making and information processing capabilities. It is responsible for monitoring and managing the traffic conditions of its intersection, including vehicle arrival and departure, lane occupancy, traffic light status, etc. It can make appropriate control decisions based on its own status and the surrounding environment, and exchange information with other intelligent agents to achieve collaborative control between trunk intersections.

[0118] (12) Intelligent agent software structure configuration: including perception module, decision-making module, execution module and communication module; the perception module is responsible for collecting traffic data; the decision-making module makes decisions based on the proposed algorithm; the execution module is responsible for controlling traffic facilities such as traffic lights; the communication module is responsible for exchanging information with other intelligent agents;

[0119] (13) Establishment of cooperative control mechanism: The goal of cooperative control is to enable all intersections on the trunk line to achieve the optimal traffic state on the basis of robustness; the proposed deep learning algorithm is used to train the agent so that it can select the appropriate cooperative control strategy according to the current traffic state;

[0120] (14) System scalability configuration: The system adopts a modular configuration, with each agent being an independent module, which has good scalability and can be easily added, deleted or replaced as the city develops;

[0121] Step S2, Robust Configuration of Arterial Intersections: Considering the uncertainties of arterial traffic flow, such as fluctuations in vehicle arrival rates and changes in road conditions, a robust optimization strategy is introduced. By proposing modeling of uncertain fluctuations in arterial traffic flow, setting robustness targets for coordinated signal control at arterial intersections, and establishing robustness constraints, the robustness of the control system is improved, as detailed below:

[0122] (21) Modeling of uncertain fluctuations in trunk traffic flow; To effectively address the uncertain fluctuations in trunk traffic flow, the influence of real-time environmental factors such as vehicle arrival rate, road conditions, and weather conditions is considered; a robust modeling strategy based on deep reinforcement learning is proposed, and a negative binomial moving average randomization algorithm is introduced. The robustness of the system is enhanced by dynamically adjusting the model parameters in conjunction with real-time environmental perception information; the specific traffic flow modeling formula is as follows:

[0123]

[0124] In the formula: Q m It represents the maximum traffic flow on the main line, k represents the number of vehicles arriving per unit time, and p... k β is the probability of k vehicles arriving per unit time, and β is the maximum traffic flow Q on the main road. m The difference between the number of vehicles arriving per unit time and the number of vehicles arriving per unit time, δ BThe vehicle arrival fluctuation parameter has a value range of [0,2], δ W This is a weather condition parameter, with a value range of [0,1], δ C The parameter represents the road condition variation, with values ​​ranging from [0,1], δ Env The environmental perception information parameter represents the impact of real-time environmental changes, including road conditions and accidents, and its value ranges from [0,1].

[0125] (22) Robustness Objective Setting: Minimizing vehicle delay is set as the primary robustness objective, and minimizing traffic congestion is set as the secondary objective. A robustness objective function S is proposed. R This is to ensure that the performance of the system control strategy remains within an acceptable range even when uncertainties exist.

[0126]

[0127] In the formula: Q represents the uncertain and fluctuating traffic flow. m R is the maximum traffic flow on the main road, R is the red light duration at the intersection, a is the robustness parameter, and ε is the maximum traffic flow on the main road. R,d It is a robust parameter for vehicle delays, with a value range of [0, a], ε R,j It is a robust parameter for traffic congestion, with a value range of [0, a], δ B V is the vehicle arrival fluctuation parameter. Q The average speed on the trunk line under uncertain and fluctuating traffic flow;

[0128] (23) Robustness constraints: To limit the system's behavior under uncertainties and ensure the system's stability and security, a red light time R is set at... Fluctuations are implemented to avoid traffic chaos caused by frequent and large-scale switching of traffic light cycles;

[0129] Step S3: Robust multi-agent model configuration; A robust mainline-intelligent agent-actor-critic (RMIAC) network framework is built, and a robust joint traffic state space definition, action control parameter definition, and robust reward function definition are proposed, as follows:

[0130] (31) Robust Mainline-Intelligent Agent-Actor-Critic (RMIAC) network framework construction; The robust Mainline-Intelligent Agent-Actor-Critic (RMIAC) network framework is constructed, including the definition of robust traffic joint traffic state space, the definition of action control parameters and the definition of robust reward function; During training, based on the proposed algorithm, the agent is trained in a "centralized learning, distributed execution" manner; During testing, the intersection agent outputs the optimal control action through its independent Actor network to ensure rapid response;

[0131] (32) Robust Joint Traffic State Space Definition: The ratio of the maximum queue length to the number of vehicles in each phase lane group during the previous signal cycle at the intersection is used as the normalized queue feature, and the arrival flow rate of each phase lane group per unit time during the previous signal cycle at the intersection is used as the normalized flow rate feature. The state features of the local intersection agent m and the adjacent intersection agent n are considered simultaneously, and the joint feature is used as the state representation of the local agent. Considering robustness, a joint traffic state space is proposed.

[0132] In the formula, This represents the maximum queue length of the i-th phase of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control cycle. This represents the arrival flow rate per unit time of the i-th phase of agent m in the previous control cycle; This represents the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control cycle. The ε represents the arrival flow rate per unit time of the i-th phase of the neighboring agent n of agent m during the previous control cycle; a is a robustness parameter, ε R,d It is a robust parameter for vehicle delays, with a value range of [0, a], ε R,j It is a robust parameter for traffic congestion, with a value range of [0, a].

[0133] (33) Definition of robust action control parameters: The maximum cycle time of the key intersection is selected as the cycle time of the trunk system. The local agent selects the green light ratio of each phase of the local intersection based on the input joint traffic state vector. Considering robustness, robust action control parameters are proposed.

[0134]

[0135] In the formula, This represents the ratio of the green light duration to the cycle duration in the i-th phase of agent m; This represents the maximum queue length of the i-th phase of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control cycle. denoted by , representing the arrival flow rate per unit time of the i-th phase of agent m within the previous control cycle; 'a' is a robustness parameter.

[0136] (34) Definition of Robust Reward Function: Considering the reward value of agent m at the local intersection and the reward value of agent n at the adjacent intersection, a robust reward function is proposed to address robustness.

[0137]

[0138] In the formula, This represents the arrival flow rate of agent m in the i-th phase during the previous control cycle. This represents the average vehicle delay of agent m in the i-th phase during the previous control cycle. This represents the arrival flow rate of agent n in the i-th phase during the previous control cycle. This represents the average vehicle delay of agent n in the i-th phase during the previous control cycle; This represents the maximum queue length of the i-th phase of agent m during the previous control cycle. This represents the number of vehicles corresponding to the maximum queue length in the i-th phase of agent m during the previous control cycle. This represents the maximum queue length of the i-th phase of the neighboring agent n of agent m during the previous control cycle. The number of vehicles corresponding to the maximum queue length of the i-th phase of the neighboring agent n of agent m in the previous control cycle; a is the robustness parameter; S R It is a robust objective function;

[0139] Step S4: Configuration of collaborative control strategy for arterial intersections; Deep reinforcement learning algorithms are used to train the agents, and a robust multi-agent deep deterministic policy gradient algorithm (CR-MADDPG) is proposed. This algorithm enables the agents to select appropriate control strategies based on the current traffic conditions, allowing each agent to maximize its own interests while considering the impact on the overall traffic flow of the arterial road. Through information exchange and cooperation among the agents, collaborative control of the arterial intersections is achieved, as detailed below:

[0140] (41) A policy update formula considering robustness is proposed:

[0141] To improve the decision-making ability of intelligent agents in complex traffic environments, a robust policy update formula is proposed. This formula, based on the traditional policy gradient algorithm, incorporates a robustness factor and adversarial noise, enhancing the adaptability and stability of the policy in uncertain and fluctuating traffic flows. The specific formula is as follows:

[0142]

[0143] Where: μ i (s i |θ i ) is the state s of agent i. i The strategy is as follows; α is the learning rate; a is the robustness parameter; It concerns the policy network parameters θ i gradient; ∈ i It is adversarial noise, used to combat external uncertainties and environmental disturbances, and to ensure the robustness of strategy updates;

[0144] (42) Propose an agent objective function that considers robustness:

[0145] To effectively guide agents in making decisions in complex and dynamic traffic environments, a robust agent objective function is proposed. This objective function comprehensively considers the agent's long-term reward, traffic flow stability, and robustness, aiming to maintain superior performance even when facing uncertainties such as traffic flow fluctuations and equipment failures. The specific formula is as follows:

[0146]

[0147] In the formula: J i (θ i ) is the objective function of agent i; For trajectory τ i The expected value; T is the time termination step; γ is the discount factor; Is Take action in a state The reward; S R is the robustness objective function; 'a' is the robustness parameter; this objective function improves the system's stability and anti-interference ability by optimizing long-term rewards and considering robustness.

[0148] (43) Propose a loss function that considers robustness:

[0149] To further improve the learning efficiency of multi-agent systems in non-ideal environments, a robust loss function is proposed. This loss function, in addition to considering the standard Q-learning error, incorporates a robust adjustment term, comprehensively considering the different challenges faced by agents in cooperation and competition, ensuring the system can operate stably in uncertain environments. The specific loss function is as follows:

[0150]

[0151] in:

[0152] In the formula: L i (φ i ) is the loss function of agent i; It is an experience replay pool; Q i (s,a∣φ i ) is the value function, where φ i These are the network parameters of the value function; y i It is the target value; These are the parameters of the target network; γ is the discount factor; S j ,a j It is a small batch of experience data extracted from the experience pool; 'a' is the robustness parameter; this loss function improves the robustness and stability of the system in complex and dynamic traffic environments by comprehensively considering the cooperation and competition of multiple agents.

[0153] Step S5: Online Learning and Adaptive Adjustment; During actual operation, the agent can learn online based on real-time traffic data and continuously optimize the control strategy; at the same time, the agent can adaptively adjust the control parameters according to changes in traffic flow, improving the adaptability and robustness of the control system, as detailed below:

[0154] (51) Initialize based on the robust multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4:

[0155] Initialize the policy network parameters θ for all agents. i and value function network parameters φ i and target network parameters Set to φ i The same; this initialization step lays the foundation for the system's learning process and ensures that the agent can adaptively learn in dynamic environments; the specific steps are as follows:

[0156] θ i ,φ i initialization;

[0157] (52) Based on the robustness-considered multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4, empirical data was collected:

[0158] At each time step t, each agent, according to the current policy Choose an action The action selection takes into account not only the current state. Robustness parameters were also added to ensure that the agent could cope with fluctuations in traffic flow; the experience each agent gained... Stored in the experience replay pool This provides data support for subsequent learning and parameter updates; the specific steps are as follows:

[0159] Experience collection: Store in experience pool

[0160] (53) Based on the robust multi-agent deep policy determination gradient algorithm (CR-MADDPG) proposed in step S4, from the experience pool Randomly select a small batch of samples to update network parameters:

[0161] From the experience replay pool Randomly sample small batches of empirical data and update the agent's value function network parameters φ based on this experience. i Policy network parameters θ i and target network parameters During the update process, in addition to traditional policy updates and value function optimization, a robust objective function S is also introduced. R As a pre-tuning term, this ensures that the agent can maintain high-quality decision-making when facing complex dynamic traffic flows; this step continues until the policy network converges, ensuring that the system can adapt to various traffic changes and optimize signal control; the specific steps are as follows:

[0162] Small batch sampling: Update network parameters φ_i, θ_i, φ_i^.

[0163] In summary, this invention is based on the distributed system construction of modular intelligent agents. Through independent decision-making and information interaction mechanisms, it improves global coordination capabilities while ensuring rapid response at each intersection. It innovates robust configuration and effectively addresses uncertainties such as fluctuations in vehicle arrival rates, weather and road conditions through dynamic traffic flow modeling, parameter adaptive objective functions, and red light time constraints.

[0164] Furthermore, the robust multi-agent model configured in this invention integrates traffic state data from adjacent intersections with a multi-level feedback mechanism, and fuses local and environmental perception information in the joint traffic state space definition to achieve optimal balance of global traffic flow. It adopts a deep reinforcement learning algorithm, introduces robustness parameters and adversarial noise, and improves decision robustness in complex environments by enhancing the stability of policy updates and the anti-interference ability of the loss function.

[0165] In addition, the online learning mechanism of this invention supports real-time data acquisition and network parameter updates, enabling the control strategy to continuously adapt to the dynamically changing traffic environment. While improving the efficiency of trunk line traffic, it significantly enhances the system's adaptability to sudden events and environmental disturbances.

[0166] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for multi-agent arterial intersection signal coordination control considering robustness, characterized in that, Comprising the following steps: Step S1, multi-agent arterial intersection system construction: each intersection on the traffic artery is regarded as an agent, which obtains real-time traffic data in real time, and performs data cleaning and feature extraction to provide input for subsequent uncertainty fluctuation modeling; Step S2, robustness configuration of arterial intersection: uncertainty fluctuation modeling of arterial traffic flow, robustness target setting and robustness constraint condition construction, specifically including: Step S21, trunk traffic flow uncertainty fluctuation modeling: considering the influence of vehicle arrival rate, road condition, weather condition and real-time environmental factors, a robust modeling strategy based on deep reinforcement learning is constructed, and a negative binomial moving average randomization algorithm is introduced , and the trunk traffic flow uncertainty fluctuation modeling is performed through the following formula: ， wherein, is the maximum traffic flow of the trunk road; is the number of vehicles arriving per unit time; is the probability of a vehicle arriving per unit time; is the difference between the maximum traffic flow of the trunk road and the number of vehicles arriving per unit time; is the vehicle arrival fluctuation parameter; is the weather condition parameter; is the road condition change parameter; is the environmental perception information parameter;​​ Step S22, robustness target setting, setting the minimization of vehicle delay and traffic congestion degree as the robustness target, dynamically adjusting the parameters of the objective function according to real-time perception data, and the robustness objective function is: ， wherein, is an uncertain fluctuating traffic flow, is a maximum arterial traffic flow, is a red light time at the intersection, is a robustness parameter, is a vehicle delay robustness parameter, is a traffic congestion degree robustness parameter, is an arterial average speed under the uncertain fluctuating traffic flow; Step S23, Robustness constraint, set red light time In fluctuate within a range; Step S3, robust multi-agent model configuration: through the traffic flow uncertainty fluctuation model, robustness target setting and robustness constraint condition in step S2, a robust arterial multi-agent network framework is built, the robust joint traffic state space, action control parameter and robust reward function are defined, and specifically including: Step S31, building a robust arterial multi-agent network framework, updating the strategy network parameters of the agent in a centralized learning manner, and outputting control actions in a distributed execution manner; Step S32, robust joint traffic state space definition, the robust joint traffic state space is defined as: ， in, Represents intelligent agents No. The maximum queue length of the phase in the previous control cycle; Indicates the first intelligent agent Phase in the previous control cycle The number of vehicles corresponding to the maximum queue length; Represents intelligent agents No. Phase in the previous control cycle The rate of arrival per unit time within the time limit For road capacity; Represents intelligent agents Neighboring agents No. Phase in the previous control cycle Maximum queue length within; Represents intelligent agents Neighboring agents No. Phase in the previous control cycle The number of vehicles corresponding to the maximum queue length; Represents intelligent agents Neighboring agents No. The phase arrival rate per unit time in the previous control cycle; It is a robustness parameter; These are robust parameters for vehicle delays; It is a robust parameter for traffic congestion levels; Step S33, robust action control parameter definition, the robust action control parameter is defined as: ， in, Represents intelligent agents The The ratio of the green light duration to the cycle duration for each phase. It is a robustness parameter; Step S34, a robust reward function is defined, said robust reward function is defined as: ， in, Represents intelligent agents No. Phase in the previous control cycle Incoming traffic within; Represents intelligent agents No. The average vehicle delay during the previous control cycle; Represents intelligent agents No. Phase in the previous control cycle Incoming traffic within; Represents intelligent agents No. Phase in the previous control cycle Average vehicle delay within the area; It is a robust objective function; Step S4, configuration of coordinated control strategy of arterial intersection: taking the robust joint traffic state space and action control parameter as the input of the deep reinforcement learning algorithm to design the observation space and action selection strategy of the agent, using the deep reinforcement learning algorithm to train the agent in a multi-agent collaborative manner, constructing a multi-agent deep deterministic policy gradient algorithm CR-MADDPG considering robustness, and embedding the robust reward function to drive the agent to learn a robust strategy and achieve global robustness optimization; Step S5, online learning and adaptive adjustment: taking the CR-MADDPG algorithm in step S4 as the initial strategy for online learning, collecting experience according to real-time traffic data, and randomly extracting a batch of samples to update the network parameters of the agent for online learning and strategy fine-tuning. 2.The multi-agent arterial intersection signal cooperative control method with robustness consideration according to claim 1, wherein, The multi-agent arterial intersection system construction in step S1 specifically includes: Step S11, single intersection agent construction: each intersection is regarded as an independent agent, and each agent has independent decision-making ability and information processing ability; Step S12, configuration of agent software structure: the agent includes a perception module, a decision module, an execution module and a communication module; the perception module is configured to collect traffic data, the decision module is configured to generate control instructions based on the multi-agent deep deterministic policy gradient algorithm considering robustness, the execution module is configured to adjust the signal light timing, and the communication module is configured to transmit adjacent intersection state information; Step S13, establishment of coordinated control mechanism: the agent is trained by using a deep learning algorithm, so that the agent can select a suitable coordinated control strategy according to the current traffic state; Step S14, system scalability configuration: each agent is configured as an independent module for expansion. 3.The method of claim 1, wherein, The multi-agent deep deterministic policy gradient algorithm considering robustness constructed in step S4 specifically includes: Step S41, construct a strategy updating formula considering robustness: ， wherein, is an agent a policy in a state ; is a learning rate; is a robustness parameter, is a gradient with respect to policy network parameters ; is adversarial noise; Step S42, construct an agent objective function considering robustness: ， wherein, is the objective function of the agent ; is the expectation over trajectories ; is the time terminal step, is the discount factor; is the reward for taking action in state is the robustness objective function; Step S43, construct a loss function considering robustness: ， wherein, is the loss function of the agent ; is the experience replay pool, is the value function; is the value function network parameter; is the target value; is the parameter of the target network; is the discount factor; and is the batch of experience data extracted from the experience replay pool.

4. The method of claim 1, wherein, The online learning and adaptive adjustment in the step S5 specifically includes: Step S51, initialize the policy network parameters of all agents and the value function network parameters , and set the target network parameters to be the same as ; Step S52, each agent updates its policy based on the current policy selects an action , and experiences a reward , and stores the experience into an experience replay pool . Step S53, randomly sampling mini-batch of experience data from the experience replay pool and updating the value function network parameters of the agent based on the experience , the policy network parameters and the target network parameters until the policy network converges.

Images