Expressway multi-ramp cooperative traffic control method and system

Abstract

The invention discloses an expressway multi-ramp cooperative traffic control method and system in the technical field of intelligent traffic control, and the method comprises the steps: carrying out the cooperative control of variable speed limit and ramp metering based on a multi-agent reinforcement learning framework, collecting the average speed, lane occupancy rate, ramp queuing length and upstream and downstream traffic flow data of a confluence region, and carrying out the cooperative control of the variable speed limit and ramp metering. And constructing a multi-dimensional state vector. According to the method, through demonstration-guidance training and multi-agent cooperative control, the traffic control performance in a multi-ramp scene is effectively improved, the method has high practicability and popularization value, and in practical application, the method can dynamically adjust ramp control and speed limiting strategies according to real-time traffic flow data, and traffic congestion and queuing length are remarkably reduced. Through collaborative decision-making among multiple agents, quick response and accurate control of a complex traffic environment are realized, and the scheme also has relatively good robustness and can adapt to control requirements under different traffic conditions.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic control technology, specifically a method and system for coordinated traffic management of multiple ramps on highways. Background Technology

[0002] With the continuous growth of highway traffic flow, merging areas with multiple entrance ramps have become high-incidence areas for traffic congestion and accidents due to frequent vehicle weaving. Traditional single-point control strategies (such as independent speed limit control or ramp metering) are difficult to cope with complex traffic scenarios involving dynamic coupling of multiple areas. Although existing reinforcement learning-based control methods have a certain degree of adaptability, they still have the following problems: insufficient multi-agent cooperation, failing to effectively integrate speed limit and metering control; unstable training process, making policy convergence difficult in non-stationary traffic environments; and a single state representation, failing to fully utilize micro and macro traffic information. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-ramp cooperative traffic management method and system for highways, in order to solve the problems mentioned above, such as insufficient multi-agent cooperation, failure to effectively integrate speed limit and metering control, unstable training process, difficulty in strategy convergence in non-stationary traffic environments, and single state representation, which fails to make full use of micro and macro traffic information.

[0004] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for coordinated traffic management of multiple ramps on highways, comprising: Based on a multi-agent reinforcement learning framework, the variable speed limit and ramp metering are controlled collaboratively. Data on average speed in merging areas, lane occupancy rate, ramp queue length and upstream and downstream traffic flow are collected to construct a multi-dimensional state vector. A teacher model is established, based on the demonstration-guided training model, and traditional control algorithms are used as the teacher model to guide the agent's policy optimization; By integrating microscopic vehicle data with macroscopic traffic conditions, a feature representation method is used to deploy the trained teacher model to the actual traffic control system, generating speed limit and ramp metering instructions in real time based on multidimensional state vectors.

[0005] As a further aspect of the present invention: the multi-agent framework includes a speed limit controller and a ramp metering controller, and adopts a "centralized training-distributed execution" architecture to achieve multi-regional collaborative management and control.

[0006] As a further aspect of the present invention: the demonstration-guided training uses Wasserstein distance to measure the difference between the student agent and the teacher's strategy, and dynamically adjusts the guidance intensity through a time-varying decay factor.

[0007] As a further aspect of the present invention, the feature characterization method includes the fusion of multi-dimensional information such as average speed in the merging zone, lane occupancy rate, ramp queue length, and upstream and downstream traffic flow.

[0008] In a second aspect, the present invention provides a multi-ramp collaborative traffic control system for highways, the system comprising: The traffic condition perception module is used to collect multi-source traffic data; The demonstrator module integrates the PI-ALINEA ramp control algorithm and the SPSC speed limit control algorithm; A multi-agent reinforcement learning controller, based on the DG-MATD3 algorithm, achieves collaborative decision-making; The instruction execution module is used to issue speed limit and traffic light control instructions.

[0009] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention effectively improves traffic management performance in multi-ramp scenarios through demonstration-guided training and multi-agent collaborative control, demonstrating high practicality and promotional value. In practical applications, this method can dynamically adjust ramp control and speed limit strategies based on real-time traffic flow data, significantly reducing traffic congestion and queue length. Through collaborative decision-making among multiple agents, it achieves rapid response and precise control in complex traffic environments. Furthermore, this scheme exhibits good robustness and can adapt to control requirements under different traffic conditions. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the demonstration-guided training process of the present invention; Figure 3 This is a comparison diagram of the collaborative control effects of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] Example: Please see Figures 1-3 In this embodiment of the invention, a method for coordinated traffic management of multiple ramps on a highway includes: S1: Based on a multi-agent reinforcement learning framework, it coordinates the control of variable speed limits and ramp metering, collects average speed in merging areas, lane occupancy rates, ramp queue lengths and upstream and downstream traffic flow data, and constructs a multi-dimensional state vector. S2: Establish a teacher model based on the demonstration-guided training model, and use traditional control algorithms as the teacher model to guide the agent's policy optimization; The PI-ALINEA algorithm is used as the teacher model for ramp control, such as... Figure 2 As shown, the SPSC algorithm, used as a speed-limiting control teacher model, is trained based on the DG-MATD3 algorithm. It utilizes the teacher model to provide demonstration actions and dynamically adjusts the guidance intensity by measuring policy differences using the Wasserstein distance. The teacher model parameters are set as follows: Control period: T = 60s; Ramp red light duration range: = [0, 50]s; Teacher-student strategy difference threshold: = 10; Termination of guidance round: = 50 S3: A feature representation method that integrates microscopic vehicle data and macroscopic traffic conditions is used to deploy the trained teacher model to the actual traffic control system and generate speed limit and ramp metering instructions in real time based on multidimensional state vectors.

[0013] Construction of a multi-agent collaborative control framework: A speed limit controller and a ramp metering controller are set up, and a "centralized training-distributed execution" architecture is adopted to achieve multi-region collaborative optimization through global information sharing; Demonstration-guided training mechanism: Traditional control algorithms such as PI-ALINEA and SPSC are introduced as teacher models. The policy difference is measured by Wasserstein distance, and a time-varying decay factor is designed to dynamically adjust the guidance intensity, thereby improving training stability and policy performance. In the demonstration-guided training mechanism, the teacher model provides demonstration actions, and the student agent performs training through minimal... The guidance is derived from the strategy difference (Wasserstein distance), and the guidance strength dynamically decays with each training epoch, specifically due to a time-varying decay factor. control: in, For the current training round, To terminate the guidance round, and To adjust the parameters.

[0014] Multi-source traffic state representation: Integrating multi-dimensional features such as average speed in merging zones, lane occupancy rate, ramp queue length, and upstream and downstream traffic flow, a state input that comprehensively reflects traffic dynamics is constructed. Reward Function Design: Combining traffic efficiency and safety indicators, a multi-objective reward function is designed to guide the agent to achieve collaborative optimization and control. The collaborative control effects are compared, for example... Figure 3 As shown; This invention effectively improves traffic management performance in multi-ramp scenarios through demonstration-guided training and multi-agent collaborative control, demonstrating high practicality and promotional value. In practical applications, this method can dynamically adjust ramp control and speed limit strategies based on real-time traffic flow data, significantly reducing traffic congestion and queue length. Through collaborative decision-making among multiple agents, it achieves rapid response and precise control in complex traffic environments. Furthermore, this scheme exhibits good robustness and can adapt to control requirements under different traffic conditions.

[0015] Preferably, the multi-agent framework includes a speed limit controller and a ramp metering controller, and adopts a "centralized training-distributed execution" architecture to achieve multi-region collaborative management and control.

[0016] Preferably, the demonstration-guided training uses the Wasserstein distance metric to measure the difference between the student agent and the teacher's strategy, and dynamically adjusts the guidance intensity through a time-varying decay factor.

[0017] Preferably, the feature characterization method includes the fusion of multi-dimensional information such as average speed in the merging zone, lane occupancy rate, ramp queue length, and upstream and downstream traffic flow.

[0018] A multi-ramp collaborative traffic control system for highways, the system comprising: The traffic condition perception module is used to collect multi-source traffic data; The demonstrator module integrates the PI-ALINEA ramp control algorithm and the SPSC speed limit control algorithm; A multi-agent reinforcement learning controller, based on the DG-MATD3 algorithm, achieves collaborative decision-making; The instruction execution module is used to issue speed limit and traffic light control instructions.

[0019] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for coordinated traffic management of multiple ramps on highways, characterized in that, include: Based on a multi-agent reinforcement learning framework, the variable speed limit and ramp metering are controlled collaboratively. Data on average speed in merging areas, lane occupancy rate, ramp queue length and upstream and downstream traffic flow are collected to construct a multi-dimensional state vector. A teacher model is established, based on the demonstration-guided training model, and traditional control algorithms are used as the teacher model to guide the agent's policy optimization. By integrating microscopic vehicle data with macroscopic traffic conditions, a feature representation method is used to deploy the trained teacher model to the actual traffic control system, generating speed limit and ramp metering instructions in real time based on multidimensional state vectors.

2. The multi-ramp coordinated traffic control method for highways according to claim 1, characterized in that: The multi-agent framework includes a speed limit controller and a ramp metering controller, and adopts a "centralized training-distributed execution" architecture to achieve multi-region collaborative management and control.

3. The multi-ramp coordinated traffic control method for highways according to claim 2, characterized in that: The demonstration-guided training uses the Wasserstein distance metric to measure the difference between the student agent and the teacher's strategy, and dynamically adjusts the guidance intensity through a time-varying decay factor.

4. The multi-ramp coordinated traffic control method for highways according to claim 3, characterized in that: The feature characterization method includes the fusion of multi-dimensional information such as average speed in the merging zone, lane occupancy rate, ramp queue length, and upstream and downstream traffic flow.

5. A multi-ramp cooperative traffic control system for highways, applied to the multi-ramp cooperative traffic control method for highways as described in any one of claims 1-4, characterized in that: The system includes; The traffic condition perception module is used to collect multi-source traffic data; The demonstrator module integrates the PI-ALINEA ramp control algorithm and the SPSC speed limit control algorithm; A multi-agent reinforcement learning controller, based on the DG-MATD3 algorithm, achieves collaborative decision-making; The instruction execution module is used to issue speed limit and traffic light control instructions.

Images