A multi-intersection bus priority control method and device, a terminal and a storage medium

Abstract

This invention provides a method, device, terminal, and storage medium for bus priority control at multiple intersections. The method includes: acquiring the approach lane status of each approach lane at each intersection within a target area at the current time, and the traffic light prediction model for each intersection; determining the target bus closest to its current intersection on each approach lane, and acquiring the bus status and bus prediction model for the target bus at the current time; predicting the traffic light phase status of each intersection at the next time based on the approach lane status, bus status, and traffic light prediction model; and predicting the bus acceleration corresponding to each target bus at the next time based on the approach lane status, bus status, and bus prediction model. This application can adjust the traffic light phase status and bus acceleration using traffic light prediction models and bus prediction models in a multi-intersection environment, thereby improving bus traffic efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method, device, terminal, and storage medium for bus priority control at multiple intersections. Background Technology

[0002] Traffic signal control, as a crucial means of urban traffic management, is vital for regulating traffic flow and optimizing road resources. Public transport priority signal control is an important branch of traffic signal control, aiming to improve the operational efficiency and punctuality of public transport vehicles, thereby enhancing the reliability and attractiveness of the public transport system. With the continuous emergence and application of new technologies such as autonomous driving, vehicle-to-everything (V2X) communication, and data analytics, more possibilities have been provided for solving urban traffic problems. These new technologies offer new tools and methods for optimizing traffic management and public transport systems. The development of autonomous driving technology enables vehicles to more accurately follow traffic rules and control commands, while V2X communication technology enables real-time communication between vehicles and between vehicles and infrastructure. How to effectively utilize complex traffic data to formulate effective signal control strategies has become a key research focus in the field of traffic management.

[0003] Bus signal priority control and bus speed guidance are two important means of implementing bus priority control. Bus signal priority control adjusts the signal scheme at intersections by acquiring the operating status of buses and combining it with the traffic signal timing system, allowing buses to pass through intersections with priority. Bus speed guidance dynamically adjusts the speed of buses to match the phase of the traffic lights at the intersection ahead, thereby reducing waiting time and improving traffic efficiency.

[0004] However, most traffic signal control methods focus on optimizing a single signal control or vehicle speed, without paying attention to the synergistic optimization between the two. This limitation makes it difficult to fully utilize real-time traffic information for comprehensive optimization in practical applications, thereby restricting the improvement of overall traffic efficiency and public transport service quality.

[0005] Therefore, existing technologies have shortcomings and need to be improved and developed. Summary of the Invention

[0006] This application provides a method, device, terminal, and storage medium for bus priority control at multiple intersections, in order to solve the technical problem in related technologies where the focus on single signal control optimization or vehicle speed optimization when controlling traffic lights limits the overall traffic efficiency.

[0007] To achieve the above objectives, this application adopts the following technical solution: The first aspect of this application provides a method for bus priority control at multiple intersections, including: Obtain the current state of each approach lane at each intersection within the target area, as well as the trained traffic light prediction model for each intersection. Identify the target bus closest to the intersection in each approach lane at the current moment, and obtain the bus status and trained bus prediction model corresponding to the target bus at the current moment; Based on the aforementioned approach lane status, bus status, and traffic light prediction model, the traffic light phase status of each intersection at the next moment is predicted, and based on the aforementioned approach lane status, bus status, and bus prediction model, the bus acceleration corresponding to each target bus at the next moment is predicted.

[0008] In one embodiment, the approach lane status includes: the current traffic light status corresponding to each intersection, the vehicle position in the approach lane, the current speed and current acceleration of the target bus in the approach lane that is closest to the intersection, the current distance between the target bus and the intersection, and the approach lane number. The bus status includes: the speed of the vehicle in front of the target bus, the speed of the vehicle behind the target bus, the acceleration of the vehicle in front, the acceleration of the vehicle behind, the first distance between the target bus and the vehicle in front, the second distance between the target bus and the vehicle behind, the current distance between the target bus and the intersection, and the current traffic light status.

[0009] In one embodiment, the training steps for the traffic light prediction model and the bus prediction model include: In a pre-built simulation platform, a multi-intersection traffic environment simulation scenario corresponding to the target area is constructed, and the state space, hybrid action space, and reward function corresponding to the traffic lights and buses are set. The simulation is performed in the multi-intersection traffic environment simulation scenario based on the state space, hybrid action space and reward function corresponding to the traffic lights and buses. Construct initial traffic light prediction models and initial bus prediction models; Acquire the first training data corresponding to each of the traffic light prediction models and the second training data corresponding to each of the bus prediction models obtained during the simulation process; The traffic light prediction model is trained based on the first training data to obtain a trained traffic light prediction model, and the bus prediction model is trained based on the second training data to obtain a trained bus prediction model.

[0010] In one embodiment, acquiring the first training data corresponding to each of the traffic light prediction models and the second training data corresponding to each of the bus prediction models obtained during the simulation process includes: Obtain the set of intersections and the set of approach lanes in the target area of ​​the multi-intersection traffic environment simulation scenario. At each time step in the simulation process, determine the number of buses closest to the intersection in each approach lane of the approach lane set, and form a bus set based on the number of buses. The current global state corresponding to the current time step is obtained based on the approach lane status of all intersections in the intersection set and the bus status of all buses in the bus set. The current global state is used as the first training data, and the bus state of each bus is used as the second training data for the corresponding bus prediction model.

[0011] In one embodiment, the reward function of the traffic light is obtained by normalizing a preset vehicle delay reward function and a vehicle throughput reward function; the reward function of the bus is obtained based on the deviation of the bus's speed from a predetermined speed threshold and the deviation of the bus's acceleration from a predetermined acceleration threshold.

[0012] In one embodiment, the traffic light prediction model is trained based on the first training data to obtain a trained traffic light prediction model, and the bus prediction model is trained based on the second training data to obtain a trained bus prediction model, including: Initialize the first policy network parameters and the first value network parameters of the traffic light prediction model, and initialize the second policy network parameters and the second value network parameters of the bus prediction model; At each time step, the traffic light prediction model selects a phase action based on the current global state through the first strategy network parameters, and the bus prediction model selects an acceleration action based on its own bus state through the second strategy network parameters. After executing phase action and acceleration action, the multi-intersection traffic environment simulation scenario returns the reward and the next global state. The current global state, phase action, acceleration action, reward returned by the multi-intersection traffic environment simulation scenario, and the next global state are stored as trajectory data in the experience pool. The PPO pruning method is used to calculate the preset objective function to update the parameters of the first policy network and the second policy network. The preset mean squared error loss function is used to update the parameters of the first value network and the second value network, so as to obtain the trained traffic light prediction model and bus prediction model. In the training process of the traffic light prediction model and the bus prediction model, a generalized dominance estimation algorithm is used to calculate a preset dominance function.

[0013] In one embodiment, the method further includes: When the experience pool is updated, batch data in the experience pool is randomly sampled, and a priority experience replay strategy is adopted to increase the sampling weight of data in the experience pool whose reward value is greater than a preset reward value or whose time series difference error is greater than a preset error to a preset weight.

[0014] A second aspect of this application provides a multi-intersection bus priority control device, comprising: The acquisition module is used to acquire the status of each approach lane of each intersection in the target area at the current time, as well as the trained traffic light prediction model for each intersection. The determination module is used to determine the target bus closest to the intersection in each approach lane at the current time, and to obtain the bus status and trained bus prediction model corresponding to the target bus at the current time. The prediction module is used to predict the phase state of the traffic lights at each intersection at the next moment based on the state of the approach lane, the bus state, and the traffic light prediction model, and to predict the bus acceleration corresponding to each target bus at the next moment based on the state of the approach lane, the bus state, and the bus prediction model.

[0015] A third aspect of this application provides a terminal, the terminal including a memory, a processor, and a multi-intersection bus priority control program stored in the memory and executable on the processor. When the processor executes the multi-intersection bus priority control program, it implements the steps of the multi-intersection bus priority control method as described above.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a multi-intersection bus priority control program. When executed by a processor, the multi-intersection bus priority control program implements the steps of the multi-intersection bus priority control method as described above.

[0017] The beneficial effects of this invention are as follows: This embodiment of the invention obtains the state of each approach lane at each intersection within the target area at the current moment, as well as the trained traffic light prediction model for each intersection; determines the target bus closest to its current intersection on each approach lane, and obtains the bus state and trained bus prediction model corresponding to the target bus at the current moment; based on the approach lane state, bus state, and traffic light prediction model, it predicts the traffic light phase state of each intersection at the next moment, and based on the approach lane state, bus state, and bus prediction model, it predicts the bus acceleration corresponding to each target bus at the next moment. This application can adjust the traffic light phase state and bus acceleration using traffic light prediction model and bus prediction model in a multi-intersection environment, improving bus traffic efficiency. Attached Figure Description

[0018] Figure 1 This is a flowchart of a preferred embodiment of the multi-intersection bus priority control method of the present invention.

[0019] Figure 2 This is the overall framework of the multi-intersection bus priority control model in this invention.

[0020] Figure 3 This is the simulation platform architecture in this invention.

[0021] Figure 4 This is the phase scheme for the signal lights in this invention.

[0022] Figure 5 This is the training process of the MAPPO algorithm in this invention.

[0023] Figure 6 This invention compares evaluation metrics under low flow rates.

[0024] Figure 7 This is a comparison of evaluation indicators under medium flow levels in this invention.

[0025] Figure 8 This invention compares the evaluation indicators under high flow rates.

[0026] Figure 9 This invention compares the evaluation indicators of public transport vehicles and private vehicles under different overall traffic flow levels at multiple intersections.

[0027] Figure 10 This invention compares the overall evaluation indicators under different overall traffic flow levels at multiple intersections.

[0028] Figure 11 This invention compares the evaluation indicators of public buses and private vehicles under different public transport flow levels at multiple intersections.

[0029] Figure 12 This invention compares the overall evaluation indicators under different bus traffic flow levels at multiple intersections.

[0030] Figure 13 This invention compares the average waiting time for training rounds of different methods at multiple intersections.

[0031] Figure 14 This invention compares the average training round loss time of different methods for multiple intersections.

[0032] Figure 15 This invention compares the average passage time of different methods for multiple intersections during training rounds.

[0033] Figure 16This invention compares the evaluation indicators of public buses and private vehicles at multiple intersections.

[0034] Figure 17 This invention compares the overall evaluation indicators for multiple intersections.

[0035] Figure 18 This is a functional principle block diagram of a preferred embodiment of the multi-intersection bus priority control device of the present invention.

[0036] Figure 19 This is a functional principle block diagram of a preferred embodiment of the terminal in this invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0038] The current system for bus priority control at signalized intersections has the following shortcomings: First, most signalized intersection bus priority control methods focus on optimizing either signal control or vehicle speed, paying less attention to the synergistic optimization between the two. This limitation makes it difficult to fully utilize real-time traffic information for comprehensive optimization in practical applications, thus restricting the improvement of overall traffic efficiency and public transport service quality.

[0039] Secondly, existing bus priority control strategies have certain limitations. For example, while fixed-time bus signal priority control is simple and easy to implement, it lacks responsiveness to real-time traffic conditions and performs poorly when traffic flow fluctuates significantly. Adaptive control, although able to adjust according to traffic conditions, still faces limitations in handling complex factors such as traffic flow fluctuations and random bus arrivals, affecting its effectiveness. Deep reinforcement learning-based methods have demonstrated good adaptability and flexibility, but currently do not fully consider the relationship between signal timing and bus speed, which limits their potential for practical applications.

[0040] Third, current research mainly focuses on bus priority control at single intersections, while research on multi-intersection scenarios is relatively limited. In urban transportation networks, bus operations typically involve multiple intersections, and the optimization of a single intersection may be affected by adjacent intersections, making it difficult to maximize overall traffic efficiency. Therefore, extending bus priority control from single intersections to multiple intersections has become an important research direction.

[0041] Therefore, this invention proposes a bus priority control strategy for signalized intersections based on deep reinforcement learning in a connected bus environment. It uses the Proximal Policy Optimization (PPO) algorithm and more accurate multi-source traffic state information to design an optimization function with delay, network throughput and bus speed stability as indicators. At the same time, it optimizes the signal scheme and bus speed to improve the efficiency of bus operation while taking into account the traffic benefits of private vehicles.

[0042] Please see Figure 1 The multi-intersection bus priority control method described in this embodiment of the invention includes the following steps: Step S10: Obtain the status of each approach lane of each intersection in the target area at the current time, as well as the trained traffic light prediction model for each intersection.

[0043] Specifically, the traffic light prediction model can also be called a traffic light agent. Each multi-intersection corresponds to an independent traffic light prediction model, which is responsible for dynamically adjusting the local signal phase. The core of the traffic light prediction model is the policy network (Actor) and the value network (Critic). The policy network outputs phase control actions, and the value network evaluates the global state value to optimize the policy.

[0044] like Figure 1 As shown, the multi-intersection bus priority control method further includes the following steps: Step S20: Determine the target bus closest to the intersection on each approach lane at the current time, and obtain the bus status and trained bus prediction model corresponding to the target bus at the current time.

[0045] Specifically, the bus prediction model can also be called a bus agent. Each bus corresponds to an independent bus prediction model, which is responsible for dynamically adjusting the driving acceleration. The model also includes an Actor network (outputting acceleration actions) and a Critic network (combining global state to evaluate the value of actions), and shares global Critic network parameters with the traffic light agent to achieve collaborative optimization.

[0046] like Figure 1 As shown, the multi-intersection bus priority control method further includes the following steps: Step S30: Based on the approach lane status, bus status, and traffic light prediction model, predict the traffic light phase status of each intersection at the next moment, and based on the approach lane status, bus status, and bus prediction model, predict the bus acceleration corresponding to each target bus at the next moment.

[0047] Specifically, the traffic light prediction model and the bus prediction model in this application work collaboratively based on the MAPPO algorithm's "centralized training - distributed execution" framework, and the two models are independent of each other. Both the traffic light prediction model and the bus prediction model have their own independent policy networks (Actors) responsible for their respective local decisions, but they share a global value network (Critic). The global value network (Critic) is input to the global state (…). (Including the states of all intersections and buses), output the global state value V ( This is used to evaluate the impact of the combination of actions from two models on overall traffic efficiency.

[0048] In this embodiment of the application, the approach lane status includes: the current traffic light status corresponding to each intersection, the vehicle position in the approach lane, the current speed and current acceleration of the target bus in the approach lane that is closest to the intersection, the current distance between the target bus and the intersection, and the approach lane number; the bus status includes: the speed of the vehicle in front of the target bus, the speed of the vehicle behind the target bus, the acceleration of the vehicle in front, the acceleration of the vehicle behind, the first distance between the target bus and the vehicle in front, the second distance between the target bus and the vehicle behind, the current distance between the target bus and the intersection, and the current traffic light status.

[0049] Specifically, the input data for the traffic light prediction model includes: First, the status of all entrance lanes within the area ( ); among which, the status of a single import lane This includes: the current status of the traffic lights ( tl Vehicle position in the import lane (); veh Matrix representation, cell length 7.5m); the current speed of the nearest target bus in the approach lane to the intersection ( v ) and acceleration ( ac ), the current distance between the target bus and the intersection ( d ) and the import lane number ( lane ).

[0050] Second, the status of the nearest bus routes in each direction within the area ( ), This represents the set of all buses closest to the intersection on all approach lanes; the status of a single bus. This includes the speeds of the vehicles in front and behind the bus. , ), acceleration ( , ), the distance between the target bus and the vehicles in front and behind ( , ) and its current distance from the intersection it is located at ( ), Current traffic light status (tl).

[0051] The output data of the traffic light prediction model includes: discrete actions ( : The value can be 0 or 1. 0 means to extend the current phase time (for example, extend by 1 second, up to a maximum of 40 seconds), and 1 means to switch to the next phase (a yellow light needs to be executed for 3 seconds before switching).

[0052] The input data for the bus prediction model includes: First, the direct observation status of the bus itself ( ): This includes the speeds of the vehicles in front and behind the bus. , ), acceleration ( , ), the distance between the target bus and the vehicles in front and behind ( , ) and its current distance from the intersection it is located at ( ), Current traffic light status (tl).

[0053] Second, the condition of the approach lanes at the intersection. , This represents the set of approach lanes at intersection k, i.e., the status of all approach lanes at intersection k where the bus is currently located, including traffic flow distribution and traffic light phases for each approach lane, and is compared with the input data of the traffic light prediction model. same.

[0054] The output data of the bus prediction model is: continuous action ( ): Bus acceleration, with a range of values. It is used to dynamically adjust the bus speed to match the traffic light phase at the intersection ahead, reducing waiting time.

[0055] This application adopts a global state sharing strategy, that is, the decisions of both models depend on the state information of the other (e.g., the traffic light needs to know the position of the bus to determine whether to extend the green light, and the bus needs to know the phase of the traffic light to adjust its acceleration), rather than relying solely on its own local information.

[0056] This invention addresses the problem of bus priority control at multiple intersections. Considering the complexity, randomness, and instability of real-world traffic environments, this invention leverages the strong learning capabilities and scalability of deep reinforcement learning to propose a bus priority control method based on the MAPPO algorithm. This method combines bus signal priority control at multiple intersections with a bus speed guidance strategy, aiming to optimize bus speeds, improve bus throughput, reduce waiting times, and alleviate traffic congestion while ensuring overall traffic efficiency on arterial roads.

[0057] In this embodiment of the application, the training steps for the traffic light prediction model and the bus prediction model include: Step S100: Construct a multi-intersection traffic environment simulation scenario corresponding to the target area in the pre-built simulation platform, and set the state space, hybrid action space and reward function corresponding to the traffic lights and buses; Step S200: Simulate the traffic environment simulation scenario in the multi-intersection traffic environment based on the state space, hybrid action space and reward function corresponding to the traffic lights and buses; Step S300: Construct the initial traffic light prediction model and the initial bus prediction model; Step S400: Obtain the first training data corresponding to each of the traffic light prediction models and the second training data corresponding to each of the bus prediction models obtained during the simulation process; Step S500: Train the traffic light prediction model based on the first training data to obtain a trained traffic light prediction model; train the bus prediction model based on the second training data to obtain a trained bus prediction model.

[0058] This invention proposes a collaborative control framework based on the MAPPO algorithm, constructing a closed-loop "perception-decision-execution" system. In a multi-intersection environment, traffic light prediction models and bus prediction models sense information such as the status of traffic lights and vehicles in the approach lanes, adjust signal duration and bus speed, and calculate rewards. The MAPPO algorithm updates and optimizes the model's strategy based on the model's actions and the rewards obtained. During the training process, information from all models is integrated and centrally trained, and then the learning strategy is implemented through distributed execution to update actions, achieving the optimization goal and improving bus traffic efficiency.

[0059] This invention considers the mutual influence of traffic flow between adjacent intersections, and integrates the traffic state information of adjacent intersections (such as traffic light status, vehicle position at approach lanes, etc.) with the state information of the intersection itself to construct a more comprehensive state space. In terms of action space, it combines traffic light phase control (discrete action) and bus acceleration control (continuous action) to form a hybrid action space. It adopts a multi-objective reward function, introduces experience replay technology and a global state sharing mechanism to improve the convergence speed and global optimization capability of the algorithm, and realizes the coordinated control of bus signal priority and speed guidance at multiple intersections.

[0060] Specifically, this application first establishes a multi-intersection traffic environment. For example... Figure 2 and Figure 3As shown, a multi-intersection traffic environment is set up with three consecutive intersections on a main urban road. All intersecting roads are eight-lane dual carriageways. The lane functions of the east, west, south, and north entrances of each intersection are consistent: each includes one left-turn lane, one right-turn lane, and two through lanes. The number of exit lanes matches the number of entrance lanes. Simultaneously, it is assumed that all buses are connected vehicles, capable of obtaining real-time information such as their own position, speed, and acceleration, as well as the status of traffic lights at each intersection (phase, remaining green light time, etc.), via V2X communication. While other vehicles are not directly controlled by speed guidance, they are indirectly affected by changes in the speed of buses ahead. To achieve coordinated control among multiple intersections, traffic information from adjacent intersections is also needed to assist in decision-making at the intersection. The traffic environment constructed in this invention is also based on the following fundamental assumptions: First, the impact of pedestrians and non-motorized vehicles on vehicle operation is not considered; Second, all vehicles entering this traffic environment follow traffic rules, without considering the occurrence of traffic accidents; Third, there is no delay or malfunction in obtaining vehicle-related traffic information (location, speed, traffic light status, etc.).

[0061] After establishing the multi-intersection traffic environment, a state space is set up. Specifically, in a multi-intersection scenario, the state space needs to integrate the local state of each intersection with the shared state of its neighbors to achieve global collaborative optimization.

[0062] Define the state space of a single entrance lane i as follows: (1) in, This represents the status information on the i-th import lane; Indicates the current status of the traffic lights at the intersection; Indicates the position of the vehicle in import lane i; represents the speed of the bus closest to the intersection on the current road; ac represents the acceleration of the bus closest to the intersection on the current road; d represents the distance from the bus closest to the intersection on the current road to the intersection; lane represents the current lane number.

[0063] Define the state space of a single bus j as follows: (2) in, This indicates the speed of the vehicle adjacent to bus j in front of it; This indicates the speed of the vehicle immediately following bus j. This represents the acceleration of the vehicle immediately preceding bus j. This represents the acceleration of the vehicle immediately following bus j. This indicates the distance between bus j and the adjacent vehicle in front of it; This indicates the distance between bus j and the adjacent vehicle behind it; This represents the distance between bus j and the nearest intersection; if there are no vehicles in front or behind, insert a null value; tl represents the signal status of the nearest intersection.

[0064] For the state space of the traffic light agent, each intersection k corresponds to a traffic light prediction model, and its state space... It consists of the states of all approach lanes and the states of the nearest buses in each direction within the target area (i.e., multiple coordinated signalized intersections) corresponding to the simulation space, and is expressed as follows: (3) Where i represents the entrance lane number within the area; This represents the state space of each entrance lane i within the region; This represents the set of entrance lanes within the target area; This represents the state space of the nearest bus j in each direction at the current intersection; This represents the group of buses closest to the intersection on all approach lanes at the current intersection.

[0065] For the state space of bus agent j, each bus is considered an independent agent, and its state space is... It includes its own directly observed state and the state of the intersection it is located at, expressed as follows: (4) in, This represents the state space directly observed by bus j at present; Let i represent the state space of the approach lane i at intersection k where bus j is located; This represents the set of approach lanes at intersection k.

[0066] This application further establishes a hybrid action space. The hybrid action space combines discrete traffic light control actions with continuous bus speed guidance actions to achieve multi-agent collaborative decision-making.

[0067] Specifically, for the action space of the traffic light agent, a fixed phase sequence scheme is adopted, and the phase set is defined as follows: ,in, Indicates a straight line running north-south; Indicates turning left from north to south; Indicates a straight line running east-west; This indicates turning left from east or west. For example... Figure 4 As shown, right-turn traffic uses a permission control mode and does not have a separate right-turn phase. (The following refers to the actions of the traffic light AI.) The value can be 0 or 1. 0 means to extend the current phase time (extend by 1 second, up to a maximum of 40 seconds), and 1 means to switch to the next phase (a yellow light needs to be executed for 3 seconds before switching).

[0068] For the motion space of the intelligent agent on the bus, the acceleration of the bus... As a continuous action, the action space range is set to This ensures that buses can quickly adjust their speed to match the traffic light phases, while avoiding the impact of sudden acceleration or deceleration on passenger comfort and traffic safety.

[0069] This application further refines the reward function. A multi-objective reward function is adopted, taking into account public transport priority, the efficiency of private vehicle traffic, and overall traffic flow, and a branch road discount factor is introduced to differentiate the priority of main roads and branch roads.

[0070] Specifically, the reward function for the traffic light agent takes into account both vehicle delay and throughput at the signalized intersection, and its expression is as follows: (5) in, express Cumulative average delay per vehicle over a given period of time; Indicates the number of buses; Indicates the number of vehicles in the community; Buses are all delayed; This represents the average delay per vehicle for private vehicles; 'a' represents the weight of buses, where a>1 is used to prioritize bus signals; 't' represents the end time of the statistical time window. This indicates the duration of the statistical time window.

[0071] (6) in, express Throughput reward within a given time period; This indicates the number of buses that have exited the intersection; This indicates the number of vehicles exiting the intersection.

[0072] The reward function for the traffic light agent is then: (7) in, and It is a normalization coefficient used to unify two types of rewards to the same order of magnitude, thereby achieving balance and comparability between rewards.

[0073] The reward function of the bus agent is determined by the speed of bus j. With the maximum speed limit The deviation, and the acceleration of bus j. With maximum acceleration The deviation is constituted. The speed and acceleration in the reward are the speed and acceleration of all buses J located on the same road as the bus agent. The reward for the bus agent. Represented as: (8) (9) (10) reward function The goal is to increase the average speed of buses and keep it within the maximum speed limit. Within this speed range, higher speeds improve operational efficiency and reduce the risk of collisions due to excessive speed. Reward function. This is to reduce the frequent acceleration and deceleration changes of buses. and The reward function Encouraging models to better control vehicle speed promotes collaborative control, thereby improving bus speed and operational efficiency, as well as enhancing traffic safety.

[0074] The regional multi-intersection reward function introduces reward information sharing among agents, comprehensively considers the reward values ​​of different entering directions, and takes into account the differences in priority between buses and private vehicles entering from different directions, introducing a branch road discount factor to correlate the reward values ​​of the main road and branch roads with their respective priorities.

[0075] Specifically, the expression for the main path reward function is: (11) in, This indicates the cumulative average delay for vehicles on the main road; Indicates the throughput of vehicles on the main road; Represents the reward function for the intelligent agent of the bus on the main road; represents the normalization coefficient; M represents the number of buses on the main road.

[0076] The expression for the branch reward function is: (12) in, This indicates the cumulative average delay for vehicles on the branch road; Indicates the throughput of vehicles on the branch road; Represents the reward function for the bus agent on the branch road; represents the normalization coefficient; N represents the number of buses on the branch roads.

[0077] The expression for the global total reward function is: (13) in, This represents the branch discount factor.

[0078] The MAPPO algorithm in this application adopts a "centralized training-distributed execution" framework. Each agent uses an independent Actor network to make action decisions, while sharing a global Critic network to evaluate the global state value. Figure 5 As shown.

[0079] Specifically, the Actor network input for the traffic light agent is the global state. The output is the probability distribution of phase actions. The global state includes the observation states of all traffic light agents and all bus agents: (14) Where K represents the traffic light agent and J represents the bus agent; the Actor network input of the bus agent is its own observed state. The output is the probability distribution of acceleration actions. The input to the Critic network is the global state. The output is the global state value V( This is used to calculate the advantage function and update the value network, ensuring that the policy optimization direction aligns with the global optimal objective.

[0080] In one embodiment of this application, step S400 specifically includes: Step S410: Obtain the set of intersections and the set of approach lanes in the target area of ​​the multi-intersection traffic environment simulation scenario. In each time step of the simulation process, determine the number of buses closest to the intersection in each approach lane of the approach lane set, and form a bus set based on the number of buses. Step S420: Based on the approach lane status of all intersections in the intersection set and the bus status of all buses in the bus set, obtain the current global state corresponding to the current time step; Step S430: Use the current global state as the first training data, and use the bus state of each bus as the second training data for the corresponding bus prediction model.

[0081] In one embodiment of this application, the reward function of the traffic light is obtained by normalizing a preset vehicle delay reward function and a vehicle throughput reward function; the reward function of the bus is obtained based on the deviation of the bus's speed from a predetermined speed threshold and the deviation of the bus's acceleration from a predetermined acceleration threshold. Specifically, as shown in formulas (5), (6), (7), (8), (9), and (10).

[0082] In this embodiment of the application, step S500 specifically includes: Step S510: Initialize the first policy network parameters and the first value network parameters of the traffic light prediction model, and initialize the second policy network parameters and the second value network parameters of the bus prediction model; Step S520: In each time step, the traffic light prediction model selects a phase action based on the current global state through the first strategy network parameters, and the bus prediction model selects an acceleration action based on its own bus state through the second strategy network parameters. Step S530: After executing the phase action and acceleration action, the multi-intersection traffic environment simulation scenario returns the reward and the next global state. The current global state, phase action, acceleration action, reward returned by the multi-intersection traffic environment simulation scenario, and the next global state are stored as trajectory data in the experience pool. Step S540: Calculate the preset objective function using the PPO pruning method to update the parameters of the first policy network and the second policy network, and update the parameters of the first value network and the second value network using the preset mean squared error loss function to obtain the trained traffic light prediction model and bus prediction model.

[0083] In the training process of the traffic light prediction model and the bus prediction model, a generalized dominance estimation algorithm is used to calculate a preset dominance function.

[0084] The algorithm is trained as follows:

[0085] For example, the first step is to initialize the parameters. This involves initializing the policy network parameters of the traffic light agent. and value network parameters Policy network parameters of the bus intelligent agent and value network parameters Set hyperparameters: learning rate Discount factor =0.95, batch size =256, PPO trimming parameters =0.2, number of training rounds =150, simulation duration per round .

[0086] The second step is to collect training data. At each time step t, the traffic light agent calculates training data based on the current global state. (The set of all intersection states and bus states), through a policy network. Select phase action The bus's intelligent agent adjusts its settings based on its own status. Through policy network Select acceleration action After the action is performed, the environment returns a reward. and the next global state , trajectory data Stored in the experience pool.

[0087] The third step is to calculate the advantage function. The generalized advantage estimation (GAE) algorithm is used to calculate the advantage function, considering the cumulative effect of multi-step rewards. The expression is as follows: (15) In the formula, It is a timing difference error. It's a GAE parameter. For the value network to the global state Value estimate.

[0088] The fourth step is to update the policy network and the value network. The policy network update uses the PPO pruning method to calculate the objective function, limiting the differences between the old and new policies. The expression is as follows: (16) In the formula, n represents the batch size; This represents the advantage estimate for the current step; "Clip" is a hyperparameter representing the magnitude of the control strategy update; "clip" is the clipping function. It is the strategy ratio, expressed as follows: (17) In the formula, Indicates the current policy given state Select action The probability of; This represents the probability of the old strategy.

[0089] Value network updates use the mean squared error loss function, expressed as follows: (18) In the formula, Represents the value estimate of the next global state (usually using the old value network); This represents the global state value prediction at step t. The Critic network has the following parameters: Below, the global state at time t The value estimate is used as the predicted value, compared with the target value, to calculate the prediction bias. That is, the value network parameters are updated through gradient descent. .

[0090] Step 5, iterative training. Repeat steps 2 through 4 until the model converges or reaches the maximum number of training epochs.

[0091] In this embodiment of the application, the method further includes: When the experience pool is updated, batch data in the experience pool is randomly sampled, and a priority experience replay strategy is adopted to increase the sampling weight of data in the experience pool whose reward value is greater than a preset reward value or whose time series difference error is greater than a preset error to a preset weight.

[0092] Specifically, this application also adopts an experience replay mechanism, which introduces an experience pool to store trajectory data from multiple rounds, and randomly samples batch data each time it is updated to break the correlation between data and improve training stability. At the same time, a priority experience replay strategy is adopted to assign higher sampling weights to data with large reward values ​​or large temporal difference errors, thereby improving training efficiency.

[0093] In addition, it also adopts Greedy exploration strategy, early training phase As the number of training rounds increases, the value linearly decreases to 0.05, balancing the agent's exploration and utilization to avoid getting trapped in local optima.

[0094] Therefore, the core mechanism of this application is: information sharing + value binding + reward association, namely, firstly, global state sharing: the decisions of both agents depend on the other's state information (e.g., a traffic light needs to know the bus's position to determine whether to extend the green light, and a bus needs to know the signal phase to adjust its acceleration), rather than relying solely on its own local information; secondly, unified value assessment: the policy updates of both agents depend on the advantage function calculated by the shared Critic algorithm. This refers to the actions of traffic lights adjusting phase and buses adjusting acceleration; the quality of these actions needs to be evaluated through a global traffic benefit assessment. Third, the reward function is related to the reward of the traffic light agent. This includes delays in both public buses and private vehicles. Rewards for intelligent agents on buses ( This includes speed and speed limit deviation, acceleration stability, and the final global reward. It also relates to the reward results of both types of intelligent agents.

[0095] Compared to existing multi-intersection bus priority control technologies, the advantages of this invention are mainly reflected in the following aspects: First, it achieves coordinated optimization of signal and vehicle speed: overcoming the limitations of independent optimization of signal control and vehicle speed guidance in existing research, a multi-agent collaborative framework is constructed based on the MAPPO algorithm. The traffic light agent and the bus agent interact in real time, synchronously adjusting signal timing and vehicle speed parameters, avoiding bus stops and waiting caused by signal and speed mismatch, and significantly improving bus operation efficiency. In high-traffic scenarios, bus waiting time is reduced by 59% compared to the fixed timing method and by 55% compared to the DDQN method.

[0096] Second, it demonstrates outstanding global optimization capabilities: By employing a global state sharing mechanism and a regional coordinated reward function, it addresses the problem of "local optimization leading to cascading congestion" in traditional multi-agent reinforcement learning. Real-time sharing of traffic state information among multiple intersections and the Critic network evaluate policy value from a global perspective ensure that optimization actions at each intersection align with overall traffic efficiency. In a scenario with three consecutive intersections, the overall vehicle queue length is reduced by 52% compared to the IPPO method, and the efficiency of social vehicle traffic is improved by 11%.

[0097] Third, it exhibits strong adaptability and stability: By employing the pruning mechanism of the MAPPO algorithm and GAE advantage estimation, the policy update magnitude is limited, training variance is reduced, and the model can be stably converged under low, medium, and high traffic scenarios. Even when the proportion of bus traffic increases from 2.5% to 7.5%, the model still maintains bus priority, and bus vehicle loss time is reduced by at least 23%, far superior to the IPPO method.

[0098] Fourth, balancing benefits for all parties: The reward function design incorporates the weight of public transport vehicles and the discount factor for side roads, ensuring public transport priority while avoiding excessive sacrifice of the interests of private vehicles. Simulation results show that in medium-flow scenarios, compared with the fixed timing method, this invention reduces the travel time of public transport vehicles by 23% and the travel time of private vehicles by 16%, achieving a balance between the benefits of public transport and private vehicles.

[0099] To further quantify and verify the aforementioned advantages, a simulation of three consecutive intersections was conducted, keeping the bus traffic proportion (5%) constant. Under three overall road network traffic levels—low (2000 veh / h), medium (4000 veh / h), and high (6000 veh / h)—the performance of three deep reinforcement learning methods in terms of waiting time, loss time, and travel time was compared. First, the optimization effects of different deep reinforcement learning algorithms were compared by calculating the average waiting time, average loss time, and average travel time for both private vehicles and buses in each training round. Figure 6 , Figure 7 and Figure 8As shown, the multi-intersection bus priority control model based on the MAPPO algorithm exhibits excellent performance under different traffic flow levels, especially in reducing waiting time, time loss, and travel time. Compared with fixed timing schemes, DDQN, and IPPO algorithms, the MAPPO algorithm has significant advantages in all indicators, indicating that the multi-intersection bus priority control model based on the MAPPO algorithm has high application value.

[0100] The final average evaluation metric over 5 training rounds is as follows: Figure 9 and Figure 10 As shown in the data from public transport vehicles, under low, medium, and high traffic flow levels, all evaluation indicators of the bus priority control model based on the MAPPO algorithm are lower than those of the Fixed, DDQN, and IPPO methods. Especially in high traffic flow scenarios, the MAPPO model has the most significant advantages compared with other deep reinforcement learning models. It guarantees the right-of-way for buses without affecting the operation of other vehicles, thus improving the efficiency of public transport.

[0101] Among them, MAPPO is a multi-agent proximal policy optimization algorithm, Fixed is a fixed timing scheme, DDQN is a dual-deep Q-network algorithm, and IPPO is an independent proximal policy optimization algorithm.

[0102] Data from private vehicles shows that, under low, medium, and high traffic flow levels, the MAPPO-based bus priority control model maintains stable private vehicle traffic efficiency, significantly higher than the Fixed scheme. Compared to other reinforcement learning models, the MAPPO-based bus priority control model, through the coordinated optimization of bus signal priority and bus speed guidance, not only avoids reducing private vehicle traffic efficiency by prioritizing bus traffic but also improves overall traffic efficiency by ensuring bus priority.

[0103] From the perspective of overall evaluation indicators, Figure 10 In high-traffic scenarios, the bus priority control model based on the MAPPO algorithm reduced queuing time and queue length by 52% and 66%, respectively. These significant improvements demonstrate that the bus priority control model effectively reduces queuing time and length under high traffic conditions, showing a greater advantage over other methods. The MAPPO-based bus priority control model not only significantly optimizes the efficiency of bus traffic but also considers the overall traffic benefits of private vehicles, indicating that the proposed model has strong adaptability and can make optimal decisions based on different traffic conditions.

[0104] The overall traffic flow of the road network remained constant at 4000 veh / h, but the proportion of bus traffic was varied, and the average evaluation metrics of each method over the final 5 training rounds were compared. Regarding buses (see...),... Figure 11The waiting time of the MAPPO model was 43.9 seconds (62%) when the bus flow rate was 100 veh / h, but dropped sharply to 15.3 seconds at 200 and 300 veh / h, only 23% and 24% of the Fixed model, respectively. This indicates that the optimization effect of the MAPPO model is more significant under high bus flow rates. The lost time and travel time show a decreasing trend, especially at a bus flow rate of 300 veh / h, where the lost time and travel time of the MAPPO model are 65% and 23% lower than the Fixed model, respectively. A horizontal comparison of the optimization effect of the MAPPO model under different bus flow rates reveals that as the proportion of bus flow in the overall flow increases, the efficiency of bus traffic also improves.

[0105] Regarding private vehicles (see) Figure 11 The MAPPO model demonstrates a significant advantage in social vehicle evaluation metrics across all bus traffic volumes. For instance, at a bus speed of 200 veh / h, the waiting time for social vehicles is 20.3 seconds, which is 35% of that achieved by the Fixed method. While other methods, such as the IPPO method, also have low social vehicle evaluation metrics, their optimization performance is still inferior to that of the MAPPO method.

[0106] In terms of overall traffic efficiency (see...) Figure 12 The MAPPO model reduced the queuing time to 10.4s at a bus flow rate of 200veh / h, which is 39% of the Fixed method. At a bus flow rate of 300veh / h, the queuing time was 17.4s, which is only 66% of the Fixed method. However, it should be noted that the queuing time is the lowest at a bus flow rate of 200veh / h. This may be because the overall traffic efficiency is high at this bus flow rate, while the queue length is inevitably increased due to the increase in the proportion of buses.

[0107] In summary, the bus priority control model based on the MAPPO algorithm can improve bus operating efficiency in both diverse overall traffic flow scenarios and diverse bus traffic flow scenarios. Compared with the IPPO model, which does not share states and rewards, it is easier to achieve globally optimal control, providing a reliable solution for balancing bus priority and overall traffic efficiency.

[0108] Furthermore, through scenario-based simulations, the performance of four deep reinforcement learning methods in terms of waiting time, loss time, and passage time was compared across all simulation training rounds. For example... Figure 13 , Figure 14 , Figure 15 As shown, both MAPPO models converge slower than the DDQN method, but their final convergence results are better. The MAPPO-bus priority model shows a slightly better final convergence result than the MAPPO-non-priority model for bus vehicle evaluation indicators, but a slightly worse result for non-bus vehicle evaluation indicators.

[0109] The average evaluation index of the final 5 training rounds was compared using a scenario of three consecutive intersections on an urban main road. Fixed signal control, DDQN method, IPPO method (each intersection runs its own PPO algorithm independently) and the method of this invention MAPPO (including bus priority and bus non-priority) were compared. The results are shown in Figure (Table 5).

[0110] exist Figure 16 In comparison with Fixed, DDQN, IPPO, and MAPPO-non-priority models, the MAPPO-bus priority model performs better across all indicators for buses, showing superior performance in reducing bus waiting time, lost time, and travel time. However, in the comparison of evaluation indicators for private vehicles, the MAPPO-non-priority model shows slightly better waiting time and lost time compared to the MAPPO-bus priority model, and significantly better performance than other methods. This indicates that in real-world complex intersection environments, the MAPPO model exhibits better adaptability and performance for more diverse intersection shapes and lane distributions. Furthermore, granting buses signal priority significantly improves bus traffic efficiency while having minimal impact on private vehicles.

[0111] Combination Figure 17 Data shows that the MAPPO model significantly outperforms other models in terms of overall vehicle queuing time and queue length. Although the MAPPO-bus priority model is slightly longer in terms of overall queuing time and queue length after granting buses signal priority, this difference is negligible compared to the overall improvement in bus traffic efficiency. This fully validates that in multi-intersection scenarios, the MAPPO-bus priority model, through a coordinated optimization strategy based on the MAPPO algorithm for bus signal priority and bus speed guidance, can effectively improve bus operating efficiency and alleviate traffic congestion at intersections without affecting overall vehicle traffic efficiency.

[0112] Furthermore, firstly, research can be extended to complex traffic networks. Current research mainly focuses on scenarios with continuous multiple intersections, and does not address the bus priority control problem in large-scale road networks. Future research could combine graph neural network technology to design bus priority control models suitable for city-level road networks. In addition, further research is needed on the path game behavior of buses and other vehicles to avoid global imbalances caused by local optimization.

[0113] Secondly, further collaborative optimization of multimodal traffic flow can be achieved. This invention mainly focuses on the collaborative control of buses and private vehicles, without considering the impact of non-motorized vehicles, pedestrians, and other traffic participants. In the future, multimodal traffic flow modeling methods can be introduced, and multi-objective reward functions can be designed to achieve a more comprehensive balance between traffic efficiency and safety.

[0114] Third, it enables multi-source data fusion and intelligent decision-making. Current model training relies on idealized data generated by simulation environments, and practical applications must address issues such as sensor noise and communication latency. With the development of intelligent transportation systems, more and more traffic data can be collected and utilized, such as real-time vehicle location, speed, and trajectories. Future research can further explore how to fuse multi-source data to achieve more accurate traffic condition perception and intelligent decision-making. Through deep learning and data mining techniques, the potential information within traffic data can be extracted, providing stronger support for public transport priority control strategies.

[0115] Fourth, integration with other intelligent transportation technologies. Public transport priority control strategies can be combined with other intelligent transportation technologies, such as autonomous driving, to achieve more efficient traffic management and control. Future research can further explore how to combine public transport priority control strategies with other intelligent transportation technologies to leverage their respective advantages and jointly improve the operational efficiency and safety of urban traffic.

[0116] In one embodiment, such as Figure 18 As shown, based on the above-described multi-intersection bus priority control method, the present invention also provides a multi-intersection bus priority control device, comprising: The acquisition module 100 is used to acquire the status of each approach lane of each intersection in the target area at the current time, as well as the trained traffic light prediction model for each intersection. The determination module 200 is used to determine the target bus closest to the intersection in each approach lane at the current time, and to obtain the bus status and trained bus prediction model corresponding to the target bus at the current time. The prediction module 300 is used to predict the phase state of the traffic lights at each intersection at the next moment based on the state of the approach lane, the state of the bus, and the traffic light prediction model, and to predict the bus acceleration corresponding to each target bus at the next moment based on the state of the approach lane, the state of the bus, and the bus prediction model.

[0117] It should be noted that the foregoing explanation of the embodiment of the bus priority control method for multiple intersections also applies to the bus priority control device for multiple intersections in this embodiment, and will not be repeated here.

[0118] This invention discloses a bus priority control device for multiple intersections. It acquires the current state of each approach lane at each intersection within a target area, along with a trained traffic light prediction model for each intersection. It then determines the nearest target bus to the intersection on each approach lane at the current time, and acquires the bus state and trained bus prediction model corresponding to that target bus at the current time. Based on the approach lane state, bus state, and traffic light prediction model, it predicts the traffic light phase state at each intersection at the next time, and also predicts the bus acceleration corresponding to each target bus at the next time. This application enables the adjustment of traffic light phase state and bus acceleration using traffic light prediction models and bus prediction models in a multi-intersection environment, improving bus traffic efficiency.

[0119] Figure 19 A schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal may include: The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0120] When the processor 502 executes the program, it implements the multi-intersection bus priority control method provided in the above embodiments.

[0121] Furthermore, the terminal also includes: Communication interface 503 is used for communication between memory 501 and processor 502.

[0122] The memory 501 is used to store computer programs that can run on the processor 502.

[0123] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0124] If the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0125] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0126] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0127] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-intersection bus priority control method.

[0128] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0129] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0130] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0131] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can read and execute instructions from or in conjunction with such an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). In addition, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically by optically scanning paper or other media, then editing, interpreting or otherwise processing them as necessary, and then storing them in computer memory.

[0132] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0133] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0135] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for bus priority control at multiple intersections, characterized in that, include: Obtain the current state of each approach lane at each intersection within the target area, as well as the trained traffic light prediction model for each intersection. Identify the target bus closest to the intersection in each approach lane at the current moment, and obtain the bus status and trained bus prediction model corresponding to the target bus at the current moment; Based on the aforementioned approach lane status, bus status, and traffic light prediction model, the traffic light phase status of each intersection at the next moment is predicted, and based on the aforementioned approach lane status, bus status, and bus prediction model, the bus acceleration corresponding to each target bus at the next moment is predicted.

2. The multi-intersection bus priority control method according to claim 1, characterized in that, The status of the approach lanes includes: the current traffic light status at each intersection, the vehicle positions in the approach lanes, the current speed and acceleration of the target bus closest to the intersection in the approach lane, the current distance between the target bus and the intersection, and the approach lane number. The bus status includes: the speed of the vehicle in front of the target bus, the speed of the vehicle behind the target bus, the acceleration of the vehicle in front, the acceleration of the vehicle behind, the first distance between the target bus and the vehicle in front, the second distance between the target bus and the vehicle behind, the current distance between the target bus and the intersection, and the current traffic light status.

3. The bus priority control method at multiple intersections according to claim 1, characterized in that, The training steps for the traffic light prediction model and the bus prediction model include: In a pre-built simulation platform, a multi-intersection traffic environment simulation scenario corresponding to the target area is constructed, and the state space, hybrid action space, and reward function corresponding to the traffic lights and buses are set. The simulation is performed in the multi-intersection traffic environment simulation scenario based on the state space, hybrid action space and reward function corresponding to the traffic lights and buses. Construct initial traffic light prediction models and initial bus prediction models; Acquire the first training data corresponding to each of the traffic light prediction models and the second training data corresponding to each of the bus prediction models obtained during the simulation process; The traffic light prediction model is trained based on the first training data to obtain a trained traffic light prediction model, and the bus prediction model is trained based on the second training data to obtain a trained bus prediction model.

4. The bus priority control method at multiple intersections according to claim 3, characterized in that, Acquiring the first training data corresponding to each of the traffic light prediction models and the second training data corresponding to each of the bus prediction models obtained during the simulation process, including: Obtain the set of intersections and the set of approach lanes in the target area of ​​the multi-intersection traffic environment simulation scenario. At each time step in the simulation process, determine the number of buses closest to the intersection in each approach lane of the approach lane set, and form a bus set based on the number of buses. The current global state corresponding to the current time step is obtained based on the approach lane status of all intersections in the intersection set and the bus status of all buses in the bus set. The current global state is used as the first training data, and the bus state of each bus is used as the second training data for the corresponding bus prediction model.

5. The bus priority control method at multiple intersections according to claim 3, characterized in that, The reward function for the traffic lights is obtained by normalizing a preset vehicle delay reward function and a vehicle throughput reward function; the reward function for the buses is obtained based on the deviation between the bus speed and a predetermined speed threshold, and the deviation between the bus acceleration and a predetermined acceleration threshold.

6. The bus priority control method at multiple intersections according to claim 4, characterized in that, The traffic light prediction model is trained based on the first training data to obtain a trained traffic light prediction model, and the bus prediction model is trained based on the second training data to obtain a trained bus prediction model, including: Initialize the first policy network parameters and the first value network parameters of the traffic light prediction model, and initialize the second policy network parameters and the second value network parameters of the bus prediction model; At each time step, the traffic light prediction model selects a phase action based on the current global state using the first strategy network parameters, while the bus prediction model selects an acceleration action based on its own bus state using the second strategy network parameters. After executing phase action and acceleration action, the multi-intersection traffic environment simulation scenario returns the reward and the next global state. The current global state, phase action, acceleration action, reward returned by the multi-intersection traffic environment simulation scenario, and the next global state are stored as trajectory data in the experience pool. The PPO pruning method is used to calculate the preset objective function to update the parameters of the first policy network and the second policy network. The preset mean squared error loss function is used to update the parameters of the first value network and the second value network, so as to obtain the trained traffic light prediction model and bus prediction model. In the training process of the traffic light prediction model and the bus prediction model, a generalized dominance estimation algorithm is used to calculate a preset dominance function.

7. The bus priority control method at multiple intersections according to claim 6, characterized in that, The method further includes: When the experience pool is updated, batch data in the experience pool is randomly sampled, and a priority experience replay strategy is adopted to increase the sampling weight of data in the experience pool whose reward value is greater than a preset reward value or whose time series difference error is greater than a preset error to a preset weight.

8. A bus priority control device for multiple intersections, characterized in that, include: The acquisition module is used to acquire the status of each approach lane of each intersection in the target area at the current time, as well as the trained traffic light prediction model for each intersection. The determination module is used to determine the target bus closest to the intersection in each approach lane at the current time, and to obtain the bus status and trained bus prediction model corresponding to the target bus at the current time. The prediction module is used to predict the phase state of the traffic lights at each intersection at the next moment based on the state of the approach lane, the bus state, and the traffic light prediction model, and to predict the bus acceleration corresponding to each target bus at the next moment based on the state of the approach lane, the bus state, and the bus prediction model.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and a multi-intersection bus priority control program stored in the memory and executable on the processor. When the processor executes the multi-intersection bus priority control program, it implements the steps of the multi-intersection bus priority control method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a multi-intersection bus priority control program, which, when executed by a processor, implements the steps of the multi-intersection bus priority control method as described in any one of claims 1-7.

Images