Training method of signal lamp control model, signal lamp control method, device, equipment, medium and product

Abstract

The embodiment of the invention provides a training method of a signal lamp control model, a signal lamp control method, a device, equipment, a medium and a product, and relates to the technical field of urban road traffic. The method comprises the following steps: acquiring a plurality of first traffic data, constructing a state characterization matrix and a reward function according to the plurality of first traffic data, and performing model training according to the plurality of first traffic data, the state characterization matrix and the reward function to obtain a signal lamp control model. According to the method, in a model training process, a state representation matrix is utilized to dynamically quantify passing priorities of buses in all directions, and meanwhile, a reward function is utilized to guide the model training process, so that a signal lamp control model obtained through training can recognize and preferentially respond to bus passing demands, and signal lamp actions are dynamically adjusted; therefore, the collaborative optimization of multi-direction bus priority requests in a complex scene is realized, and the coordination capability and response efficiency of a bus priority system in a complex urban traffic environment are improved.

Description

Technical Field

[0001] This application relates to the field of urban road traffic technology, and in particular to a training method for a traffic light control model, a traffic light control method, a device, equipment, medium, and product. Background Technology

[0002] In complex urban road networks, public buses frequently face dynamically changing traffic flows at multiple intersections, resulting in significant delays that constitute a major component of overall bus operation delays. Therefore, adjusting bus traffic flow through intelligent signal control strategies is crucial for improving public transport service quality and urban traffic efficiency. In this context, accurately controlling traffic lights to achieve scientific priority passage for public buses is particularly important.

[0003] In existing technologies, traffic light control is achieved through a combination of preset strategies and real-time responses. A fixed-duration signal cycle and phase switching scheme is pre-defined based on historical traffic flow data and executed periodically in areas with stable traffic patterns. For traffic fluctuation scenarios, road surface sensors capture vehicle arrival information in real time. When queuing vehicles in a specific direction or the approach of priority public transport vehicles are detected, a green light extension or phase insertion mechanism is immediately triggered to control the switching of traffic light states.

[0004] However, existing technologies face the problem of multi-directional conflicts when handling bus priority requests. When buses from different directions approach an intersection simultaneously, existing technologies struggle to dynamically evaluate the urgency of priority for buses in each direction and cannot make decisions based on real-time traffic conditions. Especially in complex road networks, bus priority requests at multiple intersections are intertwined. Simple first-come-first-served or fixed priority rules often lead to local optimization that affects overall traffic efficiency, resulting in some buses gaining priority while causing severe traffic congestion in other directions. Summary of the Invention

[0005] This application provides a method for training a traffic light control model, a traffic light control method, a device, equipment, medium, and product to solve the problem of multi-directional conflicts faced by the prior art when handling bus priority passage requests.

[0006] In a first aspect, embodiments of this application provide a method for training a traffic light control model, comprising:

[0007] Acquire multiple first traffic data; wherein the multiple first traffic data are used to represent the traffic status of multiple intersections within a preset first time period, and the end time of the first time period is earlier than the current time;

[0008] Based on the multiple first traffic data, a state representation matrix and a reward function are constructed; wherein, the state representation matrix is ​​used to represent the passage priority of multiple buses at the multiple intersections;

[0009] A traffic light control model is obtained by training the model based on the multiple first traffic data, the state representation matrix, and the reward function; wherein, the reward function is used to guide the model training process so that the actions output by the traffic light control model prioritize the passage of the multiple buses according to the traffic priority, and the traffic light control model is used to represent the mapping relationship between the traffic state of the multiple intersections and the actions.

[0010] In one possible design, constructing the state representation matrix and reward function based on the plurality of first traffic data includes:

[0011] Based on the multiple first traffic data, the average speed, queue length, traffic light phase, and bus status are calculated; wherein, the average speed and queue length refer to the speed and length of multiple traffic flows at the multiple intersections, respectively; the traffic light phase is used to indicate the status of the multiple traffic lights at a first time point, which is any time point within the first time period; and the bus status is used to indicate the traffic priority of the multiple buses.

[0012] The average speed, the queue length, the traffic light phase, and the bus status are fused to obtain the state characterization matrix.

[0013] The reward function is obtained by fitting a function to the plurality of first traffic data.

[0014] In one possible design, the step of performing function fitting based on the plurality of first traffic data to obtain the reward function includes:

[0015] Based on the multiple first traffic data, queue density and bus delay are calculated; wherein, queue density is used to represent the congestion severity of the multiple traffic flows, and bus delay refers to the deviation between the actual arrival time and the preset arrival time of the multiple buses;

[0016] The reward function is obtained by weighted summation of the queue density and the bus delay.

[0017] In one possible design, the step of training the model based on the plurality of first traffic data, the state representation matrix, and the reward function to obtain the traffic light control model includes:

[0018] Based on the multiple first traffic data, an action space and an action value function are constructed; wherein, the action space is a set of multiple executable actions of the multiple traffic lights, and the action value function is used to represent the mapping relationship between the multiple executable actions and the value obtained after executing the multiple executable actions;

[0019] The traffic light control model is obtained by training the model based on the state representation matrix, the reward function, the action space, and the action value function.

[0020] In one possible design, constructing the action space and action value function based on the plurality of first traffic data includes:

[0021] Based on the multiple first traffic data, an initial action space and action value function are constructed; wherein, the initial action space is the set of all actions of the multiple traffic lights;

[0022] The initial action space is filtered according to multiple preset constraints to obtain the action space.

[0023] Secondly, embodiments of this application provide a traffic light control method, including:

[0024] Acquire multiple second traffic data; wherein, the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time;

[0025] The multiple second traffic data are input into the traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at the multiple intersections so that multiple buses at the multiple intersections can pass according to traffic priority, and the traffic light control model is obtained according to the training method of the traffic light control model described in any of the first aspects;

[0026] Perform the aforementioned traffic light control action.

[0027] In one possible design, after performing the traffic light control action, the following is also included:

[0028] Obtain the execution result after performing the traffic light control action;

[0029] Calculate the deviation between the execution result and the preset expected result;

[0030] Based on the deviation, the traffic light control model is optimized.

[0031] Thirdly, embodiments of this application provide a training device for a traffic light control model, the device comprising:

[0032] The first acquisition module is used to acquire multiple first traffic data and multiple second traffic data; wherein, the multiple first traffic data are used to represent the traffic status of multiple intersections within a preset first time period, the end time of the first time period is earlier than the current time, and the multiple second traffic data are used to represent the traffic status of the multiple intersections at the current time.

[0033] A construction module is used to construct a state representation matrix and a reward function based on the plurality of first traffic data; wherein, the state representation matrix is ​​used to represent the passage priority of multiple buses at the plurality of intersections;

[0034] The training module is used to train the model based on the multiple first traffic data, the state representation matrix, and the reward function to obtain a traffic light control model; wherein, the reward function is used to guide the model training process so that the actions output by the traffic light control model prioritize the passage of the multiple buses according to the traffic priority, and the traffic light control model is used to represent the mapping relationship between the traffic states of the multiple intersections and the actions.

[0035] In one possible design, the building block includes:

[0036] The calculation unit is used to calculate the average speed, queue length, traffic light phase, and bus status based on the plurality of first traffic data; wherein the average speed and queue length refer to the speed and length of the multiple traffic flows at the plurality of intersections, respectively; the traffic light phase is used to indicate the status of the plurality of traffic lights at a first time point, the first time point being any time point within the first time period; and the bus status is used to indicate the passage priority of the plurality of buses.

[0037] The feature fusion unit is used to perform feature fusion on the average speed, the queue length, the traffic light phase, and the bus status to obtain the status representation matrix;

[0038] The function fitting unit is used to perform function fitting based on the plurality of first traffic data to obtain the reward function.

[0039] In one possible design, the function fitting unit includes:

[0040] A calculation component is used to calculate queue density and bus delay based on the plurality of first traffic data; wherein, the queue density is used to represent the congestion severity of the plurality of traffic flows, and the bus delay refers to the deviation between the actual arrival time of the plurality of buses and the preset arrival time;

[0041] A weighted component is used to perform a weighted summation of the queue density and the bus delay to obtain the reward function.

[0042] In one possible design, the training module includes:

[0043] A construction unit is configured to construct an action space and an action value function based on the plurality of first traffic data; wherein, the action space is a set of multiple executable actions of the plurality of traffic lights, and the action value function is used to represent the mapping relationship between the plurality of executable actions and the value obtained after executing the plurality of executable actions;

[0044] The model training unit is used to train the model based on the state representation matrix, the reward function, the action space, and the action value function to obtain the traffic light control model.

[0045] In one possible design, the building unit includes:

[0046] A component is constructed to build an initial action space and an action value function based on the plurality of first traffic data; wherein, the initial action space is the set of all actions of the plurality of traffic lights;

[0047] A filtering component is used to filter the initial action space according to multiple preset constraints to obtain the action space.

[0048] Fourthly, embodiments of this application provide a traffic light control device, the device comprising:

[0049] The second acquisition module is used to acquire multiple second traffic data; wherein the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time;

[0050] An action determination module is used to input the multiple second traffic data into a traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at the multiple intersections so that multiple buses at the multiple intersections can pass according to traffic priority, and the traffic light control model is obtained based on the training device for the traffic light control model described in the third aspect;

[0051] The action execution module is used to execute the traffic light control action.

[0052] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0053] The memory stores computer-executable instructions;

[0054] When the processor executes the computer execution instructions stored in the memory, it is used to implement the training method of the traffic light control model as described in any of the first aspects, or to implement the traffic light control method as described in any of the second aspects.

[0055] Sixthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement a training method for a traffic light control model as described in any of the first aspects, or to implement a traffic light control method as described in any of the second aspects.

[0056] In a seventh aspect, this application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement a training method for a traffic light control model as described in any of the first aspects, or to implement a traffic light control method as described in any of the second aspects.

[0057] This application provides a training method, control method, device, equipment, medium, and product for a traffic light control model. By constructing a state representation matrix containing bus passage priorities and designing a reward function that guides bus priority, it solves the problem of multi-directional bus request conflicts. During model training, this method dynamically quantifies the passage priorities of buses in each direction using the state representation matrix. Simultaneously, the reward function guides the model training process, enabling the final trained traffic light control model to intelligently identify and prioritize bus passage demands, dynamically adjusting traffic light actions to ensure the rapid passage of high-priority buses. This achieves collaborative optimization of multi-directional bus priority requests in complex scenarios, improving the coordination capability and response efficiency of the bus priority system in complex urban traffic environments. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] Figure 1 This is a schematic diagram illustrating the training method of the traffic light control model and the application scenario of the traffic light control method provided in the embodiments of this application.

[0060] Figure 2 A flowchart illustrating the training method for the traffic light control model provided in this application embodiment;

[0061] Figure 3 The intersection layout phase diagram provided in the embodiments of this application;

[0062] Figure 4 This is a schematic diagram of a three-way intersection arterial road network layout provided in an embodiment of this application;

[0063] Figure 5 This is a diagram illustrating the calculation architecture of the action value function provided in the embodiments of this application.

[0064] Figure 6 A schematic diagram of a collaborative control framework for centralized learning and decentralized decision-making provided in an embodiment of this application;

[0065] Figure 7 A flowchart illustrating the traffic light control method provided in this application embodiment. Figure 1 ;

[0066] Figure 8 A flowchart illustrating the traffic light control method provided in this application embodiment. Figure 2 ;

[0067] Figure 9 A schematic diagram of the structure of a training device for a traffic light control model provided in an embodiment of this application;

[0068] Figure 10 This is a schematic diagram of the structure of a traffic light control device provided in an embodiment of this application;

[0069] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0070] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0072] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply difference. It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more.

[0073] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation; the embodiments of this application do not specifically limit this. Furthermore, the training method, traffic light control method, device, equipment, medium, and product for a traffic light control model provided in the embodiments of this application are merely examples; a training method, traffic light control method, device, equipment, medium, and product for a traffic light control model may also include more or less content.

[0074] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0075] Road network: refers to the transportation network system composed of urban road intersections and road segments connecting these intersections. It covers all travel routes within a specific area, including roads of different levels such as arterial roads, secondary roads, and local roads. Through signal control at intersections and coordinated operation between roads, the road network can guide the orderly flow of vehicles and pedestrians, forming a complete traffic operation system.

[0076] One-hot encoded vectors are a coding method used to represent discrete categorical data. They convert each possible category into a binary vector, where only the bit corresponding to that category is "true," and all other bits are "false." For example, in traffic signal phase control, if an intersection has four traffic lights, the state of each light can be represented by a four-dimensional vector, with the bit corresponding to the green light being 1 and the bit corresponding to the non-green light being 0. This coding method facilitates machine learning models in identifying and processing categorical variables while avoiding misleading calculations caused by numerical magnitude.

[0077] A replay buffer is a data structure used in reinforcement learning algorithms to store the agent's experience interacting with its environment. It records the actions taken by the controller in different states, the rewards received, and the new states after transitions. During training, the algorithm randomly draws past experiences from the buffer to learn from, breaking the temporal correlation between data points, improving sample utilization, and stabilizing the training process. This mechanism is similar to humans summarizing experiences by recalling past events, enabling the controller to efficiently learn policies from historical data and avoiding training fluctuations caused by the correlation of consecutive samples.

[0078] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0079] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0080] To clearly understand the technical solution of this application, the existing technical solutions will first be described in detail. In complex urban road networks, buses frequently face dynamically changing traffic flows at multiple intersections, and the resulting vehicle delays have become a major component of bus operation delays. How to adjust the bus travel process through intelligent signal control strategies has become a key issue in improving the quality of bus services and urban traffic efficiency. Against this backdrop, accurately controlling traffic lights to achieve scientific priority passage for buses is particularly important.

[0081] Current technologies control traffic lights by combining preset strategies with real-time responses. However, when buses from different directions approach an intersection simultaneously, existing technologies struggle to dynamically assess the urgency of priority for each direction and cannot make decisions based on real-time traffic conditions. Especially in complex road networks, bus priority requests from multiple intersections intertwine, and simple first-come-first-served or fixed-priority rules often lead to localized optimization that negatively impacts overall traffic efficiency, resulting in some buses gaining priority while causing severe congestion in other directions. Therefore, existing technologies face the problem of multi-directional conflicts when handling bus priority requests.

[0082] Therefore, addressing the issue of multi-directional conflicts encountered by existing technologies in handling bus priority requests, this research found that a priority evaluation method can be adopted to solve this problem. This method optimizes traffic light control strategies by calculating the passage priority of buses in each direction, allowing higher-priority buses to pass first, thus achieving intelligent collaborative decision-making for multi-directional bus priority requests: ① A multi-dimensional evaluation model can be constructed, comprehensively considering parameters such as vehicle passenger capacity, delay time, and route importance, combined with real-time traffic data, to establish a quantifiable passage priority calculation framework, enabling automated sorting and decision-making for conflict requests. ② A regional linkage control framework can be established, enabling adjacent intersections to coordinate the handling of bus priority requests through information sharing and collaborative algorithms. A combination of distributed decision-making and centralized coordination ensures both local response speed and overall road network efficiency, preventing chain congestion caused by priority decisions at a single intersection. ③ A signal control system with autonomous learning capabilities can be developed, automatically adjusting signal timing schemes based on historical data and real-time traffic conditions. The system can identify changes in traffic flow patterns, activate special control strategies during peak hours when public transport priority is needed, and revert to the normal optimization mode during off-peak hours, thus enabling free allocation of resources.

[0083] Specifically, a multi-dimensional information system can be established by comprehensively analyzing the vehicle operation status, bus operation characteristics, and signal control status in all directions of the intersection. Based on this, a quantitative evaluation system for bus priority demand can be designed to optimize the traffic light control decision model that considers both bus traffic efficiency and overall traffic operation benefits. This enables the system to automatically generate appropriate traffic light control schemes based on real-time traffic conditions, ensuring the rapid passage of high-priority buses while maintaining the coordinated operation of traffic flow in all directions of the intersection.

[0084] This application discloses a training method, control method, device, equipment, medium, and product for a traffic light control model. By constructing a state representation matrix containing bus passage priorities and designing a reward function that guides bus priority, it solves the coordination problem of multi-directional bus conflict requests. During model training, the method dynamically quantifies the passage priorities of buses in each direction using the state representation matrix. Simultaneously, it guides the model training process through the reward function, enabling the final trained traffic light control model to intelligently identify and prioritize bus passage demands, dynamically adjust traffic light actions, and ensure the rapid passage of high-priority buses. This achieves collaborative optimization of multi-directional bus priority requests in complex scenarios, improving the coordination capability and response efficiency of the bus priority system in complex urban traffic environments.

[0085] Based on the above-mentioned inventive discovery, the technical solution of this application is proposed.

[0086] The following describes the training method of the traffic light control model and the application scenarios of the traffic light control method provided in the embodiments of the present invention. Figure 1 This document illustrates the training method for the traffic light control model provided in this application, as well as a schematic diagram of the application scenario for the traffic light control method. Figure 1 As shown, this application scenario includes a user terminal 101 and a server 102. The user terminal 101 collects multiple first traffic data and multiple second traffic data, and sends them to the server 102. The server 102 constructs a state representation matrix and a reward function based on the multiple first traffic data, and then trains a model based on the multiple first traffic data, the state representation matrix, and the reward function to obtain a traffic light control model. The server 102 inputs the multiple second traffic data into the traffic light control model to obtain the traffic light control action, and then executes the traffic light control action.

[0087] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0088] Figure 2 This is a flowchart illustrating the training method for the traffic light control model provided in an embodiment of this application. Figure 2 As shown, in this embodiment, the execution entity of this invention is a server. Therefore, the training method for the traffic light control model provided in this embodiment includes the following steps:

[0089] S201. Obtain multiple first traffic data; wherein, the multiple first traffic data are used to represent the traffic status of multiple intersections within a preset first time period, and the end time of the first time period is earlier than the current time.

[0090] Specifically, traffic flow information can be collected in real time based on detection equipment installed at intersections, including dynamic parameters such as bus location, speed, passenger status, and queue length of private vehicles. Combined with scheduled timetable data provided by the public transport coordination center, this step is used to build the spatiotemporal feature foundation required for model training. Through the digital representation of historical traffic conditions, data support is provided for the subsequent establishment of a priority evaluation system and optimization of signal control strategies, so that the training process can accurately reflect the spatiotemporal distribution characteristics and conflict patterns of multi-directional bus priority requests in complex road networks.

[0091] Among them, detection equipment refers to hardware devices installed at road intersections to collect traffic parameters. These devices mainly include geomagnetic induction coils, video detection cameras, microwave radars, and vehicle positioning terminals. These devices can acquire traffic flow characteristic data such as vehicle position, speed, and queue length in real time, providing accurate perception input for signal control.

[0092] For example, static road network data and dynamic traffic data can be obtained through detection equipment, and then the static road network data and dynamic traffic data can be integrated to obtain multiple primary traffic data.

[0093] The method for obtaining static road network data is as follows: Select the spatial range of the signal-controlled road network, obtain the static information within the selected road network area, including the number of lanes at intersections, the functional settings of each lane (the functional settings refer to whether the lane is for left turn, straight or right turn, etc.), the location of bus stops, and bus routes. Figure 3 The intersection layout phase diagram provided in the embodiments of this application, such as Figure 3 As shown, the intersection's phase layout includes a north-south straight-ahead phase P0, a south-to-north left-turn phase P1, an east-west straight-ahead phase P2, and an east-west left-turn phase P4. The north-south straight-ahead phase P0 controls the passage of north-south vehicles going straight, allowing them to pass through the intersection during the green light. The north-south left-turn phase P1 is dedicated to north-south left-turning vehicles and is separated from the straight-ahead phase to avoid conflicts. The east-west straight-ahead phase P2 manages east-west straight-ahead vehicles, with its release staggered with the north-south phase. The east-west left-turn phase P4 independently releases east-west left-turning vehicles, typically equipped with a dedicated left-turn arrow light. These phases are switched at different times to ensure orderly passage of vehicles from different directions, improving intersection efficiency.

[0094] It can acquire overall traffic flow data, traffic light data, and public transport operation data, and then integrate the overall traffic flow data, traffic light data, and public transport operation data to obtain dynamic traffic data.

[0095] Overall traffic flow data is obtained by using roadside sensors deployed around intersections, such as loop detectors, radar sensors, millimeter-wave radar, lidar, ultrasonic sensors, cameras, and integrated radar-visual systems, as well as data from navigation applications, to obtain the average speed S of each segment of traffic flow. Alm and queue length L Qlm Where the entrance lane number l∈Ψ, the traffic flow number m∈M1, Ψ is the set of entrance lanes at the intersection, which usually includes the south entrance lane, north entrance lane, east entrance lane and west entrance lane for a cross intersection, and M1 is the set of traffic flows for entrance lane l.

[0096] The traffic light data is obtained by directly acquiring the current green light phase information through the data interface provided by the traffic signal controller.

[0097] The bus operation data is obtained as follows: the real-time location of the bus is obtained through the onboard positioning device, which provides the distance to the nearest intersection; the number of passengers in the bus is obtained in real time through the automatic passenger counting system installed inside the bus; and the delay is calculated by comparing the schedule with the current time when the bus arrives at the station. When running between two stations, the calculation formula can be expressed as:

[0098] SD = max(0, SDV) s +SD t )

[0099] SD represents flight delays. t It is the delay from the nearest stop S to the current location of the bus at time t, and its calculation formula is: TT t and TD t V represents the travel time and distance from leaving station S to time t. s This is the planned operating speed of the bus, typically 20 km / h. Only late buses are assigned a specific delay time to account for different priorities; therefore, the delay for buses that are not late is set to 0.

[0100] Figure 4 This is a schematic diagram of a three-intersection arterial road network layout provided in an embodiment of this application, such as... Figure 4 As shown, the road network includes three intersections, multiple bus routes, and multiple stops. The multiple bus routes include bus routes W1, E0, N2, and E3, and the multiple stops include stops OE1, E1, S2, and N2. Stop OE1 indicates that buses on route OE1 will stop at stop OE1. Stops W0, OE1, and OE3 indicate that buses on routes W0, OE1, and OE3 will stop at this stop.

[0101] S202. Based on multiple first traffic data, construct a state representation matrix and a reward function; wherein, the state representation matrix is ​​used to represent the passage priority of multiple buses at multiple intersections.

[0102] Specifically, by combining data such as the real-time location, delay duration, passenger volume, and traffic density in each direction at intersections, a state representation matrix can be constructed through feature weighted fusion and spatiotemporal correlation analysis. This matrix quantifies the urgency of bus passage in different directions. Simultaneously, based on the principle of balancing bus priority efficiency and the impact of private vehicle passage, a reward function that comprehensively considers the benefits of priority passage and the costs of traffic congestion is designed. This step transforms the raw traffic parameters into machine-understandable decision-making criteria, enabling the model training process to automatically learn how to adjust traffic light actions in scenarios with multiple conflict requests, thereby achieving network-level bus priority collaborative control.

[0103] S203. The traffic light control model is obtained by training the model based on multiple first traffic data, state representation matrix and reward function; wherein, the reward function is used to guide the model training process so that the action output by the traffic light control model prioritizes the passage of multiple buses according to the passage priority, and the traffic light control model is used to represent the mapping relationship between traffic state and action of multiple intersections.

[0104] Specifically, a reinforcement learning framework can be used, with the state representation matrix as the input feature. By continuously trying different traffic light control actions and receiving feedback from the reward function, the model parameters can be gradually optimized. This step is used to enable the control model to automatically learn the complex mapping relationship between traffic state and traffic light actions, so that when faced with multi-directional bus priority requests, it can dynamically generate traffic light actions that take into account both bus priority efficiency and overall traffic flow based on real-time traffic conditions.

[0105] This embodiment provides a training method for a traffic light control model. By constructing a state representation matrix containing bus passage priorities and designing a reward function that guides bus priority, it solves the coordination problem of multi-directional bus conflict requests. During model training, this method dynamically quantifies the passage priorities of buses in each direction using the state representation matrix. Simultaneously, it guides the model training process through the reward function, enabling the finally trained traffic light control model to intelligently identify and prioritize bus passage demands, dynamically adjust traffic light actions, and ensure the rapid passage of high-priority buses. This achieves collaborative optimization of multi-directional bus priority requests in complex scenarios, improving the coordination capability and response efficiency of the bus priority system in complex urban traffic environments.

[0106] In one possible design, S202, based on multiple first traffic data, constructs a state representation matrix and a reward function, including:

[0107] S2021. Based on multiple primary traffic data, calculate the average speed, queue length, traffic light phase, and bus status; where average speed and queue length refer to the speed and length of multiple traffic flows at multiple intersections, respectively; traffic light phase is used to indicate the status of multiple traffic lights at the first time point, which is any time point within the first time period; and bus status is used to indicate the passage priority of multiple buses.

[0108] Specifically, by using vehicle trajectory data and traffic light timing information collected by roadside sensing devices, the average speed of traffic flow in each direction within a certain time interval can be calculated as the average speed. The number of vehicles continuously stopped behind the stop line can be used as the queue length. The current traffic light color switching cycle can be recorded as the phase state. In combination with the real-time location, passenger capacity, and planned deviation of buses, the traffic priority can be calculated. This step is used to transform the raw traffic parameters into quantitative indicators with clear physical meaning, providing standardized input for constructing a feature matrix that reflects the comprehensive operating status of the road network, so that subsequent models can accurately identify the priority decision basis under different traffic situations.

[0109] For example, the formulas for calculating average speed and queue length are:

[0110] SD A ={S Alm}, l∈ψ, m∈M l

[0111] L Q ={Q Qlm}, l∈ψ, m∈M l

[0112] Among them, SD A and L Q S represents the average speed and queue length, respectively. Alm and L Qlm Let l be the speed and length of the traffic flow in each road segment, Ψ be the entrance lane number, m be the traffic flow number, and M1 be the traffic flow set of the entrance lane.

[0113] Traffic light phase P c It can be represented by a one-hot encoded vector. In the one-hot encoded vector, the position shown as 1 is the green light, and the position shown as 0 is the red light or the yellow light.

[0114] The bus status is determined by the distance D from the bus approaching the intersection to the stop line. pd , SD of train schedule pd Passenger occupancy rate O pd Composition, Bus Status S Bus It can be represented as:

[0115] S Bus ={D pd SD pd O pd}, p∈P, b∈Γ

[0116] Where P is the phase set, and for a four-phase signal scheme p∈{0,1,2,3}. Γ is the bus set defined by |Γ|=3, meaning that the information of the three most urgent buses in each phase p is placed in S. Bus Medium. Urgency level is indicated by SD. pd O pd / ((D pd The calculation is performed using +δ), where δ = 1 to avoid invalid calculations. Therefore, each intersection can simultaneously consider 12 bus requests from different directions.

[0117] S2022. Feature fusion is performed on average speed, queue length, traffic light phase, and bus status to obtain a state representation matrix.

[0118] Specifically, by using weighted aggregation and spatiotemporal correlation analysis, features such as traffic flow speed, queuing status, traffic light status, and bus priority levels in different directions at each intersection can be combined in multiple dimensions. The spatiotemporal correlation features are integrated using matrix coding. This step is used to generate a comprehensive state representation that can simultaneously reflect the local traffic situation and the global road network correlation. This provides the signal control model with complete environmental information that includes the collaborative decision-making required for multiple intersections, enabling the model to accurately evaluate the impact of different traffic light actions on the overall traffic flow.

[0119] S2023. Based on multiple primary traffic data, a function is fitted to obtain the reward function.

[0120] Specifically, by analyzing the correlation between bus delays and the efficiency of private vehicle traffic in historical traffic data, a quantitative evaluation system can be established that focuses on improving bus punctuality while also taking into account the rights of private vehicles. This system transforms the balance between the benefits of bus priority and the costs of traffic congestion into a mathematical expression. This step is used to construct directional indicators to guide model optimization, enabling the training process to automatically identify traffic light control actions that ensure rapid bus passage while avoiding overall road network congestion. Ultimately, this forms an intelligent decision-making mechanism with multi-objective coordination capabilities.

[0121] The technical effect of this solution in this embodiment is that it improves the accuracy and adaptability of traffic light control by deeply integrating multi-dimensional traffic features. This method extracts key indicators such as average speed, queue length, traffic light phase, and bus status from historical traffic data. Through feature fusion, it constructs a dynamically perceived state representation matrix, which accurately depicts the correlation between intersection traffic flow dynamics and bus priority demand. Simultaneously, based on a reward function fitted from historical data, the generated traffic light control model can respond in real time to the passage demand of high-priority buses, ultimately achieving bus priority passage. This intelligent control method not only improves the punctuality and operational efficiency of buses but also reduces traffic congestion at intersections.

[0122] In one possible design, S2023, a reward function is obtained by fitting multiple sets of first traffic data, including:

[0123] S20231. Calculate queue density and bus delay based on multiple primary traffic data; where queue density is used to represent the severity of congestion in multiple traffic flows, and bus delay refers to the deviation between the actual arrival time and the preset arrival time of multiple buses.

[0124] Specifically, the ratio of the queue length of vehicles in each direction to the lane detection length can be used as the queue density by collecting the queue length of vehicles in the detection equipment and the detection length of the detector. At the same time, combined with the location information of buses and timetable data, the difference between the actual travel time and the planned time can be calculated as the delay time. This step is used to quantitatively evaluate the real-time status of traffic operation efficiency and public transport service quality, and to provide key indicators reflecting the road network operation status for the reward function.

[0125] S20232. The reward function is obtained by weighted summation of queuing density and bus delay.

[0126] Specifically, based on the principle of balancing public transport priority strategy and social vehicle traffic demand, dynamic weight coefficients can be set for queuing density and public transport delay respectively. The two indicators can be combined into an evaluation function through linear combination. This step is used to establish a comprehensive evaluation system that takes into account both public transport efficiency and social equity, so that the model can explore appropriate weight ratios during training, and finally form an intelligent signal control scheme that can effectively reduce public transport delays and control the degree of social vehicle congestion.

[0127] For example, the design of the reward function should consider two points: prioritizing public transport vehicles to reduce bus delays and minimize the adverse impact on other traffic flows; and improving overall traffic efficiency when the number of buses is low. Therefore, the reward can be defined as a weighted sum of queue density and bus delays:

[0128]

[0129] Where r represents the reward. and These represent the lanes of entrance lane l in the road network. Queue length and detector detection length, SD k and O k Let w1 and w2 represent the bus delay and passenger number of bus k in the road network, respectively, and w1 and w2 are two negative weights for queue length and bus delay, respectively.

[0130] The technical effect of this solution in this embodiment is that by establishing a reward mechanism based on queuing density and bus delays, a scientific balance between bus priority and traffic efficiency is achieved. Queuing indicators reflecting road congestion and delay indicators reflecting bus punctuality are organically combined, and a comprehensive reward function is generated through intelligent weighted calculation. This enables the signal control system to automatically identify the optimal decision point: when bus delays are severe, priority is given to ensuring bus traffic; when bus operations are stable, the focus is on alleviating road congestion. This dynamic trade-off mechanism avoids traffic imbalance caused by excessive priority while ensuring the reliability of bus services, ultimately achieving synergistic optimization of bus priority and overall road network efficiency.

[0131] In one possible design, S203 trains a traffic light control model based on multiple first traffic data, a state representation matrix, and a reward function, including:

[0132] S2031. Based on multiple first traffic data, construct an action space and an action value function; wherein, the action space is a set of multiple executable actions of multiple traffic lights, and the action value function is used to represent the mapping relationship between multiple executable actions and the value obtained after executing multiple executable actions.

[0133] Specifically, based on the phase configuration scheme and switching rules of the intersection traffic lights, control commands such as green light duration adjustment and phase switching can be organized into a set of discretized optional actions. At the same time, by analyzing the response effects of historical traffic flow to different signal strategies, an evaluation function reflecting the long-term benefits of the actions can be established. This step is used to provide complete decision elements for the reinforcement learning framework, enabling the training process to systematically explore the effects of various traffic light control actions, and ultimately obtain an intelligent decision-making model that can select traffic light control actions based on real-time traffic conditions.

[0134] For example, Figure 5 The action value function calculation architecture diagram provided in the embodiments of this application is as follows: Figure 5 As shown, the input state representation matrix vector, for the average velocity S A and queue length L Q Features are obtained through standardization and flattening. The current phase directly uses a one-hot encoded vector to represent the bus state S. Bus After standardization, a convolutional operation is performed followed by activation using a Leaky Rectified Linear Unit (LeakyReLU). The output is flattened and then the extracted features are output using a fully connected layer. The average speed, queue length, current phase, and bus state features are concatenated, activated using a fully connected layer and LeakyReLU, and then the state value and action advantage value are output from the fully connected layer and summed to obtain the calculated action value.

[0135] S2032. Train the model based on the state representation matrix, reward function, action space and action value function to obtain the traffic light control model.

[0136] Specifically, reinforcement learning algorithms can be used to continuously try different signal control schemes in the action space, perceive changes in the traffic environment by combining the state representation matrix, evaluate the control effect using the reward function, and iteratively optimize the decision strategy based on the action value function. This step is used to enable the model to learn autonomously how to make the optimal signal control decision in complex traffic environments, so that it can automatically generate control schemes that improve public transportation efficiency and ensure overall traffic flow based on real-time traffic conditions.

[0137] For example, a controller can be set up on each traffic light to control the action of the traffic light. The controller can be based on the state representation matrix, reward function, action space, and action value function, and adopt a collaborative control framework of centralized learning and decentralized decision-making using Q-value Mixing Network (QMIX). Figure 6 This is a schematic diagram of a collaborative control framework for centralized learning and decentralized decision-making provided in an embodiment of this application, as shown below. Figure 6 As shown, controller i uses an action value function estimation algorithm to estimate the current state at time t. Obtain the value of all actions and, in conjunction with signal constraints, select the action with the highest value. get The action value of multiple controllers and global state In the input hybrid network, the combined actions of each controller are processed through multiple layers of fully connected networks. The action values ​​are non-linearly combined into the global value Q. tot (S t ,Λ t Similarly, the target network is given the subsequent state. Finally, the global target network is obtained. It can be expressed as:

[0138]

[0139] Among them, S t and S t ′ represents the current and next global states at time t, respectively, Λ t and Λ t ′ represents the set of actions at the current moment and the next moment, respectively, and Θ and Θ are the sets of actions at the current moment and the next moment, respectively. - These are the parameters for the main network and the target network, respectively. N is the number of intersections, and Q is the number of intersections. tot and They are respectively Figure 6 The global value of the target network and the global value of the main network.

[0140] The loss calculation function during model training can be expressed as:

[0141]

[0142] Among them, y tot For the overall goal value, For loss, r t S represents the target value of the current action, γ is the discount factor, and S bat This represents the number of training samples.

[0143] The model training process may include the following steps:

[0144] (1) Initialize key parameters, including replay buffer, parameters of the main network and target network, and time counter.

[0145] (2) Each training round contains multiple time steps. For each time step t, each controller observes the current state. It can greedily choose an action with a probability of (1-ξ) based on the value, or independently and randomly choose an action with a probability of ξ, where ξ is a preset parameter.

[0146] (3) Execute joint actions Transfer to a new state And receive the corresponding reward r.

[0147] (4) All agents < r t Experiences are stored in a replay buffer; if the buffer is full, older samples are discarded. Each experience is also assigned a priority weight so that experiences are replayed according to priority.

[0148] (5) When the number of stored experiences exceeds the number of small batch samples, a small sample will be drawn from the buffer according to priority.

[0149] (6) Calculate the loss for each mini-batch of samples and use these gradients to update the main network.

[0150] (7) After a certain number of updates, the target network synchronizes with the main network.

[0151] (8) The loop of steps (2)-(7) continues until the number of training sets reaches the preset number and the algorithm is observed to have converged to a suitable strategy. The network parameters of this strategy are stored and can be used to achieve efficient coordination of bus signal priority.

[0152] During model training, the settings of key hyperparameters have a decisive impact on algorithm performance. A series of parameter configurations validated by data and optimized empirically can be adopted. The discount factor balances the short-term and long-term goals of the agent by adjusting the discount rate of future rewards. A higher discount factor value makes the controller focus more on long-term rewards, which is suitable for complex environments with delayed rewards. The learning rate controls the magnitude of policy network parameter updates. While a larger learning rate can accelerate initial convergence, it may cause training instability. When a greedy strategy is adopted, the exploration rate ξ dynamically adjusts the trade-off between exploration and exploitation, generally causing the exploration rate to gradually decay as training progresses, eventually leading the policy to converge to a deterministic decision. The target network update frequency controls the period of target network parameter updates to ensure training stability. The empirical replay mechanism includes two key parameters: replay buffer capacity and mini-batch size. The replay buffer capacity ensures that sufficiently diverse state transition samples are stored for training. The mini-batch size specifies the number of samples extracted from the replay buffer for each learning step, affecting learning stability and computational efficiency. These parameters were initially set to default values ​​from open-source frameworks, and then repeatedly fine-tuned empirically through experimentation to optimize performance. In addition, the two negative weights in the reward function are set to normalize the two parts of the function, so that the controller can balance the queue length and the bus late arrival time.

[0153] The technical effect of this solution in this embodiment is as follows: by constructing a complete intelligent decision-making framework, the training effect and decision quality of the traffic light control model are improved. Traffic state representation, reward mechanisms, and the signal control action space are organically integrated. The effectiveness of different control strategies is accurately evaluated through action value functions, enabling the model to autonomously learn the optimal traffic light control strategy. This systematic training method ensures that the generated traffic light control model can not only quickly respond to public transport priority demands but also comprehensively consider the overall operating status of the road network, making scientific and reasonable control decisions in complex traffic environments.

[0154] In one possible design, S2031, based on multiple primary traffic data, constructs an action space and action value function, including:

[0155] S20311. Based on multiple first traffic data, construct an initial action space and action value function; wherein, the initial action space is the set of all actions of multiple traffic lights.

[0156] Specifically, based on intersection signal control specifications and historical timing schemes, operations such as extending or shortening green light times and switching phases in each phase can be organized into a complete set of initial actions. At the same time, by combining the correlation between traffic flow parameters and signal control effects, a preliminary action value evaluation mechanism can be established. This step is used to form the basic decision framework for model training, providing a complete solution library containing all possible control options and preliminary value judgment criteria for subsequent optimization.

[0157] S20312. Based on multiple preset constraints, filter the initial motion space to obtain the motion space.

[0158] Specifically, based on the safety and efficiency requirements of traffic signal control, by setting constraints such as minimum green light time and maximum red light waiting time, executable action combinations that meet actual traffic management needs can be selected from the initial action space. This step is used to ensure that the generated signal control scheme not only meets the basic requirements of road traffic safety, but also effectively improves traffic operation efficiency, and avoids the model exploring unrealistic or potentially unsafe control strategies during the training process.

[0159] For example, an initial action space can be constructed based on all actions of a traffic light. Then, an invalid action masking algorithm and signal constraints can be used to filter the initial action space to obtain the final action space. The invalid action masking algorithm adds a large negative value to the original action value to form a new action value. Because it selects the action with the highest action value, other invalid actions with lower values ​​will be masked out.

[0160] Signal constraints include two aspects: skip rule constraints and signal time constraints. Phase skipping rules are set based on engineering experience and driver acceptance: when queuing is detected in a phase, that phase cannot be skipped twice consecutively; when the queue exceeds a certain threshold, that phase cannot be skipped. Therefore, once a phase cannot be skipped, the phases following it in the basic phase scheme will be invalid. Phase skipping refers to the situation where, during signal control, if there is currently no vehicle demand for a phase, the traffic signal control system can temporarily skip the green light for that phase and directly switch to the next demanding phase, thereby reducing invalid green light time and improving the overall efficiency of the intersection. Signal time constraints include the shortest green light time and the longest green light time, calculated using the following formula:

[0161]

[0162]

[0163] Among them, G max and G min D represents the shortest green light time and the longest green light time, respectively. Cross and V Ped L represents the length of the pedestrian crossing and the speed of the pedestrians, respectively. QP Let |P| be the maximum queue length of phase P. For example, if phase P is a straight-through phase of a branch, then the maximum value of the queue length of a straight-through phase of a bidirectional branch is taken. |P| is the size of the phase set P.

[0164] The technical effect of this solution in this embodiment is that by introducing constraints to intelligently filter the initial action space, the safety and feasibility of signal control are effectively improved. By combining traffic engineering rules with intelligent decision-making, while maintaining control flexibility, signal actions that do not meet actual traffic management requirements are automatically filtered out, ensuring that the generated signal timing scheme satisfies both public transport priority needs and fully complies with traffic safety regulations. This constraint optimization mechanism enables the final signal light control model to dynamically adjust the allocation of right-of-way in each direction while ensuring traffic order, achieving a dual guarantee of public transport priority and traffic safety.

[0165] Figure 7 A flowchart illustrating the traffic light control method provided in this application embodiment. Figure 1 In this embodiment, the traffic light control method includes:

[0166] S701. Obtain multiple second traffic data; wherein, the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time.

[0167] Specifically, detection devices installed at intersections can collect real-time information on the driving status of vehicles in all directions and the current phase of traffic lights. Combined with the location and passenger data reported by buses, parameters reflecting the latest traffic conditions can be dynamically obtained. This step provides accurate and real-time decision-making basis for the traffic light control model, ensuring that the generated signal control actions can adapt to the current road network operation status and achieve dynamic optimization of bus priority control.

[0168] S702. Input multiple second traffic data into the traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at multiple intersections so that multiple buses at multiple intersections can pass according to traffic priority. The traffic light control model is based on... Figure 2 The signal light control model was obtained through training methods.

[0169] Specifically, parameters such as real-time vehicle location, speed, and traffic light status can be standardized and then input into a pre-trained control model. The model can then automatically calculate appropriate traffic light control actions based on the current traffic conditions. This step converts real-time traffic data into executable signal control commands, enabling intersection traffic lights to dynamically respond to actual traffic demands while ensuring priority for public transport and maintaining overall traffic efficiency.

[0170] S703, execute traffic light control actions.

[0171] Specifically, the control commands output by the model can be converted into specific signal light phase switching commands and timing parameters. The display status and duration of the signal lights in each direction can be adjusted in real time through traffic signal control equipment. This step is used to transform the intelligent decision-making results into actual road traffic management measures, ensuring that the public transport priority strategy is implemented accurately, while maintaining the safe and orderly passage of traffic in each direction at the intersection.

[0172] The technical effect of this solution in this embodiment is that by applying the trained intelligent control model to actual traffic light control, efficient traffic flow with bus priority is achieved. Utilizing real-time traffic data to drive the pre-trained model to generate traffic light control strategies, the system can intelligently adjust the traffic light status according to current road conditions, ensuring the rapid passage of high-priority buses. This AI-based real-time decision-making mechanism shortens bus waiting times, improves intersection efficiency, and provides an effective solution for urban intelligent traffic management.

[0173] Figure 8 A flowchart illustrating the traffic light control method provided in this application embodiment. Figure 2 In this embodiment, in Figure 7 Based on the provided embodiments, the traffic light control method is further explained. The traffic light control method includes:

[0174] S801. Obtain multiple second traffic data; wherein, the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time.

[0175] S802. Input multiple second traffic data into the traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at multiple intersections so that multiple buses at multiple intersections can pass according to traffic priority. The traffic light control model is based on... Figure 2 The signal light control model was obtained through training methods.

[0176] S803, execute traffic light control actions.

[0177] S801-S803 are similar to S701-S703, and will not be described again in this embodiment.

[0178] S804. Obtain the execution result after executing the traffic light control action.

[0179] Specifically, the detection equipment can collect feedback data in real time, such as the traffic efficiency in each direction after the traffic lights are adjusted and the actual transit time of buses. It can also record the changes in traffic status after the control actions are implemented. This step is used to evaluate the actual effect of the signal control strategy, provide real-world verification data for subsequent model optimization, and ensure that the control strategy can continuously adapt to the changing characteristics of traffic flow.

[0180] For example, an evaluation index system for executing traffic light control actions can be designed by combining learning performance evaluation metrics for artificial intelligence algorithms and application performance evaluation metrics for traffic control. In this evaluation index system, learning performance metrics include cumulative reward and joint action value. Cumulative reward is the instantaneous reward accumulated at each time step of a complete round, while joint action value reflects the quality of the agent's joint actions under specific states and strategies. Both metrics can be plotted as curves of change with training step size and ensemble size, reflecting learning performance and observing convergence. Traffic control application performance metrics include average delay per person, average queue length, and average bus delay time, which can intuitively reflect the impact of executing traffic light control actions on the overall road network and bus operation.

[0181] Table 1 shows the execution results provided in the embodiments of this application. As shown in Table 1, the change rate of the proposed method in terms of average delay per person, queue length, and delay time was calculated using timed coordinated signal control as a benchmark. Several other general methods were also selected for comparison.

[0182] Table 1 Execution Results

[0183]

[0184] As shown in Table 1, the proposed method achieves satisfactory performance in reducing the average delay per person, the average delay per car, and the delay time of buses. While it ranks second only to adaptive signal control in reducing the average delay per bus and queue length, it still shows a significant improvement over other methods. The superior results of the proposed method compared to independent reinforcement learning demonstrate the advantages of its efficient multi-intersection collaborative mechanism. The superior delay time results compared to adaptive signal control also prove that the method handles multi-directional conflicting bus priority requests, achieving more efficient second-level adaptive bus signal priority decision-making. The improvement compared to linear credit allocation reinforcement learning indicates that the proposed method, by employing nonlinear credit allocation, can better clarify the contribution of each agent, thereby improving learning performance and leading to better application results. Therefore, simulations verify that the proposed multi-agent reinforcement learning-driven arterial road cooperative bus signal priority control method can achieve a dual improvement in road network traffic efficiency and bus reliability.

[0185] S805. Calculate the deviation between the execution result and the preset expected result.

[0186] Specifically, the difference between the actual travel time of public buses and the planned timetable, as well as the gap between the traffic efficiency of private vehicles and the expected indicators, can be compared to quantitatively analyze the degree of deviation of the control effect. This step is used to objectively evaluate the implementation effectiveness of traffic light control actions, provide data support for model parameter adjustment, and ensure that the subsequent optimization direction is consistent with the actual traffic improvement needs.

[0187] S806. Optimize the traffic light control model based on the deviation.

[0188] Specifically, by analyzing the deviation data of the control effect, the model parameters and decision logic can be adjusted to make the signal timing scheme closer to the actual traffic demand. This step is used to continuously improve the adaptability and accuracy of the control model, ensure that the public transport priority strategy maintains an appropriate implementation effect in the dynamically changing traffic environment, and balance the right of way for traffic in all directions.

[0189] The technical effect of this solution in this embodiment is that it continuously improves the adaptability and accuracy of the traffic light control system through real-time feedback and dynamic optimization mechanisms. After executing control actions, actual effect data is collected and compared with the expected target. Based on the deviation, the control model parameters are automatically adjusted, enabling the system to continuously learn and improve. This closed-loop optimization design gives the traffic light control model self-improvement capabilities, allowing it to continuously optimize decision-making strategies as traffic conditions change. This ensures the real-time effectiveness of bus priority control and enhances the system's ability to cope with complex traffic environments.

[0190] Figure 9 This is a schematic diagram of the structure of a training device for a traffic light control model provided in an embodiment of this application. Figure 9 As shown, the training device for the traffic light control model includes:

[0191] The first acquisition module 901 is used to acquire multiple first traffic data and multiple second traffic data; wherein, the multiple first traffic data are used to represent the traffic status of multiple intersections within a preset first time period, the end time of the first time period is earlier than the current time, and the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time.

[0192] Module 902 is used to construct a state representation matrix and a reward function based on multiple first traffic data; wherein, the state representation matrix is ​​used to represent the passage priority of multiple buses at multiple intersections.

[0193] Training module 903 is used to train the model based on multiple first traffic data, state representation matrix and reward function to obtain traffic light control model; wherein, reward function is used to guide the model training process so that the action output by traffic light control model prioritizes the passage of multiple buses according to the passage priority, and traffic light control model is used to represent the mapping relationship between traffic state and action of multiple intersections.

[0194] In one possible design, module 902 includes:

[0195] The calculation unit is used to calculate the average speed, queue length, traffic light phase, and bus status based on multiple first traffic data. The average speed and queue length refer to the speed and length of multiple traffic flows at multiple intersections, respectively. The traffic light phase is used to indicate the status of multiple traffic lights at a first time point, which is any time point within a first time period. The bus status is used to indicate the passage priority of multiple buses.

[0196] The feature fusion unit is used to fuse the features of average speed, queue length, traffic light phase and bus status to obtain a state representation matrix.

[0197] The function fitting unit is used to fit a function based on multiple first traffic data to obtain a reward function.

[0198] In one possible design, the function fitting unit includes:

[0199] The calculation component is used to calculate queue density and bus delay based on multiple primary traffic data; where queue density is used to represent the severity of congestion among multiple traffic flows, and bus delay refers to the deviation between the actual arrival time and the preset arrival time of multiple buses.

[0200] The weighted component is used to perform a weighted summation of queue density and bus delay to obtain the reward function.

[0201] In one possible design, training module 903 includes:

[0202] The construction unit is used to construct an action space and an action value function based on multiple first traffic data. The action space is a set of multiple executable actions of multiple traffic lights, and the action value function is used to represent the mapping relationship between multiple executable actions and the value obtained after executing multiple executable actions.

[0203] The model training unit is used to train the model based on the state representation matrix, reward function, action space, and action value function to obtain the traffic light control model.

[0204] In one possible design, the building blocks include:

[0205] A component is constructed to build an initial action space and an action value function based on multiple first traffic data; wherein, the initial action space is the set of all actions of multiple traffic lights.

[0206] The filtering component is used to filter the initial motion space according to multiple preset constraints to obtain the motion space.

[0207] The training device for the traffic light control model provided in this embodiment can execute... Figure 2The technical solution of the training method embodiment for a traffic light control model shown herein, its implementation principle and technical effect are similar to Figure 2 The training method for the traffic light control model shown is similar to that of the example shown, and will not be described in detail here.

[0208] Figure 10 This is a schematic diagram of a traffic light control device provided in an embodiment of this application. Figure 10 As shown, the traffic light control device includes:

[0209] The second acquisition module 1001 is used to acquire multiple second traffic data; wherein the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time.

[0210] Action determination module 1002 is used to input multiple second traffic data into the traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at multiple intersections so that multiple buses at multiple intersections can pass according to traffic priority, and the traffic light control model is based on... Figure 9 The signal light control model was obtained from the training device.

[0211] The action execution module 1003 is used to execute traffic light control actions.

[0212] The traffic light control device provided in this embodiment can perform... Figure 7 and Figure 8 The technical solution of the traffic light control method embodiment shown herein, its implementation principle and technical effects are similar to Figure 7 and Figure 8 The embodiment of the traffic light control method shown is similar and will not be described in detail here.

[0213] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application. Figure 11 As shown, the electronic device 110 includes at least one processor 1101 and a memory 1102. The electronic device 110 also includes a communication component 1103. The processor 1101, the memory 1102, and the communication component 1103 are connected via a bus 1104.

[0214] In the specific implementation process, at least one processor 1101 executes computer execution instructions stored in memory 1102, so that at least one processor 1101 is used to implement a training method for a traffic light control model and a traffic light control method of the above embodiment.

[0215] The specific implementation process of processor 1101 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0216] In the above embodiments, it should be understood that the processor 1101 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0217] The memory 1102 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage.

[0218] Bus 1104 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1104 can be divided into address bus, data bus, control bus, etc. For ease of illustration, bus 1104 in the accompanying drawings of this application is not limited to only one bus or one type of bus.

[0219] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0220] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, these instructions are used to implement a cache optimization method as described in the above embodiments. In the specific implementation of the aforementioned traffic light control model training method and traffic light control method, each module can be implemented as a processor.

[0221] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0222] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0223] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement a training method for a traffic light control model and a traffic light control method as described in the above embodiments.

[0224] The computer program is stored in a readable storage medium, and at least one processor can read the computer program from the readable storage medium and execute the computer program to perform the scheme provided in any of the above embodiments.

[0225] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0226] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for a traffic light control model, characterized in that, include: Acquire multiple first traffic data; wherein the multiple first traffic data are used to represent the traffic status of multiple intersections within a preset first time period, and the end time of the first time period is earlier than the current time; Based on the multiple first traffic data, a state representation matrix and a reward function are constructed; wherein, the state representation matrix is ​​used to represent the passage priority of multiple buses at the multiple intersections; A traffic light control model is obtained by training the model based on the multiple first traffic data, the state representation matrix, and the reward function; wherein, the reward function is used to guide the model training process so that the actions output by the traffic light control model prioritize the passage of the multiple buses according to the traffic priority, and the traffic light control model is used to represent the mapping relationship between the traffic state of the multiple intersections and the actions.

2. The training method for the traffic light control model according to claim 1, characterized in that, The step of constructing a state representation matrix and a reward function based on the plurality of first traffic data includes: Based on the multiple first traffic data, the average speed, queue length, traffic light phase, and bus status are calculated; wherein, the average speed and queue length refer to the speed and length of multiple traffic flows at the multiple intersections, respectively; the traffic light phase is used to indicate the status of the multiple traffic lights at a first time point, which is any time point within the first time period; and the bus status is used to indicate the traffic priority of the multiple buses. The average speed, the queue length, the traffic light phase, and the bus status are fused to obtain the state characterization matrix. The reward function is obtained by fitting a function to the plurality of first traffic data.

3. The training method for the traffic light control model according to claim 2, characterized in that, The step of fitting the reward function based on the plurality of first traffic data includes: Based on the multiple first traffic data, queue density and bus delay are calculated; wherein, queue density is used to represent the congestion severity of the multiple traffic flows, and bus delay refers to the deviation between the actual arrival time and the preset arrival time of the multiple buses; The reward function is obtained by weighted summation of the queue density and the bus delay.

4. The training method for the traffic light control model according to claim 1, characterized in that, The step of training a traffic light control model based on the plurality of first traffic data, the state representation matrix, and the reward function includes: Based on the multiple first traffic data, an action space and an action value function are constructed; wherein, the action space is a set of multiple executable actions of the multiple traffic lights, and the action value function is used to represent the mapping relationship between the multiple executable actions and the value obtained after executing the multiple executable actions; The traffic light control model is obtained by training the model based on the state representation matrix, the reward function, the action space, and the action value function.

5. The training method for the traffic light control model according to claim 4, characterized in that, The step of constructing the action space and action value function based on the plurality of first traffic data includes: Based on the multiple first traffic data, an initial action space and action value function are constructed; wherein, the initial action space is the set of all actions of the multiple traffic lights; The initial action space is filtered according to multiple preset constraints to obtain the action space.

6. A traffic light control method, characterized in that, include: Acquire multiple second traffic data; wherein, the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time; The multiple second traffic data are input into the traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at the multiple intersections so that multiple buses at the multiple intersections can pass according to traffic priority, and the traffic light control model is obtained by the training method of the traffic light control model according to any one of claims 1 to 5; Perform the aforementioned traffic light control action.

7. The traffic light control method according to claim 6, characterized in that, After executing the traffic light control action, the method further includes: Obtain the execution result after performing the traffic light control action; Calculate the deviation between the execution result and the preset expected result; Based on the deviation, the traffic light control model is optimized.

8. A training device for a traffic light control model, characterized in that, The device includes: The first acquisition module is used to acquire multiple first traffic data and multiple second traffic data; wherein, the multiple first traffic data are used to represent the traffic status of multiple intersections within a preset first time period, the end time of the first time period is earlier than the current time, and the multiple second traffic data are used to represent the traffic status of the multiple intersections at the current time. A construction module is used to construct a state representation matrix and a reward function based on the plurality of first traffic data; wherein, the state representation matrix is ​​used to represent the passage priority of multiple buses at the plurality of intersections; The training module is used to train the model based on the multiple first traffic data, the state representation matrix, and the reward function to obtain a traffic light control model; wherein, the reward function is used to guide the model training process so that the actions output by the traffic light control model prioritize the passage of the multiple buses according to the traffic priority, and the traffic light control model is used to represent the mapping relationship between the traffic states of the multiple intersections and the actions.

9. A traffic light control device, characterized in that, The device includes: The second acquisition module is used to acquire multiple second traffic data; wherein the multiple second traffic data are used to represent the traffic status of multiple intersections at the current time; An action determination module is used to input the multiple second traffic data into a traffic light control model to obtain traffic light control actions; wherein, the traffic light control actions are used to control multiple traffic lights at the multiple intersections so that multiple buses at the multiple intersections can pass according to traffic priority, and the traffic light control model is obtained by the training device for the traffic light control model according to claim 8; The action execution module is used to execute the traffic light control action.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; When the processor executes the computer execution instructions stored in the memory, it is used to implement the training method of the traffic light control model as described in any one of claims 1 to 5, or to implement the traffic light control method as described in any one of claims 6 to 7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the training method for the traffic light control model as described in any one of claims 1 to 5, or to implement the traffic light control method as described in any one of claims 6 to 7.

12. A computer program product, characterized in that, The system includes a computer program, which, when executed by a processor, is used to implement the training method for the traffic light control model as described in any one of claims 1 to 5, or to implement the traffic light control method as described in any one of claims 6 to 7.

Images