Traffic signal control method, equipment and medium

Abstract

The invention discloses a traffic signal control method and device and a medium, and relates to the technical field of artificial intelligence traffic planning. The method comprises the following steps: acquiring multi-source data of an intersection; performing trajectory tracking on the multi-source data to obtain traffic flow parameters of the intersection; inputting the traffic flow parameters into a trained reinforcement learning signal control model to obtain a signal control instruction output by the reinforcement learning signal control model; and controlling signal lamps at the intersection to execute the signal control instruction. Therefore, the control response speed of the traffic signal can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence traffic planning, and particularly relates to a traffic signal control method and device and medium. BACKGROUND

[0002] With the development of artificial intelligence and computer vision technology, video detection technology begins to be applied to traffic perception. Existing video-based traffic signal control systems mostly only use video flow instead of coil flow, and the control logic is still essentially inductive control or adaptive control, with long optimization period and unable to cope with sudden and short-term traffic flow impact, and with lagging control response. SUMMARY

[0003] The embodiments of the present application provide a traffic signal control method, device and medium, to solve the technical problem of how to improve the control response speed of traffic signals.

[0004] In a first aspect, the embodiments of the present application provide a traffic signal control method, which comprises: acquiring multi-source data of an intersection; performing trajectory tracking on the multi-source data to obtain traffic flow parameters of the intersection, wherein the traffic flow parameters are used to indicate real-time traffic states of the intersection; inputting the traffic flow parameters into a trained reinforcement learning signal control model to obtain a signal control instruction output by the reinforcement learning signal control model; and controlling a signal lamp of the intersection to execute the signal control instruction.

[0005] In a second aspect, the embodiments of the present application further provide a traffic signal control device, which comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the traffic signal control method of the first aspect.

[0006] In a third aspect, the embodiments of the present application further provide a computer storage medium storing computer executable instructions, and the computer executable instructions are executed to implement the traffic signal control method of the first aspect.

[0007] The traffic signal control method, device and medium provided by the embodiments of the present application have the following beneficial effects: In the embodiment of the present application, the traffic flow parameters for indicating the real-time traffic state of the intersection can be obtained by trajectory tracking on the multi-source data of the intersection, laying a foundation for rapid response. Then the traffic flow parameters can be input into the trained reinforcement learning signal control model to obtain a signal control instruction, and finally the signal light can be controlled to execute the signal control instruction. In this way, the reinforcement learning signal control model can output an optimal or near-optimal signal control instruction within milliseconds based on the current real-time traffic state of the intersection, so that the control response speed of the traffic signal can be improved, and the decision-making can be intelligentized and real-time. BRIEF DESCRIPTION OF DRAWINGS

[0008] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a traffic signal control method provided by an embodiment of the present application; Figure 2 A structural schematic diagram of a traffic signal control device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0009] To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making any creative efforts fall within the scope of protection of the present application.

[0010] The embodiments of the present application provide a traffic signal control scheme, and the technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings.

[0011] Figure 1 A flowchart of a traffic signal control method provided by an embodiment of the present application. As shown in Figure 1 The traffic signal control method provided by an embodiment of the present application specifically includes the following steps: Step 101, obtaining multi-source data of an intersection.

[0012] In the embodiment of the present application, the multi-source data of the intersection can be obtained, which is data of different sources based on monitoring the traffic conditions of the intersection. In this way, comprehensive traffic information of the intersection can be obtained, which facilitates subsequent control of the traffic signal.

[0013] In step 102, trajectory tracking is performed on the multi-source data to obtain a traffic flow parameter of the intersection, wherein the traffic flow parameter is used to indicate a real-time traffic state of the intersection.

[0014] In the embodiments of the present application, trajectory tracking can be performed on the multi-source data to obtain a traffic flow parameter of the intersection, so as to provide a data basis for real-time signal control.

[0015] In step 103, the traffic flow parameter is input into a trained reinforcement learning signal control model to obtain a signal control instruction output by the reinforcement learning signal control model.

[0016] In the embodiments of the present application, the reinforcement learning signal control model has an agent as its core. The agent is an entity in the reinforcement learning framework that can perceive the state of the environment, perform actions, and obtain rewards from the environment. In the embodiments of the present application, the agent can select a control instruction of a signal lamp according to the traffic state of the intersection, and maximize the cumulative reward through continuous attempts. It learns the optimal control strategy by interacting with the environment (i.e., the traffic flow of the intersection).

[0017] In the embodiments of the present application, the traffic flow parameter can be processed by using the reinforcement learning signal control model. In this way, the reinforcement learning signal control model can output an optimal or nearly optimal signal control instruction within milliseconds based on the real-time traffic state of the current intersection, so as to improve the response speed of signal control. Moreover, the signal timing is no longer fixed, but is dynamically generated based on real-time traffic demand, and even personalized priority passing services can be provided for special vehicles (ambulances, fire engines), public transport priority, and pedestrian crossing, so as to realize dynamic and personalized control.

[0018] In step 104, the signal lamp of the intersection is controlled to execute the signal control instruction.

[0019] In actual application, the generated signal control instruction (such as "eastern import straight phase extension 10 seconds") can be issued to the signal control machine through a TCP / IP protocol or a special serial port protocol. The signal control machine can execute the signal control instruction to drive the signal lamp to execute based on the signal control instruction. In this way, the signal of the intersection can be optimized, the possible problems of the intersection can be solved, and the sudden and short-term traffic flow impact can be coped with.

[0020] In the embodiments of the present application, the above steps can form an efficient automated closed loop: perception → obtaining traffic flow parameters → decision making using a reinforcement learning signal control model → execution → re-perception → … This cycle can run quickly (e.g., several decision-making cycles can be completed every second). While the traditional scheme may adjust the scheme only once every few minutes, or only make a response once under certain triggering conditions (such as the inductive coil being triggered). At the same time, high-frequency iteration means that the signal can be continuously fine-tuned to closely follow the changes in traffic flow, and even suppress impending congestion in advance. The reinforcement learning signal control model described above has certain short-term prediction capabilities. The reinforcement learning signal control model can learn that “when a certain traffic flow pattern appears, congestion is likely to occur in the next 20 seconds”. Therefore, it can adjust the signal phase in advance before the congestion actually occurs, thereby killing the congestion in the cradle.

[0021] In the embodiments of the present application, the multi-source data of the intersection can be tracked by trajectory tracking, and then traffic flow parameters indicating the real-time traffic state of the intersection can be obtained, laying the foundation for rapid response. The traffic flow parameters described above can be input into the trained reinforcement learning signal control model to obtain signal control instructions, and finally the signal light can be controlled to execute the signal control instructions. In this way, the reinforcement learning signal control model can output an optimal or near-optimal signal control instruction within milliseconds based on the current real-time traffic state of the intersection, and therefore, the control response speed of the traffic signal can be improved, and the decision-making can be intelligent and real-time.

[0022] In one possible implementation, the multi-source data of the intersection is obtained by: obtaining high-resolution video streams of the intersection corresponding to the high-definition network camera; obtaining position information of traffic participants corresponding to the intersection collected by the millimeter wave radar; obtaining 3D point cloud data corresponding to the intersection collected by the laser radar; spatiotemporal fusion of the high-resolution video streams, the position information, and the 3D point cloud data is performed by a spatiotemporal synchronization calibration technique to obtain multi-source data of the intersection after spatiotemporal fusion.

[0023] In actual applications, various data collected by sensors deployed in each direction of the intersection can be obtained. The sensors include cameras (such as high-definition network cameras), millimeter wave radars, and some laser radars that may be deployed at the intersection, without limitation.

[0024] In the above embodiment, the high-definition network camera is deployed at the high point of the intersection, covering the intersection entrance, exit, turning area and pedestrian crossing, and can be used to obtain high-resolution video streams. The millimeter wave radar can be deployed at the same site as the camera for accurate measurement of the position information of traffic participants, i.e., the position, speed and distance of motor vehicles, non-motor vehicles and pedestrians, which can make up for the lack of video perception in bad weather (rain, fog, night). The laser radar is an optional item in practical application and can be selected for key and complex intersections. It can be used to generate high-precision 3D point cloud data that can accurately outline the contours of traffic participants and solve the problem of vehicle occlusion. After obtaining the above various data, a unified, redundant and reliable perception field can be formed by using a space-time synchronous calibration technology to fuse data from different sources. In this way, high-precision multi-sensor data fusion can be achieved, dynamic environment adaptability can be supported, system real-time performance and robustness can be improved, and hardware dependence and cost can be reduced. The space-time synchronous calibration technology is a key technology for multi-sensor systems, which aims to simultaneously calibrate the spatial geometric relationship (external parameter) and time delay parameter between sensors to ensure the accuracy and real-time performance of data fusion.

[0025] In practical application, in order to realize the control method of the traffic signal, the existing camera stand can be upgraded by adding an edge computing server and algorithm software, without the need for large-scale road construction and coil burying. Therefore, the method provided in the embodiments of the present application has lower overall deployment cost than the traditional adaptive system, and is more convenient to maintain.

[0026] In one possible implementation, the trajectory tracking on the multi-source data to obtain the traffic flow parameters of the intersection includes: trajectory tracking on the multi-source data to obtain trajectory data corresponding to traffic participants of the intersection; parameter extraction on the trajectory data to obtain the traffic flow parameters of the intersection.

[0027] In the above embodiment, the multi-source data can be subjected to trajectory tracking to obtain trajectory data corresponding to traffic participants of the intersection. Then, the traffic flow parameters of the intersection can be obtained by parameter extraction on the trajectory data. The traffic participants can include motor vehicles (such as buses and trucks), non-motor vehicles (such as bicycles and electric vehicles), pedestrians, etc. In practical application, the traffic flow parameters can include traffic volume, queue length, headway, time occupancy, space occupancy, average speed, travel time, lane density, etc., without limitation. In practical application, the traffic flow parameters can be obtained by parameter extraction on the trajectory data by using a traffic flow parameter extraction algorithm, which can avoid missing information.

[0028] In the prior art, it is difficult to accurately detect various traffic participants such as motor vehicles, non-motor vehicles and pedestrians at the same time. However, the method provided by the present application can not only accurately identify various traffic participants and solve the problem of single perception dimension in the prior art, but also solve the problem that few systems can effectively detect and give priority to the traffic demand of pedestrians and non-motor vehicles, pay attention to vulnerable groups and improve the satisfaction of residents.

[0029] In one possible implementation, the trajectory tracking of the multi-source data to obtain the trajectory data corresponding to the traffic participants at the intersection comprises: inputting the multi-source data into a target detection model to detect and classify the traffic participants at the intersection; adopting a tracking model to generate a globally unique identifier for each traffic participant and realize continuous trajectory tracking of the traffic participant within the intersection; obtaining the trajectory data corresponding to the traffic participant through continuous trajectory tracking of the traffic participant.

[0030] In actual application, the target detection model can be an improved YOLOv7 or DETR deep learning model, which can detect and classify motor vehicles (passenger cars and trucks), non-motor vehicles (bicycles and electric vehicles) and pedestrians in real time. In actual application, the tracking model can be a DeepSORT or a Transformer-based tracker, which can generate a globally unique identifier (such as ID) for each traffic participant and realize continuous trajectory tracking of the traffic participant within the entire intersection. In the case of continuous trajectory tracking of the traffic participant, the corresponding trajectory data can be obtained.

[0031] In one possible implementation, after the trajectory tracking of the multi-source data to obtain the traffic flow parameters at the intersection, the method further comprises: adopting a behavior recognition model to recognize the trajectory data and determine whether a traffic event occurs at the intersection; in the case that a traffic event occurs at the intersection, executing a solution corresponding to the traffic event according to the category of the traffic event.

[0032] In the above embodiment, after obtaining the traffic flow parameters based on the trajectory data, the behavior recognition model can be used to recognize the trajectory data to determine whether a traffic event occurs at the intersection. In actual application, the traffic event can be an abnormal parking, reverse driving, congestion, pedestrian running a red light, vehicle occupying a non-motor vehicle lane and the like. In this way, resources can be saved, and the multi-source data does not need to be processed additionally. In addition, the traffic event on the road can be monitored, which is helpful to ensure safety.

[0033] In practical applications, motion features (such as speed, acceleration, and trajectory) and spatial features (such as location and relationship with traffic facilities) of each traffic participant can be extracted from trajectory data. Then, a trained behavior recognition model (such as one based on Long Short-Term Memory (LSTM) or spatiotemporal graph convolutional networks) is used to classify the behavior of each traffic participant, for example: normal driving, parking, driving against traffic, running a red light, etc. Among these, behaviors can be judged according to predefined rules or models to detect abnormal events. For example: abnormal parking refers to a vehicle stopping in a non-permitted parking area for more than a certain period of time; driving against traffic refers to a vehicle moving in the opposite direction to the prescribed direction; congestion refers to a situation where the average speed of vehicles is below a threshold and the vehicle density is above a threshold within a certain area, and this situation persists for a period of time. In practical applications, end-to-end behavior recognition models can be used, directly inputting multi-source data and outputting the type and location of traffic events. Alternatively, appearance features can be extracted from video frames in multi-source data, and trajectory data can be used simultaneously, fusing the two for behavior recognition. For example, trajectories can be used to locate regions of interest, and then video features of those regions can be extracted for behavior recognition. The specific method for determining traffic events is not specifically limited.

[0034] When a traffic incident is identified at an intersection, a corresponding solution can be implemented based on the incident's category. First, multiple detection results for the same incident can be verified to avoid false alarms. For example, this can be done through multi-camera perspectives or sensor fusion to confirm the incident. Then, events from different detection points are aggregated to avoid duplicate alarms. For example, abnormal parking of the same vehicle might be captured by multiple cameras and needs to be merged into a single event. Next, the traffic incident can be processed according to its category. For example, driving against traffic and pedestrians running red lights could immediately cause accidents, and the nearest traffic police or patrol car can be automatically notified via SMS, app push notifications, etc. For events affecting traffic flow (such as congestion), variable message signs can be used to issue warnings to other drivers. For vehicles with abnormal parking or driving against traffic, license plates can be recorded and reported. Finally, traffic incidents can be stored in a database, and the processing status can be recorded. This ensures the safety of traffic participants while improving the speed of traffic incident identification, shortening incident response time, and achieving refined management.

[0035] In one possible implementation, the construction process of the reinforcement learning signal control model is as follows: A state space is constructed based on the real-time traffic status information of the intersection; An action space is constructed based on the control commands given to the traffic lights at the intersection; The reward function is constructed based on a multi-objective optimization function, wherein the reward function aims to simultaneously minimize global vehicle total delay, total queue length, and number of stops, and maximize traffic efficiency. The deep reinforcement learning algorithm is used to pre-train the reinforcement learning signal control model in a simulation environment, and then the model is migrated to the intersection for online fine-tuning.

[0036] In the above embodiment, the state space can be defined as real-time traffic state information extracted from the holographic perception layer, which is a high-dimensional vector including: the number of vehicles in each lane queue, the number of pedestrians / non-motor vehicles waiting for traffic in each direction, the average waiting time of vehicles, the upstream vehicle speed and density, the current signal phase state, the historical delay index, etc. The historical delay index refers to the historical congestion of the intersection. If the intersection is often congested, the value of the historical delay index is larger. Through the above data, the reinforcement learning signal control model can better understand the traffic conditions.

[0037] In the above embodiment, the action space can be defined as the control instruction of the signal light, including: extending the current phase, shortening the current phase, and switching to the next specified phase. The action space is simple and clear, and only involves three operations, which helps to reduce the complexity of actions. It should be noted that in order to avoid frequent switching, a minimum green light time needs to be set to ensure safety, i.e. to prevent the signal light from changing too fast and causing safety hazards such as insufficient pedestrian crossing time.

[0038] In the above embodiment, the reward function is the core of the optimization of the reinforcement learning signal control model, which can be designed as a multi-objective optimization function, aiming to simultaneously minimize global vehicle total delay, total queue length, and number of stops, and maximize traffic efficiency. For example, the reward function R can be designed as: R =-α * (total delay) - β * (total queue length) - γ * (number of stops) + δ * (number of passing vehicles * weight) Wherein, α, β, γ, δ are adjustable weight coefficients, which can be adjusted according to the optimization emphasis of different intersections.

[0039] In this way, negative rewards (penalties) can be used to minimize undesirable factors, and positive rewards can be used to maximize traffic efficiency. At the same time, the weight coefficients allow for adjustment according to the needs of different intersections, which can increase the flexibility of the model.

[0040] In practical applications, a deep reinforcement learning algorithm such as Proximal Policy Optimization (PPO) or Deep Deterministic Policy Gradient (DDPG) can be used to train and online learn the reinforcement learning signal control model. The Proximal Policy Optimization is a policy gradient-based reinforcement learning algorithm that limits the policy update amplitude to ensure training stability. The Deep Deterministic Policy Gradient is a model-free policy algorithm for continuous action space, which combines the ideas of deterministic policy gradient and deep Q network.

[0041] Because the real-world training cost is high and dangerous, the reinforcement learning signal control model can be first pre-trained in a virtual environment generated by traffic simulation software such as SUMO and Vissim, which can save real-world costs and allow the model to explore dangerous scenarios on a large scale to accelerate convergence. Then, it is migrated to the real intersection for online fine-tuning to adapt the model to changes in the real environment (such as sensor errors and traffic randomness) and improve robustness.

[0042] In one possible implementation, the inputting the traffic flow parameters into the trained reinforcement learning signal control model to obtain the signal control instruction output by the reinforcement learning signal control model comprises: receiving traffic state information transmitted by adjacent intersections of the intersection; inputting the traffic flow parameters and the traffic state information into the trained reinforcement learning signal control model to obtain the signal control instruction output by the reinforcement learning signal control model.

[0043] In practical applications, the above traffic signal control method can not only be independently applied to a single intersection, but also can realize regional signal coordination control, that is, connecting the reinforcement learning signal control models (agents) of multiple intersections through a cloud platform to realize regional coordination optimization (Multi-agent RL). Regional coordination optimization refers to a reinforcement learning method in which multiple agents learn and optimize strategies to achieve global goals through cooperation, competition, or interaction in the same environment. In Multi-agent RL, each agent has its own state, behavior, and reward function, and the agents learn and make decisions through observation, interaction, and cooperation. This method needs to handle the interaction and cooperation between agents and how to learn and optimize global strategies. Multi-agent RL has a way of implementation, centralized training, decentralized execution (CTDE): during training, global information can be accessed, and during execution, each agent can only access local information. Each intersection still has a local agent that makes quick decisions in real time based on the state of the intersection, which can ensure the response speed and reliability of the system (even if the network is interrupted, a single intersection can still operate independently). When training the above agents, the training method is a multi-agent reinforcement learning algorithm such as MAPPO or MADDPG, rather than a single-agent PPO or DDPG.

[0044] In the above embodiment, the state input of the agent of each intersection is no longer limited to the local, but also includes the traffic state information of the adjacent intersection, for example, the number of vehicles queuing at the upstream intersection (predicting the upcoming traffic flow), the congestion state of the downstream intersection (avoiding traffic overflow), the current phase and duration of the adjacent intersection, etc. At this time, the signal control instruction of the signal light of the intersection can be determined based on the traffic state information of the adjacent intersection and the traffic flow parameters of the intersection itself. In this way, the agent not only learns to optimize local traffic, but also learns how to coordinate with upstream and downstream neighbors to achieve regional-level optimization. For example, it may learn that "sometimes sacrificing a little local efficiency to let the incoming vehicle fleet pass quickly can ultimately benefit everyone."

[0045] At the same time, in this cooperative control scenario, the global coordinator of the cloud platform can have global information and optimize the overall goal rather than the local goal. The cloud platform coordinator can influence the decision-making of the intersection agents in the following ways: coordinating the green light start time between adjacent intersections to form a green wave; adjusting the signal cycle of related intersections to avoid congestion caused by mismatched cycles; coordinating bus priority, emergency vehicle passage, and other strategies at the regional level. Communication and information sharing mechanisms can also be established between intersections and the cloud platform, for example, each intersection regularly uploads state vectors to the cloud platform, and the cloud platform distributes coordination parameters or strategy suggestions to the downstream, etc., without limitation.

[0046] In practical applications, some typical application scenarios are as follows: (1) The main road into the city is under great pressure during the morning peak, and the main road out of the city is under great pressure during the evening peak. At this time, the cloud platform detects the tidal flow mode, can adjust the phase time and phase sequence of the related intersections, realizes the "green wave" coordination at the regional level, and improves the traffic efficiency of the main road. (2) Intersection A is congested and is about to spread to adjacent intersections B and C. At this time, the cloud platform predicts the congestion diffusion path and can instruct upstream intersections B and C to extend the red light appropriately to limit vehicle access to the congestion area and provide more emptying time for the congestion area A.

[0047] (3) Emergency vehicles need to have priority, such as ambulances that need to quickly pass through multiple intersections. For example, the cloud platform obtains the real-time position and predetermined route of the ambulance, and can adjust the signal phase of the intersection along the way in advance to form a "green channel". The intelligent agent of each intersection receives the priority instruction and adjusts the local phase sequence to ensure that the emergency vehicle has priority.

[0048] In this way, while maintaining the fast response capability of the local intelligent agent, the global perspective is introduced to coordinate the decisions of each party, avoiding the suboptimal situation of "every man for himself".

[0049] In one possible implementation, after the control of the signal lamp of the intersection executing the signal control instruction, the method further comprises: monitoring the traffic state change after the signal lamp of the intersection executing the signal control instruction executing action; determining a reward value according to the traffic state change; feeding back the reward value to the reinforcement learning signal control model to realize online adaptation and continuous optimization of the reinforcement learning signal control model.

[0050] In practical applications, the reinforcement learning model has online learning capability and can continuously adapt to long-term changes in traffic flow patterns to continuously improve performance. Online learning means that the model is updated while running in the real environment, rather than being fixed after only training in simulation. Therefore, after controlling the signal lamp of the intersection to execute the signal control instruction, the traffic state change after the execution action can be continuously monitored, and the reward value can be fed back to the reinforcement learning model to realize online adaptation and continuous optimization of the reinforcement learning signal control model.

[0051] In real-world applications, traffic state changes refer to the changes in traffic conditions at the intersection and its surrounding area after the traffic signal executes the signal control instruction (e.g., extending the current phase, shortening the current phase, switching to the next phase). These changes can include: the increase or decrease in the number of vehicles in each lane, the change in the average or total time that vehicles wait at the intersection, the number of vehicles that pass through the intersection in a time step, whether the speed or density of upstream vehicles changes, the change in the number of pedestrians and non-motor vehicles waiting, etc.

[0052] The reward value is a comprehensive evaluation of the above traffic state changes, i.e., the value of the reward function. This value is the direct basis for the reinforcement learning signal control model to learn and optimize. The design of the reward function aims to evaluate the good or bad of the action, i.e., whether the action has a positive or negative impact on the traffic conditions. For example, the agent decides to extend the current phase by 10 seconds when there are still many vehicles in the north-south straight phase. The traffic state changes (monitored results) include positive changes, such as the basic emptying of the north-south queued vehicles, and the passing of 20 vehicles; and negative changes, such as the increase of the east-west vehicle waiting time by about 200 seconds and the increase of the east-west queued length by 8 vehicles. Assume a weight: α = 0.01, β = 0.1, γ = 0.5, δ = 0.1.

[0053] The reward value R = -0.01 * (200) - 0.1 * (8) - 0.5 * (15) + 0.1 * (20) = -8.3 This negative reward value (-8.3) is a clear "effect" signal that tells the reinforcement learning signal control model that "your decision to extend the phase is a bad decision from a global perspective, as it helps the north-south direction but causes more harm to the east-west direction, leading to a decrease in overall efficiency." The reinforcement learning signal control model can continuously adjust its strategy (i.e., the mapping from state to action) based on these feedback (reward value and new state) to make better decisions in the future.

[0054] In real-world applications, an updated specific process is as follows: the reinforcement learning signal control model (agent) selects an action A t based on the current state S t (e.g., extending the green light by 5 seconds). The environment (intersection) changes the traffic state from S t to S t under the influence of the action A {t+1} and produces a reward value R t(Calculated by the reward function). The agent stores the experience (S_t, A_t, R_t, S_{t+1}) into the memory. The agent uses these experience data (possibly a batch of data) to update the model parameters, thereby improving the policy. Through a large number of such interactions, the agent gradually learns what actions should be taken in what state to obtain the maximum long-term cumulative reward.

[0055] It should be noted that, during online learning, due to the complexity of the real environment, model updating needs to be cautious to avoid unstable policies from causing adverse effects on actual traffic. Some safety mechanisms can be used, such as limiting the amplitude of policy updating, or using a combination of offline learning and online fine-tuning.

[0056] The above is the method embodiment of the present application. Based on the same inventive concept, the embodiments of the present application also provide a deep learning-based city intersection holographic perception and intelligent signal cooperative control system, which comprises a holographic perception layer, an edge computing layer, an intelligent decision-making layer, and a cooperative execution layer.

[0057] The holographic perception layer is composed of a plurality of high-performance sensor units deployed in each direction of the intersection. Each sensor unit comprises: High-definition network camera: deployed at a high point, covering the intersection entrance, exit, turning area, and pedestrian crossing, for obtaining high-resolution video streams, mainly for target detection, classification, and tracking.

[0058] Millimeter wave radar: deployed at the same site as the camera, for accurately measuring the position, speed, and distance of the target under all-weather conditions, making up for the lack of video perception in bad weather (rain, fog, night).

[0059] Laser radar (optional): optional at key complex intersections, for generating high-precision 3D point cloud data, accurately outlining the contours of traffic participants, and solving the problem of vehicle occlusion.

[0060] The above holographic perception layer can fuse the data of different sensors through spatiotemporal synchronization calibration technology to form a unified, redundant, and reliable perception field.

[0061] In actual application, the edge computing layer is composed of edge computing servers deployed near the intersection. It has a built-in holographic perception algorithm module for processing the raw data of the perception layer. The core algorithms include: (1) Multi-target detection and tracking algorithm: based on improved YOLOv7 or DETR and other deep learning models, real-time detection and classification of motor vehicles (passenger cars, trucks), non-motor vehicles (bicycles, electric vehicles), pedestrians and other targets. Using DeepSORT or a tracker based on Transformer, a globally unique ID is generated for each target, and continuous trajectory tracking of the target within the entire intersection range is achieved.

[0062] (2) Traffic flow parameter extraction algorithm: based on the trajectory data of the target, real-time calculation of micro and macro parameters including: traffic flow, queue length, headway, time occupancy, space occupancy, average speed, travel time, lane density, etc.

[0063] (3) Traffic event detection algorithm: based on a behavior recognition model, real-time detection of abnormal parking, reverse driving, congestion, pedestrian running a red light, and vehicles occupying non-motor vehicle lanes.

[0064] (4) Signal light state recognition algorithm: through a small CNN network, real-time recognition of the red, green and yellow states of the signal lights in each direction of the current intersection, as feedback input for system decision-making, to ensure a closed loop for system state awareness.

[0065] In actual application, the intelligent decision-making layer can run on an edge computing server or a regional cloud control platform, and the core is a reinforcement learning signal control model. The construction process of the reinforcement learning signal control model is as described in the above method embodiment, which will not be repeated here.

[0066] In actual application, the collaborative execution layer includes a network protocol converter and a signal control machine. The signal control instructions generated by the above edge computing server can be issued to the signal control machine through TCP / IP protocol or a special serial port protocol. The signal control machine executes the signal control instructions to drive the signal light switching. At the same time, the system state and control results are recorded in real time and fed back to the decision-making layer for continuous learning and optimization of the reinforcement learning signal control model.

[0067] Based on the same inventive concept as the above method embodiment, the embodiments of the present application also provide a traffic signal control device, the structure of which is as shown in Figure 2 .

[0068] Figure 2 A traffic signal control device provided by the embodiments of the present application has an internal structure as shown in Figure 2 . The device includes: at least one processor 201; and a memory 202 in communication connection with the at least one processor; The memory 202 stores instructions executable by the at least one processor 201, and the at least one processor 201 executes the instructions to enable the at least one processor 201 to perform the traffic signal control method described above.

[0069] Some embodiments of the present application provide a non-transitory computer storage medium corresponding to Figure 1 that stores computer executable instructions configured to perform the traffic signal control method described above.

[0070] Each of the embodiments of the present application is described in a progressive manner, and the same or similar parts of each of the embodiments can be referred to each other. Each of the embodiments mainly describes the difference from other embodiments. In particular, the embodiments of the Internet of Things device and medium are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0071] The system and medium provided by the embodiments of the present application are one-to-one corresponding to the method, and therefore, the system and medium have similar beneficial technical effects to the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be described here.

[0072] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one flow or multiple flows and / or blocks

[0074] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0076] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0077] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.

[0078] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition provided herein, computer readable media excludes transitory media, such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0080] The above description is merely illustrative of the application, and not restrictive. Various modifications and changes can become apparent to those skilled in the art. Incorporating any modification, equivalent substitution, improvement, etc. within the spirit and principle of the application, shall be included in the scope of the claims of the application.

Claims

1. A method for controlling traffic signals, characterized in that, include: Obtain multi-source data from the intersection; Trajectory tracking is performed on the multi-source data to obtain traffic flow parameters of the intersection, wherein the traffic flow parameters are used to indicate the real-time traffic status of the intersection; The traffic flow parameters are input into the trained reinforcement learning signal control model to obtain the signal control commands output by the reinforcement learning signal control model. The traffic lights at the intersection are controlled to execute the signal control commands.

2. The method according to claim 1, characterized in that, The acquisition of multi-source data at the intersection includes: Acquire high-resolution video streams from high-definition network cameras corresponding to the intersections; Obtain the location information of traffic participants corresponding to the intersection collected by millimeter-wave radar; Acquire the 3D point cloud data corresponding to the intersection collected by the lidar; Using spatiotemporal synchronization calibration technology, the high-resolution video stream, the location information, and the 3D point cloud data are spatiotemporally fused to obtain multi-source data of the intersection after spatiotemporal fusion.

3. The method according to claim 1, characterized in that, The step of tracking the multi-source data to obtain the traffic flow parameters of the intersection includes: Trajectory tracking is performed on the multi-source data to obtain trajectory data corresponding to traffic participants at the intersection; By extracting parameters from the trajectory data, the traffic flow parameters of the intersection are obtained.

4. The method according to claim 3, characterized in that, The step of tracking the trajectory of the multi-source data to obtain the trajectory data corresponding to the traffic participants at the intersection includes: The multi-source data is input into the target detection model to detect and classify traffic participants at the intersection; A tracking model is used to generate a globally unique identifier for each traffic participant and to achieve continuous trajectory tracking of the traffic participants within the intersection area; By continuously tracking the trajectories of the traffic participants, the trajectory data corresponding to the traffic participants is obtained.

5. The method according to claim 3, characterized in that, After performing trajectory tracking on the multi-source data to obtain the traffic flow parameters of the intersection, the method further includes: A behavior recognition model is used to identify the multi-source data to determine whether a traffic incident has occurred at the intersection; In the event of a traffic incident at the intersection, a solution corresponding to the traffic incident is executed according to the category of the traffic incident.

6. The method according to claim 1, characterized in that, The construction process of the reinforcement learning signal control model is as follows: A state space is constructed based on the real-time traffic status information of the intersection; An action space is constructed based on the control commands given to the traffic lights at the intersection; Based on a multi-objective optimization function, a reward function is constructed, wherein the purpose of the reward function is to simultaneously minimize the total global vehicle delay, total queue length, and number of stops, while maximizing traffic efficiency. Based on deep reinforcement learning algorithms, the reinforcement learning signal control model is pre-trained in a simulation environment and then transferred to the intersection for online fine-tuning.

7. The method according to claim 1, characterized in that, The step of inputting the traffic flow parameters into the trained reinforcement learning signal control model to obtain the signal control commands output by the reinforcement learning signal control model includes: Receive traffic status information transmitted from adjacent intersections of the intersection; The traffic flow parameters and traffic state information are input into the trained reinforcement learning signal control model to obtain the signal control commands output by the reinforcement learning signal control model.

8. The method according to claim 1, characterized in that, After the traffic lights at the intersection execute the signal control command, the method further includes: Monitor the changes in traffic status at the intersection after the traffic lights execute the signal control commands; The reward value is determined based on the changes in traffic conditions; The reward value is fed back to the reinforcement learning signal control model to achieve online adaptation and continuous optimization of the reinforcement learning signal control model.

9. A traffic signal control device, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a traffic signal control method as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, a traffic signal control method as described in any one of claims 1-8 is implemented.

Images