Self-adaptive traffic light control system based on multi-target vehicle detection and tracking

Abstract

The invention belongs to the technical field of traffic light control, and discloses a self-adaptive traffic light control system based on multi-target vehicle detection and tracking, which comprises a video stream decoding module, a YOLOv5 target detection module, a ByteTrack multi-target tracking module, a WebSocket data transmission module and a data processing and display module. By detecting and tracking the lane vehicles in real time, the traffic flow condition of the current intersection can be accurately obtained, and the traffic light duration can be dynamically adjusted according to real-time data. By means of the characteristic, traffic light control is more accurate, traffic jam and waiting time are effectively reduced, and road passing efficiency is improved. According to the invention, existing camera resources and embedded equipment are used for processing, so that high hardware cost is avoided. Compared with an intelligent traffic light system adopting high-tech means such as laser and ultrasonic waves, the intelligent traffic light system is lower in deployment and maintenance cost and easier to popularize and apply in a wide range.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic light control, and particularly relates to an adaptive traffic light control system based on multi-target vehicle detection and tracking. Background Art

[0002] In the field of traffic signal control, traffic lights, as a key tool for regulating road traffic flow, their intelligent level directly affects road traffic efficiency and traffic safety. Most traditional traffic lights adopt a fixed-duration control scheme, and this static management method is difficult to adapt to the real-time changing traffic flow conditions, easily leading to traffic congestion and resource waste. With the development of technology, some traffic light systems have begun to introduce a time-periodic duration adjustment strategy, which, although improving flexibility to a certain extent, still fails to achieve true intelligence.

[0003] Further attempts include using high-tech means such as lasers and ultrasonic waves to monitor lane vehicle information, and then dynamically adjusting the traffic light duration. For example, some high-end intelligent transportation systems precisely measure vehicle speed, distance and other parameters through laser or ultrasonic devices, and select a preset duration scheme based on this. However, these schemes often rely on expensive hardware devices, increasing the cost of system deployment and maintenance, and restricting their application in a wider range.

[0004] Through the above analysis, the problems and defects existing in the prior art are as follows: (1) Limitations of the fixed-duration and time-periodic duration schemes: Both of these schemes fail to fully consider the changes in real-time traffic flow, resulting in serious congestion during peak traffic hours and waste of green light time during off-peak hours.

[0005] (2) High hardware costs: The high equipment costs of intelligent traffic light systems using high-tech means such as lasers and ultrasonic waves limit their application in economically underdeveloped regions or projects with limited budgets.

[0006] (3) System complexity and maintenance difficulty: The introduction of high-tech devices increases the system complexity and maintenance difficulty, and requires professional technical personnel for regular maintenance and calibration. Summary of the Invention

[0007] Aiming at the problems existing in the prior art, the present invention provides an adaptive traffic light control system based on multi-target vehicle detection and tracking.

[0008] The present invention is implemented as follows. An adaptive traffic light control system based on multi-target vehicle detection and tracking includes: A video stream decoding module, a YOLOv5 target detection module, a ByteTrack multi-target tracking module, a WebSocket data transmission module, and a data processing and display module.

[0009] The video stream decoding module, connected to the YOLOv5 object detection module, is used to process with an efficient video codec library; it supports decoding four RTSP video streams with a resolution of 1080p simultaneously, ensuring smooth decoding of the video stream and providing high-quality image input for subsequent object detection.

[0010] The YOLOv5 object detection module, connected to the ByteTrack multi-object tracking module, is used to deploy the YOLOv5 object detection algorithm in the RKNN environment and perform model conversion using the RKNN-Toolkit tool; it detects vehicle objects from video frames using the YOLOv5 model and outputs the position, category, and confidence information of the objects, providing an accurate detection basis for multi-object tracking.

[0011] The ByteTrack multi-object tracking module, connected to the WebSocket data transmission module, is used to implement the ByteTrack multi-object tracking algorithm by combining strategies such as the Kalman filter and the Hungarian algorithm; it receives the output of the YOLOv5 module, performs multi-object tracking, handles occlusion problems, retains low-confidence detection boxes, and improves the robustness and accuracy of tracking.

[0012] The WebSocket data transmission module, connected to the data processing and display module, is used to establish a real-time communication channel using the WebSocket protocol; it transmits the tracking results output by the ByteTrack module to the client in real time, supporting flexible distribution and display of data.

[0013] The data processing and display module, connected to the WebSocket data transmission module, is used to perform data processing and analysis on the client side and provide a graphical user interface; after receiving the data transmitted by the WebSocket, it calculates traffic flow and vehicle residence time statistics information, and displays the video stream, detection results, and statistical data in real time, supporting user interaction operations.

[0014] Furthermore, the video stream decoding module: Decodes four RTSP video streams with a resolution of 1080p using an efficient video codec library.

[0015] Adopts other libraries or tools that support RTSP video stream decoding, such as GStreamer.

[0016] Furthermore, the YOLOv5 object detection module: Deploys the YOLOv5 object detection algorithm in the RKNN environment, detects the decoded video frames, and outputs the position, category, and confidence information of vehicle objects.

[0017] Use other efficient target detection algorithms, such as Faster R-CNN and SSD, and select a suitable deployment environment based on the characteristics of the algorithm.

[0018] Furthermore, the ByteTrack multi-target tracking module: The ByteTrack algorithm is used to track multiple detected vehicle targets, handle occlusion problems, and maintain the continuity of target trajectories.

[0019] Use other multi-target tracking algorithms, SORT and DeepSORT.

[0020] Furthermore, the WebSocket data transmission module: A real-time communication channel is established between the client and the server through the WebSocket protocol, and the tracking results are transmitted to the client in real time.

[0021] Alternative method: Use other real-time communication protocols, such as MQTT and WebSocket Secure (WSS), to ensure the security and reliability of data transmission.

[0022] Furthermore, the data processing and display module: After the client receives the transmitted data, it performs further processing and analysis to calculate the number of vehicles in the lane, traffic volume, vehicle dwell time, and vehicle speed statistics, and displays the video stream, detection results, and statistical data in real time through a graphical user interface.

[0023] Other data processing and analysis methods, such as machine learning and deep learning, are used to mine more valuable information. At the same time, the design of the GUI can also be customized according to user needs.

[0024] Another object of the present invention is to provide an adaptive traffic light control method based on multi-target vehicle detection and tracking, comprising: Step 1: The video stream decoding module uses an efficient video codec library for processing; it supports decoding four 1080p resolution RTSP video streams at the same time, ensuring smooth decoding of the video stream and providing high-quality image input for subsequent target detection.

[0025] Step 2: Deploy the YOLOv5 target detection algorithm in the RKNN environment through the YOLOv5 target detection module, and use the RKNN-Toolkit tool to convert the model; use the YOLOv5 model to detect vehicle targets from video frames, output the target's location, category, and confidence information, and provide an accurate detection basis for multi-target tracking.

[0026] Step 3: Implement the ByteTrack multi-object tracking algorithm by combining strategies such as the Kalman filter and the Hungarian algorithm in the ByteTrack multi-object tracking module; receive the output of the YOLOv5 module, perform multi-object tracking, handle occlusion problems, retain low-confidence detection boxes, and improve the robustness and accuracy of tracking.

[0027] Step 4: Establish a real-time communication channel using the WebSocket protocol through the WebSocket data transmission module; transmit the tracking results output by the ByteTrack module to the client in real time to support flexible distribution and display of data.

[0028] Step 5: Perform data processing and analysis on the client through the data processing and display module to provide a graphical user interface; after receiving the data transmitted by WebSocket, calculate traffic flow and vehicle detention time statistics, and display the video stream, detection results, and statistical data in real time to support user interaction operations.

[0029] Another object of the present invention is to provide a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the adaptive traffic light control method based on multi-object vehicle detection and tracking.

[0030] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the adaptive traffic light control method based on multi-object vehicle detection and tracking.

[0031] Another object of the present invention is to provide an information data processing terminal for implementing the adaptive traffic light control system based on multi-object vehicle detection and tracking.

[0032] Combined with the above technical solutions and solved technical problems, the advantages and positive effects of the technical solution to be protected by the present invention are as follows: The innovation of the present invention lies in that by using existing camera resources and advanced artificial intelligence and computer vision algorithms, real-time detection and tracking of vehicles in the lane are achieved, and then the optimal traffic light duration is dynamically calculated according to the actual situation of the current intersection. This solution not only avoids high hardware costs but also improves the flexibility and adaptability of the system, which is an effective way to achieve low-cost transformation of intelligent traffic lights.

[0033] The present invention aims to solve the problems existing in the existing traffic light control system, where the fixed-duration and time-periodic-duration schemes cannot adapt to the real-time traffic flow changes, and the high hardware costs brought by high-tech means. By utilizing the existing camera resources and advanced artificial intelligence and computer vision algorithms, it realizes the real-time detection and tracking of lane vehicles, dynamically calculates and adjusts the traffic light duration, so as to improve the road traffic efficiency and traffic safety, and at the same time reduce the deployment and maintenance costs of the intelligent traffic light system.

[0034] The present invention proposes an intelligent traffic light control system based on camera and artificial intelligence technologies. Its core lies in integrating multiple modules such as video stream decoding, object detection, multi-object tracking, real-time data transmission, and data processing and display, in order to achieve the real-time detection and tracking of lane vehicles, and dynamically adjust the traffic light duration according to the real-time traffic flow situation.

[0035] Real-time and accuracy: Through efficient video stream decoding, fast object detection and multi-object tracking algorithms, it realizes the real-time detection and tracking of lane vehicles, ensuring the accuracy and timeliness of data.

[0036] Low cost and easy deployment: By utilizing the existing camera resources and embedded devices, it avoids high hardware costs and reduces the deployment and maintenance difficulties of the intelligent traffic light system.

[0037] Flexibility and scalability: It supports the simultaneous processing of multiple video streams, real-time data transmission and flexible distribution, as well as user interaction operations, providing convenience for the further expansion and application of the system.

[0038] Compared with the existing technology, the present invention has remarkable advantages and positive effects, which are specifically manifested in the following aspects: 1. Real-time adaptation to traffic flow changes: By real-time detecting and tracking lane vehicles, the present invention can accurately obtain the traffic flow situation at the current intersection and dynamically adjust the traffic light duration according to the real-time data. This feature makes the traffic light control more precise, effectively reduces traffic congestion and waiting time, and improves the road traffic efficiency.

[0039] 2. Cost reduction: The present invention uses the existing camera resources and embedded devices for processing, avoiding high hardware costs. Compared with the intelligent traffic light systems using high-tech means such as lasers and ultrasonic waves, the deployment and maintenance costs of the present invention are lower, and it is easier to promote and apply on a wide scale.

[0040] 3. Improvement of system flexibility and scalability: The present invention supports the simultaneous processing of multiple video streams, real-time data transmission and flexible distribution, as well as user interaction operations. These features make the system more flexible, capable of being customized and expanded according to actual needs, and meeting the traffic signal control requirements in different scenarios.

[0041] 4. Enhance traffic safety: By adjusting the traffic light duration in real time, the present invention can better adapt to traffic flow changes and reduce the occurrence of traffic accidents. Meanwhile, the provided data processing and display functions help traffic managers grasp the intersection situation in real time and make decisions promptly, further improving the traffic safety level.

[0042] Promote the construction of smart cities: As an important part of the intelligent transportation system, the present invention can be integrated and coordinated with other smart city applications to jointly promote the urban intelligentization process. By providing real-time and accurate traffic data, the present invention helps optimize urban traffic planning and management, and improves the urban operation efficiency and residents' quality of life.

[0043] Most of the current traffic light control systems on the market adopt fixed-duration or time-period-based duration adjustment strategies, failing to achieve true intelligence and real-time adaptability. By integrating technologies such as video stream decoding, object detection, multi-object tracking, real-time data transmission, and data processing and display, the present invention realizes the dynamic adjustment of traffic light duration based on real-time traffic flow data, filling the technical gap in this field at home and abroad.

[0044] The design of traditional traffic light control systems often relies on fixed-duration or time-period-based adjustment strategies, which to a certain extent reflects a technical bias, that is, it is considered difficult to accurately obtain and process real-time traffic flow data. The technical solution of the present invention successfully overcomes this technical bias by introducing efficient object detection, multi-object tracking, and real-time data transmission technologies, proving that the dynamic adjustment of traffic light duration based on real-time traffic flow data is feasible and effective. This not only improves the intelligence level of the traffic light control system but also provides new ideas and methods for the development of future intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is the structural block diagram of the adaptive traffic light control system based on multi-object vehicle detection and tracking provided by an embodiment of the present invention.

[0046] Figure 2 is the flowchart of the adaptive traffic light control method based on multi-object vehicle detection and tracking provided by an embodiment of the present invention.

[0047] Figure 3 is the detailed flowchart of the adaptive traffic light control method based on multi-object vehicle detection and tracking provided by an embodiment of the present invention.

[0048] Figure 4 is the detection and tracking result diagram provided by an embodiment of the present invention.

[0049] Figure 5 is the graphical user interface provided by an embodiment of the present invention.

[0050] Figure 1 Among them: 1. Video stream decoding module; 2. YOLOv5 object detection module; 3. ByteTrack multi-object tracking module; 4. WebSocket data transmission module; 5. Data processing and display module. Specific implementation manners

[0051] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] As Figure 1 shown, an adaptive traffic light control system based on multi-object vehicle detection and tracking provided by an embodiment of the present invention includes: Video stream decoding module 1, YOLOv5 object detection module 2, ByteTrack multi-object tracking module 3, WebSocket data transmission module 4, and data processing and display module 5.

[0053] The video stream decoding module 1 is connected to the YOLOv5 object detection module 2 and is used for processing by adopting an efficient video codec library; it supports simultaneous decoding of four RTSP video streams with a resolution of 1080p to ensure smooth decoding of the video stream and provide high-quality image input for subsequent object detection.

[0054] The YOLOv5 object detection module 2 is connected to the ByteTrack multi-object tracking module 3 and is used for deploying the YOLOv5 object detection algorithm in the RKNN environment and performing model conversion by using the RKNN-Toolkit tool; the YOLOv5 model is used to detect vehicle targets from video frames and output the position, category and confidence information of the targets, providing an accurate detection basis for multi-object tracking.

[0055] The ByteTrack multi-object tracking module 3 is connected to the WebSocket data transmission module 4 and is used for implementing the ByteTrack multi-object tracking algorithm by combining strategies such as a Kalman filter and a Hungarian algorithm; receiving the output of the YOLOv5 module, performing multi-object tracking, handling occlusion problems, retaining low-confidence detection frames, and improving the robustness and accuracy of tracking.

[0056] The WebSocket data transmission module 4 is connected to the data processing and display module 5 and is used for establishing a real-time communication channel by adopting the WebSocket protocol; the tracking results output by the ByteTrack module are transmitted to the client in real time, supporting flexible distribution and display of data.

[0057] The data processing and display module 5, which is connected to the WebSocket data transmission module 4, is used to process and analyze data on the client side, provide a graphical user interface; after receiving the data transmitted by WebSocket, calculate traffic flow, vehicle residence time statistics, and display the video stream, detection results, and statistical data in real time, and support user interaction operations.

[0058] The video stream decoding module provided by the embodiment of the present invention: Decode four RTSP video streams with a resolution of 1080p using an efficient video codec library.

[0059] Adopt other libraries or tools that support RTSP video stream decoding, such as GStreamer.

[0060] The YOLOv5 object detection module provided by the embodiment of the present invention: Deploy the YOLOv5 object detection algorithm in the RKNN environment, detect the decoded video frames, and output the position, category, and confidence information of vehicle targets.

[0061] Adopt other efficient object detection algorithms, such as Faster R-CNN, SSD, and select a suitable deployment environment according to the characteristics of the algorithm.

[0062] The ByteTrack multi-object tracking module provided by the embodiment of the present invention: Use the ByteTrack algorithm to perform multi-object tracking on the detected vehicle targets, handle occlusion problems, and maintain the continuity of target trajectories.

[0063] Adopt other multi-object tracking algorithms, such as SORT, DeepSORT.

[0064] The WebSocket data transmission module provided by the embodiment of the present invention: Establish a real-time communication channel between the client and the server through the WebSocket protocol, and transmit the tracking results to the client in real time.

[0065] Alternative method: Adopt other real-time communication protocols, such as MQTT, WebSocket Secure (WSS), to ensure the security and reliability of data transmission.

[0066] The data processing and display module provided by the embodiment of the present invention: After receiving the transmitted data on the client side, perform further processing and analysis, calculate the number of vehicles in the lane, traffic flow, vehicle residence time, vehicle speed statistics, and display the video stream, detection results, and statistical data in real time through the graphical user interface.

[0067] Adopt other data processing and analysis methods, such as machine learning and deep learning, to mine more valuable information; at the same time, the design of the GUI can also be customized according to user needs.

[0068] As Figure 2 shown, an adaptive traffic light control method based on multi-target vehicle detection and tracking provided by an embodiment of the present invention includes: S101, process through a video stream decoding module using an efficient video codec library; support simultaneous decoding of four RTSP video streams with a resolution of 1080p to ensure smooth decoding of the video stream and provide high-quality image input for subsequent object detection.

[0069] S102, deploy the YOLOv5 object detection algorithm in the RKNN environment through the YOLOv5 object detection module, and use the RKNN-Toolkit tool for model conversion; use the YOLOv5 model to detect vehicle targets from video frames and output the position, category, and confidence information of the targets to provide an accurate detection basis for multi-target tracking.

[0070] S103, implement the ByteTrack multi-target tracking algorithm through the ByteTrack multi-target tracking module in combination with strategies such as the Kalman filter and the Hungarian algorithm; receive the output of the YOLOv5 module, perform multi-target tracking, handle occlusion problems, retain low-confidence detection boxes, and improve the robustness and accuracy of tracking.

[0071] S104, establish a real-time communication channel through the WebSocket data transmission module using the WebSocket protocol; transmit the tracking results output by the ByteTrack module to the client in real time to support flexible distribution and display of data.

[0072] S105, perform data processing and analysis on the client through the data processing and display module to provide a graphical user interface; after receiving the data transmitted by WebSocket, calculate traffic flow and vehicle detention time statistical information, and display the video stream, detection results, and statistical data in real time to support user interaction operations.

[0073] The adaptive traffic light control method of the present invention first decodes the RTSP video stream through the video stream decoding module using an efficient video codec library. The decoding module supports simultaneous processing of multiple video streams with a resolution of 1080p to ensure the smoothness and stability of decoding. The high-quality video decoding output provides a clear image input for the subsequent object detection module, laying the foundation for accurate detection of vehicle targets.

[0074] The YOLOv5 object detection algorithm was deployed based on the RKNN environment through the YOLOv5 object detection module. During the model deployment process, the RKNN-Toolkit tool was used to optimize and convert the YOLOv5 model, improving the model's running efficiency and compatibility. This module can accurately detect vehicle targets from video frames and output the position, category, and confidence information of the targets, providing reliable input data for the subsequent multi-object tracking module.

[0075] Based on the ByteTrack multi-object tracking algorithm, combined with strategies such as the Kalman filter and the Hungarian algorithm, complex problems such as vehicle occlusion and multi-object separation were addressed. This module receives the results detected by YOLOv5 and performs optimized tracking, while retaining detection boxes with low confidence to improve the robustness of the tracking algorithm. Through multi-object tracking, the system can generate the motion trajectories of vehicles in real time, ensuring the accuracy of traffic flow data.

[0076] To achieve real-time communication, the system established an efficient data transmission channel through the WebSocket protocol. The WebSocket data transmission module is responsible for transmitting the results of the multi-object tracking module to the client in real time, ensuring the rapid update of data in a low-latency environment. The flexibility of this module supports the efficient distribution of data and is suitable for different client requirements and scenario displays.

[0077] The client receives the real-time data transmitted by WebSocket through the data processing and display module, analyzes and calculates the statistical information of vehicle targets, such as traffic flow and vehicle residence time. The client provides an intuitive graphical user interface (GUI) to display the video stream, detection results, and statistical data in real time, and supports user interaction with the system, improving the usability and user experience of the system.

[0078] The system integrates all the above functional modules to form a complete adaptive traffic light control system based on multi-object vehicle detection and tracking. By analyzing and processing traffic flow data, the system can intelligently adjust the switching time of traffic lights, optimize traffic flow distribution, reduce vehicle detention, improve traffic efficiency, and ultimately achieve dynamic and intelligent traffic light control.

[0079] Specific embodiments of the present invention: I. System architecture The system architecture of the present invention mainly consists of a video stream decoding module, a YOLOv5 object detection module, a ByteTrack multi-object tracking module, a WebSocket data transmission module, a data processing and display module, and an embedded control device. These modules work together to achieve real-time detection and tracking of vehicles in the lane and dynamic adjustment of traffic light durations.

[0080] AsFigure 3 As shown in Figure 3 , the following are the implementations of the second step.

[0081] 1. Video stream decoding Implementation method: Use an efficient video codec library (such as FFmpeg) to decode four RTSP video streams with a resolution of 1080p.

[0082] Alternative method: Use other libraries or tools that support RTSP video stream decoding, such as GStreamer, etc.

[0083] 2. Object detection Implementation method: Deploy the YOLOv5 object detection algorithm in the RKNN environment to detect the decoded video frames and output the position, category, and confidence information of vehicle objects.

[0084] Alternative method: Use other efficient object detection algorithms, such as Faster R-CNN, SSD, etc., and select a suitable deployment environment according to the characteristics of the algorithm.

[0085] 3. Multi-object tracking Implementation method: Use the ByteTrack algorithm to perform multi-object tracking on the detected vehicle objects, handle problems such as occlusion, and maintain the continuity of the object trajectories.

[0086] Alternative method: Use other multi-object tracking algorithms, such as SORT, DeepSORT, etc., to improve the accuracy and robustness of tracking.

[0087] 4. Data transmission Implementation method: Establish a real-time communication channel between the client and the server through the WebSocket protocol, and transmit the tracking results (including the number of vehicles, categories, residence time, etc.) to the client in real time.

[0088] Alternative method: Use other real-time communication protocols, such as MQTT, WebSocket Secure (WSS), etc., to ensure the security and reliability of data transmission.

[0089] 5. Data processing and display Implementation method: After receiving the transmitted data on the client side, perform further processing and analysis, calculate statistical information such as the number of vehicles in the lane, traffic flow, vehicle residence time, vehicle speed, etc., and display the video stream, detection results, and statistical data in real time through a graphical user interface (GUI).

[0090] Alternative method: Use other data processing and analysis methods, such as machine learning, deep learning, etc., to mine more valuable information; at the same time, the design of the GUI can also be customized according to user needs.

[0091] 6. Adjustment of Traffic Light Duration Implementation Method: Based on the obtained real-time traffic flow data, use the self-developed control algorithm to calculate the optimal traffic light duration, and perform real-time regulation on the existing traffic light devices through the embedded control device.

[0092] Alternative Method: Adopt other control algorithms or strategies to adjust the traffic light duration, such as fuzzy control, reinforcement learning, etc.; at the same time, the embedded control device can also be selected and replaced according to the actual situation.

[0093] III. Structural Implementation 1. Hardware Structure Camera: Responsible for collecting lane video streams.

[0094] Embedded Device: Equipped with a video stream decoding module, YOLOv5 object detection module, ByteTrack multi-object tracking module, WebSocket data transmission module, and control algorithm, etc.

[0095] Traffic Light Device: Receive the control signal from the embedded device to achieve dynamic adjustment of the traffic light duration.

[0096] 2. Software Structure Operating System: The embedded device runs a real-time operating system (such as RT-Thread, FreeRTOS, etc.) to ensure the real-time performance and stability of the system.

[0097] Application Program: An application program including modules such as video stream decoding, object detection, multi-object tracking, data transmission, data processing and display, and control algorithm.

[0098] Communication Protocol: Adopt the WebSocket protocol for data transmission to ensure the real-time performance and reliability of the data.

[0099] Alternative Implementation Method: 1. Video Stream Decoding: In addition to FFmpeg, other libraries or tools that support RTSP video stream decoding can also be selected, such as GStreamer, etc.

[0100] 2. Object Detection: In addition to YOLOv5, other efficient object detection algorithms can also be selected, such as Faster R-CNN, SSD, etc., and a suitable deployment environment can be selected according to the characteristics of the algorithm.

[0101] 3. Multi-Object Tracking: In addition to ByteTrack, other multi-object tracking algorithms can also be selected, such as SORT, DeepSORT, etc., to improve the accuracy and robustness of tracking.

[0102] 4. Data Transmission: In addition to WebSocket, other real-time communication protocols such as MQTT and WebSocket Secure (WSS) can be selected to ensure the security and reliability of data transmission.

[0103] Traffic Light Duration Adjustment: In addition to the self-developed control algorithm, other control algorithms or strategies can be adopted to adjust the traffic light duration, such as fuzzy control and reinforcement learning.

[0104] As Figure 4 shown, the detection and tracking result graph.

[0105] As Figure 5 shown, the graphical user interface.

[0106] This experiment aims to verify the effectiveness and performance of the adaptive traffic light control system based on multi-object vehicle detection and tracking. The experiment design includes the following key steps: Data Collection: Use cameras to collect traffic video streams at intersections, ensuring that the video streams contain traffic conditions in different time periods (peak, off-peak, low-peak).

[0107] System Deployment: Deploy modules such as video stream decoding, YOLOv5 object detection, ByteTrack multi-object tracking, WebSocket data transmission, and data processing and display on embedded devices.

[0108] Parameter Setting: Set system parameters according to experimental requirements, such as the detection threshold of the YOLOv5 model and the tracking parameters of ByteTrack.

[0109] Real-time Testing: Connect the system to an actual traffic intersection for real-time detection and tracking, and record the process of traffic light duration adjustment and traffic flow changes.

[0110] Data Analysis: Collect experimental data, including vehicle detection accuracy, tracking accuracy, traffic light duration adjustment, etc., and conduct statistical analysis.

[0111] Vehicle Detection Accuracy: Calculate the detection accuracy by comparing the vehicle targets detected by the YOLOv5 model with the actual vehicle targets. Assume that during peak hours, the detection accuracy reaches over 92%, during off-peak hours it reaches over 95%, and during low-peak hours it is close to 97%. Tracking Accuracy: Use the ByteTrack algorithm to track the detected vehicle targets and calculate the tracking accuracy. Assume that the tracking accuracy can still be maintained above 90% under occlusion.

[0112] Traffic light duration adjustment: By comparing the traffic light durations before and after the experiment, evaluate the system's dynamic adaptability to traffic flow. Assume that the system can effectively shorten the red light duration, increase the green light duration, and reduce vehicle waiting time during peak hours; conversely, during off-peak hours, extend the red light duration, reduce the green light duration, and avoid waste of resources.

[0113] Example 1: Adaptive traffic light control during peak hours on urban arterial roads.

[0114] During peak hours of commuting in a certain urban arterial road, the traffic flow is relatively large. Especially during the morning and evening commuting periods, the phenomenon of vehicle congestion at intersections is serious, causing traffic jams. The traditional traffic light control method cannot dynamically adjust the signal switching time according to the real-time traffic flow, resulting in low traffic efficiency.

[0115] 1) Data collection and object detection Deploy high-definition cameras at multiple intersections of the arterial road and connect them to the adaptive traffic light control system of the present invention. The video stream is decoded by an efficient decoding module, and the image data is input into the YOLOv5 object detection module in real time to detect the number, type, and location of vehicles.

[0116] 2) Multi-object tracking and traffic flow analysis The ByteTrack module combines the detection results to perform multi-object tracking on each vehicle and generate the vehicle's movement trajectory data. At the same time, key indicators such as the residence time and traffic flow of vehicles at each intersection are counted.

[0117] 3) Real-time traffic light signal optimization According to the traffic flow analysis results, the system adjusts the switching time of traffic light signals. For example, when the traffic flow in a certain direction increases significantly, extend the green light duration in that direction and shorten the passing time in other directions, thereby alleviating traffic pressure.

[0118] 4) Client data display Managers can monitor the traffic flow distribution, vehicle residence time, and traffic light status at each intersection in real time through the client, and manually intervene or adjust control parameters when necessary.

[0119] Through the deployment of this system, the vehicle passing efficiency on the arterial road during peak hours has increased by about 30%, and the average vehicle waiting time has been reduced by about 40%, effectively alleviating the traffic congestion problem.

[0120] Example 2: Intelligent traffic light management during school arrival and departure times in the school area.

[0121] There is a situation of mixed vehicles and pedestrians on the roads around a certain school during school arrival and departure times. Especially when students pass through the zebra crossing in the morning, the vehicle retention phenomenon is serious, posing a high safety risk.

[0122] 1) Video surveillance and target detection Deploy cameras at the main intersections around the school and connect them to the system of the present invention. After the video stream is decoded, the YOLOv5 model is used to detect vehicle and pedestrian targets, accurately identify the vehicle detention areas and the density of pedestrians passing through the zebra crossing.

[0123] 2) Multi-target tracking and pedestrian priority judgment The ByteTrack module tracks the movement trajectories of vehicles and pedestrians, and calculates the pedestrian density and passing speed near the zebra crossing in real time. When a large number of students are detected passing through the zebra crossing, the system triggers the pedestrian priority mode and extends the red light time to ensure safety.

[0124] 3) Real-time control and data feedback The system automatically adjusts the traffic light switching rules according to the vehicle and pedestrian flows. For example, in areas with dense pedestrians, the traffic light control system extends the time for pedestrians to pass; in other time periods, it resumes the normal vehicle passing mode to reduce the ineffective waiting of vehicles.

[0125] 4) Safety tips and monitoring The client interface displays the real-time monitoring screen and statistical information, and reminds drivers and pedestrians of the changes in traffic signals through voice or text prompts to enhance safety.

[0126] The safety of students crossing the street during school arrival and dismissal hours is significantly improved, and the vehicle detention time caused by the concentrated passage of students is reduced by about 50%. The intelligent adjustment of the system not only meets the safety needs of pedestrians but also improves the vehicle passing efficiency.

[0127] It should be noted that the embodiments of the present invention can be implemented through hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated designed hardware. Those of ordinary skill in the art can understand that the above devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code is provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits of programmable hardware devices such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable logic devices such as field programmable gate arrays, or can be implemented by software executed by various types of processors, or can be implemented by a combination of the above hardware circuits and software, such as firmware.

[0128] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be covered by the protection scope of the present invention.

Claims

1. An adaptive traffic light control system based on multi-object vehicle detection and tracking, characterized in that Including: Video stream decoding module, YOLOv5 object detection module, ByteTrack multi-object tracking module, WebSocket data transmission module, data processing and display module; The video stream decoding module, connected to the YOLOv5 object detection module, is used to process with an efficient video codec library; It supports decoding four RTSP video streams with a resolution of 1080p simultaneously, ensuring smooth decoding of the video stream and providing high-quality image input for subsequent object detection; The YOLOv5 object detection module, connected to the ByteTrack multi-object tracking module, is used to deploy the YOLOv5 object detection algorithm in the RKNN environment and perform model conversion using the RKNN-Toolkit tool; Detect vehicle objects from video frames using the YOLOv5 model and output the position, category, and confidence information of the objects, providing an accurate detection basis for multi-object tracking; The ByteTrack multi-object tracking module, connected to the WebSocket data transmission module, is used to implement the ByteTrack multi-object tracking algorithm by combining strategies such as the Kalman filter and the Hungarian algorithm; Receive the output of the YOLOv5 module, perform multi-object tracking, handle occlusion problems, retain low-confidence detection boxes, and improve the robustness and accuracy of tracking; The WebSocket data transmission module, connected to the data processing and display module, is used to establish a real-time communication channel using the WebSocket protocol; Transmit the tracking results output by the ByteTrack module to the client in real time, supporting flexible distribution and display of data; The data processing and display module, connected to the WebSocket data transmission module, is used to perform data processing and analysis on the client side and provide a graphical user interface; After receiving the data transmitted by WebSocket, calculate traffic flow and vehicle residence time statistics, and display the video stream, detection results, and statistical data in real time, supporting user interaction operations.

2. The adaptive traffic signal control system based on multi-target vehicle detection and tracking according to claim 1, characterized in that, The said video stream decoding module: Decode four RTSP video streams with a resolution of 1080p using an efficient video codec library; Adopt other libraries or tools that support RTSP video stream decoding, such as GStreamer.

3. The adaptive traffic signal control system based on multi-target vehicle detection and tracking according to claim 1, wherein The said YOLOv5 object detection module: Deploy the YOLOv5 object detection algorithm in the RKNN environment, detect the decoded video frames, and output the position, category, and confidence information of vehicle objects; Adopt other efficient object detection algorithms, such as Faster R-CNN and SSD, and select a suitable deployment environment according to the algorithm characteristics.

4. The adaptive traffic signal control system based on multi-target vehicle detection and tracking according to claim 1, characterized in that, The said ByteTrack multi-object tracking module: Use the ByteTrack algorithm to perform multi-object tracking on the detected vehicle objects, handle occlusion problems, and maintain the continuity of object trajectories; Adopt other multi-object tracking algorithms, such as SORT and DeepSORT.

5. The adaptive traffic signal control system based on multi-target vehicle detection and tracking according to claim 1, wherein The said WebSocket data transmission module: Establish a real-time communication channel between the client and the server through the WebSocket protocol and transmit the tracking results to the client in real time; Alternative approach: Adopt other real-time communication protocols, such as MQTT and WebSocket Secure (WSS), to ensure the security and reliability of data transmission.

6. The adaptive traffic light control system based on multi-target vehicle detection and tracking according to claim 1, wherein, The data processing and display module: After receiving the transmitted data at the client side, it performs further processing and analysis, calculates the number of vehicles in the lane, traffic flow, vehicle residence time, and vehicle speed statistics information, and real-time displays the video stream, detection results, and statistical data through the graphical user interface. Adopt other data processing and analysis methods, such as machine learning and deep learning, to mine more valuable information; at the same time, the design of the GUI can also be customized according to user requirements.

7. An adaptive traffic light control method based on multi-object vehicle detection and tracking for implementing the adaptive traffic light control system based on multi-object vehicle detection and tracking according to any one of claims 1-6, characterized in that, The adaptive traffic light control method based on multi-object vehicle detection and tracking includes: Step 1, process through the video stream decoding module using an efficient video codec library; support simultaneous decoding of four RTSP video streams with a resolution of 1080p to ensure smooth decoding of the video stream and provide high-quality image input for subsequent object detection. Step 2, deploy the YOLOv5 object detection algorithm in the RKNN environment through the YOLOv5 object detection module, and use the RKNN-Toolkit tool for model conversion; use the YOLOv5 model to detect vehicle objects from video frames and output the position, category, and confidence information of the objects, providing an accurate detection basis for multi-object tracking. Step 3, implement the ByteTrack multi-object tracking algorithm through the ByteTrack multi-object tracking module by combining strategies such as the Kalman filter and the Hungarian algorithm; receive the output of the YOLOv5 module, perform multi-object tracking, handle occlusion problems, retain low-confidence detection boxes, and improve the robustness and accuracy of tracking. Step 4, establish a real-time communication channel through the WebSocket data transmission module using the WebSocket protocol; transmit the tracking results output by the ByteTrack module to the client in real-time, supporting flexible distribution and display of data. Step 5, perform data processing and analysis at the client through the data processing and display module to provide a graphical user interface; after receiving the data transmitted by WebSocket, calculate traffic flow, vehicle residence time statistics information, and real-time display the video stream, detection results, and statistical data, supporting user interaction operations.

8. A computer device, characterized in that, The computer device includes a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor executes the steps of the adaptive traffic light control method based on multi-object vehicle detection and tracking as claimed in claim 7.

9. A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the adaptive traffic light control method based on multi-object vehicle detection and tracking as claimed in claim 7.

10. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the adaptive traffic light control system based on multi-object vehicle detection and tracking as claimed in any one of claims 1-6.

Images