Underground garage congestion detection method based on target tracking

By using RTSP and YOLOv5 models in underground garages for real-time object detection and filter technology target tracking, combined with custom congestion index calculation, the problems of large amount of congestion detection and environmental complexity in underground garages are solved, real-time and accurate congestion monitoring and index calculation reduce hardware costs.

CN119091613APending Publication Date: 2024-12-06SHANGHAI INTELLIGENT & CONNECTED VEHICLE R & D CENTER CO LTD +1

Patent Information

Application Number
CN202411070815.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When performing congestion detection in indoor underground garages, the calculation volume is large and difficult to adapt to complex environments, resulting in processor overload and traditional congestion index calculation methods that are not applicable.

Method used

RTSP is used to obtain video data of underground garages, use YOLOv5 model to perform real-time object detection and frame skipping processing, combine filter technology to improve target tracking accuracy, and design a custom congestion index calculation method to consider the complexity of underground garage environment and traffic changes .

Benefits of technology

Real-time object detection and tracking of multi-way video in underground garages is realized, which reduces the system's calculation volume, ensures real-time and accuracy, and adapts to congestion on different roads, improves system flexibility and adaptability, and reduces hardware costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091613A_ABST
    Figure CN119091613A_ABST
Patent Text Reader

Abstract

The invention relates to an underground garage congestion detection method based on target tracking. The method comprises the following steps: acquiring underground garage video data; performing target detection on the video data to identify a vehicle; performing target tracking on the identified vehicle; and based on a target tracking result, judging a current congestion state, and if congestion is judged to occur, calculating a congestion index and outputting the congestion index. Compared with the prior art, the method has the advantages that real-time target detection and target tracking of multiple paths of videos can be realized in an underground garage scene, methods of frame loss, frame skipping and data updating speed control are adopted, and congestion judgment and congestion index calculation methods are designed, so that the system calculation amount can be reduced on the premise of ensuring the real-time performance and accuracy, and the system reliability is improved. And the traffic condition of the underground garage is effectively monitored and analyzed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of traffic congestion detection, and in particular to an underground garage congestion detection method based on target tracking. Background Art

[0002] With the continuous advancement of computer vision and deep learning technology, the technology of traffic congestion detection system based on target tracking has become increasingly mature. The core modules of such systems mainly include target tracking and congestion judgment. The currently widely used target tracking method is to combine target detection and target tracking technology to obtain more accurate target information. For example, Chinese patent CN116310946A uses the YOLOv4 model and Tensorrt acceleration technology for target detection, and uses Kalman filtering and Hungarian algorithm for target tracking to obtain the pixel coordinates of the same target within a duration. In addition, the current common congestion judgment method is mainly to obtain the congestion index by calculating various indicators, where the calculation indicators include vehicle flow, vehicle speed and vehicle density. For example, Chinese patent CN110688922A calculates the congestion index by vehicle speed and vehicle density to judge the level of traffic congestion.

[0003] Although congestion detection for outdoor traffic is relatively mature, congestion detection for indoor underground garages is still in the development stage. Traditional underground garage congestion detection mainly relies on manual monitoring and related indicator analysis based on image processing. However, with the increasing demand for intelligent parking management systems, more and more deep learning technologies are being applied to underground garage congestion detection. In actual application, when migrating the outdoor traffic congestion detection system to the underground garage, there are mainly the following problems:

[0004] 1. For the target tracking module, the amount of deep learning calculation is large. Target detection and target tracking on multiple video data at the same time may cause processor overload, which may cause program crash or system freeze.

[0005] 2. The underground garage environment is complex, and when the traffic volume is large, there are many emergencies. For example, the administrator may temporarily use one side of the aisle for traffic diversion or temporary occupation, making the traditional congestion index calculation method unsuitable for this scenario. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an underground garage congestion detection method based on target tracking, which can realize real-time target detection and target tracking of multi-channel videos, and effectively monitor and analyze the traffic conditions of the underground garage.

[0007] The purpose of the present invention can be achieved by the following technical solution: A method for detecting congestion in an underground garage based on target tracking, comprising the following steps:

[0008] S1. Obtain underground garage video data;

[0009] S2, performing target detection on the video data to identify the vehicle;

[0010] S3, target tracking for the identified vehicle;

[0011] S4, based on the target tracking result, determine the current congestion state, if it is determined that congestion occurs, execute step S5, otherwise return to step S1;

[0012] S5. Calculate the congestion index and output it.

[0013] Furthermore, the step S1 specifically utilizes RTSP (Real Time Streaming Protocol) technology to obtain multi-channel video data of the underground garage camera, and performs frame loss processing when pulling the video data.

[0014] Furthermore, the step S2 specifically utilizes a pre-trained YOLOv5 model to perform real-time target detection on multiple videos, and uses frame skipping during the detection process to reduce the amount of calculation.

[0015] Furthermore, step S3 includes the following steps:

[0016] S31, assigning a corresponding ID to each identified vehicle;

[0017] S32, combining filter technology and increasing the confidence threshold of target tracking to obtain the target motion trajectory of each identified vehicle.

[0018] Furthermore, the step S4 specifically selects the lane area within the camera field of view as ROI (region of interest) for each of the multiple videos, stores the data of each vehicle in the ROI area based on the target tracking result, and calculates the information of each frame of each video based on these data to determine and select roads that meet the custom congestion conditions.

[0019] Furthermore, when processing a frame of data from a certain camera, step S4 includes the following steps:

[0020] S41, processing the vehicle information data of the current frame according to a preset data processing frequency;

[0021] First, obtain the target tracking result of this frame, including the vehicle pixel coordinate position and ID. According to the vehicle pixel coordinate and the set ROI pixel coordinate area, determine whether the current vehicle is in the ROI. If not, determine the next vehicle. If the vehicles in the ROI in this frame have been traversed, wait for the next frame of data input;

[0022] If the vehicle is in the ROI, the ID, appearance time and existence time of each vehicle are stored. Then, according to the pixel coordinates of the current vehicle and the pixel coordinates of the previously traversed vehicles, it is determined whether the vehicle is in the same lane, and the corresponding output value not_on_the_same_side is 1 or 0, where 1 means that the vehicle is not in the same lane, and 0 means that the vehicle is in the same lane;

[0023] S42, counting the information of the vehicle in the current frame;

[0024] After traversing all the vehicles in this frame, calculate the number and existence time of the current vehicles and the number and existence time of the vehicles that disappeared compared to the previous frame;

[0025] S43, judging congestion status;

[0026] First, the user-defined preset value usr_value corresponding to each road is obtained, where usr_value=0 indicates that the congestion of the double-sided lanes is defined as true congestion, and usr_value=1 indicates that the congestion of the single-sided lane is defined as true congestion;

[0027] The usr_value value and the not_on_the_same_side value are ORed together. If the result is greater than 0, the congestion indicator congestion_one is output as 1. At the same time, based on the data of each frame, the average speed, relative delay, and number of vehicles in the current frame of all vehicles appearing in the previous m frames are calculated. When the preset congestion conditions are met, the current frame is judged to be in a congested state.

[0028] Furthermore, the preset congestion conditions in step S43 are specifically: the average speed of all vehicles appearing in the previous m frames is less than 1-2 m / s, the relative delay is less than 0-1, the number of vehicles in the current frame is greater than 2-4, and the congestion_one value is 1.

[0029] Furthermore, the value range of m is 30s to 60s.

[0030] Furthermore, the calculation indicators for calculating the congestion index in step S5 include vehicle flow, vehicle density, average speed, relative delay and number of overtime vehicles.

[0031] Furthermore, the step S5 specifically performs weighted averaging on the calculation indicators through prior knowledge to calculate the congestion index.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] The present invention first obtains underground garage video data, then performs target detection on the video data to identify vehicles, then performs target tracking on the identified vehicles, and finally judges the current congestion state based on the target tracking result. If congestion is judged to occur, the congestion index is further calculated and output. Among them, when acquiring video data, the frame loss technology is used to reduce the amount of data input without affecting the judgment of congestion results; when performing target detection on video data, the pre-trained YOLOv5 model is used to perform real-time target detection on multiple videos, which can ensure the accuracy of identifying vehicles in underground garage scenes, and the frame skipping method is used in the detection process to reduce the amount of calculation; and the filter technology is combined to accurately track the motion trajectory of each identified vehicle, and the tracking error rate is reduced by increasing the confidence threshold of target tracking. In this way, the system calculation amount can be reduced while ensuring real-time performance and accuracy, and real-time target detection and target tracking of multiple videos can be effectively realized.

[0034] The present invention selects the lane area within the camera's field of view as ROI for multi-channel videos, stores the data of each vehicle in each ROI area based on the target tracking results, and then calculates the information of each frame of each road based on the vehicle data to judge and select roads that meet the custom congestion conditions, and calculate the corresponding congestion index. Therefore, the complexity of the underground garage environment is fully considered, and a congestion judgment and congestion index calculation method suitable for underground planning library scenes is designed. Considering the complexity of the situation faced by the garage when the traffic volume is large, different roads have different requirements for single-sided or double-sided congestion, that is, true congestion. Therefore, a custom preset value usr_value corresponding to each road is also designed to facilitate users to customize the congestion requirements corresponding to each road according to actual conditions, thereby improving the flexibility and adaptability of the system.

[0035] The present invention has strong portability and can be deployed on an edge computing unit or an existing server. In addition, the input of the present invention only requires connecting a camera and does not require additional hardware facilities, which can minimize costs in application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0037] Figure 2 Schematic diagram of the process of congestion status judgment and congestion index calculation in the embodiment. DETAILED DESCRIPTION

[0038] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Example

[0040] like Figure 1 As shown, a method for detecting congestion in an underground garage based on target tracking comprises the following steps:

[0041] S1. Obtain underground garage video data;

[0042] S2, performing target detection on the video data to identify the vehicle;

[0043] S3, target tracking for the identified vehicle;

[0044] S4, based on the target tracking result, determine the current congestion state, if it is determined that congestion occurs, execute step S5, otherwise return to step S1;

[0045] S5. Calculate the congestion index and output it.

[0046] This embodiment applies the above solution, mainly including:

[0047] 1. Video data acquisition: Use RTSP technology to obtain video data from underground garage cameras. When pulling video data, use frame dropping technology to reduce the amount of data input without affecting the judgment of congestion results.

[0048] 2. Target detection and vehicle identification: The YOLOv5 model trained with underground garage environment data is used to achieve 16-way real-time target detection. At the same time, during the detection process, the frame skipping method is used to reduce the amount of calculation and avoid target inference for each frame.

[0049] 3. Target tracking and unique ID assignment: Combine Kalman filter and correlation filter technology to accurately obtain the target motion trajectory and reduce the error rate by increasing the confidence threshold of target tracking.

[0050] 4. Congestion index calculation: For 16 videos, the lane area within the camera's field of view is selected as the ROI. Based on the target tracking results, the data of each vehicle in each ROI area is stored. Based on these data, the information of each frame of each road is calculated, and the roads that meet the custom congestion conditions are screened out, and the congestion index is calculated. The calculation indicators of the congestion index include vehicle flow, vehicle density, average speed, relative delay and number of overtime vehicles.

[0051] 5. Result output: Finally, the congestion information is sent to the server through the UDP protocol.

[0052] Among them, steps 1 to 3 realize real-time target detection and target tracking of 16 videos.

[0053] Step 4 is to design a method for judging the congestion state and calculating the congestion index based on the underground garage. At the same time, the frequency F of data processing is controlled (the frequency F of data processing is set to 1s / time in this embodiment) to reduce the amount of system calculation, such as Figure 2 The specific steps are as follows: for a frame of data from a camera on a certain road, the frame number of this frame is greater than m, where m is the congestion state in the number of seconds before the calculation, and the value range of m is 30s to 60s.

[0054] 4.1. Process the vehicle information data of the current frame. First, obtain the target tracking results of this frame, including the vehicle pixel coordinate position and ID. According to the vehicle pixel coordinates and the set ROI pixel coordinate area, determine whether the current vehicle is in the ROI. If it is not in the ROI, judge the next vehicle. If the vehicles in the ROI in this frame have been traversed, wait for the next frame of data to be input. If the vehicle is in the ROI, store the ID, appearance time and existence time of each vehicle. After that, determine whether the vehicle is in the same side lane based on the current vehicle pixel coordinates and the previously traversed vehicle pixel coordinates. The output value not_on_the_same_side is 1 or 0-1 means that the vehicle is not in the same side lane, and 0 means that the vehicle is in the same side lane.

[0055] 4.2. Count the information of vehicles in the current frame. After traversing all the vehicles in this frame, calculate the number and existence time of the current vehicles and the number and existence time of the vehicles that disappeared compared with the previous frame.

[0056] 4.3. Congestion status judgment and congestion index calculation. First, considering the complexity of the situation faced by the garage when the traffic volume is large, different roads have different requirements for whether congestion on one side or both sides is true congestion. Therefore, this embodiment reserves a UDP interface to receive the value usr_value corresponding to each road defined by the administrator, where 0 indicates that congestion on both sides of the lane is defined as true congestion, and 1 indicates that congestion on one side of the lane is true congestion.

[0057] If the usr_value value and not_on_the_same_side value are ORed and the result is greater than 0, the output congestion indicator congestion_one is 1. At the same time, based on the data of each frame, the average speed of all vehicles appearing in the previous m frames, the relative delay, and the number of vehicles in the current frame are calculated. If the preset congestion conditions are met, the current frame is judged to be in a congested state. The congestion judgment conditions can be set as: the average speed of all vehicles appearing in the previous m frames is less than 1~2m / s, the relative delay is less than 0~1, the number of vehicles in the current frame is greater than 2~4, and the congestion_one value is 1.

[0058] After that, the indicator vehicle flow, vehicle density, average speed, relative delay and number of overtime vehicles are calculated, and the congestion index is calculated by weighted average based on prior knowledge. Finally, the congestion information is sent to the server through the UDP protocol.

[0059] In summary, this solution is based on the YOLOv5 model. By creating and training a vehicle dataset for an underground garage, the target detection model can be well applied to the underground garage scenario. At the same time, the method of dropping frames, skipping frames and controlling the data update speed can reduce the system calculation amount while ensuring real-time and accuracy, and can achieve real-time target detection and target tracking for 16-channel videos. In addition, this solution comprehensively considers the complexity of the underground garage environment, designs a congestion judgment and congestion index calculation method suitable for this scenario, and reserves a UDP interface for receiving the congestion requirements corresponding to each road defined by the administrator, which improves the flexibility of the system. This solution is applied in practice, with the advantages of strong portability and low hardware cost. This solution can be deployed on edge computing units or existing servers; the input connection only requires a camera, and no additional hardware facilities are required, which can minimize the cost in application.

Claims

1. A method for detecting congestion in an underground garage based on target tracking, characterized in that: The following steps are involved: S1. Obtaining underground garage video data; S2, performing target detection on the video data to identify the vehicle; S3, target tracking for the identified vehicle; S4, based on the target tracking result, determine the current congestion state, if it is determined that congestion occurs, execute step S5, otherwise return to step S1; S5. Calculate the congestion index and output it.

2. According to the target tracking-based underground garage congestion detection method of claim 1, it is characterized in that: The step S1 specifically uses RTSP technology to obtain multi-channel video data from the camera in the underground garage, and performs frame loss processing when pulling the video data.

3. The underground garage congestion detection method based on target tracking according to claim 1 is characterized in that: The step S2 specifically utilizes the pre-trained YOLOv5 model to perform real-time target detection on multiple videos, and uses frame skipping during the detection process to reduce the amount of calculation.

4. The underground garage congestion detection method based on target tracking according to claim 1 is characterized in that: The step S3 comprises the following steps: S31, assigning a corresponding ID to each identified vehicle; S32, combining filter technology and increasing the confidence threshold of target tracking to obtain the target motion trajectory of each identified vehicle.

5. The underground garage congestion detection method based on target tracking according to claim 4 is characterized in that: The step S4 specifically selects the lane area within the camera field of view as ROI for each of the multiple videos, stores the data of each vehicle in the ROI area based on the target tracking result, and calculates the information of each frame of each video based on these data to determine and select roads that meet the custom congestion conditions.

6. The underground garage congestion detection method based on target tracking according to claim 5 is characterized in that: When processing a frame of data from a certain camera, step S4 includes the following steps: S41, processing the vehicle information data of the current frame according to a preset data processing frequency; First, obtain the target tracking result of this frame, including the vehicle pixel coordinate position and ID. According to the vehicle pixel coordinate and the set ROI pixel coordinate area, determine whether the current vehicle is in the ROI. If not, determine the next vehicle. If the vehicles in the ROI in this frame have been traversed, wait for the next frame of data input; If the vehicle is in the ROI, the ID, appearance time and existence time of each vehicle are stored. Then, according to the pixel coordinates of the current vehicle and the pixel coordinates of the previously traversed vehicles, it is determined whether the vehicle is in the same lane, and the corresponding output value not_on_the_same_side is 1 or 0, where 1 means that the vehicle is not in the same lane, and 0 means that the vehicle is in the same lane; S42, counting the information of the vehicle in the current frame; After traversing all the vehicles in this frame, calculate the number and existence time of the current vehicles and the number and existence time of the vehicles that disappeared compared to the previous frame; S43, congestion status determination; First, the user-defined preset value usr_value corresponding to each road is obtained, where usr_value=0 indicates that the congestion of the double-sided lanes is defined as true congestion, and usr_value=1 indicates that the congestion of the single-sided lane is defined as true congestion; The usr_value value and the not_on_the_same_side value are ORed together. If the result is greater than 0, the congestion indicator congestion_one is output as 1. At the same time, based on the data of each frame, the average speed, relative delay, and number of vehicles in the current frame of all vehicles appearing in the previous m frames are calculated. When the preset congestion conditions are met, the current frame is judged to be in a congested state.

7. The underground garage congestion detection method based on target tracking according to claim 6 is characterized in that: The preset congestion conditions in step S43 are specifically: the average speed of all vehicles in the previous m frames is less than 1-2 m / s, the relative delay is less than 0-1, the number of vehicles in the current frame is greater than 2-4, and the congestion_one value is 1.

8. The underground garage congestion detection method based on target tracking according to any one of claims 6 to 7, characterized in that: The value range of m is 30s to 60s.

9. The underground garage congestion detection method based on target tracking according to claim 1 is characterized in that: The calculation indexes for calculating the congestion index in step S5 include vehicle flow, vehicle density, average speed, relative delay and number of overtime vehicles.

10. The underground garage congestion detection method based on target tracking according to claim 9 is characterized in that: The step S5 specifically performs weighted averaging on the calculation indicators through prior knowledge to calculate the congestion index.

Citation Information

Patent Citations

  • Traffic jam detection system and method based on deep learning

    CN110688922A

  • Intelligent traffic jam detection method and system based on multi-target tracking detection

    CN116310946A

Cited By

  • Traffic situation analysis method and device

    CN119296326A