Traffic accident detection method based on deep learning

Abstract

The invention provides a traffic accident detection method based on deep learning, and the method comprises the following steps: S1, obtaining an accessed real-time video, and verifying whether a scene is normally shot, and does not have short-range and long-range stretching and angle deviation or not according to the scene id of the real-time video, so as to prevent a detection region under normal shooting from changing; s2, judging the vehicle parking information of the accessed real-time video, and detecting whether a static vehicle exists or not; s3, filtering out vehicles outside the detection area by loading a detection area configuration file; and S4, predicting whether a road traffic accident occurs through a region accident identification algorithm according to the vehicle stop time, the vehicle stop number and whether the frame regions of the adjacent vehicles are overlapped. According to the traffic accident detection method based on deep learning, vehicle collision accidents in two accident high-incidence scenes of urban crossroads and high-speed key road sections can be detected and analyzed, and an alarm is given in time.

Description

Technical Field

[0001] This invention relates to a traffic accident detection method, and more particularly to a traffic accident detection method based on deep learning. Background Technology

[0002] With the development of the times, the number of cars in my country is increasing, and the road environment is becoming increasingly worse. Statistics show that on average, someone loses their life in a traffic accident every day in my country. In 2023, there were 10 million traffic accidents nationwide, resulting in 100,000 deaths, 2 million injuries, and direct economic losses of 10 billion yuan. Geographically, traffic accidents mainly occur on urban roads and highways, accounting for 70% of the total; while the accident rate is relatively low on rural roads and mountain roads. Traffic accidents can lead to cascading secondary collisions, further causing traffic congestion and worsening the traffic environment. Therefore, detecting the location and time of traffic accidents as soon as possible and promptly handling accidents and clearing roads can help alleviate urban traffic congestion. Summary of the Invention

[0003] This invention provides a traffic accident detection method based on deep learning, which solves the problem of detecting vehicle collision accidents based on road surveillance videos. The technical solution is as follows: A deep learning-based traffic accident detection method includes the following steps: S1: Obtain the accessed real-time video and verify whether the scene is being captured normally based on the scene ID of the real-time video, without near-far stretching or angle shift, in order to prevent changes in the detection area under normal shooting conditions. S2: Determine the vehicle stationary information from the incoming real-time video to detect whether there are stationary vehicles; S3: By loading the detection area configuration file, vehicles outside the detection area are filtered out; S4: Based on the vehicle's dwell time, the number of vehicles, and whether the border areas of adjacent vehicles overlap, a regional accident recognition algorithm is used to predict whether a road traffic accident has occurred.

[0004] Furthermore, in step S1, the scene ID of the accessed real-time video is obtained, and the scene ID is used to verify whether near-far scene stretching and angle shift have occurred. If so, the accident recognition algorithm is not enabled.

[0005] Furthermore, in step S2, the accessed real-time video includes video streams with three resolutions: 1920×1080, 2048×1536, and 1536×2048.

[0006] In step S2, the vehicle stationary information is divided into two lists: stationary and moving. The first list is the structured result from the previous frame, `prev_objs`, and the second list is the result for all currently observed stationary vehicles, `still_objs`. `prev_objs = [{'cls':cls_value, 'score': score_value, 'bbox': bbox_value}, …]` means that the structured results need to be updated to `prev_objs` after the results of the current frame are processed. `still_objs = [{'ID': id_value, 'still_frame_num': still_frame_num_value, 'cls': cls_value, 'score': score_value, 'bbox': bbox_value}, …]`, where `ID` is a unique identifier for the vehicle, used to determine whether the vehicle in consecutive frames is the same vehicle; `still_frame_num` records the number of frames the vehicle remains stationary, starting when it begins to stop and incrementing by one for each frame. `cls` represents the target category; `score` represents the confidence score, indicating the reliability of the recognition result, with a value of 0 or 1; and `bbox` represents the coordinates of the target bounding box, the rectangle containing the identified target.

[0007] Maintaining still_objs requires the following three operations: (1) Add: Add a vehicle to the list when it is found to be parked; (2) Deletion: When a parked vehicle is found to start moving, it is deleted from the list; (3) Update: When comparing the IoU of each vehicle in each frame, the vehicle's bbox needs to be updated in real time.

[0008] Furthermore, in step S3, the detection area configuration file is used to determine the range of the detection area, and the coordinates of the detection area are stored in the database.

[0009] Furthermore, in step S4, the thresholds related to the vehicle's dwell time, the number of vehicles, and whether the border areas of adjacent vehicles overlap are adjusted through the configuration of the regional accident recognition algorithm.

[0010] Furthermore, in step S4, whether the border regions of adjacent vehicles overlap is determined using a vehicle target detection model, which employs the EfficientDet-D7 deep learning algorithm.

[0011] In step S4, in addition to the basic vehicle target detection model, the regional accident recognition algorithm also introduces a weighted bidirectional feature pyramid network and a composite scaling method as auxiliary methods.

[0012] The deep learning-based traffic accident detection method detects vehicle collisions based on road surveillance videos, enabling timely detection and handling, effectively reducing the probability of secondary collisions in cities, alleviating traffic congestion, and ensuring people's normal travel order. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the deep learning-based traffic accident detection method. Detailed Implementation

[0014] like Figure 1 As shown, the deep learning-based traffic accident detection method, for urban road intersections or normal driving lanes of highways, if there are two or more vehicles that stop in the detection area for more than a set time threshold, and the rectangular borders of the stopped vehicles overlap, then it is predicted that the vehicles stopped abnormally due to a traffic accident.

[0015] Specifically, it includes the following steps: S1: Obtain the accessed real-time video, and verify whether the scene is being captured normally based on the scene ID of the real-time video, without near-far stretching or angle shift, to prevent changes in the detection area under normal shooting conditions; the scene ID generally represents the ID number of the video source, and the ID will change when the video source changes or the shooting position of the same video source changes.

[0016] When the camera is recording normally, a detection area can be manually planned based on the scene captured by the camera. This detection area is implemented by loading a detection area configuration file. The detection area, as the region to be detected, can be marked with a thick red border. An area incident recognition algorithm is then used to determine whether an incident has occurred.

[0017] Obtain the scene ID of the incoming real-time video. Based on the scene ID, verify whether the scene has experienced near-far stretching or angular shift. Considering that in real-world scenarios, intersection cameras may automatically stretch and zoom (e.g., intelligent far-field stretching to capture license plates of illegally parked vehicles) or automatically rotate (e.g., cameras set to periodically patrol preset points), both of these situations will cause changes in the detection area scene. Therefore, it is necessary to first verify whether shifts or near-far changes have occurred using the scene ID. If so, the accident recognition algorithm will not be activated to avoid misjudgments.

[0018] S2: Determine the vehicle stationary information from the incoming real-time video to detect whether there are stationary vehicles; The incoming real-time video is processed in frames. The number of frames processed per second is configurable; in this embodiment, a processing rate of 3 frames per second can be used. Furthermore, based on the resolution of mainstream cameras, three video stream resolutions can be selected: 1920×1080, 2048×1536, and 1536×2048.

[0019] Detecting the presence of stationary vehicles involves judging the presence of vehicle stillness information in the frame. This information can be divided into two lists: stationary and moving vehicles. Therefore, we primarily maintain two lists: the structured result from the previous frame (prev_objs) and the result for all currently observed stationary vehicles (still_objs). `prev_objs = [{'cls':cls_value, 'score': score_value, 'bbox': bbox_value}, …]` means that the structured results need to be updated to `prev_objs` after the results of the current frame are processed. still_objs = [{'ID': id_value, 'still_frame_num': still_frame_num_value, 'cls': cls_value, 'score': score_value, 'bbox': bbox_value}, …], where ID is a unique identifier for the vehicle, used to determine whether the vehicle in two consecutive frames is the same vehicle. The identification method uses IoU (Intersection over Union), Siamese network, or image fingerprint; still_frame_num is used to record the number of frames the vehicle is stationary, starting when it begins to stop and incrementing by one for each frame it stops.

[0020] cls represents the target category, such as motor vehicle or non-motor vehicle; The score represents the confidence score, which indicates the reliability of the recognition result, and takes a value of 0 or 1; bbox represents the coordinates of the target bounding box, the rectangle containing the identified target.

[0021] Furthermore, maintaining still_objs requires the following three operations: (1) Add: Add a vehicle to the list when it is found to be parked; (2) Deletion: When a parked vehicle is found to start moving, it is deleted from the list; (3) Update: When comparing the IoU of each vehicle in each frame, the vehicle's bbox needs to be updated in real time.

[0022] Using the structured result of the previous frame (prev_objs) and the results of all currently observed stationary vehicles (still_objs), the system records the changes in vehicle conditions after accurate identification, providing basic data support for the analysis of vehicle accidents.

[0023] S3: By loading the detection area configuration file, vehicles outside the detection area are filtered out; The detection area configuration file is used to determine the scope of the detection area. The detection area is manually planned by the operator based on the scene captured by the camera, and the coordinates of this detection area (the coordinates of the four points of a quadrilateral) are stored in the database. Defining a region of interest in monitoring is existing technology and will not be elaborated upon in this invention.

[0024] Vehicles outside the detection area are filtered out by comparing the center point of the vehicle in still_objs with the coordinates of the detection area.

[0025] S4: Based on the vehicle's dwell time, the number of vehicles, and whether the border areas of adjacent vehicles overlap, a regional accident recognition algorithm is used to predict whether a road traffic accident has occurred. The aforementioned regional accident recognition algorithm is an artificial intelligence-based solution that automatically detects and identifies traffic accidents by analyzing road surveillance video. This algorithm typically employs computer vision and deep learning models to process video streams within the monitored area in real time, quickly identifying accident signs and issuing alerts to assist traffic management departments in responding promptly.

[0026] In this embodiment, the duration and number of vehicles stopped can be set, such as: For urban road intersections, if more than one vehicle remains stopped for more than 5 minutes, and there are still vehicles moving normally within the area, and the number of stopped vehicles in the current frame is between 1 and 4, then if the borders of the stopped vehicles overlap, it is determined to be a road traffic accident; if the borders of the stopped vehicles do not overlap, it is determined to be a suspected road traffic accident.

[0027] For highway sections, if a vehicle remains stationary for more than 5 minutes, and there are more than 1 but no more than 5 stationary vehicles (excluding highway traffic jams), and other vehicles are still moving normally within the area, and the number of stationary vehicles in the current frame is between 1 and 4, then if the borders of the stationary vehicles overlap, it is considered a road traffic accident; if the borders of the stationary vehicles do not overlap, it is considered a suspected road traffic accident.

[0028] If a traffic accident is determined to have occurred, an alarm should be issued immediately.

[0029] This step predicts whether a road traffic accident has occurred by analyzing the duration of vehicle dwell time, the number of dwelling vehicles, and whether the vehicle boundary areas overlap. By configurable adjustment of the threshold, scenario-based recognition and analysis can be achieved at intersections with varying traffic volumes.

[0030] Furthermore, to determine whether the bounding box regions of adjacent vehicles overlap, a vehicle target detection model is used as the core. This model employs the EfficientDet-D7 deep learning algorithm, which can more accurately identify targets, especially accurately identifying details such as the location and damage after a vehicle collision. EfficientDet-D7 is the largest model in Google's EfficientDet series, designed specifically for high-precision target detection and suitable for scenarios with relatively abundant resources. EfficientDet-D7 demonstrates good recognition accuracy: with 326B FLOPS and 52M parameters, it achieved a state-of-the-art mAP (mAP) of 51.0 on the COCO2017 validation dataset. With approximately 52 million parameters and approximately 325 billion FLOPs, it reaches the state-of-the-art level in single-model, single-scale testing.

[0031] The regional accident recognition algorithm is based on deep learning theory. It selects a network model suitable for traffic accident detection and chooses training parameters according to the characteristics of traffic accidents in the actual application scenario. The training process of the regional accident recognition algorithm involves collecting video sources of traffic accidents at various intersections and highways to train the network model. The model supports the analysis and judgment of vehicle collisions from different angles and directions. In addition to using EfficientNet as the backbone network, two auxiliary algorithms are also introduced: (1) Weighted Bidirectional Feature Pyramid Network (BiFPN): Multi-scale feature fusion is achieved through learnable weights, which improves the efficiency of feature utilization.

[0032] (2) Compound Scaling Method: This method unifies the scaling of the resolution, depth, and width of the backbone network, feature network, and prediction network, balancing accuracy and efficiency. The compound scaling method is a convolutional neural network (CNN) model scaling technique proposed by Google AI in 2019. It aims to improve both the accuracy and computational efficiency of the model by balancing the scaling of the network's depth, width, and resolution. In this embodiment, the present invention can process images quickly. Tested on a Tesla P4 card using the EfficientNet-D7 model (150MB in size, 850MB in VRAM), the detection time for a single region was 30ms, achieving an accuracy of 90.3% in urban intersection scenarios and 95.6% in key highway sections.

[0033] This invention can detect and analyze vehicle collision accidents in two high-accident scenarios: urban intersections and key sections of highways, and issue timely warnings.

Claims

1. A traffic accident detection method based on deep learning, comprising the following steps: S1: Obtain the accessed real-time video and verify whether the scene is being captured normally based on the scene ID of the real-time video, without near-far stretching or angle shift, in order to prevent changes in the detection area under normal shooting conditions. S2: Determine the vehicle stationary information from the incoming real-time video to detect whether there are stationary vehicles; S3: By loading the detection area configuration file, vehicles outside the detection area are filtered out; S4: Based on the vehicle's dwell time, the number of vehicles, and whether the border areas of adjacent vehicles overlap, a regional accident recognition algorithm is used to predict whether a road traffic accident has occurred.

2. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S1, the scene ID of the accessed real-time video is obtained, and the scene is checked for near-far stretching and angle shift based on the scene ID. If these occur, the accident recognition algorithm is not enabled.

3. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S2, the accessed real-time video includes video streams with three resolutions: 1920×1080, 2048×1536, and 1536×2048.

4. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S2, the vehicle stationary information is divided into two lists: stationary and moving. The first list is the structured result from the previous frame, `prev_objs`, and the second list is the result for all currently observed stationary vehicles, `still_objs`. `prev_objs = [{'cls':cls_value, 'score': score_value, 'bbox': bbox_value},…]` means that the structured results need to be updated to `prev_objs` after the current frame's results are processed. still_objs = [{'ID': id_value, 'still_frame_num': still_frame_num_value,'cls': cls_value, 'score': score_value, 'bbox': bbox_value}, …], where ID is a unique identifier for the vehicle, used to determine whether the vehicle in two consecutive frames is the same vehicle; still_frame_num records the number of frames the vehicle remains stationary, starting when it begins to stop and incrementing by one for each frame; cls represents the target category; score represents the confidence score, indicating the reliability of the recognition result, with a value of 0 or 1; bbox represents the coordinates of the target bounding box, the rectangle containing the recognized target.

5. The traffic accident detection method based on deep learning according to claim 4, characterized in that: Maintaining still_objs requires the following three operations: (1) Add: Add a vehicle to the list when it is found to be parked; (2) Deletion: When a parked vehicle is found to start moving, it is deleted from the list; (3) Update: When comparing the IoU of each vehicle in each frame, the vehicle's bbox needs to be updated in real time.

6. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S3, the detection area configuration file is used to determine the range of the detection area, and the coordinates of the detection area are stored in the database.

7. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S4, the thresholds related to the vehicle's dwell time, the number of vehicles, and whether the border areas of adjacent vehicles overlap are adjusted through the configuration of the regional accident recognition algorithm.

8. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S4, to determine whether the border regions of adjacent vehicles overlap, a vehicle target detection model is used, which employs the EfficientDet-D7 deep learning algorithm.

9. The traffic accident detection method based on deep learning according to claim 1, characterized in that: In step S4, in addition to the basic vehicle target detection model, the regional accident recognition algorithm also introduces a weighted bidirectional feature pyramid network and a composite scaling method as auxiliary methods.

Images