Highway traffic accident staged discovery method based on monitoring video
By applying video image target recognition and target tracking technology in highway traffic accident surveillance videos, combining convolutional neural networks and multi-objective tracking algorithms to identify and judge the accident stage, the problem of low efficiency of artificial video patrol in the existing technology is solved. It has achieved timely detection and effective handling of highway traffic accidents.
Patent Information
- Application Number
- CN202411625976.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-02-14
AI Technical Summary
In the prior art, the discovery of highway traffic accidents relies on manual video patrols, which has a large workload, low efficiency, and is difficult to effectively identify and distinguish different stages of the accident, resulting in waste of emergency response resources.
Using a phased discovery method of highway traffic accidents based on surveillance video, the trajectory of vehicles, personnel and vehicles related to traffic accidents is identified through video image target recognition and target tracking technology, and combining convolutional neural network and multi-objective tracking algorithm to determine the occurrence of the accident. stage.
It has achieved timely detection and phase identification of highway traffic accidents, improved the timeliness and pertinence of emergency responses, saved emergency response resources, and avoided the expansion of secondary accidents and accident damage.
Smart Images
Figure CN119445847A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and more specifically, to a method for discovering highway traffic accidents in stages based on surveillance videos. Background Art
[0002] It is very important to detect highway traffic accidents in a timely manner. If the discovery is delayed and not handled in time, on the one hand, it may cause secondary accidents or delayed rescue of the injured, and on the other hand, it will prolong the road occupancy time and affect the efficiency of highway traffic. In addition to reporting, highway management departments mainly actively detect traffic accidents through video patrols. However, manual video patrols are labor-intensive, inefficient, and heavily dependent on personal responsibility. There is an urgent need for an AI recognition algorithm for highway traffic accidents based on video.
[0003] Highway traffic accidents are divided into several stages. The first is the accident stage, such as before and after the vehicle collision, which is characterized by the fact that the traffic order has not been disrupted and the traffic objects are easy to identify and track. The second is the initial stage of the accident, which is characterized by the instantaneous disorder of the traffic order, but the vehicles have not yet accumulated (that is, the congestion does not seem serious), and there are not many pedestrians on the highway (drivers and passengers leaving the vehicle). The traffic objects are still easy to identify but not easy to track. The third is the stage where the accident has not been handled for a long time, which is characterized by serious congestion on the highway, and traffic objects are not easy to identify or track. The fourth is the stage where the accident is handled. It is characterized by the gradual recovery of traffic order and the reduction of congestion, which is similar to the initial stage of the accident, but there is no accident vehicle.
[0004] The traffic management department has different focuses on different stages of different traffic accidents. In the accident stage, because the speed of vehicles on the road is still relatively high, the focus should be on saving people and preventing secondary accidents. In the early stage of the accident, temporary traffic order should be established while saving people. When the accident is not handled for a long time, due to traffic congestion, the relevant accident vehicles should be transferred as soon as possible to ease the traffic. At the stage when the accident is handled, no more emergency response resources should be invested. Summary of the invention
[0005] In order to address the deficiencies in the prior art, the present invention provides a method for discovering highway traffic accidents in stages based on surveillance videos. The method is based on video image target recognition and target tracking technology, and by identifying vehicles, personnel, and vehicle trajectories related to traffic accidents, the stages of traffic accidents can be determined while the accidents are discovered according to the characteristics of different stages of highway traffic accidents, so as to effectively handle the accidents and save emergency response resources.
[0006] As a first aspect of the present invention, a method for discovering highway traffic accidents in stages based on surveillance video is provided, comprising the following steps: Step S1: Obtaining surveillance video of the highway; Step S2: determining the traffic accident stage of the expressway according to the surveillance video of the expressway; Step S3: when the vehicle running tracks in the monitoring video intersect and then stop, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video after the vehicle running tracks stop, it is determined that the highway is in the stage where a traffic accident has just occurred; Step S4: when the number of vehicles and pedestrians in the monitoring video increases, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the initial stage of a traffic accident; Step S5: When traffic congestion appears in the monitoring video, and the accident vehicle and accident debris are identified from the monitoring video, and multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the highway is in a stage where a traffic accident has occurred for a long time and has not been dealt with.
[0007] Further, when the vehicle running tracks in the monitoring video intersect and then stop, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video after the vehicle running tracks stop, it is determined that the highway is in the stage where a traffic accident has just occurred, and further includes: A multi-target tracking algorithm is applied to track the vehicles in the surveillance video to obtain the vehicle running trajectory. When the vehicle running trajectory intersects and then stops, it is suspected that the highway is in the stage where a traffic accident has just occurred. After the vehicle's running trajectory comes to a standstill, the monitoring video is identified using a highway accident vehicle, accident debris and accident person identification algorithm based on convolutional neural network training. When the accident vehicle, accident debris and accident person are identified from the monitoring video, it is determined that the highway is in the stage where a traffic accident has just occurred.
[0008] Furthermore, when the number of vehicles and pedestrians in the monitoring video increases, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video, and it is determined that the highway is in the initial stage of a traffic accident, the method further includes: When the vehicle running tracks do not stop after crossing, or the accident vehicle, accident debris and accident personnel are not identified from the monitoring video, the number of vehicles and pedestrians in the monitoring video is calculated. When the number of vehicles and pedestrians in the monitoring video increases, it is suspected that the highway is in the early stage of a traffic accident, and then it is determined whether the accident vehicle, accident debris and accident personnel can be identified from the monitoring video; When the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the initial stage of a traffic accident; When the accident vehicle, accident debris and accident persons are not identified from the monitoring video, the movement trajectory of the pedestrian in the monitoring video is tracked. When the movement trajectory of the pedestrian is the movement trajectory of the stranded persons in the accident, it is determined that the highway is in the initial stage of a traffic accident.
[0009] Furthermore, the number of vehicles and pedestrians in the surveillance video is calculated, and when the number of vehicles and pedestrians in the surveillance video increases, it is suspected that the highway is in the initial stage of a traffic accident, and further includes: By tracking the running trajectories of the vehicles in the monitoring video, the number of vehicles passing through the expressway per unit time is calculated, and the number of vehicles and pedestrians in a single frame of the monitoring video is calculated; If any two of the following three conditions are met: the number of vehicles passing through the expressway per unit time decreases, the number of vehicles in a single frame of the surveillance video increases, and the number of pedestrians in a single frame of the surveillance video increases, then the expressway is suspected to be in the initial stage of a traffic accident.
[0010] Furthermore, when traffic congestion occurs in the monitoring video, and accident vehicles and accident debris are identified from the monitoring video, and multiple vehicle avoidance tracks concentrated at several points appear in the monitoring video, it is determined that the highway is in a stage where a traffic accident has occurred for a long time and has not been handled, the method further includes: When the number of vehicles and pedestrians in the monitoring video does not increase, determining whether traffic congestion occurs in the monitoring video; If no traffic congestion occurs in the monitoring video, it is determined that the expressway is in the stage of traffic accident handling completion or there is no traffic accident; if traffic congestion occurs in the monitoring video, it is determined whether the accident vehicle and accident debris can be identified from the monitoring video; If the accident vehicle and accident debris are not identified from the monitoring video, it is determined that the expressway is in the stage of traffic accident handling completion or there is no traffic accident; if the accident vehicle and accident debris are identified from the monitoring video, it is determined whether multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video; If multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the expressway is in a stage where a traffic accident has occurred and has not been dealt with for a long time; if multiple vehicle avoidance trajectories concentrated at several points do not appear in the monitoring video, it is determined that the expressway is in a stage where the traffic accident has been dealt with or there is no traffic accident.
[0011] The method for discovering highway traffic accidents in stages based on surveillance video provided by the present invention has the following advantages: traffic accidents can be discovered in a timely manner according to the video image features at different stages when a traffic accident occurs on a highway, thereby improving the timeliness of intervention and disposal by the highway traffic management department, effectively avoiding secondary accidents and the expansion of accident damage, and accelerating the restoration of highway traffic order. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.
[0013] Figure 1 The present invention provides a flow chart of a method for discovering highway traffic accidents in stages based on surveillance videos.
[0014] Figure 2 A flowchart of a specific implementation method of a method for discovering highway traffic accidents in stages based on surveillance video provided by the present invention. DETAILED DESCRIPTION
[0015] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0016] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method, structure, characteristics and effects of a staged discovery method for highway traffic accidents based on surveillance video proposed by the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.
[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0018] In this embodiment, a method for discovering highway traffic accidents in stages based on surveillance video is provided. Figure 1 As shown, the method for discovering highway traffic accidents in stages based on surveillance videos includes: Step S1: Obtaining surveillance video of the highway; It should be noted that the monitoring video of the expressway is accessed through the interface module and pre-processed, such as image denoising and resolution adjustment.
[0019] Step S2: determining the traffic accident stage of the expressway according to the surveillance video of the expressway; Step S3: when the vehicle running tracks in the monitoring video intersect and then stop, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video after the vehicle running tracks stop, it is determined that the highway is in the stage where a traffic accident has just occurred; Preferably, Figure 2 As shown, when the vehicle running tracks in the monitoring video intersect and then stop, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video after the vehicle running tracks stop, it is determined that the highway is in the stage where a traffic accident has just occurred, and further includes: Apply a multi-target tracking algorithm (such as Deep SORT) to track vehicles, pedestrians and other moving targets in the surveillance video to obtain the running trajectories of vehicles, pedestrians and other moving targets. When the running trajectories of vehicles intersect and then stop, it is suspected that the highway is in the stage of a traffic accident. After the vehicle's running trajectory comes to a standstill, the highway accident vehicle, accident debris and accident personnel recognition algorithm based on convolutional neural networks ResNet and VGG training is used to perform target recognition on the monitoring video (the image after the track crosses and stops). When the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the stage where a traffic accident has just occurred.
[0020] Step S4: when the number of vehicles and pedestrians in the monitoring video increases, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the initial stage of a traffic accident; Preferably, Figure 2 As shown, when the number of vehicles and pedestrians in the monitoring video increases, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video, and it is determined that the highway is in the initial stage of a traffic accident, the method further includes: When the vehicle running tracks do not stop after crossing, or the accident vehicle, accident debris and accident personnel are not identified from the monitoring video, the number of vehicles and pedestrians in the monitoring video is calculated. When the number of vehicles and pedestrians in the monitoring video increases, it is suspected that the highway is in the early stage of a traffic accident, and then it is determined whether the accident vehicle, accident debris and accident personnel can be identified from the monitoring video; Specifically, Figure 2 As shown, the number of vehicles and pedestrians in the monitoring video is calculated. When the number of vehicles and pedestrians in the monitoring video increases, it is suspected that the highway is in the early stage of a traffic accident, and further includes: By tracking the running trajectories of the vehicles in the monitoring video, the number of vehicles passing through the expressway per unit time is calculated, and the number of vehicles and pedestrians in a single frame of the monitoring video is calculated; If any two of the following three conditions are met: the number of vehicles passing through the expressway per unit time decreases, the number of vehicles in a single frame of the surveillance video increases, and the number of pedestrians in a single frame of the surveillance video increases, then the expressway is suspected to be in the initial stage of a traffic accident.
[0021] It should be noted that the initial stage of a traffic accident is characterized by the beginning of traffic disorder and an increase in the number of vehicles and pedestrians, but the accident vehicle, accident debris, and casualties may be obscured and unable to be identified.
[0022] When the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the initial stage of a traffic accident; When the accident vehicle, accident debris and accident persons are not identified from the monitoring video, the movement trajectory of the pedestrian in the monitoring video is tracked. When the movement trajectory of the pedestrian is the movement trajectory of the stranded persons in the accident, it is determined that the highway is in the initial stage of a traffic accident.
[0023] It should be noted that the characteristics of people in the early stage of a traffic accident are: first, people on the highway in the early stage of the accident are generally non-highway workers, who do not wear reflective vests and other signs, and can be excluded by the highway worker identification algorithm; second, people in the early stage of the accident will obviously stay near the vehicle, emergency lane, middle green belt, etc. The movement trajectory of the accident stranded people is obviously different from the movement trajectory of general pedestrians who walk along the highway or cross the highway. By tracking the movement trajectory of pedestrians in the video for a long time, it can be confirmed that it is the early stage of a traffic accident.
[0024] Step S5: When traffic congestion appears in the monitoring video, and the accident vehicle and accident debris are identified from the monitoring video, and multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the highway is in a stage where a traffic accident has occurred for a long time and has not been dealt with.
[0025] Preferably, Figure 2 As shown, when traffic congestion occurs in the monitoring video, and accident vehicles and accident debris are identified from the monitoring video, and multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the highway is in a stage where a traffic accident has occurred for a long time and has not been handled, the method further includes: When the number of vehicles and pedestrians in the monitoring video does not increase, determining whether traffic congestion occurs in the monitoring video; If no traffic congestion occurs in the monitoring video, it is determined that the expressway is in the stage of traffic accident handling completion or there is no traffic accident; if traffic congestion occurs in the monitoring video, it is determined whether the accident vehicle and accident debris can be identified from the monitoring video; If the accident vehicle and accident debris are not identified from the monitoring video, it is determined that the expressway is in the stage of traffic accident handling completion or there is no traffic accident; if the accident vehicle and accident debris are identified from the monitoring video, it is determined whether multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video; If multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the expressway is in a stage where a traffic accident has occurred and has not been dealt with for a long time; if multiple vehicle avoidance trajectories concentrated at several points do not appear in the monitoring video, it is determined that the expressway is in a stage where the traffic accident has been dealt with or there is no traffic accident.
[0026] In response to the need to discover traffic accidents as soon as possible during highway video patrols, the present invention provides a method for discovering highway traffic accidents in stages based on surveillance videos, which supports the identification of traffic accidents at different stages to improve the pertinence and efficiency of emergency response and save emergency response resources.
[0027] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.
Claims
1. A method for discovering highway traffic accidents in stages based on surveillance video, characterized in that: The method for discovering highway traffic accidents in stages based on surveillance video comprises the following steps: Step S1: Obtaining surveillance video of the highway; Step S2: determining the traffic accident stage of the expressway according to the surveillance video of the expressway; Step S3: when the vehicle running tracks in the monitoring video intersect and then stop, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video after the vehicle running tracks stop, it is determined that the highway is in the stage where a traffic accident has just occurred; Step S4: when the number of vehicles and pedestrians in the monitoring video increases, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the initial stage of a traffic accident; Step S5: When traffic congestion appears in the monitoring video, and the accident vehicle and accident debris are identified from the monitoring video, and multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the highway is in a stage where a traffic accident has occurred for a long time and has not been dealt with.
2. The method for discovering highway traffic accidents in stages based on surveillance video according to claim 1 is characterized in that: When the vehicle running tracks in the monitoring video intersect and then stop, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video after the vehicle running tracks stop, it is determined that the highway is in the stage where the traffic accident has just occurred, and further includes: A multi-target tracking algorithm is applied to track the vehicles in the surveillance video to obtain the vehicle running trajectory. When the vehicle running trajectory intersects and then stops, it is suspected that the highway is in the stage where a traffic accident has just occurred. After the vehicle's running trajectory comes to a standstill, the monitoring video is identified using a highway accident vehicle, accident debris and accident person identification algorithm based on convolutional neural network training. When the accident vehicle, accident debris and accident person are identified from the monitoring video, it is determined that the highway is in the stage where a traffic accident has just occurred.
3. The method for discovering highway traffic accidents in stages based on surveillance video according to claim 2 is characterized in that: When the number of vehicles and pedestrians in the monitoring video increases, and the accident vehicle, accident debris and accident personnel are identified from the monitoring video, and it is determined that the highway is in the initial stage of a traffic accident, the method further includes: When the vehicle running tracks do not stop after crossing, or the accident vehicle, accident debris and accident personnel are not identified from the monitoring video, the number of vehicles and pedestrians in the monitoring video is calculated. When the number of vehicles and pedestrians in the monitoring video increases, it is suspected that the highway is in the early stage of a traffic accident, and then it is determined whether the accident vehicle, accident debris and accident personnel can be identified from the monitoring video; When the accident vehicle, accident debris and accident personnel are identified from the monitoring video, it is determined that the highway is in the initial stage of a traffic accident; When the accident vehicle, accident debris and accident persons are not identified from the monitoring video, the movement trajectory of the pedestrian in the monitoring video is tracked. When the movement trajectory of the pedestrian is the movement trajectory of the stranded persons in the accident, it is determined that the highway is in the initial stage of a traffic accident.
4. The method for discovering highway traffic accidents in stages based on surveillance video according to claim 3 is characterized in that: The calculating of the number of vehicles and pedestrians in the monitoring video, when the number of vehicles and pedestrians in the monitoring video increases, it is suspected that the highway is in the initial stage of a traffic accident, further comprising: By tracking the running trajectories of the vehicles in the monitoring video, the number of vehicles passing through the expressway per unit time is calculated, and the number of vehicles and pedestrians in a single frame of the monitoring video is calculated; If any two of the following three conditions are met: the number of vehicles passing through the expressway per unit time decreases, the number of vehicles in a single frame of the surveillance video increases, and the number of pedestrians in a single frame of the surveillance video increases, then the expressway is suspected to be in the initial stage of a traffic accident.
5. The method for discovering highway traffic accidents in stages based on surveillance video according to claim 3 is characterized in that: When traffic congestion occurs in the monitoring video, and accident vehicles and accident debris are identified from the monitoring video, and multiple vehicle avoidance tracks concentrated at several points appear in the monitoring video, it is determined that the highway is in a stage where a traffic accident has occurred for a long time and has not been handled, the method further includes: When the number of vehicles and pedestrians in the monitoring video does not increase, determining whether traffic congestion occurs in the monitoring video; If no traffic congestion occurs in the monitoring video, it is determined that the expressway is in the stage of traffic accident handling completion or there is no traffic accident; if traffic congestion occurs in the monitoring video, it is determined whether the accident vehicle and accident debris can be identified from the monitoring video; If the accident vehicle and accident debris are not identified from the monitoring video, it is determined that the expressway is in the stage of traffic accident handling completion or there is no traffic accident; if the accident vehicle and accident debris are identified from the monitoring video, it is determined whether multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video; If multiple vehicle avoidance trajectories concentrated at several points appear in the monitoring video, it is determined that the expressway is in a stage where a traffic accident has occurred and has not been dealt with for a long time; if multiple vehicle avoidance trajectories concentrated at several points do not appear in the monitoring video, it is determined that the expressway is in a stage where the traffic accident has been dealt with or there is no traffic accident.