A Multi-Factor Fusion Real-Time Traffic Accident Recognition Method Based on Video Analysis

Abstract

This invention provides a multi-element fusion real-time traffic accident recognition method based on video analysis. Addressing the issue of low accuracy in traffic accident recognition in urban road scenarios, this method utilizes the Ascend 710 high-performance computing chip. It employs an improved YOLOv5 network model to detect vehicles and pedestrians, and uses Bytetrack real-time tracking technology to determine the motion state and trajectory of the targets. Through multi-element fusion, traffic accidents are identified and reported to the traffic control center in real time. The traffic accident event is then automatically distributed to the nearest traffic police officer for handling, forming a closed loop. This improves the efficiency of handling traffic accidents on urban roads, ensuring traffic safety and smooth traffic flow.

Description

Technical Field

[0001] This invention relates to the field of image intelligent analysis technology, and in particular to a real-time traffic accident recognition method based on video analysis and multi-element fusion. Background Technology

[0002] With the advancement of urbanization and the improvement of people's living standards in my country, automobiles have become a necessity of life, but they are also accompanied by increasingly frequent traffic accidents. In order to detect and deal with traffic accidents in a timely manner, a complete automated real-time traffic accident detection system is needed. Traffic accident identification is a difficult problem in the field of traffic video recognition, and it is of great significance to traffic management. According to the relevant laws of my country, a road traffic accident refers to an event in which a vehicle causes personal injury or property damage due to fault or accident while driving on the road. The following elements must be present to constitute a traffic accident: (1) There must be stationary vehicles after the traffic accident occurs; (2) It must be caused by a vehicle; (3) It occurs on the road; (4) It occurs while in motion; (5) An incident occurs. This refers to the occurrence of phenomena such as collision, crushing, scraping, overturning, and falling off the vehicle; (6) The cause of the incident is human; (7) There must be damage consequences.

[0003] Video-based traffic accident detection primarily focuses on vehicle-to-vehicle accidents, specifically categorized into single-vehicle accidents and multi-vehicle accidents. The main characteristics of a traffic accident on the road typically include the vehicle changing from a moving state to a stationary state, the parties involved opening their doors and exiting the vehicle, inspecting the damage and negotiating with each other, and subsequent events such as vehicles swerving to avoid the accident.

[0004] Currently, there are two main methods for video-based traffic accident detection. The first method determines whether a traffic accident has occurred by judging whether there are stationary vehicles in the image. This method only uses the stationary motion state of vehicles to identify traffic accidents, which can easily lead to false detections. This is because if there is congestion, vehicles will also be stationary for a period of time, resulting in false detections and misidentifying non-traffic accidents as traffic accidents. The second method obtains road features, environmental features, and traffic flow features, and then inputs them into a pre-trained traffic accident recognition model to obtain the traffic accident information recognition results on the road. This method uses a traffic accident recognition model for recognition, which has a large computational load, poor real-time performance, and due to the complexity of urban roads, such as obstruction and the angle of monitoring equipment, it has poor generalization for untrained scenarios, which can easily lead to false detections and missed detections.

[0005] Therefore, it is necessary to analyze road surveillance videos based on the elements constituting a traffic accident in order to improve the accuracy of traffic accident identification and to promptly and automatically report traffic accident incidents. Summary of the Invention

[0006] This invention provides a multi-element fusion real-time traffic accident identification method based on video analysis. By acquiring real-time video streams using electronic police cameras and roadside monitoring equipment in the middle of urban roads, the method automatically identifies whether multiple elements of a traffic accident meet the conditions for traffic accident occurrence based on video analysis, and automatically reports traffic accident incidents. This enables traffic regulatory departments to promptly detect and handle traffic accident events, effectively ensuring traffic safety and smooth traffic flow.

[0007] This invention provides a multi-element fusion real-time traffic accident recognition method based on video analysis, comprising:

[0008] Configure traffic accident detection areas: For each surveillance video, configure multiple traffic accident detection areas, wherein the traffic accident detection areas are arbitrary polygonal areas;

[0009] Accessing real-time video streams: Using the RTSP protocol to access real-time video streams from road monitoring, and then outputting the bitstream to the decoding module;

[0010] Ascend DVPP hardware decoding: The received bitstream is input frame by frame into the Ascend DVPP hardware decoding module, which uses a dedicated decoding chip to decode and output the decoded monitoring image data.

[0011] Ascend NPU Target Detection: Based on Ascend NPU, an improved Yolov5 network model is used to perform target detection on each frame of decoded surveillance image data, and output vehicle and pedestrian target detection information;

[0012] Vehicle and pedestrian target tracking: The Bytetrack tracking algorithm is used to track vehicles and pedestrians separately and obtain tracking information for each target;

[0013] Multi-factor fusion calculation of traffic accident probability: For each frame of monitoring image data, the element value is calculated based on nine pre-designed elements and the tracking information of each target. Then, the weighted fusion calculation of all element values ​​is performed to obtain the probability of a traffic accident.

[0014] Determining the type of traffic accident: Based on the number of traffic accidents detected in a monitoring scene, determine whether it is a multi-vehicle traffic accident;

[0015] Identify vehicle information in traffic accidents: Use a license plate detection algorithm model to identify the license plates of vehicles involved in traffic accidents. If the license plate is obscured, identify the vehicle type information.

[0016] Reporting traffic accident information: The identified traffic accident type, vehicle information, and accident location are reported to the traffic command center so that traffic regulatory departments can handle the traffic accident in a timely manner.

[0017] In an optional embodiment, the preprocessing of the target detection involves converting the format of the monitoring image data from YUV to RGB format using an NPU.

[0018] In an optional embodiment, the improved Yolov5 network model replaces the cspdarknet backbone network in Yolov5 with a shufflenet lightweight network.

[0019] In an optional embodiment, the nine pre-designed elements include the duration of vehicle stationary, the motion state of the vehicle before it stops, the intersection state of the stationary vehicle and the moving vehicle, the intersection state of the stationary vehicle's trajectory before it stops with the trajectory of the moving vehicle, the duration of the stationary vehicle's intersection with other stationary vehicles, the acceleration change state of the stationary vehicle before it stops, the change state of the stationary vehicle's motion direction before it stops, the state of the first appearance of people around the stationary vehicle, and the state of pedestrians around the stationary vehicle.

[0020] Accordingly, the element values ​​are calculated based on the pre-designed nine elements and the tracking information of each target. Then, a weighted fusion calculation is performed on all element values ​​to obtain the probability of a traffic accident, including:

[0021] Based on the tracking information of each target, the stationary vehicles and the duration of vehicle stationary time in the monitoring scene are determined as the first element, and it is determined whether the first element is greater than the preset duration threshold.

[0022] If so, calculate elements two through nine, as follows:

[0023] The second element, MT, is calculated based on the tracking information of stationary vehicles before they come to a stop.

[0024] The number of intersections between stationary and moving vehicles is calculated based on the tracking information of stationary and moving vehicles and is used as the third factor MN.

[0025] The fourth element, TN, is calculated based on the tracking information of stationary vehicles and moving vehicles, which is the number of times the trajectory of the stationary vehicle before it comes to a stop intersects with the trajectory of the moving vehicle.

[0026] The fifth element, ST, is calculated based on the tracking information of stationary vehicles, showing the duration of intersection between stationary vehicles and other stationary vehicles.

[0027] The sixth element, MA, is calculated based on the tracking information of stationary vehicles before they come to a complete stop.

[0028] The change in the direction of motion of a stationary vehicle before it comes to a stop is calculated based on the tracking information of the stationary vehicle as the seventh element MD;

[0029] The eighth element, FN, is calculated based on the tracking information of stationary vehicles and pedestrians to determine whether anyone appears around a stationary vehicle for the first time.

[0030] The number of times pedestrians were detected around stationary vehicles was calculated based on the tracking information of stationary vehicles and pedestrians, and this number was used as the ninth element, PN.

[0031] The probability of a traffic accident is calculated as P = V * Transpose(W) based on the obtained monitoring scene element vector V = [MT MN TN ST MA MD FN PN] and the preset element coefficient vector W = [w1 w2 w3 w4 w5 w6 w7 w8].

[0032] If P is greater than the preset traffic accident threshold PT, then a traffic accident is determined to have occurred; otherwise, it is determined that no traffic accident has occurred.

[0033] In an optional embodiment, after determining the stationary vehicles and their stationary duration in the monitoring scene based on the tracking information of each target as the first element, the method further includes:

[0034] If the original stationary vehicle state is detected to change from stationary to moving, then the motion of the stationary vehicle is determined based on the tracking information of the current frame and the previous frame. Specifically, the smaller of the width and height of the detection box of the stationary vehicle in the current frame is selected as the reference value, and the product of the reference value and the first threshold coefficient and the second threshold coefficient is calculated to obtain the first threshold and the second threshold. If the displacement of the stationary vehicle between the current frame and the previous frame is less than the first threshold, or the current speed of the stationary vehicle is less than the second threshold, then it is determined that the stationary vehicle has not moved.

[0035] If a new stationary vehicle's ID is detected to have changed, then the tracking information of the new stationary vehicle and the tracking information of previously cached stationary vehicles are used to determine whether the new stationary vehicle is a previously cached stationary vehicle. Specifically, the crossover ratio (CRR) between the newly detected stationary vehicle and the previously cached stationary vehicles is calculated. If the CRR is greater than a preset CRR threshold, then the newly detected stationary vehicle and the previously cached stationary vehicle are determined to be the same target, and the stationary duration of the previously cached stationary vehicles is continued to be accumulated.

[0036] This invention provides a real-time traffic accident recognition method based on video analysis and multi-element fusion. Addressing the issue of low accuracy in traffic accident recognition in urban road scenarios, this method utilizes the Ascend 710 high-performance computing chip. It employs an improved YOLOv5 network model to detect vehicles and pedestrians, and uses Bytetrack real-time tracking technology to determine the movement state and trajectory of the targets. Through multi-element fusion, traffic accidents are identified and reported in real-time to the traffic control center. The traffic control center automatically forwards the traffic accident information to the nearest traffic police officer for handling, forming a closed loop. This improves the efficiency of handling traffic accidents on urban roads, ensuring traffic safety and smooth traffic flow. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of a scenario architecture upon which this disclosure is based;

[0039] Figure 2 A flowchart illustrating a multi-element fusion real-time traffic accident recognition method based on video analysis, provided in an embodiment of this disclosure;

[0040] Figure 3 This is a flowchart of the multi-factor fusion calculation process for traffic accident determination provided in this disclosure. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] This application addresses urban road traffic accidents by implementing traffic accident recognition using an Ascend 710 edge computing device. It connects to existing surveillance video sources, performs hardware decoding using the Ascend DVPP device, then uses DVPP's hardware image processing capabilities for image scaling, followed by NPU-based color gamut conversion via AIPP, and finally uses the NPU to perform Yolov5 model inference to detect vehicles and pedestrians. The detected targets are then input into the Bytetrack tracking module for real-time tracking, acquiring the target's motion state and trajectory. A multi-factor fusion method is then designed to combine various elements of vehicles and pedestrians for calculation and judgment, determining the probability of a traffic accident and identifying different accident types such as single-vehicle and multi-vehicle accidents. The traffic accident type and vehicle information are then reported to the platform, notifying the command center for timely response.

[0043] Figure 1 This is a schematic diagram of a scenario architecture upon which this disclosure is based, such as... Figure 1 As shown, the scenario architecture upon which this disclosure is based may include a traffic accident recognition device 1 and a monitoring system 2.

[0044] The traffic accident identification device 1 is hardware or software that can interact with the monitoring device 2 via a network, and it can be used to execute the traffic accident identification methods described in the following embodiments.

[0045] When the traffic accident recognition device 1 is hardware, it can be an electronic device with computing capabilities. When the traffic accident recognition device 1 is software, it can be installed in an electronic device with computing capabilities. Such electronic devices include, but are not limited to, servers, cloud computing, and desktop computers.

[0046] Specifically, the monitoring device 2 can be a hardware device with shooting function, such as a bullet camera, a PTZ camera, or a micro-camera, while the traffic accident recognition device 1 can be a server integrated or installed on the monitoring device 2.

[0047] Traffic accident recognition device 1 can operate on monitoring device 2 and provide traffic accident recognition services to monitoring device 2. Furthermore, traffic accident recognition device 1 uses its display or display component to display the traffic accident recognition results to the user.

[0048] Of course, in other application scenarios, the traffic accident recognition device 1 can also be integrated into a server for processing vehicle video. In this case, the monitoring device 2 can be a device that can communicate and interact with the aforementioned traffic accident recognition device 1 via a network, including bullet cameras, PTZ cameras, and micro-monitoring cameras. The monitoring device 2 can then send real-time video streams to the traffic accident recognition device 1, so that the traffic accident recognition device 1 can perform traffic accident recognition on the real-time video stream using the method shown below.

[0049] The following will further explain the multi-factor fusion real-time traffic accident recognition method based on video analysis provided in this application:

[0050] Figure 2 This is a flowchart illustrating a multi-element fusion real-time traffic accident recognition method based on video analysis, provided in Embodiment 1 of this disclosure. Figure 2 As shown in the embodiments of this disclosure, a multi-element fusion real-time traffic accident recognition method based on video analysis is provided, comprising:

[0051] Configure traffic accident detection areas: For each surveillance video, configure multiple traffic accident detection areas, wherein the traffic accident detection areas are arbitrary polygonal areas;

[0052] Accessing real-time video streams: Using the RTSP protocol to access real-time video streams from road monitoring, and then outputting the bitstream to the decoding module;

[0053] Ascend DVPP hardware decoding: The received bitstream is input frame by frame into the Ascend DVPP hardware decoding module, which uses a dedicated decoding chip to decode and output the decoded monitoring image data.

[0054] Ascend NPU Target Detection: Based on Ascend NPU, an improved Yolov5 network model is used to perform target detection on each frame of decoded surveillance image data, and output vehicle and pedestrian target detection information;

[0055] To maximize computational performance, given the hardware characteristics of Ascend devices, a DVV+AIPP scheme was adopted for object detection preprocessing. Based on this scheme, the original image is scaled down to the model input size using DVPP hardware scaling. At this point, the image format is still YUV, while inference computation uses RGB format. AIPP then utilizes its NPU to convert the image format from YUV to RGB. This means that the entire image preprocessing stage in object detection is handled in hardware, significantly improving preprocessing performance.

[0056] To address the performance bottleneck of edge device detection algorithms, this paper proposes replacing the CSP Darknet backbone network in YOLOv5 with the lightweight ShuffleNet network. This improves speed while leveraging the multi-channel fusion features of Shuffle convolution in ShuffleNet to further enhance pedestrian feature extraction capabilities, thus enabling better detection of small targets. Since pedestrians are much smaller than vehicles, ShuffleNet improves performance while enhancing the detection of small targets.

[0057] Since the output of the YOLOv5 model includes the outputs of three feature maps, all targets need to be merged and NMS calculated on the CPU. To improve post-processing performance, this implementation combines the three post-processing steps into one, allowing direct computation within the NPU. Furthermore, the NMS operator is encapsulated using the Ascend ACL language and directly embedded into the YOLOv5 model network, also computed on the NPU. When dealing with a large number of targets, using the NPU to compute NMS offers a significant performance improvement over the CPU.

[0058] Vehicle and pedestrian target tracking: The Bytetrack tracking algorithm is used to track vehicles and pedestrians separately and obtain tracking information for each target;

[0059] Multi-factor fusion calculation of traffic accident probability: For each frame of monitoring image data, the element value is calculated based on nine pre-designed elements and the tracking information of each target. Then, the weighted fusion calculation of all element values ​​is performed to obtain the probability of a traffic accident.

[0060] To determine whether a traffic accident has occurred, it is necessary to start from the elements that constitute a traffic accident. By analyzing the elements that constitute a traffic accident, multiple elements can be selected, and it is necessary to ensure that the selected elements are orthogonal. Only in this way can a traffic accident be more accurately identified from a space of all elements.

[0061] Based on the analysis of real traffic accidents in various scenarios, the following nine elements were selected to characterize a traffic accident. Although there are many elements, not every traffic accident necessarily contains all nine elements. When identifying a traffic accident, it is only necessary to determine whether most of the elements are met.

[0062] The nine pre-selected elements include the duration of vehicle stationary position, the vehicle's motion state before stationary position, the intersection state between stationary and moving vehicles, the intersection state between the stationary vehicle's trajectory before stationary position and the trajectory of the moving vehicle, the duration of intersection between stationary and other stationary vehicles, the acceleration change state of the stationary vehicle before stationary position, the change state of the stationary vehicle's motion direction before stationary position, the first appearance of people around the stationary vehicle, and the presence of pedestrians around the stationary vehicle.

[0063] First factor: Vehicle stationary time

[0064] This element refers to the time Tsv that is continuously counted from the moment vehicle Vs is detected to be stationary. Once Vs is detected to start moving, Tsv is reset to 0. Because in traffic accidents, there will definitely be stationary vehicles present, and they will remain stationary for a long time, usually at least a few minutes. Therefore, the duration of stationary status is selected as the first element and is a prerequisite element.

[0065] The second element: the vehicle's state of motion before it came to a complete stop.

[0066] This element refers to whether Vs was in motion before coming to a stop, and the duration of motion must meet a certain time threshold Tmv. Because traffic accidents occur while in motion, and at least one of the vehicles involved in a traffic accident changes from motion to a stop, this element is selected to determine the state of motion before coming to a stop.

[0067] The third element: the intersection state of stationary and moving vehicles.

[0068] This element refers to whether Vs intersects with a moving vehicle Vm. After a traffic accident, if traffic flow is heavy, vehicles in adjacent lanes will pass by, potentially causing Vs and Vm to intersect. This element is introduced to exclude scenarios where a stationary vehicle's actions are merely a stopping event rather than a traffic accident. Because after a traffic accident, the speed of vehicles around the stationary vehicle will slow down, the probability of Vs and Vm intersecting will greatly increase. Therefore, this element is selected to determine whether any vehicles pass by the accident vehicle after the accident.

[0069] The fourth element: the intersection state of the trajectory of the stationary vehicle before it came to a stop with the trajectory of the moving vehicle.

[0070] This element refers to whether the trajectory of Vs before it comes to a stop intersects with the trajectory of a moving vehicle. After a traffic accident, if there are vehicles behind the accident vehicle, these vehicles will pass by Vs, and Vs's previous trajectory is highly likely to intersect with the trajectories of other moving vehicles. This element is introduced for the same reason as element 3, to exclude scenarios where the stationary vehicle is merely stopping and not involved in a traffic accident. Because after a traffic accident, some vehicles will detour, and one result of this detour is the intersection of trajectories; therefore, this element is selected to determine whether any vehicles detour around the accident vehicle after the accident.

[0071] Fifth element: Intersection time between stationary vehicles and other stationary vehicles

[0072] This element refers to whether two stationary vehicles that satisfy element 1 intersect for an extended period of time. Because if a multi-vehicle accident occurs, all vehicles involved will be stationary, and an intersection is possible, which could last for a considerable time. Therefore, this element is introduced to determine whether a multi-vehicle accident has occurred.

[0073] Sixth element: The state of acceleration change of a stationary vehicle before it comes to a complete stop.

[0074] This element refers to the change in acceleration before Vs comes to a stop. Because when a traffic accident occurs, the vehicle may experience a sudden increase or decrease in acceleration, this is used as an element to identify traffic accidents.

[0075] Seventh element: The change in the direction of motion of a stationary vehicle before it comes to a complete stop.

[0076] This element refers to the change in the direction of motion of Vs before it comes to a stop. Because when a traffic accident occurs, the vehicle may suddenly turn from a normal direction of travel, resulting in a significant change in direction, this is used as an element to identify traffic accidents.

[0077] Eighth element: The first appearance of people around a stationary vehicle.

[0078] This element refers to the detection of person P, and P being detected first in the vicinity of Vs. Because people may get out of vehicles during a traffic accident, the first detection of P would be in the vicinity of Vs. This element can exclude situations where there are people around the vehicle, but they are pedestrians passing by Vs, rather than people getting out of the vehicle.

[0079] Ninth element: Pedestrian presence around stationary vehicles.

[0080] This element refers to the prolonged detection of people around Vs. Because after a traffic accident, people typically get out of their vehicles and surround the wrecked vehicles to assess the situation, their presence will be detected around Vs for an extended period. Therefore, this is considered an element for identifying traffic accidents.

[0081] like Figure 3 As shown, the next step is to calculate the element values ​​based on the nine pre-selected elements and the tracking information of each target. Then, a weighted fusion calculation is performed on all the element values ​​to obtain the probability of a traffic accident, including:

[0082] Based on the tracking information of each target, the stationary vehicles and the duration of vehicle stationary time in the monitoring scene are determined as the first element, and it is determined whether the first element is greater than the preset duration threshold.

[0083] If so, calculate elements two through nine, as follows:

[0084] The second element, MT, is calculated based on the tracking information of stationary vehicles before they come to a stop.

[0085] The number of intersections between stationary and moving vehicles is calculated based on the tracking information of stationary and moving vehicles and is used as the third factor MN.

[0086] The fourth element, TN, is calculated based on the tracking information of stationary vehicles and moving vehicles, which is the number of times the trajectory of the stationary vehicle before it comes to a stop intersects with the trajectory of the moving vehicle.

[0087] The fifth element, ST, is calculated based on the tracking information of stationary vehicles, showing the duration of intersection between stationary vehicles and other stationary vehicles.

[0088] The sixth element, MA, is calculated based on the tracking information of stationary vehicles before they come to a complete stop.

[0089] The change in the direction of motion of a stationary vehicle before it comes to a stop is calculated based on the tracking information of the stationary vehicle as the seventh element MD;

[0090] The eighth element, FN, is calculated based on the tracking information of stationary vehicles and pedestrians to determine whether anyone appears around a stationary vehicle for the first time.

[0091] The number of times pedestrians were detected around stationary vehicles was calculated based on the tracking information of stationary vehicles and pedestrians, and this number was used as the ninth element, PN.

[0092] The probability of a traffic accident, P = V * Transpose(W), is calculated based on the obtained monitoring scene element vector V = [MT MN TN ST MA MD FN PN] and the preset element coefficient vector W = [w1 w2 w3 w4 w5 w6 w7 w8]. The second to ninth elements correspond to coefficients w1 to w8, respectively. The coefficients of each element are different according to their importance. For example, the number of times pedestrians are detected around a stationary vehicle is more important than other elements, so its coefficient w8 can be larger than other coefficient values.

[0093] If P is greater than the preset traffic accident threshold PT, then a traffic accident is determined to have occurred; otherwise, it is determined that no traffic accident has occurred.

[0094] As can be seen from the nine elements of a traffic accident mentioned above, the entire process revolves around stationary vehicles. This is crucial for accurate accident identification, as the first element plays a vital role as a prerequisite. Therefore, the accurate identification and continuous timing of stationary vehicles are critical to the accuracy of accident recognition. Obstruction by objects (including vehicles, pedestrians, and moving objects) can break the target tracking chain, resulting in loss of the target. Re-detection triggers a restart of the timing, leading to inaccurate calculation of stationary duration and hindering accurate accident identification. To overcome the problem of broken tracking chains, feature-matching tracking algorithms such as Deepsort can be used. However, these algorithms involve model inference, increasing tracking time. Real-time tracking becomes impossible, especially during peak hours when there is a large amount of vehicle data on the road. Therefore, this paper uses a geometric approach to address the inaccurate calculation of stationary duration, primarily through two strategies, focusing on correcting for previously stationary vehicles that suddenly become moving targets.

[0095] 1) Determine whether the motion state of a stationary vehicle has actually changed.

[0096] This mainly considers the situation where the tracking result changes from a static state to a moving state due to occlusion or other reasons. Although the tracking result shows a moving state, the target has not actually moved. The inaccurate target detection is caused by partial occlusion. To solve this problem, the following formula is used to determine whether the target has actually moved.

[0097] Ref_len=min(box.width,box.height)

[0098] If Mov_len <Ref_len*f1 or Speed<Ref_len*f2:

[0099] Static_state

[0100] Else:

[0101] Move_state

[0102] Where box is the target detection bounding box in the current frame, Move_len is the displacement of the target between the current frame and the previous frame, Speed ​​is the target's current velocity, and f1 and f2 are the first threshold coefficient and the second threshold coefficient, respectively. If a target was previously stationary, and the tracking result in the current frame shows a moving state, then after being determined by the above formula, if it is found to be stationary, then it is still considered to be stationary.

[0103] 2) Determine if the target is re-ID

[0104] When a target remains stationary for an extended period, other targets inevitably pass in front of it, obscuring it. When the stationary target is detected and tracked again, its ID has changed. To address this, an Interchange of Union (IOU) operation is performed between the newly detected stationary target NS and the previously cached stationary target PS. If the IOU value is greater than a threshold, the newly detected stationary target NS and the previously cached stationary target PS are considered the same target. Therefore, when NS is detected again, it is considered PS, and the stationary duration of PS is continuously increased. This avoids the inaccurate stationary target duration issue caused by Re-ID.

[0105] Determining the type of traffic accident: Based on the number of traffic accidents detected in a monitoring scene, determine whether it is a multi-vehicle traffic accident;

[0106] If more than one vehicle in the scene satisfies P>PT, it is considered a multi-vehicle accident; otherwise, it is considered a single-vehicle accident.

[0107] Identify vehicle information in traffic accidents: Use a license plate detection algorithm model to identify the license plates of vehicles involved in traffic accidents. If the license plate is obscured, identify the vehicle type information.

[0108] Reporting traffic accident information: The identified traffic accident type, vehicle information, and accident location are reported to the traffic command center so that traffic regulatory departments can handle the traffic accident in a timely manner.

[0109] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A multi-element fusion real-time traffic accident recognition method based on video analysis, characterized in that, include: Configure traffic accident detection areas: For each surveillance video, configure multiple traffic accident detection areas, wherein the traffic accident detection areas are arbitrary polygonal areas; Accessing real-time video streams: Using the RTSP protocol to access real-time video streams from road monitoring, and then outputting the bitstream to the decoding module; Ascend DVPP hardware decoding: The received bitstream is input frame by frame into the Ascend DVPP hardware decoding module, which uses a dedicated decoding chip to decode and output the decoded monitoring image data. Ascend NPU Target Detection: Based on Ascend NPU, an improved Yolov5 network model is used to perform target detection on each frame of decoded surveillance image data, and output vehicle and pedestrian target detection information; Vehicle and pedestrian target tracking: The Bytetrack tracking algorithm is used to track vehicles and pedestrians separately and obtain tracking information for each target; Multi-factor fusion calculation of traffic accident probability: For each frame of monitoring image data, the element value is calculated based on nine pre-designed elements and the tracking information of each target. Then, the weighted fusion calculation of all element values ​​is performed to obtain the probability of a traffic accident. Determining the type of traffic accident: Based on the number of traffic accidents detected in a monitoring scene, determine whether it is a multi-vehicle traffic accident; Identify vehicle information in traffic accidents: Use a license plate detection algorithm model to identify the license plates of vehicles involved in traffic accidents. If the license plate is obscured, identify the vehicle type information. Reporting traffic accident information: Report the identified traffic accident type, vehicle information, and accident location to the traffic command center so that traffic regulatory authorities can handle the traffic accident in a timely manner; The nine pre-designed elements include the duration of vehicle stationary position, the motion state of the vehicle before it stops, the intersection state of stationary and moving vehicles, the intersection state of the trajectory of the stationary vehicle before it stops with the trajectory of the moving vehicle, the duration of intersection of the stationary vehicle with other stationary vehicles, the acceleration change state of the stationary vehicle before it stops, the change state of the motion direction of the stationary vehicle before it stops, the state of the first appearance of people around the stationary vehicle, and the state of the presence of pedestrians around the stationary vehicle. Accordingly, the element values ​​are calculated based on the nine pre-designed elements and the tracking information of each target. Then, a weighted fusion calculation is performed on all element values ​​to obtain the probability of a traffic accident, including: Based on the tracking information of each target, the stationary vehicles and the duration of vehicle stationary time in the monitoring scene are determined as the first element, and it is determined whether the first element is greater than the preset duration threshold. If so, calculate elements two through nine, as follows: The second element, MT, is calculated based on the tracking information of stationary vehicles before they come to a stop. The number of intersections between stationary and moving vehicles is calculated based on the tracking information of stationary and moving vehicles and is used as the third factor MN. The fourth element, TN, is calculated based on the tracking information of stationary vehicles and moving vehicles, which is the number of times the trajectory of the stationary vehicle before it comes to a stop intersects with the trajectory of the moving vehicle. The fifth element, ST, is calculated based on the tracking information of stationary vehicles, showing the duration of intersection between stationary vehicles and other stationary vehicles. The sixth element, MA, is calculated based on the tracking information of stationary vehicles before they come to a complete stop. The change in the direction of motion of a stationary vehicle before it comes to a stop is calculated based on the tracking information of the stationary vehicle as the seventh element MD; The eighth element, FN, is calculated based on the tracking information of stationary vehicles and pedestrians to determine whether anyone appears around a stationary vehicle for the first time. The number of times pedestrians were detected around stationary vehicles was calculated based on the tracking information of stationary vehicles and pedestrians, and this number was used as the ninth element, PN. The probability of a traffic accident is calculated as P = V * Transpose(W) based on the obtained monitoring scene element vector V = [MT MN TN ST MA MD FN PN] and the preset element coefficient vector W = [w1 w2 w3 w4 w5 w6 w7 w8]. If P is greater than the preset traffic accident threshold PT, then a traffic accident is determined to have occurred; otherwise, no traffic accident is determined to have occurred.

2. The multi-element fusion real-time traffic accident recognition method based on video analysis according to claim 1, characterized in that, The preprocessing for target detection involves converting the format of the monitoring image data from YUV to RGB using an NPU.

3. The multi-element fusion real-time traffic accident recognition method based on video analysis according to claim 1, characterized in that, The improved Yolov5 network model replaces the cspdarknet backbone network in Yolov5 with the shufflenet lightweight network.

4. The multi-element fusion real-time traffic accident recognition method based on video analysis according to claim 1, characterized in that, After determining the stationary vehicles and their stationary duration in the monitoring scene based on the tracking information of each target as the first element, the method further includes: If the original stationary vehicle state is detected to change from stationary to moving, then the motion of the stationary vehicle is determined based on the tracking information of the current frame and the previous frame. Specifically, the smaller of the width and height of the detection box of the stationary vehicle in the current frame is selected as the reference value, and the product of the reference value and the first threshold coefficient and the second threshold coefficient is calculated to obtain the first threshold and the second threshold. If the displacement of the stationary vehicle between the current frame and the previous frame is less than the first threshold, or the current speed of the stationary vehicle is less than the second threshold, then it is determined that the stationary vehicle has not moved. If a new stationary vehicle's ID is detected to have changed, then the tracking information of the new stationary vehicle and the tracking information of previously cached stationary vehicles are used to determine whether the new stationary vehicle is a previously cached stationary vehicle. Specifically, the crossover ratio (CRR) between the newly detected stationary vehicle and the previously cached stationary vehicles is calculated. If the CRR is greater than a preset CRR threshold, then the newly detected stationary vehicle and the previously cached stationary vehicle are determined to be the same target, and the stationary duration of the previously cached stationary vehicles is continued to be accumulated.

Images