Traffic accident identification processing method, device and equipment based on monitoring video stream

Abstract

The invention provides a traffic accident recognition processing method, device and equipment based on a monitoring video stream, and relates to the technical field of image processing, and the method comprises the steps: carrying out the traffic accident recognition of the monitoring video stream based on a deep learning target detection model, and obtaining a target video frame of a recognized traffic accident; aiming at each target video frame, taking a plurality of continuous frames containing the target video frame in the monitoring video stream as a judgment set of the target video frame, traversing video frames with traffic accident detection frames in the judgment set, and executing a first operation until all the traffic accident detection frames of the target video frame have confirmation marks, or judging that all the video frames with the traffic accident detection frames are traversed; and obtaining a traffic accident identification result of the target video frame based on the traffic accident detection frame with the confirmation mark in the target video frame. By adopting the technical scheme, the problem of high false detection rate of the existing traffic accident identification mode is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device and equipment for identifying and processing traffic accidents based on monitoring video streams. Background Art

[0002] As one of the main means of monitoring road traffic incidents today, monitoring video streams helps managers obtain more comprehensive road traffic information in real time, so as to respond promptly to traffic accidents and make corresponding decisions. The commonly used accident identification method based on monitoring video streams is manual monitoring and identification.

[0003] However, manual monitoring and identification consumes a lot of human resources and time, and it is difficult to quickly detect and respond to accidents. Based on this, accident identification based on deep learning models has been further developed. With the help of convolutional neural networks, traffic accident data sets are constructed and models are trained. With video streams as input, the trained target detection model is used to identify whether a traffic accident has occurred in the analysis area.

[0004] However, the existing traffic accident recognition based on deep learning has a high false positive rate. Summary of the invention

[0005] The present invention provides a traffic accident identification and processing method, device and equipment based on monitoring video stream, which are used to solve the problem of high misdetection rate of traffic accidents in existing traffic accident identification methods.

[0006] In a first aspect, the present invention provides a traffic accident identification and processing method based on a surveillance video stream, comprising: performing traffic accident identification on a surveillance video stream based on a deep learning target detection model, and obtaining a target video frame in which the traffic accident is identified; for each target video frame, taking multiple continuous frames containing the target video frame in the surveillance video stream as a judgment set of the target video frame, wherein the target video frame is the last frame of the multiple continuous frames; traversing the video frames with traffic accident detection frames in the judgment set and performing a first operation until all traffic accident detection frames with the target video frame are marked with confirmation marks, or all video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation comprises: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame with the same accident type as the first traffic accident detection frame and without a confirmation mark in the subsequent frame of the video frame, determining the detection frame accurately identified in each first traffic accident detection frame and each second traffic accident detection frame, and the accurately identified detection frame is marked with a confirmation mark; based on the traffic accident detection frame with a confirmation mark in the target video frame, obtaining the traffic accident identification result of the target video frame.

[0007] Optionally, based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame in the subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and has no confirmation mark, each first traffic accident detection frame and each second traffic accident detection frame that are accurately identified are determined, including: for each first traffic accident detection frame, determining a set of detection frames in each second traffic accident detection frame that have the same accident type as the first traffic accident detection frame; determining an intersection-and-union ratio of the first traffic accident detection frame with each detection frame in the detection frame set based on the coordinate information of the first traffic accident detection frame and the coordinate information of each detection frame in the detection frame set; determining the first traffic accident detection frame and the accurately identified detection frame in the detection frame set based on the detection frames in the detection frame set whose corresponding intersection-and-union ratios are greater than or equal to a first threshold.

[0008] Optionally, based on the detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than or equal to a first threshold, the first traffic accident detection frame and the accurately identified detection frames in the detection frame set are determined, including: if the number of detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than or equal to the first threshold is greater than or equal to a second threshold, then it is determined that the first traffic accident detection frame and the detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than the first threshold are both accurately identified detection frames.

[0009] Optionally, the accident types include four categories: pedestrian falling, two-wheeled vehicle falling, vehicle damage, and vehicle collision.

[0010] Optionally, the method also includes: if the traffic accident detection box with a confirmation mark in the target video frame includes a vehicle collision detection box, then based on the historical tracking information of each vehicle target in the vehicle collision detection box and the detection box information of each vehicle target, determining the accident probability of each vehicle target; based on the accident probability of each vehicle target, further confirming whether there is a false detection in the vehicle collision detection box.

[0011] Optionally, based on the historical tracking information of each vehicle target in the vehicle collision detection frame and the detection frame information of each vehicle target, the accident probability of each vehicle target is determined, including: for each vehicle target, based on the historical tracking information of each vehicle target, determining the acceleration change value, speed change value, angle change value, and collision angle value of the vehicle target with other vehicle targets in the vehicle collision detection frame within a historical period; determining the judgment vector of the vehicle target based on the acceleration change value, speed change value, angle change value, collision angle value, and the intersection and union ratio of the detection frame of the vehicle target and the detection frames of other vehicle targets; determining the accident probability of the vehicle target based on the judgment vector and factor coefficient of the vehicle target.

[0012] In a second aspect, the present invention provides a traffic accident identification and processing device based on a surveillance video stream, comprising: a first processing module, for performing traffic accident identification on a surveillance video stream based on a deep learning target detection model, and obtaining a target video frame for identifying a traffic accident; a second processing module, for taking, for each target video frame, a plurality of consecutive frames containing the target video frame in the surveillance video stream as a judgment set of the target video frame, wherein the target video frame is the last frame of the plurality of consecutive frames; a third processing module, for traversing the video frames with traffic accident detection frames in the judgment set and performing a first operation until all the traffic accident detection frames in the target video frame are marked with a confirmation mark, or the judgment set contains a plurality of consecutive frames. All video frames with traffic accident detection frames are traversed; wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame in the subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and has no confirmation mark, determine the accurately identified detection frame in each first traffic accident detection frame and each second traffic accident detection frame, and the accurately identified detection frame is marked with a confirmation mark; a fourth processing module is used to obtain the traffic accident recognition result of the target video frame based on the traffic accident detection frame with a confirmation mark in the target video frame.

[0013] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the above-mentioned traffic accident identification and processing methods based on surveillance video streams.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned traffic accident identification and processing methods based on surveillance video streams.

[0015] In a fifth aspect, the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned traffic accident identification and processing methods based on monitoring video streams.

[0016] The traffic accident identification and processing method, device and equipment based on the monitoring video stream provided by the present invention obtains the target video frame for identifying the traffic accident based on the target detection model, and constructs a judgment set containing multiple continuous frames for each target video frame by time backtracking to assist in the recognition and confirmation of the target video frame. Furthermore, since the position usually does not change further after the traffic accident occurs, the present invention combines the coordinate information of the detection frame in the judgment set to determine whether the recognition result of the target detection model is accurate, adds a confirmation mark to the detection frame that is accurately identified, and uses the detection frame with the confirmation mark as the final traffic accident recognition result. The present invention performs a second confirmation on the recognition result of the target detection model through the coordinate information of the detection frame, filters out the recognition errors, reduces the recognition false detection rate, and improves the reliability of the traffic accident recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the traffic accident identification and processing method based on monitoring video stream provided by the present invention.

[0019] Figure 2 This is a flow chart for identifying traffic accidents provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the process of false detection filtering provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of a traffic accident identification and processing device based on monitoring video stream provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Figure 1FIG. 1 is a flow chart of a traffic accident identification and processing method based on monitoring video stream provided by the present invention. Figure 1 As shown, the method includes the following steps S101 to S104.

[0025] Step S101: Perform traffic accident recognition on the monitoring video stream based on the deep learning target detection model to obtain the target video frame in which the traffic accident is recognized.

[0026] As an optional embodiment, the target detection model can be any deep learning model for identifying video frames in surveillance videos where traffic accidents occur. For example, R-CNN series models, YOLO series models, SSD models, RetinaNet models, EfficientDet models, Cascade R-CNN models, CornerNet models, etc. The present invention does not impose any further restrictions on the models used for traffic accident identification. In addition, as described in the background technology, how to identify traffic accidents in surveillance video streams based on deep learning target detection models and obtain target video frames that identify traffic accidents is a prior art and will not be repeated here.

[0027] As an optional embodiment, the surveillance video stream is composed of a series of continuous video frames, which are usually captured at a speed of multiple frames per second. The surveillance video stream is usually encoded to reduce the file size and improve the transmission efficiency.

[0028] As an optional embodiment, as mentioned above, since the surveillance video stream is usually encoded to improve transmission efficiency, when the surveillance video stream is input into the deep learning-based target detection model for traffic accident identification, the surveillance video stream needs to be decoded into frame-by-frame image data.

[0029] As an optional embodiment, the target detection model includes a detection frame and a recognition result in the identified target video frame, and the recognition result includes the identified accident type and the detection frame coordinate information.

[0030] As an optional embodiment, the present invention does not further limit the training method and model structure of the target detection model. By collecting the image data of the car accident scene under the monitoring perspective, a traffic accident data set that meets the training requirements can be constructed. The target detection algorithm based on deep learning is trained with the constructed accident data set, and the model parameters are debugged until the accuracy requirements are met to obtain the applicable traffic accident detection weights, thereby completing the training of the target detection model.

[0031] As an optional embodiment, as mentioned above, the monitoring video stream is composed of a series of continuous video frames, which are usually captured at a speed of multiple frames per second, but for the sake of ease of explanation and understanding, only one frame per second is used for explanation. For example, the monitoring video stream has a total of 10 seconds, and there is one frame of image data per second, so there are a total of 10 frames of image data. YOLO8 is selected as the target detection model. The above embodiment can be understood as inputting a 10-second monitoring video stream into the YOLO8 model, and the YOLO8 model performs traffic accident recognition on the 10-second monitoring video stream. The recognition result is to determine that a traffic accident has occurred in the video frames of 5 seconds, 6 seconds, 7 seconds and 8 seconds. The video frames of 5 seconds, 6 seconds, 7 seconds and 8 seconds in this embodiment are the target video frames in the above embodiment.

[0032] Step S102: for each target video frame, a plurality of continuous frames including the target video frame in the monitoring video stream are used as a judgment set of the target video frame, wherein the target video frame is the last frame of the plurality of continuous frames.

[0033] As an optional embodiment, multiple consecutive frames including the target video frame can be obtained by backtracking. During the backtracking process, it is only necessary to ensure that the target video frame is the last frame of multiple consecutive frames. The present invention does not limit the backtracking length and can be flexibly set based on the monitoring video stream.

[0034] As an optional embodiment, the judgment set of the target video frame can be determined by setting the number of backtracked frames or the backtracked time. For example, if the target video frame is the first frame of the fifth second, the five seconds before the fifth second can be backtracked, and all consecutive frames within the five seconds before the fifth second and the frame of the fifth second are used as the judgment set of the target video frame.

[0035] As an optional embodiment, multiple consecutive frames can also be separated by setting a scene frame threshold The target video frame and its previous The present invention does not further limit the specific size of the scene frame threshold, assuming that the scene frame threshold is 4, and the target video frame is the 5th frame, then the judgment set of the target video frame can be expressed as [2, 3, 4, 5]; the target video frame is the 6th frame, then the judgment set of the target video frame can be expressed as [3, 4, 5, 6]; the target video frame is the 7th frame, then the judgment set of the target video frame can be expressed as [4, 5, 6, 7]; the target video frame is the 8th frame, then the judgment set of the target video frame can be expressed as [5, 6, 7, 8]. It should be particularly noted that, for the convenience of description, the following text will be explained with only one frame corresponding to each second. Accordingly, the number in the judgment set in the embodiment of the present invention represents the video frame corresponding to that second, that is, 2 in the judgment set represents the video frame corresponding to the 2nd second, and the same will be applied hereinafter, and no further explanation will be given one by one.

[0036] As an optional embodiment, in actual application, the setting of the scene frame threshold is determined by the frames per second (fps) of the video. For example, for a video with fps=30 Hz, the scene frame threshold can be set to 12.

[0037] Step S103, traverse the video frames with traffic accident detection frames in the judgment set and perform the first operation until all the traffic accident detection frames in the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed. Wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame with the same accident type as the first traffic accident detection frame and without a confirmation mark in the subsequent frame of the video frame, determine the detection frame that is accurately identified in each first traffic accident detection frame and each second traffic accident detection frame, and the detection frame that is accurately identified is marked with a confirmation mark.

[0038] As an optional embodiment, the traffic accident detection frame is added to the target frame in which a traffic accident is preliminarily determined after the target detection model recognizes the surveillance video stream. The present invention does not further limit the method of adding, and at least needs to include the accident type and the coordinate information of the detection frame.

[0039] In an exemplary embodiment, the accident types include four categories: pedestrian fall, two-wheeled vehicle fall, vehicle damage, and vehicle collision.

[0040] As an optional embodiment, the types of pedestrian and two-wheeled vehicle falling down include the falling down of pedestrian and two-wheeled vehicle targets after a traffic accident. Vehicle damage includes the overturning and serious damage of the vehicle after the accident. Vehicle collision includes the contact and scratching between vehicles in the traffic accident.

[0041] As an optional embodiment, each type of accident corresponds to a detection frame. That is to say, if two types of accidents, pedestrian falling and vehicle collision, occur in a video frame, two types of traffic accident detection frames will appear at corresponding positions.

[0042] As an optional embodiment, the target video frame has a detection frame. As mentioned above, 5, 6, 7, and 8 are all judged by the target detection model as traffic accidents, that is, they have detection frames. For the judgment set [2, 3, 4, 5], only the video frame at the 5th second has a detection frame. For the judgment set [3, 4, 5, 6], the video frames corresponding to 5 and 6 have detection frames. For the judgment set [4, 5, 6, 7], 5, 6, and 7 all have detection frames. Correspondingly, in the judgment set [5, 6, 7, 8], each frame has a detection frame. It should be noted that the target video frame may contain multiple accident information. For example, if a person and a car collide, there will be at least two detection frames in the target video frame.

[0043] For the judgment set [2, 3, 4, 5], only the video frame at the 5th second has a traffic accident detection frame, so the judgment set cannot perform the first operation. Continue to identify and confirm the judgment set [3, 4, 5, 6]. The video frames with traffic accident detection frames in this judgment set are 5 and 6. In 6, the coordinate information of the detection frame of the same accident type as 5 is retained. Based on the coordinate information of the accident detection frame of 5 and the coordinate information of the accident detection frame of 6, determine whether the recognition result of the target detection model is accurate. If it is determined that the recognition is accurate, add a confirmation mark in the corresponding detection frame.

[0044] As an optional embodiment, for the judgment set [3, 4, 5, 6], it can be seen from the above that 3 and 4 do not contain detection frame information, so detection confirmation starts from 5. Assuming that 5 contains 1 Class A traffic accident detection frame and 6 contains 2 traffic accident detection frames, one for Class A and one for Class B, based on the coordinate information of the Class A traffic accident detection frame in 5 and the coordinate information of the Class A traffic accident detection frame in 6, confirm whether the Class A traffic accident detection frame added by the target detection model is accurate, and further mark the detection frame that is confirmed to be accurately identified.

[0045] As an optional embodiment, for the judgment set [5, 6, 7, 8], starting from 5, the recognition result is confirmed based on the coordinate information of the same type of traffic accident detection frame in 6, 7, and 8, and a confirmation mark is put on the confirmed traffic accident detection frame. It is assumed that at this time, the confirmation mark has been put on 7. Further, starting from 6 (the accident type of 6 is the same as 5), the recognition results of 7 and 8 are confirmed based on the coordinate information of the same type of traffic accident detection frame in 7 and 8. Since 7 has been marked with a confirmation mark before, at this time, confirmation is only performed based on the traffic accident detection frame of 8. After confirming that the recognition is accurate, a confirmation mark is put on the detection frame of 8.

[0046] It should be noted that the absence of a confirmation mark only applies to the current judgment set. For example, in [4, 5, 6, 7], 6 has a confirmation mark. For the judgment set [5, 6, 7, 8], when the first operation is performed starting from 5, 6, 7, and 8 do not have any confirmation marks. If 7 is marked with a confirmation mark after 5 performs the first operation, then when 6 performs the first operation, it will no longer confirm the accuracy of the accident detection box of 7, and will only confirm the accuracy of the accident detection box of 8 through the first operation.

[0047] As an optional embodiment, the prior art directly uses the recognition of the target detection model as the final recognition result of the traffic accident, but there may be certain false detections in this process. The present invention uses a time backtracking method and combines the coordinate information of the detection box to assist in verifying whether the recognition result of the target video frame is accurate for the recognition result output by the target detection model. Further, through a sliding window traversal method, it is ensured that each recognition result output by the target detection model is confirmed and verified, and it is ensured that the target frames with the accident detection box are confirmed by the first operation whether the recognition of the accident detection box of the target detection model is accurate. And by marking a confirmation mark, it indicates that the accident detection box obtained by the target detection model is accurate, that is, the target detection model is accurate in identifying traffic accidents.

[0048] Step S104: obtaining a traffic accident recognition result of the target video frame based on the traffic accident detection frame with a confirmation mark in the target video frame.

[0049] As an optional embodiment, the recognition result includes the recognized accident type and the detection frame coordinate information. It should be noted that the detection frame coordinate information is usually the corner point coordinate information of the detection frame.

[0050] As an optional embodiment, the target video frames 5, 6, 7, and 8 all have traffic accident detection frames. At the same time, through the first operation in step S103, the target video frames 5 and 6 are marked with confirmation marks. Then, based on the traffic accident detection frames with confirmation marks of the target video frames 5 and 6, the traffic accident identification results are confirmed.

[0051] As an optional embodiment, the target detection model performs traffic accident recognition on the monitoring video stream, and recognizes that the target video frames of the traffic accident are the video frames corresponding to 5, 6, 7, and 8 seconds (as described above, taking one frame per second as an example, that is, the 5th frame, the 6th frame, the 7th frame, and the 8th frame, respectively). The video frames corresponding to 5, 6, 7, and 8 seconds respectively carry the accident type and detection frame coordinate information. For example, the target detection model recognizes that the accident type in the video frame corresponding to 5 seconds is the fall of a two-wheeled vehicle, the accident type in the video frame corresponding to 6 seconds is the fall of a two-wheeled vehicle and the collision of a vehicle, the accident type in the video frame corresponding to 7 seconds is the fall of a two-wheeled vehicle, and the accident type in the video frame corresponding to 8 seconds is the fall of a two-wheeled vehicle. After passing through the above step S103, the traffic accident detection frame in the 5-second video frame and the traffic accident detection frame of the two-wheeled vehicle falling in the 6-second video frame are marked with a confirmation mark, then it is confirmed that a traffic accident of a two-wheeled vehicle falling occurred in 5 seconds and 6 seconds. The traffic accident detection frame of the 5-second video frame and the traffic accident detection frame of the two-wheeled vehicle falling down in the 6-second video frame are retained as the final traffic accident recognition results.

[0052] In the above steps S101 to S104, the target video frames of the traffic accident are obtained based on the target detection model, and a judgment set containing multiple continuous frames is constructed for each target video frame by time backtracking to assist in the recognition and confirmation of the target video frame. Furthermore, since the position usually does not change further after a traffic accident occurs, the present invention combines the coordinate information of the detection frame in the judgment set to determine whether the recognition result of the target detection model is accurate, adds a confirmation mark to the detection frame that is accurately identified, and uses the detection frame with the confirmation mark as the final recognition result of the traffic accident. The present invention performs a second confirmation on the recognition result of the target detection model through the coordinate information of the detection frame, filters out the recognition errors, reduces the recognition false detection rate, and improves the reliability of the recognition result of the traffic accident.

[0053] In an exemplary embodiment, based on the coordinate information of each first traffic accident detection frame without a confirmation mark in a video frame, and the coordinate information of each second traffic accident detection frame having the same accident type as the first traffic accident detection frame and without a confirmation mark in a subsequent frame of the video frame, determining the detection frame in each first traffic accident detection frame and each second traffic accident detection frame that is accurately identified includes: For each first traffic accident detection frame, a set of detection frames in each second traffic accident detection frame that have the same accident type as the first traffic accident detection frame is determined.

[0054] As an optional embodiment, the target video frames for identifying traffic accidents include [5, 6, 7, 8]. It should be noted that the numbers represent the video frame data corresponding to the second. 5 has detection frames A and B, 6 has detection frames A, B, and C, 7 has detection frames A and B, and 8 has detection frames A and B. The letters A, B, and C here represent traffic accident detection frames corresponding to different types of traffic accidents identified by the target detection model. The first traffic accident detection frame is the Class A traffic accident detection frame in 5, and the Class A traffic accident detection frames in 6, 7, and 8 that are the same as 5 are retained as a detection frame set.

[0055] As an optional embodiment, if 5 contains only one traffic accident detection frame, 6 does not contain a traffic accident detection frame, and 7 and 8 contain a traffic accident detection frame that is the same as 5, then the detection frame set is [7, 8]. The detection frame set [7, 8] can be understood as a set consisting of the detection frame in the 7-second video frame and the detection frame in the 8-second video frame. Each detection frame in the detection frame set has its own coordinate information.

[0056] Based on the coordinate information of the first traffic accident detection frame and the coordinate information of each detection frame in the detection frame set, an intersection-and-union ratio between the first traffic accident detection frame and each detection frame in the detection frame set is determined.

[0057] As an optional embodiment, the coordinate information of the first traffic accident detection frame is the coordinate information of the 5th second video frame, the detection frame set is [6, 7, 8], and each detection frame in the detection frame set has its own coordinate information. It should be noted that the coordinate information of the detection frame is usually the corner point information of the detection frame. The intersection and union ratio of the detection frames 5 and 6, the intersection and union ratio of the detection frames 5 and 7, and the intersection and union ratio of the detection frames 5 and 8 are calculated respectively. The intersection and union ratio of the detection frames 5 and 6 described here can be understood as the intersection and union ratio of the detection frame in the 5-second video frame and the detection frame in the 6-second video frame. The method of calculating the intersection and union ratio is a prior art and will not be repeated here.

[0058] Based on the detection frames in the detection frame set whose corresponding intersection-over-union ratios are greater than or equal to the first threshold, a first traffic accident detection frame and an accurately identified detection frame in the detection frame set are determined.

[0059] As an optional embodiment, after a traffic accident occurs, the object involved in the traffic accident generally will not move further. Therefore, if the intersection-and-union ratio of the detection box in the 5th second video frame and the detection box in the 6th second video frame is 1, it can be understood that the position of the accident object in the detection box in the 6th second video frame is not changed compared to the position of the accident object in the detection box in the 5th second video frame. Therefore, the first threshold (which can be understood as the intersection-and-union ratio threshold) can be used to confirm the recognition result of the target detection model.

[0060] As an optional embodiment, if the corresponding intersection-and-union ratio in the detection frame set is greater than or equal to the first threshold, it indicates that the position of the detection frame is relatively fixed. If the intersection-and-union ratio of the detection frame of the 5-second video frame and the detection frame of the 6-second video frame is greater than the first threshold, it indicates that the position of the detection frame of the 6-second video frame is fixed relative to the position of the detection frame of the 5-second video frame. Further, the detection frame that is accurately identified is determined based on the number of detection frames with fixed positions.

[0061] In the above embodiment, the intersection-over-union ratio is calculated through the detection frame information of different frames of the same type of traffic accident, and the overlap degree between the detection frames is quantified to ensure the accuracy of the detection frame recognized by the target detection model.

[0062] In an exemplary embodiment, determining the first traffic accident detection frame and the accurately identified detection frame in the detection frame set based on the detection frames whose corresponding intersection-over-union ratio in the detection frame set is greater than or equal to a first threshold includes: If the number of detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than or equal to the first threshold is greater than or equal to the second threshold, it is determined that the first traffic accident detection frame and the detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than the first threshold are both accurately identified detection frames.

[0063] As an optional embodiment, the second threshold value can be understood as a threshold value for the number of detection frames with fixed positions. The first traffic accident detection frame is a traffic accident detection frame in a 5-second video frame, and the detection frame set is [6, 7, 8]. As mentioned above, three intersection-and-union ratios are calculated respectively. The intersection-and-union ratio of 5 and 6 is greater than the first threshold, the intersection-and-union ratio of 5 and 7 is less than the first threshold, and the intersection-and-union ratio of 5 and 8 is greater than the first threshold. The second threshold value is 1. At this time, the number of detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than the first threshold is greater than the second threshold. Therefore, it is determined that the first traffic accident detection frame 5 and the detection frames 6 and 8 in the detection frame set whose corresponding intersection-and-union ratio is greater than the first threshold are both accurately identified detection frames.

[0064] As an optional embodiment, both the first threshold and the second threshold are set according to the frame rate per second of the surveillance video, which is not further limited in the present invention.

[0065] As an optional embodiment, for a surveillance video with fps=30 Hz, the first threshold may be set to 0.9, and the second threshold may be set to 11 or 10.

[0066] In the above embodiment, by setting the threshold of the intersection-over-union ratio, the set of detection frames that highly overlap with the first traffic accident detection frame can be effectively filtered out, thereby ensuring the accuracy of these frames in accident identification. By limiting the number of detection frames, the reliability of identification can be further enhanced, ensuring that only when the number of detection frames reaches a certain standard, it is confirmed as an accurate identification result. The accuracy and stability of traffic accident identification are improved.

[0067] In an exemplary embodiment, the method further comprises: If the traffic accident detection frame with a confirmation mark in the target video frame includes a vehicle collision detection frame, the accident probability of each vehicle target is determined based on the historical tracking information of each vehicle target in the vehicle collision detection frame and the detection frame information of each vehicle target.

[0068] Based on the accident probability of each vehicle target, it is further confirmed whether there is any false detection in the vehicle collision detection frame.

[0069] As an optional embodiment, as mentioned above, the accident types include pedestrians falling, two-wheeled vehicles falling, vehicle damage, and vehicle collision. However, in daily life, vehicle collisions are more common, and vehicle collisions include some minor scratches. Because the position changes are small, the probability of false detection is also the highest when using the target detection model for recognition. Therefore, it is hoped that further detection and confirmation of vehicle collisions will be performed. The false detection frames are filtered out to improve the credibility of recognition.

[0070] As an optional embodiment, the scheme in this embodiment only further confirms that the traffic accident detection box with a confirmation mark is a detection box of a vehicle collision. Because the position changes of the other three types of traffic accidents are more obvious, after the false detection and filtering of the above embodiment, a higher recognition accuracy rate can be achieved. Therefore, there is no need for further confirmation through the scheme in this implementation.

[0071] As an optional embodiment, the vehicle collision is at least a collision between two vehicles, so it is necessary to calculate the accident probability of each vehicle, that is, for the case of a collision between two vehicles, two accident probability values ​​will be obtained accordingly ( and ), further set the accident occurrence assessment threshold, if , If at least one condition is greater than the accident occurrence assessment threshold, then vehicle 1 ( corresponding car) and vehicle 2 ( The corresponding vehicle) has collided, that is, the vehicle collision detection box output by the target detection model is accurate and there is no false detection.

[0072] In the above embodiment, by combining the historical tracking information of the vehicle collision detection frame with the confirmation mark and the current detection frame information, the accident probability of each vehicle target can be accurately calculated. In the case of vehicle collision, the false detection rate is further reduced, and the reliability and accuracy of the overall accident identification are improved.

[0073] In an exemplary embodiment, based on the historical tracking information of each vehicle target in the vehicle collision detection frame and the detection frame information of each vehicle target, the accident probability of each vehicle target is determined, including: For each vehicle target, based on the historical tracking information of each vehicle target, the acceleration change value, speed change value, angle change value of the vehicle target in the historical period, and the collision angle value between the vehicle target and other vehicle targets in the vehicle collision detection frame are determined.

[0074] Among them, the acceleration change value of the vehicle target in the historical period refers to the acceleration change value of the vehicle target between the start and end of the historical period. For example, if you go back 10 frames (that is, there are 10 frames in the historical period), the acceleration change value refers to the acceleration change value of the vehicle target between the first frame and the last frame, that is, the change value of the acceleration of the vehicle target in the last frame relative to the acceleration in the first frame.

[0075] Similarly, the speed change value of the vehicle target in the historical period refers to the speed change value of the vehicle target between the start and end of the historical period. For example, if we go back 10 frames (i.e., there are 10 frames in the historical period), the speed change value refers to the speed change value of the vehicle target between the first frame and the last frame, i.e., the change value of the vehicle target's speed in the last frame relative to the speed in the first frame.

[0076] The turning angle change value of the vehicle target in the historical period refers to the turning angle change value of the vehicle target between the start and end of the historical period. For example, if you go back 10 frames (that is, there are 10 frames in the historical period), the turning angle change value refers to the turning angle change value of the vehicle target between the first frame and the last frame, that is, the change value of the turning angle of the vehicle target in the last frame relative to the turning angle in the first frame.

[0077] The collision angle between two vehicle targets can be calculated by the center coordinates of the detection boxes of the two vehicle targets. For example, represents the collision angle between vehicle A and vehicle B, which can be calculated by the following formula :

[0078] In the formula, and Represent the center coordinates of the detection boxes of vehicle A and vehicle B respectively.

[0079] As an optional embodiment, the objects of vehicle collision are vehicle A and vehicle B respectively, and the acceleration change value, speed change value, turning angle change value, collision angle value of vehicle A and vehicle B, and the intersection-and-union ratio of the detection frame of vehicle A and the detection frame of vehicle B are obtained, and the acceleration change value, speed change value, turning angle change value, collision angle value of vehicle B and vehicle A, and the intersection-and-union ratio of the detection frame of vehicle B and the detection frame of vehicle A are obtained. For the same traffic accident, the intersection-and-union ratio of the detection frame of vehicle A and the detection frame of vehicle B is the same as the intersection-and-union ratio of the detection frame of vehicle B and the detection frame of vehicle A, and the collision angle value of vehicle A and vehicle B is the same as the collision angle value of vehicle B and vehicle A.

[0080] As an optional embodiment, the historical tracking information may be data information of video frames within a period of time before the video frame in which the vehicle collision detection frame appears, such as vehicle speed information, road condition information, etc.

[0081] The judgment vector of the vehicle target is determined based on the acceleration change value, the speed change value, the turning angle change value, the collision angle value, and the intersection and union ratio of the detection frame of the vehicle target and the detection frames of other vehicle targets.

[0082] As an optional embodiment, the judgment vector of vehicle A can be expressed as , represents the acceleration change value of vehicle A, Indicates the speed change value of vehicle A, Indicates the turning angle change of vehicle A, Indicates the collision angle between vehicle A and vehicle B. Represents the intersection-over-union ratio of the detection boxes of vehicle A and vehicle B.

[0083] Based on the judgment vector and factor coefficients of the vehicle target, the accident probability of the vehicle target is determined.

[0084] As an optional embodiment, the factor coefficient can be expressed as , which has a one-to-one correspondence with the judgment vector, that is, The factor coefficient representing the acceleration change value, The factor coefficient representing the speed change value, The factor coefficient representing the angle change value, The factor coefficient representing the collision angle value, The factor coefficient representing the intersection-joint ratio. The factor coefficient is affected by the vehicle type and the road condition at the monitoring point. Therefore, the factor coefficient corresponding to each traffic accident is different. For the same traffic accident (mostly collision type), the road condition at the monitoring point is basically the same. The factor coefficient can be manually set according to factors such as vehicle type and road condition at the monitoring point, and there is no specific restriction.

[0085] As an optional embodiment, under the same monitoring point road conditions, the factor coefficient of the acceleration change value, the factor coefficient of the speed change value, and the factor coefficient of the angle change value of a large vehicle collision are higher than those of a medium-sized vehicle collision and a small vehicle collision. Under the same accident type (such as a large vehicle collision), the factor coefficient of the acceleration change value, the factor coefficient of the speed change value, and the factor coefficient of the angle change value under congested road conditions are higher than those of medium-speed driving and high-speed driving. The specific values ​​can be flexibly set, and the present invention does not further limit this.

[0086] As an optional embodiment, based on the judgment vector and factor coefficient of the vehicle target, the accident probability of the vehicle target can be determined by the following formula: ,in represents the accident probability of the vehicle target, represents the factor coefficient, Represents the judgment vector.

[0087] In the above embodiment, by calculating multi-dimensional indicators such as acceleration, speed, corner change and track intersection angle, combined with the intersection ratio of the detection frame, an accurate judgment vector is generated, thereby accurately calculating the accident probability of the vehicle target, further improving the recognition accuracy of vehicle collision accidents.

[0088] The executor of each step in the above method can be a traffic accident identification and processing device based on a monitoring video stream. The device can be implemented through software and / or hardware. The device can be integrated into an electronic device. The electronic device can be a terminal device (such as a smart phone, a personal computer, etc.), or a server (such as a local server or a cloud server, or a server cluster, etc.), or a processor, or a chip, etc.

[0089] Obviously, the above described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. In order to better understand the above method, the above process is described below in conjunction with the embodiments, but the technical solutions of the embodiments of the present invention are not limited.

[0090] Figure 2 This is a flow chart of traffic accident identification provided by the present invention. Figure 2 As shown in FIG. 1 , the identification process includes the following 6 steps. Each step is described in detail below.

[0091] 1. Collect image data of car accident scenes from the monitoring perspective, and annotate the types of accidents that need to be detected. Such as pedestrians falling, two-wheeled vehicles falling, vehicle damage, and vehicle collision. Construct a traffic accident data set that meets the training requirements. Among them, the types of pedestrians and two-wheeled vehicles falling include pedestrians and two-wheeled vehicles falling after traffic accidents. Vehicle damage includes overturning and serious damage after the vehicle accident. Vehicle collision includes contact and scratching between vehicles in traffic accidents.

[0092] 2. Based on deep learning, the target detection algorithm is trained with the constructed accident data set, and the model parameters are debugged until the accuracy requirements are met to obtain the applicable traffic accident detection weights.

[0093] 3. Access the real-time video stream of road monitoring, decode it into frame-by-frame image data as the input of the deep learning framework, perform real-time target detection on the image based on the pre-trained weights, and obtain the recognition results of road traffic accidents, that is, the recognized accident type and detection box coordinate information.

[0094] 4. For the video frames with target detection results, enter the false detection filtering module of accident target detection. Figure 3 FIG. 1 is a flow chart of the false detection filtering process provided by the present invention, which specifically includes the following steps.

[0095] (1) Set the scene frame threshold , for the current video frame and the previous A judgment set is generated for each video frame, which stores all the detection box information contained in each video frame in sequence according to the time sequence of the video frames. At this time, all the detection boxes in the judgment set are not marked.

[0096] (2) Starting from the first frame of the judgment set, read the unmarked detection frame A, traverse the detection frames in other video frames after the video frame corresponding to the detection frame A, and retain the detection frame B with the same accident type as the detection frame A in each frame. Set the threshold , calculate the intersection over union (IoU) between detection box A and detection box B, which is higher than the threshold Then the detection frame B is considered to be fixed relative to the detection frame A. For example, if the judgment set is [A1, B1, B2, B3, B4, B5], when the first traversal is performed, all the detection frames are not marked, and the intersection and union ratios of the detection frames A1 and B1, A1 and B2, A1 and B3, A1 and B4, and A1 and B5 are calculated respectively.

[0097] (3) Setting the threshold , the number of detection frames in the judgment set that are fixed relative to the detection frame A is higher than When , it is considered that the detection frame A and the detection frames fixed relative to the detection frame A constitute the accident confirmation detection frame sequence. In the judgment set, the detection frames corresponding to the sequence are marked and divided into a group of accidents.

[0098] As an optional embodiment, the intersection-and-union ratio of detection frame A1 and detection frame B3 is higher than the intersection-and-union ratio threshold, and the intersection-and-union ratio of detection frame A1 and detection frame B4 is higher than the intersection-and-union ratio threshold. At this time, the detection frames whose positions are fixed relative to detection frame A are detection frame B3 and detection frame B4, which is higher than the detection frame quantity threshold. Then, it is confirmed that [A1, B3, B4] constitute a confirmed detection frame sequence, and a confirmation mark is given to each detection frame.

[0099] (4) The video frames in the judgment set are traversed from front to back, and the detection boxes in each frame are traversed. If the detection box is not marked, the operations of (2) and (3) are performed, otherwise skipped. Repeat this step until the detection boxes in the judgment set are traversed.

[0100] In the above embodiment, after the traversal of detection frame A1 is completed, the traversal and comparison is continued from detection frame B1. In theory, the intersection and union ratios of detection frame B2 and detection frame B3, detection frame B2 and detection frame B4, and detection frame B2 and detection frame B5 should be calculated. However, since detection frame B3 and detection frame B4 have been marked with confirmation during the traversal of detection frame A1, only steps (2) and (3) are performed on detection frame B2 and detection frame B5.

[0101] (5) For the current video frame, check whether all detection frames contain any cases marked as accidents. Keep the detection frames marked as accidents in the video frame and the corresponding accident type information. Discard the detection frames not marked as accidents in the video frame to complete the false detection filtering process.

[0102] 5. Use target detection and tracking algorithms to identify vehicles in the lane and obtain tracking information for each target. The tracking information data of the frame is used as the historical basis. When the false detection filtering module outputs the vehicle collision type accident information, the historical information of the tracked target contained in the detection area is called and recorded separately. The acceleration change value of the tracked target to be evaluated within the frame , speed change value , Angle change value , the intersection and union ratio of the target to be evaluated in the current frame and other target detection frames , Collision Angle Value As a judgment factor, form a judgment vector . According to the vehicle type detected by the algorithm, the road conditions at the monitoring point and other influencing factors, set the corresponding factor coefficient , based on the formula Calculate the probability of false detection. Set the threshold for assessing the occurrence of an accident , for the target detection results of vehicle collision type generated in the previous steps, calculate the probability , when two cars collide, there are two Value, satisfying one or more of When , it is judged that there is a collision, otherwise there is a false detection.

[0103] 6. Send alarm information and set the interval for sending alarm information , when a traffic accident occurs, if the time interval between the current accident video frame and the last sent accident video frame is greater than , send an accident alarm signal to the traffic control center. The alarm information should include: camera equipment information, camera point latitude and longitude, accident video frame, image and type in the accident detection frame, and abnormal accident time period.

[0104] The present invention uses the processing logic of dynamic frame filtering to filter out the misdetected accident targets in the moving state, effectively reducing the false alarm rate of traffic accident information and alleviating the workload of the staff in handling accident information. At the same time, in view of the problem that some normal traffic conditions are easily confused with minor traffic accidents, the accurate traffic accident scenes are screened out through the acquisition of redundant information and the processing logic of historical trajectory information, reducing the occurrence of misjudgment and achieving the generation of more accurate alarm information.

[0105] The following is a description of the traffic accident identification and processing device based on monitoring video stream provided by the present invention. The traffic accident identification and processing device based on monitoring video stream described below and the traffic accident identification and processing method based on monitoring video stream described above can be referenced to each other.

[0106] Figure 4 Schematic diagram of the structure of the traffic accident identification and processing device based on monitoring video stream provided by the present invention. Figure 4 As shown, the device comprises: The first processing module 410 is used to identify traffic accidents on the monitoring video stream based on the target detection model of deep learning, and obtain the target video frame in which the traffic accident is identified; The second processing module 420 is used for, for each target video frame, taking a plurality of consecutive frames including the target video frame in the monitoring video stream as a judgment set of the target video frame, wherein the target video frame is the last frame of the plurality of consecutive frames; The third processing module 430 is used to traverse the video frames with traffic accident detection frames in the judgment set and perform the first operation until all the traffic accident detection frames with the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame with the same accident type as the first traffic accident detection frame and without a confirmation mark in the subsequent frame of the video frame, determine the detection frame with accurate identification in each first traffic accident detection frame and each second traffic accident detection frame, and the detection frame with accurate identification is marked with a confirmation mark; The fourth processing module 440 is used to obtain a traffic accident recognition result of the target video frame based on the traffic accident detection frame with a confirmation mark in the target video frame.

[0107] The above-mentioned device, based on the target video frame obtained by the target detection model to identify the traffic accident, constructs a judgment set containing multiple continuous frames for each target video frame by time backtracking to assist in the recognition and confirmation of the target video frame. Furthermore, since the position usually does not change further after the traffic accident occurs, the present invention combines the coordinate information of the detection frame in the judgment set to determine whether the recognition result of the target detection model is accurate, adds a confirmation mark to the detection frame that is accurately identified, and uses the detection frame with the confirmation mark as the final recognition result of the traffic accident. The present invention performs a second confirmation on the recognition result of the target detection model through the coordinate information of the detection frame, filters out the recognition errors, reduces the recognition false detection rate, and improves the reliability of the recognition result of the traffic accident.

[0108] In some embodiments, the third processing module 430 is also used to determine, for each first traffic accident detection frame, a set of detection frames in each second traffic accident detection frame that have the same accident type as the first traffic accident detection frame; determine the intersection-and-union ratio of the first traffic accident detection frame and each detection frame in the detection frame set based on the coordinate information of the first traffic accident detection frame and the coordinate information of each detection frame in the detection frame set; determine the first traffic accident detection frame and the accurately identified detection frames in the detection frame set based on the detection frames in the detection frame set whose corresponding intersection-and-union ratios are greater than or equal to the first threshold.

[0109] In some embodiments, the third processing module 430 is also used to determine the first traffic accident detection frame and the accurately identified detection frame in the detection frame set based on the detection frames whose corresponding intersection-and-union ratios in the detection frame set are greater than or equal to the first threshold, and then determine that the first traffic accident detection frame and the detection frames in the detection frame set whose corresponding intersection-and-union ratios are greater than the first threshold are both accurately identified detection frames.

[0110] In some embodiments, the accident types include four categories: pedestrian fall, two-wheeled vehicle fall, vehicle damage, and vehicle collision.

[0111] In some embodiments, the device also includes a fifth processing module, which is used to determine the accident probability of each vehicle target based on the historical tracking information of each vehicle target in the vehicle collision detection frame and the detection frame information of each vehicle target if the traffic accident detection frame with a confirmation mark in the target video frame includes a vehicle collision detection frame; based on the accident probability of each vehicle target, further confirm whether there is a false detection in the vehicle collision detection frame.

[0112] In some embodiments, the fifth processing module is also used to determine, for each vehicle target, based on the historical tracking information of each vehicle target, the acceleration change value, speed change value, angle change value, and the collision angle value of the vehicle target with other vehicle targets in the vehicle collision detection frame within a historical period; determine the judgment vector of the vehicle target based on the acceleration change value, speed change value, angle change value, collision angle value, and the intersection and union ratio of the detection frame of the vehicle target and the detection frames of other vehicle targets; determine the accident probability of the vehicle target based on the judgment vector and factor coefficient of the vehicle target.

[0113] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communications interface 520 and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a traffic accident identification processing method based on a monitoring video stream, the method comprising: Based on the deep learning target detection model, the monitoring video stream is used to identify traffic accidents and obtain the target video frame in which the traffic accident is identified; For each target video frame, multiple continuous frames containing the target video frame in the monitoring video stream are used as a judgment set of the target video frame, wherein the target video frame is the last frame of the multiple continuous frames; The video frames with traffic accident detection frames in the judgment set are traversed and the first operation is performed until all the traffic accident detection frames in the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame in the subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and does not have a confirmation mark, the detection frame that is accurately identified in each first traffic accident detection frame and each second traffic accident detection frame is determined, and the detection frame that is accurately identified is marked with a confirmation mark; Based on the traffic accident detection frame with the confirmation mark in the target video frame, the traffic accident recognition result of the target video frame is obtained.

[0114] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0115] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the traffic accident identification and processing method based on the monitoring video stream provided by the above methods, and the method includes: Based on the deep learning target detection model, the monitoring video stream is used to identify traffic accidents and obtain the target video frame in which the traffic accident is identified; For each target video frame, multiple continuous frames containing the target video frame in the monitoring video stream are used as a judgment set of the target video frame, wherein the target video frame is the last frame of the multiple continuous frames; The video frames with traffic accident detection frames in the judgment set are traversed and the first operation is performed until all the traffic accident detection frames in the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame in the subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and does not have a confirmation mark, the detection frame that is accurately identified in each first traffic accident detection frame and each second traffic accident detection frame is determined, and the detection frame that is accurately identified is marked with a confirmation mark; Based on the traffic accident detection frame with the confirmation mark in the target video frame, the traffic accident recognition result of the target video frame is obtained.

[0116] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for identifying and processing traffic accidents based on surveillance video streams provided by the above methods is implemented, and the method comprises: Based on the deep learning target detection model, the monitoring video stream is used to identify traffic accidents and obtain the target video frame in which the traffic accident is identified; For each target video frame, multiple continuous frames containing the target video frame in the monitoring video stream are used as a judgment set of the target video frame, wherein the target video frame is the last frame of the multiple continuous frames; The video frames with traffic accident detection frames in the judgment set are traversed and the first operation is performed until all the traffic accident detection frames in the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame in the subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and does not have a confirmation mark, the detection frame that is accurately identified in each first traffic accident detection frame and each second traffic accident detection frame is determined, and the detection frame that is accurately identified is marked with a confirmation mark; Based on the traffic accident detection frame with the confirmation mark in the target video frame, the traffic accident recognition result of the target video frame is obtained.

[0117] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A traffic accident identification and processing method based on surveillance video stream, characterized in that: include: Based on the deep learning target detection model, the monitoring video stream is used to identify traffic accidents and obtain the target video frame in which the traffic accident is identified; For each of the target video frames, a plurality of continuous frames in the monitoring video stream that contain the target video frame are used as a judgment set of the target video frame, wherein the target video frame is the last frame of the plurality of continuous frames; Traversing the video frames with traffic accident detection frames in the judgment set and performing the first operation until all the traffic accident detection frames with the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation includes: determining the accurately identified detection frames of each first traffic accident detection frame and each second traffic accident detection frame based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame with the same accident type as the first traffic accident detection frame and without a confirmation mark in the subsequent frame of the video frame, and the accurately identified detection frame is marked with a confirmation mark; Based on the traffic accident detection frame with a confirmation mark in the target video frame, a traffic accident recognition result of the target video frame is obtained.

2. The traffic accident identification and processing method based on monitoring video stream according to claim 1 is characterized in that: The determining of the accurately identified detection frames in each of the first traffic accident detection frames and each of the second traffic accident detection frames based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame and the coordinate information of each second traffic accident detection frame in a subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and does not have a confirmation mark comprises: For each of the first traffic accident detection frames, determining a set of detection frames in each of the second traffic accident detection frames that have the same accident type as the first traffic accident detection frame; Determine, based on the coordinate information of the first traffic accident detection frame and the coordinate information of each detection frame in the detection frame set, an intersection-over-union ratio between the first traffic accident detection frame and each detection frame in the detection frame set; Based on the detection frames in the detection frame set whose corresponding intersection-over-union ratio is greater than or equal to the first threshold, the first traffic accident detection frame and the accurately identified detection frames in the detection frame set are determined.

3. The traffic accident identification and processing method based on monitoring video stream according to claim 2 is characterized in that: The determining, based on the detection frames in the detection frame set whose corresponding intersection-over-union ratio is greater than or equal to the first threshold, the first traffic accident detection frame and the accurately identified detection frame in the detection frame set includes: If the number of detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than or equal to the first threshold is greater than or equal to the second threshold, it is determined that the first traffic accident detection frame and the detection frames in the detection frame set whose corresponding intersection-and-union ratio is greater than the first threshold are both accurately identified detection frames.

4. The method for identifying and processing traffic accidents based on surveillance video streams according to any one of claims 1 to 3, characterized in that: The accident types include four categories: pedestrians falling, two-wheeled vehicles falling, vehicle damage, and vehicle collision.

5. The method for identifying and processing traffic accidents based on surveillance video stream according to claim 4 is characterized in that: The method further comprises: If the traffic accident detection frame with the confirmation mark in the target video frame includes a vehicle collision detection frame, then based on the historical tracking information of each vehicle target in the vehicle collision detection frame and the detection frame information of each vehicle target, determining the accident probability of each vehicle target; Based on the accident probability of each vehicle target, it is further confirmed whether the vehicle collision detection frame has any false detection.

6. The method for identifying and processing traffic accidents based on surveillance video stream according to claim 5, characterized in that: The determining the accident probability of each vehicle target based on the historical tracking information of each vehicle target in the vehicle collision detection frame and the detection frame information of each vehicle target includes: For each of the vehicle targets, based on the historical tracking information of each of the vehicle targets, determine the acceleration change value, speed change value, angle change value of the vehicle target within a historical period, and the collision angle value between the vehicle target and other vehicle targets in the vehicle collision detection frame; Determine a judgment vector of the vehicle target based on the acceleration change value, the speed change value, the turning angle change value, the collision angle value, and an intersection-and-combination ratio between the detection frame of the vehicle target and the detection frames of the other vehicle targets; Based on the judgment vector and factor coefficients of the vehicle target, the accident probability of the vehicle target is determined.

7. A traffic accident recognition and processing device based on monitoring video stream, characterized in that: include: The first processing module is used to identify traffic accidents on the monitoring video stream based on the target detection model of deep learning, and obtain the target video frame in which the traffic accident is identified; A second processing module is used for, for each of the target video frames, taking a plurality of consecutive frames in the monitoring video stream that contain the target video frame as a judgment set of the target video frame, wherein the target video frame is the last frame of the plurality of consecutive frames; A third processing module is used to traverse the video frames with traffic accident detection frames in the judgment set and perform a first operation until all the traffic accident detection frames in the target video frame are marked with confirmation marks, or all the video frames with traffic accident detection frames in the judgment set are traversed; wherein, for any traversed video frame, the first operation includes: based on the coordinate information of each first traffic accident detection frame without a confirmation mark in the video frame, and the coordinate information of each second traffic accident detection frame in the subsequent frame of the video frame that has the same accident type as the first traffic accident detection frame and does not have a confirmation mark, determine the detection frame that is accurately identified in each of the first traffic accident detection frames and each of the second traffic accident detection frames, and the detection frame that is accurately identified is marked with a confirmation mark; The fourth processing module is used to obtain the traffic accident recognition result of the target video frame based on the traffic accident detection frame with the confirmation mark in the target video frame.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for identifying and processing traffic accidents based on monitoring video streams as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the traffic accident identification and processing method based on monitoring video stream as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the traffic accident identification and processing method based on monitoring video stream as described in any one of claims 1 to 6 is implemented.

Images