Expressway vehicle abnormal event detection system based on monitoring video

Abstract

The invention relates to the technical field of video anomaly detection, and comprises a detection method based on multi-target tracking, and the method comprises the steps: employing a target detection and target tracking algorithm to carry out the precise detection and tracking of a vehicle in video data; according to the detection method based on the weak supervision video, vehicle abnormal behavior detection based on a video abnormal detection algorithm is adopted; according to the track classification-based detection method, a motion track-based detection algorithm is adopted to realize detection of vehicles with abnormal tracks in a monitoring video, and dichotomy of motion tracks of all vehicles in the monitoring video is realized through the track anomaly-based detection algorithm. Through combination of multiple detection modes, target detection tracking and lane line detection are combined to realize detection of vehicle positions and motion states, a two-stage network is provided, and accurate detection of abnormal events of highway vehicles is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of video anomaly detection, and more specifically to a highway vehicle abnormal event detection system based on monitoring video. Background Art

[0002] In recent years, with the rapid development of my country's economy and the continuous improvement of people's living standards, vehicles have become an indispensable means of transportation in people's lives. The rapid growth of car ownership has brought many conveniences to people's lives, but it has also brought huge pressure to road traffic and many safety hazards that cannot be ignored. Among them, highway traffic safety issues are becoming increasingly serious. The key causes of highway accidents are: abnormal traffic events induce serious driving safety hazards, and traffic accidents are not discovered and handled in a timely and accurate manner, causing secondary accident risks.

[0003] Existing video anomaly detection methods can be divided into three categories: strong supervision, unsupervised, and weak supervision according to the availability of data labels. Strongly supervised video anomaly detection methods rely on precisely labeled samples to train binary classification models to detect anomalies. However, due to the difficulty in balancing normal and abnormal samples, the model performance is limited, and it depends on data labels, so its practical application is limited. Unsupervised methods use normal data to train models and detect anomalies through self-supervision techniques such as reconstruction errors or future frame prediction. Weakly supervised methods use video-level labels to learn regression anomaly detection models through multiple instances, which are suitable for scenarios with scarce abnormal event samples, such as traffic accident detection.

[0004] Although the popularity of surveillance video has provided convenience for recording the behavior of highway vehicles, due to the huge amount of data caused by the working mode of video data collection around the clock, manual review of massive video data is often time-consuming and labor-intensive, inefficient, and has poor real-time performance. Therefore, the present invention designs a highway vehicle abnormal event detection system based on surveillance video to solve the above problems. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a highway vehicle abnormal event detection system based on surveillance video, which has the advantages of accurately detecting abnormal events such as illegal parking, wrong-way driving, reversing, occupying emergency lanes, traffic accidents, abnormal trajectories, etc. of highway vehicles, thereby improving the efficiency and safety of highway traffic management.

[0006] To achieve the above object, the present invention provides the following technical solution: a highway vehicle abnormal event detection system based on monitoring video, comprising:

[0007] The detection method based on multi-target tracking uses target detection and target tracking algorithms to accurately detect and track vehicles in video data. At the same time, the lane line detection algorithm is used to identify the lane lines of the highway, extract various possible vehicle abnormal events, build models, and mathematically define each abnormal traffic event. When the relevant parameters meet specific conditions, it is determined that an abnormal vehicle event has occurred, and based on this, an abnormal behavior recognition algorithm based on state changes is constructed;

[0008] The detection method based on weakly supervised video adopts the vehicle abnormal behavior detection based on the video anomaly detection algorithm, and utilizes the vehicle abnormal behavior recognition technology based on the video anomaly detection algorithm. The video anomaly detection model automatically learns the appearance and motion characteristics of the target in the monitoring video to realize the detection of the vehicle involved in the traffic accident.

[0009] The detection method based on trajectory classification adopts a motion trajectory detection algorithm to realize the detection of vehicles with abnormal trajectories in surveillance videos. This method realizes the binary classification of the motion trajectories of all vehicles in surveillance videos through a detection algorithm based on trajectory anomalies.

[0010] As a preferred technical solution of the present invention, the detection method based on multi-target tracking is based on multi-target detection, multi-target tracking, lane line detection and abnormal behavior recognition algorithm based on state changes, to achieve accurate detection of illegal parking, reversing, reversing, and occupying emergency lanes, providing a basis for trajectory classification.

[0011] As a preferred technical solution of the present invention, the detection method based on weakly supervised video is based on a multiple instance learning algorithm and a strong supervision algorithm respectively to achieve accurate detection of traffic accidents.

[0012] As a preferred technical solution of the present invention, the detection method based on trajectory classification is based on the model establishment of trajectory anomaly, trajectory processing and a trajectory classification method based on an autoencoder and a Gaussian mixture model to achieve detection of trajectory anomaly.

[0013] As a preferred technical solution of the present invention, it also includes algorithm comparison and ablation experiments. The algorithm comparison is used to compare other weak supervision algorithms and compare experimental results. The ablation experiment includes the establishment of an overall detection system.

[0014] As a preferred technical solution of the present invention, the detection method based on multi-target tracking frames and annotates highway video data, uses a one-stage target detection algorithm to detect vehicle target frames in real time, combines Kalman filtering and Hungarian algorithm to perform multi-target tracking, obtains vehicle ID and real-time target frame, and uses a lane line detection model to identify lane information.

[0015] As a preferred technical solution of the present invention, the detection method based on weakly supervised video uses a weakly supervised multi-instance learning method to identify normal segments in weakly labeled abnormal videos. The network predicts the abnormal scores of the video segments and uses them as derived labels after processing for the next stage of training. At the same time, the noise in the abnormal videos is filtered out to provide clearer data for the learning of the anomaly detection model. The derived labels and data are used to train a strongly supervised anomaly detection network to realize video anomaly detection.

[0016] As a preferred technical solution of the present invention, the detection method based on trajectory classification establishes an abnormal trajectory feature table, trajectory smoothing processing and a trajectory anomaly detection algorithm based on an autoencoder and a Gaussian mixture model.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. The present invention cooperates with a detection method based on multi-target tracking, a detection method based on weakly supervised video and a detection method based on trajectory classification, combines target detection and tracking with lane line detection to realize the detection of vehicle position and motion state, and proposes a two-stage network. In the first stage, a weakly supervised network is used to filter preliminary data. In the second stage, a strong supervised network event anomaly detection is used to establish a highway trajectory feature model. A trajectory anomaly detection algorithm based on an autoencoder and a Gaussian mixture model is proposed to realize trajectory classification detection. A variety of detection methods are combined to establish a complete detection system to realize accurate detection of abnormal events such as illegal parking, reverse driving, reversing, occupying emergency lanes, traffic accidents, and trajectory abnormalities of highway vehicles.

[0019] 2. The present invention uses algorithm comparison and ablation experiments. The algorithm comparison is used to compare other weak supervision algorithms and compare experimental results. The ablation experiment includes the establishment of an overall detection system, which further verifies the effectiveness and accuracy of the system in detecting abnormal events of highway vehicles, with higher detection efficiency and lower false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the overall framework diagram of the present invention;

[0021] Figure 2 A diagram of a detection method based on multi-target tracking in the present invention;

[0022] Figure 3 A roadmap for the detection method based on multi-target tracking in the present invention;

[0023] Figure 4 A roadmap for the detection method based on weakly supervised video in the present invention;

[0024] Figure 5This is a solution roadmap for the detection method based on trajectory classification in the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] like Figures 1 to 5 As shown, the present invention provides a highway vehicle abnormal event detection system based on monitoring video, comprising:

[0027] The detection method based on multi-target tracking uses target detection and target tracking algorithms to accurately detect and track vehicles in video data. At the same time, the lane line detection algorithm is used to identify the lane lines of the highway, extract various possible vehicle abnormal events, build models, and mathematically define each abnormal traffic event. When the relevant parameters meet specific conditions, it is determined that an abnormal vehicle event has occurred, and based on this, an abnormal behavior recognition algorithm based on state changes is constructed;

[0028] The detection method based on weakly supervised video adopts the vehicle abnormal behavior detection based on the video anomaly detection algorithm, and utilizes the vehicle abnormal behavior recognition technology based on the video anomaly detection algorithm. The video anomaly detection model automatically learns the appearance and motion characteristics of the target in the monitoring video to realize the detection of the vehicle involved in the traffic accident.

[0029] The detection method based on trajectory classification adopts a motion trajectory detection algorithm to realize the detection of vehicles with abnormal trajectories in surveillance videos. This method realizes the binary classification of the motion trajectories of all vehicles in surveillance videos through a detection algorithm based on trajectory anomalies.

[0030] Among them, the detection method based on multi-target tracking is based on multi-target detection, multi-target tracking, lane line detection and abnormal behavior recognition algorithm based on state changes, which can achieve accurate detection of illegal parking, reverse driving, reversing, and occupying emergency lanes, providing a basis for trajectory classification.

[0031] Through,target detection and target tracking algorithms, the vehicles in the video are detected and tracked, the lane line algorithm is used to detect the lane lines of the highway, and then various possible abnormal vehicle events are modeled and each abnormal traffic event is mathematically defined. When the relevant parameters reach certain conditions, it is considered that an abnormal vehicle event has occurred, and an abnormal behavior recognition algorithm based on state changes is established.

[0032] Among them, the detection method based on weak supervision video is based on a multi-instance learning algorithm and a strong supervision algorithm respectively to achieve accurate detection of traffic accidents.

[0033] By automatically learning the appearance and motion characteristics of the target in the video through the video anomaly detection model, the purpose of detecting anomalies in the video is achieved. According to the research progress of the existing video anomaly detection methods, the key technologies involved include video anomaly detection algorithms based on weak supervision.

[0034] Among them, the detection method based on trajectory classification is based on the model establishment of trajectory anomaly, trajectory processing and trajectory classification method based on autoencoder and Gaussian mixture model to realize the detection of trajectory anomaly.

[0035] Through,the detection algorithm based on trajectory anomaly, the binary classification of all vehicle motion trajectories in the surveillance video is achieved.

[0036] It also includes algorithm comparison and ablation experiments. The algorithm comparison is used to compare other weakly supervised algorithms and compare experimental results. The ablation experiment includes the establishment of an overall detection system.

[0037] Among them, the detection method based on multi-target tracking frames and annotates highway video data, uses a one-stage target detection algorithm to detect vehicle target frames in real time, combines Kalman filtering and Hungarian algorithm to perform multi-target tracking, obtains vehicle ID and real-time target frame, and uses lane line detection model to identify lane information.

[0038] By taking frames and annotating the highway video data under the monitoring perspective, a high-real-time one-stage target detection algorithm is used to detect the target frame of the vehicle in the monitoring screen. Then, based on the target detection results, Kalman filter prediction and Hungarian algorithm matching are used to achieve multi-target tracking and obtain the corresponding ID and real-time target frame of the vehicle. At the same time, the lane line detection model is used to detect the lane line information of the lane in the picture. The various parameters (ID, target frame, lane line coordinates) obtained by the above model can obtain the real-time position information and status information of all vehicles relative to the road. According to different changes in the state, different behavior judgment models are designed to realize the judgment of corresponding abnormal events.

[0039] Among them, the detection method based on weakly supervised video uses a weakly supervised multi-instance learning method to identify normal segments in weakly labeled abnormal videos. The network predicts the abnormal scores of video segments and uses them as derived labels after processing for the next stage of training. At the same time, the noise in the abnormal video is filtered out to provide clearer data for the learning of the anomaly detection model. The derived labels and data are used to train a strongly supervised anomaly detection network to realize video anomaly detection.

[0040] Through, a label noise filtering network is trained based on weak supervision. Here, noise refers to normal video clips in weakly labeled abnormal videos. The network is trained by adopting a weak supervision method based on multi-instance learning, and the abnormality score of each video clip is predicted. The abnormality score represents the possibility of abnormality in the video clip. After further processing, the abnormality score is used as the derived label of the corresponding video clip, and the corresponding video clip is used as the input data of the next stage training model; the network can preliminarily distinguish the normal video clip and the abnormal identification clip in the abnormal video in the training data, thereby achieving the purpose of filtering the label noise in the abnormal video, so as to help the next stage of training the abnormal detection model to better learn the abnormal behavior characteristics. In the second stage, the derived labels and corresponding data are used as supervision information to train a strongly supervised anomaly detection network, whose backbone network is the action classifier I3D. At the same time, the self-attention module is introduced into the I3D model to help the model better focus on the abnormal area in the video. Finally, the purpose of detecting abnormalities in the video is achieved through binary classification.

[0041] Among them, the detection method based on trajectory classification establishes an abnormal trajectory feature table, trajectory smoothing processing and a trajectory anomaly detection algorithm based on an autoencoder and a Gaussian mixture model.

[0042] By using the abnormal trajectory feature table to collect and organize key information in the vehicle's motion trajectory, including speed, acceleration, driving direction, etc., and perform trajectory smoothing to eliminate noise interference caused by the accuracy of monitoring equipment or environmental factors, and ensure the accuracy of trajectory data. Finally, the autoencoder is used to learn the normal trajectory and establish a model that can reconstruct the normal trajectory, while the Gaussian mixture model is used to analyze the reconstruction error. By comparing the error distribution between the actual trajectory and the reconstructed trajectory, it is determined whether there is an abnormal trajectory.

[0043] The working principle and use process of the present invention are as follows: First, the highway monitoring video is preprocessed, and then the detection method based on weak supervision video, the detection method based on trajectory classification and the detection method based on multi-target tracking are used to detect and track the vehicles in the video in real time to identify abnormal behaviors. The lane information is identified through the lane line detection model, and different behavior judgment models are designed in combination with the real-time position information and status information of the vehicle to realize the judgment of the corresponding abnormal events. Finally, the effectiveness of the present invention is verified through algorithm comparison and ablation experiments, and the overall detection system is optimized and improved.

[0044] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0045] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A highway vehicle abnormal event detection system based on surveillance video, characterized in that: include: The detection method based on multi-target tracking uses target detection and target tracking algorithms to accurately detect and track vehicles in video data. At the same time, the lane line detection algorithm is used to identify the lane lines of the highway, extract various possible vehicle abnormal events, build models, and mathematically define each abnormal traffic event. When the relevant parameters meet specific conditions, it is determined that an abnormal vehicle event has occurred, and based on this, an abnormal behavior recognition algorithm based on state changes is constructed; The detection method based on weakly supervised video adopts the vehicle abnormal behavior detection based on the video anomaly detection algorithm, and utilizes the vehicle abnormal behavior recognition technology based on the video anomaly detection algorithm. The video anomaly detection model automatically learns the appearance and motion characteristics of the target in the monitoring video to realize the detection of the vehicle involved in the traffic accident. The detection method based on trajectory classification adopts a motion trajectory detection algorithm to realize the detection of vehicles with abnormal trajectories in surveillance videos. This method realizes the binary classification of the motion trajectories of all vehicles in surveillance videos through a detection algorithm based on trajectory anomalies.

2. The highway vehicle abnormal event detection system based on monitoring video according to claim 1 is characterized by: The detection method based on multi-target tracking is based on multi-target detection, multi-target tracking, lane line detection and abnormal behavior recognition algorithm based on state change, to achieve accurate detection of illegal parking, wrong-way driving, reversing and occupying emergency lane, and provide a basis for trajectory classification.

3. The highway vehicle abnormal event detection system based on monitoring video according to claim 1 is characterized by: The detection method based on weakly supervised video is based on a multi-instance learning algorithm and a strong supervision algorithm respectively, so as to achieve accurate detection of traffic accidents.

4. The highway vehicle abnormal event detection system based on monitoring video according to claim 1 is characterized by: The detection method based on trajectory classification is based on the establishment of a trajectory anomaly model, trajectory processing, and a trajectory classification method based on an autoencoder and a Gaussian mixture model to achieve detection of trajectory anomalies.

5. The highway vehicle abnormal event detection system based on monitoring video according to claim 1 is characterized by: It also includes algorithm comparison and ablation experiments. The algorithm comparison is used to compare other weakly supervised algorithms and compare experimental results. The ablation experiment includes the establishment of an overall detection system.

6. The highway vehicle abnormal event detection system based on monitoring video according to claim 2 is characterized by: The detection method based on multi-target tracking takes frames and annotates highway video data, uses a one-stage target detection algorithm to detect vehicle target frames in real time, combines Kalman filtering and the Hungarian algorithm to perform multi-target tracking, obtains vehicle IDs and real-time target frames, and uses a lane line detection model to identify lane information.

7. The highway vehicle abnormal event detection system based on monitoring video according to claim 3 is characterized by: The detection method based on weakly supervised video uses a weakly supervised multi-instance learning method to identify normal segments in weakly labeled abnormal videos. The network predicts the abnormal scores of video segments and uses them as derived labels after processing for the next stage of training. At the same time, the noise in the abnormal video is filtered out to provide clearer data for the learning of the anomaly detection model. A strongly supervised anomaly detection network is trained using the derived labels and data to realize video anomaly detection.

8. The highway vehicle abnormal event detection system based on monitoring video according to claim 4 is characterized by: The detection method based on trajectory classification establishes an abnormal trajectory feature table, trajectory smoothing processing, and a trajectory anomaly detection algorithm based on an autoencoder and a Gaussian mixture model.

Images