Lightweight vehicle abnormal behavior detection method and system for highway scenarios

Abstract

The application discloses a highway scene-oriented lightweight vehicle abnormal behavior detection method, comprising the following steps: (1) setting a monitoring camera on a high pole at the roadside of the highway to obtain monitoring video stream data; (2) using a lightweight vehicle detection algorithm MG-YOLOv5s based on an improved YOLOv5s to detect vehicles in the monitoring video stream data, and obtaining the coordinate position of the vehicle target in the video; (3) using a vehicle tracking algorithm BoT-SORT to track the vehicle and obtain the motion trajectory of the vehicle, and then preprocessing the vehicle trajectory; (4) analyzing the motion trajectory of the vehicle to establish a trajectory model of the vehicle; (5) detecting whether the vehicle has vehicle parking and vehicle reverse abnormal behaviors through a set judgment condition. The vehicle trajectory abnormality detection can analyze and process monitoring data in real time, can quickly and accurately find the abnormal trajectory of the vehicle, and can timely notify relevant personnel to make early warning and guidance processing.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and intelligent transportation, and specifically, a lightweight vehicle abnormal behavior detection method for highway scenes is designed by using target detection and target tracking technology. BACKGROUND

[0002] In recent years, China has increased investment in transportation infrastructure, making China have the world's largest highway network. With the continuous improvement of the road network, highways have gradually become the main artery of road traffic, bearing relatively fast traffic flow. However, the average speed of the driving vehicle is fast, and when encountering abnormal situations, the driver's reaction time is short, which can easily lead to traffic safety problems. Therefore, how to use the vehicle abnormal behavior detection method to find vehicle abnormal behavior and notify relevant personnel to make early warning and remind, improve the stability and efficiency of traffic operation, and prevent traffic accidents from occurring, has become an important problem faced by China's highway vehicle abnormal detection.

[0003] At present, the highway management company has established a relatively perfect video monitoring system, and the road side monitoring has a wide field of view, which can enlarge and reduce the monitoring range, and the monitoring range of part of the road can basically cover the whole road. These monitoring videos are gathered in the monitoring room of the highway group, and the main work of the management personnel is to patrol the camera and understand the vehicle conditions in each video. With the increasing number of monitoring cameras, the monitoring management personnel cannot guarantee to conduct real-time inspection on all cameras, and if there is a missed inspection, the relevant vehicles are not handled in time, which leads to traffic accidents, the injured personnel need to be treated, and the accident scene needs to be cleaned up. If the accident scene is not handled in time, it will lead to a decrease in highway traffic capacity, and even secondary accidents may occur, causing certain economic losses and resource waste.

[0004] According to the different application scenarios, the accuracy and speed requirements of vehicle abnormal detection are different. In the highway scene, there are still problems such as real-time performance and high false alarm rate of vehicle abnormal behavior detection. In addition, there are also problems such as multi-scale vehicle detection and missing small target vehicles in the video, which need to be further studied. SUMMARY

[0005] The purpose of the application is to overcome the deficiencies of the vehicle abnormal behavior detection method in the highway scene, and the application provides a lightweight vehicle abnormal behavior detection method for the highway scene, which improves the work efficiency of the monitoring management personnel, prevents traffic accidents, improves travel safety, and realizes the intelligentization of traffic supervision.

[0006] Technical scheme: The lightweight vehicle abnormal behavior detection method for the highway scene comprises the following steps:

[0007] Step (1) sets up a monitoring camera on a high pole at the roadside of the highway to obtain monitoring video stream data;

[0008] Step (2) uses a lightweight vehicle detection algorithm MG-YOLOv5s based on YOLOv5s to detect vehicles in the monitoring video stream data and obtain the coordinate position of the vehicle target in the video; the algorithm improves the vehicle detection speed and the detection effect of small target vehicles by improving the MobileNetv3 lightweight network, the bidirectional feature pyramid and the attention mechanism module;

[0009] Step (3) uses a vehicle tracking algorithm BoT-SORT to track the vehicle and obtain the motion trajectory of the vehicle, and then pre-processes the vehicle trajectory; in order to obtain the specific position of the vehicle, the vehicle detection algorithm is used to obtain the position of the vehicle target and the appearance feature information, and the Kalman filter is used to predict the next frame of vehicle state position information, the obtained vehicle tracking frame position is formed into a trajectory, and then the vehicle trajectory is pre-processed;

[0010] Step (4) analyzes the motion trajectory of the vehicle to establish a trajectory model of the vehicle, and detects whether the vehicle has the abnormal behaviors of parking and reverse driving through the set judgment conditions;

[0011] Step (5) repeats steps (2) to (4) to obtain the motion trajectory of the vehicle, detects the motion trajectory of the vehicle, and if the detected trajectory is a parking trajectory, the trajectory is set as an abnormal trajectory, and it is determined that the vehicle has the parking behavior;

[0012] Step (6) repeats steps (2) to (4) to obtain the motion trajectory of the vehicle, detects the motion trajectory of the vehicle, and if the detected trajectory is a reverse driving trajectory, the trajectory is set as an abnormal trajectory, and it is determined that the vehicle has the reverse driving behavior.

[0013] Further, the step (3) pre-processes the vehicle trajectory as follows:

[0014] (3.1) takes the upper left corner of the monitoring picture as the vertex of the coordinate system, and establishes a two-dimensional rectangular coordinate system of the image with the horizontal direction of the image as the X-axis and the vertical direction of the image as the Y-axis; the vehicle trajectory point model formula is as follows:

[0015]

[0016] tn is the time, and id is the vehicle ID;

[0017] (3.2) uses the mean filtering method to smooth the trajectory data; a sliding window is selected, that is, the average value of the adjacent two coordinate points in the trajectory of the vehicle ID id is selected as the new trajectory point, and the specific operation method is as follows:

[0018]

[0019] wherein, the trajectory points (x t ,y t ) and (x t+1 ,y t+1 ) are adjacent trajectory points.

[0020] Further, the step (5) comprises:

[0021] Further, the step (5) comprises:

[0022] According to the parking trajectory model, the following determination is made on the parking trajectory, and the specific formula is as shown below:

[0023] Dis last3 <D1

[0024]

[0025] wherein, Dis last3 represents the sum of the Euclidean distances of the last three frames of trajectory, when the distance is less than the threshold D1, then the average distance of the vehicle trajectory points of the adjacent n frames of the trajectory is calculated, if the calculated average distance is less than the threshold D2, and the ratio of the number of times of Stop num to the total number of frames N is greater than the threshold M; wherein, Stop num is the number of two adjacent trajectory points less than the threshold D2.

[0026] Further, the step (6) comprises:

[0027] (6.1) selecting a number of D trajectory points, when the number of trajectory points exceeds D, then the farthest trajectory point is deleted and the nearest trajectory point is added; in the D trajectory points, each trajectory point is compared with the previous trajectory point;

[0028] (6.2) when the proportion is greater than M, it is considered that the trajectory belongs to the reverse trajectory, and is determined as reverse.

[0029] Further, when the positive direction is set as the vertical downward direction, the y-axis coordinate value of the vehicle trajectory point of the reverse behavior will gradually decrease; when the positive direction is set as the vertical upward direction, the y-axis coordinate of the vehicle trajectory point will gradually increase.

[0030] Further, when setting different regions, it is judged whether the trajectory point belongs to a forward lane or a reverse lane, and the size of the coordinate value is further judged to establish a reverse trajectory model; thus, according to the reverse trajectory model, the specific determination formula for the reverse trajectory is as follows:

[0031]

[0032] wherein Direction num is the number of comparisons of the y-axis coordinate values of two trajectory points, when the downward vertical direction is the positive direction, if y1>y2 in the y-axis trajectory points, Direction num is added by 1, wherein L is the total number of comparisons; and M is a threshold value.

[0033] The lightweight vehicle abnormal behavior detection system for the expressway scene comprises a data acquisition module, a vehicle trajectory processing module, a vehicle trajectory model module, and a vehicle abnormality judgment module.

[0034] The data acquisition module sets a monitoring camera on a high pole at the roadside of the expressway to acquire monitoring video stream data.

[0035] The vehicle trajectory processing module uses a lightweight vehicle detection algorithm MG-YOLOv5s based on the improved YOLOv5s to detect vehicles in the monitoring video stream data, and acquires the coordinate position of the vehicle target in the video; the algorithm improves the vehicle detection speed and the detection effect of small target vehicles by improving the MobileNetv3 lightweight network, the bidirectional feature pyramid, and the attention mechanism module;

[0036] A vehicle tracking algorithm BoT-SORT is used to track the vehicle and acquire the motion trajectory of the vehicle, and then the vehicle trajectory is preprocessed. In order to acquire the specific position of the vehicle, the vehicle detection algorithm is used to obtain the vehicle target position and appearance feature information, and Kalman filtering is used to predict the next frame vehicle state position information, the acquired vehicle tracking frame position is formed into a trajectory, and then the vehicle trajectory is preprocessed;

[0037] The vehicle trajectory model module analyzes the motion trajectory of the vehicle to establish a vehicle trajectory model, and detects whether the vehicle has vehicle parking and vehicle reverse abnormal behavior through a set judgment condition;

[0038] The vehicle abnormality judgment module repeats the steps of the vehicle trajectory processing module and the vehicle trajectory model module to acquire the motion trajectory of the vehicle, detects the motion trajectory of the vehicle, if the detection is a parking trajectory, the trajectory is set as an abnormal trajectory, and it is judged that the vehicle has a parking behavior;

[0039] If the detection is a reverse trajectory, the trajectory is set as an abnormal trajectory, and it is judged that the vehicle has a reverse behavior.

[0040] Further, the parking trajectory model in the vehicle anomaly judgment module comprises the following judgments on the parking trajectory, and specific formulas are as follows:

[0041] Dis last3 <D1

[0042]

[0043] wherein Dis last3 represents the sum of the Euclidean distances of the last three frames of trajectory, when the distance is less than a threshold D1, then the average distance of the vehicle trajectory points of the adjacent n frames of the trajectory is calculated, if the calculated average distance is less than a threshold D2, and the ratio of the number of times of Stop num to the total number of frames is greater than a threshold M; wherein Stop num is the number of two adjacent trajectory points less than the threshold D2.

[0044] Further, the reverse trajectory model in the vehicle anomaly judgment module comprises: when different regions are set, it is judged whether the trajectory point belongs to a forward lane or a reverse lane, and the size of the coordinate value is judged to establish a reverse trajectory model; thus, according to the reverse trajectory model, the specific judgment formula of the reverse trajectory is as follows:

[0045]

[0046] wherein Direction num is the number of comparison of the y-axis coordinate values of two trajectory points, when the downward direction is taken as the positive direction, if y1>y2 in the y-axis trajectory point, then Direction num is added by 1, wherein L is the total number of comparison; and M is a threshold.

[0047] Advantages: Compared with the prior art, the vehicle abnormal behavior detection method based on the expressway scene has the following advantages: the vehicle abnormal behavior detection method based on the expressway scene is aimed at the vehicle driving conditions of the accident-prone road section and different time periods. The vehicle is detected and tracked through the monitoring video, and the position, direction, trajectory and other characteristic information of the vehicle are obtained in the video frame, which provides strong support for realizing vehicle trajectory anomaly detection. The vehicle trajectory anomaly detection can quickly and accurately find the vehicle abnormal trajectory by real-time analysis and processing of the monitoring data, and timely notify the relevant personnel to make early warning and guidance processing. Not only helps to improve the stability and efficiency of traffic operation, but also can prevent traffic accidents, reduce personnel casualties and vehicle losses, so as to improve the travel safety. In addition, the intelligent traffic supervision system can help the supervision and management personnel to improve the work efficiency and reduce the pressure of manual inspection, and realize the intelligentization of traffic supervision. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 A vehicle abnormal trajectory detection algorithm flowchart is provided;

[0049] Figure 2 An MG-YOLOv5s network structure diagram is provided;

[0050] Figure 3 A vehicle behavior trajectory schematic diagram is provided. DETAILED DESCRIPTION

[0051] The technical solutions of the present application will be further described below in combination with the drawings.

[0052] As Figure 1 shown, the present application proposes a lightweight vehicle abnormal behavior detection method based on highway scenes, which mainly tracks and detects abnormal behaviors of vehicles through highway monitoring video streams, and displays whether there is a vehicle abnormal behavior in the current video in real time.

[0053] (1) A monitoring camera is arranged on a high pole at the roadside of the highway to obtain monitoring video stream data;

[0054] (2) The center point position of the vehicle tracking frame in the video stream is obtained through the MG-YOLOv5s vehicle detection algorithm and the BoT-SORT vehicle tracking algorithm, and after a series of trajectory points are obtained, the mean filter is used for smoothing processing to obtain the processed trajectory sequence. The network structure diagram of the MG-YOLOv5s algorithm is as shown in Figure 2 .

[0055] (3) As Figure 3 shown, a vehicle trajectory model is established according to the vehicle abnormal behavior in the monitoring video.

[0056] After the monitoring camera is fixed and the driving direction is determined, ID1 is the driving trajectory of the vehicle in the normal driving direction, t0 is the time when the vehicle detection frame appears in the monitoring picture, and tn is the time when the ID1 vehicle is about to drive out. The time when ID3 vehicle t0 is the driving-in frame, and tn is the driving-out frame. As can be seen from the vehicle trajectory, the driving direction of ID3 vehicle is opposite to the specified direction, and there is a reverse behavior. ID2 is the trajectory diagram of the vehicle parking, at t0, ID2 vehicle drives in the trajectory direction, at t1, the vehicle stops at this time, and until tn, the vehicle does not drive away. Through the time and the driving trajectory of the vehicle, we can obtain the driving direction of the vehicle and the coordinate position of the vehicle in the video. Analyze the motion trajectory of these vehicles, and through the set judgment condition, whether the vehicle parking, vehicle reverse abnormal behavior exists is detected.

[0057] (4) The vehicle parking detection algorithm flowchart is as shown in Figure 1As shown, according to the processed tracking trajectory and in combination with the determination condition, it is determined whether it is a vehicle parking trajectory, and further whether it is a vehicle parking behavior.

[0058] According to the vehicle driving trajectory, the trajectory points corresponding to the vehicle parking are gathered in a certain area within a certain time, and the distance between the coordinate points is very close. According to the parking trajectory model, the following specific formula is used to determine the parking trajectory:

[0059] Dis last3 <D1

[0060]

[0061] wherein Dis last3 represents the sum of the Euclidean distances of the last three frames of trajectory, when the distance is less than the threshold D1, then the average distance of the vehicle trajectory points of the adjacent n frames of the trajectory is calculated, if the calculated average distance is less than the threshold D2, and the ratio of the number of times of Stop num to the total number of frames N is greater than the threshold M; wherein Stop num is the number of two adjacent trajectory points less than the threshold D2.

[0062] The above formula is used to determine the trajectory of the vehicle parking, if the above determination is met, the trajectory is set as the vehicle parking abnormal trajectory, and it is determined that the vehicle has a parking behavior;

[0063] (5) The vehicle reverse driving detection algorithm flow chart is shown in Figure 1 According to the processed tracking trajectory and in combination with the determination condition, it is determined whether it is a vehicle reverse driving trajectory, and further whether it is a vehicle reverse driving behavior.

[0064] When the positive direction is set as vertically downward, the y-axis coordinate value of the vehicle trajectory point with reverse driving behavior will gradually decrease; when the positive direction is set as vertically upward, the y-axis coordinate of the vehicle trajectory point will gradually increase. Specifically, when different areas are set, it is determined whether the trajectory point belongs to the forward lane or the reverse lane, and then the size of the coordinate value is determined, and a reverse driving trajectory model is established. According to the reverse driving trajectory model, the specific determination formula of the reverse driving trajectory is as follows:

[0065]

[0066] wherein Direction num is the number of y-axis coordinate value comparison of two trajectory points, when the positive direction is set as vertically downward, if y1>y2 in the y-axis trajectory point, then Direction num is added by 1, wherein L is the total comparison times; and M is the threshold.

[0067] Further, the vehicle reverse driving detection algorithm has the implementation process as follows

[0068] (a) Selecting the number of D trajectory points, when the number of trajectory points exceeds D, the farthest trajectory point is deleted and the nearest trajectory point is added. In the D trajectory points, each trajectory point is compared with the previous trajectory point;

[0069] (b) When the proportion is greater than M, it is considered that the trajectory belongs to the reverse driving trajectory, and the vehicle reverse driving is determined;

[0070] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A lightweight vehicle abnormal behavior detection method for a highway scenario, characterized in that, The method comprises the following steps: Step (1): setting a monitoring camera on a high pole at the roadside of an expressway to obtain monitoring video stream data; Step (2): detecting vehicles in the monitoring video stream data by using a lightweight vehicle detection algorithm MG-YOLOv5s based on YOLOv5s to obtain the coordinate position of the vehicle target in the video; Step (3): tracking the vehicle by using a vehicle tracking algorithm BoT-SORT and obtaining the motion trajectory of the vehicle, and then preprocessing the vehicle trajectory; Step (4): analyzing the motion trajectory of the vehicle to establish a trajectory model of the vehicle; Step (5): repeating steps (2) to (4) to obtain the motion trajectory of the vehicle, detecting the motion trajectory of the vehicle, if the detected trajectory is a parking trajectory, setting the trajectory as an abnormal trajectory, and determining that the vehicle has a parking behavior; Step (6): repeating steps (2) to (4) to obtain the motion trajectory of the vehicle, detecting the motion trajectory of the vehicle, if the detected trajectory is a reverse trajectory, setting the trajectory as an abnormal trajectory, and determining that the vehicle has a reverse behavior; The step (5) comprises: According to the parking trajectory model, the following determination is made on the parking trajectory, and the specific formula is as follows: Dis last3 <D1 wherein Dis last3 represents the sum of the Euclidean distances of the last three frames of the trajectory, when the distance is less than a threshold D1, then the average distance of the vehicle trajectory points of the adjacent n frames of the trajectory is calculated, if the calculated average distance is less than a threshold D2, and the ratio of the number of times Stop num is greater than a threshold M; wherein Stop num is the number of two adjacent trajectory points less than a threshold D2. Step (6) comprises: (6.1) selecting a number of D trajectory points, when the number of trajectory points exceeds D, deleting the farthest trajectory point and adding the nearest trajectory point, and comparing each trajectory point with the previous trajectory point in the D trajectory points; (6.2) when the proportion is greater than M, it is considered that the trajectory belongs to a reverse trajectory, and it is determined to be reverse; When the positive direction is set as the vertical downward direction in the step (6), the y-axis coordinate value of the trajectory point of the vehicle with the reverse behavior gradually decreases; when the positive direction is set as the vertical upward direction, the y-axis coordinate of the vehicle trajectory point gradually increases; When different regions are set, it is determined whether the trajectory point belongs to a forward lane or a reverse lane, and the size of the coordinate value is determined to establish a reverse trajectory model; thus, according to the reverse trajectory model, the specific determination formula of the reverse trajectory is as follows: wherein Direction num is the number of times that two trajectory point y-axis coordinate values are compared, and when the vertically downward direction is the positive direction, if y1>y2 in the y-axis trajectory points, then Direction num is incremented by 1, wherein L is the total number of comparisons; and M is a threshold value.

2. The highway scenario oriented light-weight vehicle abnormal behavior detection method according to claim 1, characterized in that, The preprocessing step of the vehicle trajectory in the step (3) is as follows: (3.1) taking the upper left corner of the monitoring picture as the vertex of the coordinate system, and establishing a two-dimensional rectangular coordinate system of the image with the horizontal direction of the image as the X-axis and the vertical direction of the image as the Y-axis; the vehicle trajectory point model formula is as follows: tn is the time, and id is the vehicle ID; (3.2) the mean filtering method is adopted to smooth the trajectory data; a sliding window is selected, that is, the average value of the adjacent two coordinate points in the trajectory of the vehicle with the vehicle ID id is selected as the new trajectory point, and the specific operation method is as follows: Wherein, the trajectory points (x t ,y t ) and (x t+1 ,y t+1 ) are adjacent trajectories of the previous and the next moment.

3. The highway scenario oriented light-weight vehicle abnormal behavior detection method according to claim 1, characterized in that, In the step (5), the vehicle trajectory is obtained by repeating steps (2) to (4), the trajectory point corresponding to the parking of the vehicle is gathered in a certain region within a certain time, the distance between the coordinate points is less than a threshold value, the trajectory is set as an abnormal trajectory, and it is determined that the vehicle has a parking behavior.

4. A lightweight vehicle abnormal behavior detection system for a highway scenario, characterized by, It comprises: a data acquisition module, a vehicle trajectory processing module, a vehicle trajectory model module, and a vehicle abnormality judgment module; The data acquisition module sets a monitoring camera on a high pole at a roadside of an expressway to acquire monitoring video stream data; The vehicle track processing module detects vehicles in the monitoring video stream data using a lightweight vehicle detection algorithm MG-YOLOv5s improved based on YOLOv5s to acquire coordinate positions of the vehicles in the video; A vehicle tracking algorithm BoT-SORT is used to track the vehicles and acquire motion tracks of the vehicles, and then the vehicle tracks are preprocessed; The vehicle track model module analyzes the motion tracks of the vehicles to establish a track model of the vehicles; The vehicle abnormality judgment module repeats the steps of the vehicle track processing module and the vehicle track model module to acquire the motion tracks of the vehicles, detects the motion tracks of the vehicles, and if the tracks are detected as parking tracks, sets the tracks as abnormal tracks and determines that the vehicles have a parking behavior; If the tracks are detected as reverse tracks, sets the tracks as abnormal tracks and determines that the vehicles have a reverse behavior; The parking track model in the vehicle abnormality judgment module includes the following determinations on the parking tracks, and specific formulas are as follows: Dis last3 <D1 wherein Dis last3 represents the sum of the Euclidean distances of the last three frames of the trajectory, when the distance is less than a threshold D1, then the average distance of the vehicle trajectory points of the adjacent n frames of the trajectory is calculated, if the calculated average distance is less than a threshold D2, and the ratio of the number of times Stop num is greater than a threshold M; wherein Stop num is the number of two adjacent trajectory points less than a threshold D2. The reverse track model in the vehicle abnormality judgment module includes the following steps: when different regions are set, it is determined whether the track point belongs to a forward lane or a reverse lane, and then the size of the coordinate value is determined to establish a reverse track model; thus, according to the reverse track model, the reverse track is determined according to the following specific formula: wherein Direction num is the number of times that two trajectory point y-axis coordinate values are compared, and when the vertically downward direction is the positive direction, if y1>y2 in the y-axis trajectory points, then Direction num is incremented by 1, wherein L is the total number of comparisons; and M is a threshold value.

Images