A computer vision-based detection system and method for detecting sporadic traffic anomalies

Abstract

The present application is based on the existing highway traffic monitoring system, by processing the monitoring video image, constructs a kind of incidental traffic abnormal condition detection system and method based on computer vision. Through the difference operation between background models, only the picture containing road abnormal condition is obtained;Through the difference operation between background model and real-time video frame, the learning rate of mixed Gaussian model is adjusted, and the rate of abnormal condition into background model is improved;Through the above difference operation and reflection calculation, the preliminary discovery of abnormal condition is realized;Finally, YOLOv11 is imported for further identification and confirmation, and alarm information is sent out, prompting staff to pay attention and handle.The present application is simple to deploy, uses the existing monitoring facilities of highway, basically does not need to increase equipment investment, makes up for the functional deficiency of existing traffic event detection facilities, makes the function of traffic monitoring system more perfect, and improves the highway safety operation ability.

Description

Technical Field

[0001] This invention relates to the field of traffic anomaly detection technology, and more specifically to a computer vision-based system and method for detecting occasional traffic anomalies. Background Technology

[0002] There is considerable research and application in the detection of abnormal road traffic conditions, with most highways equipped with video traffic incident detectors on top of traffic monitoring facilities. According to the national standard "Video Traffic Incident Detector" (GB / T28789-2012), video traffic incident detectors primarily target six types of typical human and vehicle behavior events: stopping events, wrong-way driving events, pedestrian events, littering events, congestion events, and vehicle departure events. Marketed traffic incident detector products are based on this standard, and their detection functions mainly focus on the aforementioned abnormal events. They lack the ability to identify and detect road surface and environmental factors such as road collapses, rockfalls, and water accumulation, resulting in the inability to detect and handle these abnormal events promptly, leading to related accidents and hazards.

[0003] Therefore, strengthening the monitoring of road and environmental conditions and intelligently identifying occasional traffic anomalies will help to respond quickly to dangerous factors after anomalies occur, take timely measures to deal with them, minimize traffic accidents, and avoid or mitigate casualties; at the same time, it will provide technical support for building an intelligent road network perception system.

[0004] Existing technologies and methods: The main methods for detecting abnormal road traffic conditions include manual identification methods, real-time detection methods based on road sensors, indirect judgment methods based on traffic flow and other parameters, and target detection methods based on computer vision. These methods can be used as a reference for detecting occasional abnormal traffic conditions such as road collapses.

[0005] Manual identification methods rely primarily on human judgment, identifying traffic anomalies through on-site inspections or reviewing video surveillance footage. However, this method is not sustainable, inefficient, and prone to missed detections. Sensor-based detection methods utilize various sensors deployed on road surfaces and along roadsides, such as displacement sensors, stress sensors, weather sensors, visibility sensors, and vehicle detectors, to monitor relevant parameters in real time and issue alerts or warnings when anomalies are detected. While these methods provide relatively accurate data, they can only detect specific parameters, and their effectiveness depends heavily on the location and density of the sensors. Traffic flow parameter analysis methods compare traffic flow data from a specific cross-section or road segment over a certain period with historical traffic flow patterns, analyzing abnormal traffic flow characteristics to conclude that traffic anomalies have occurred. This method focuses on detecting traffic flow-related anomalies and heavily relies on historical data for the specific road segment, resulting in poor versatility. Computer vision-based target detection methods include traditional target detection methods such as background subtraction, optical flow, and frame difference, as well as various computer vision technologies such as deep learning methods based on neural networks. These methods are the main methods for traffic anomaly detection, which are based on surveillance videos and have the advantages of wide coverage and the ability to detect a wide variety of targets.

[0006] Computer vision-based object detection tasks typically divide images or videos into foreground and background. The foreground refers to the target of interest, such as people, animals, or vehicles; the background includes everything else, such as the environment and static objects. In highway traffic anomaly detection, most systems treat the road surface and surrounding environment as background factors, and moving objects like vehicles and pedestrians as the foreground targets. They focus on the abnormal movement of vehicles and pedestrians, lacking the ability to detect the condition of the road surface itself. In this case, if anomalies such as road collapses occur, the system will not issue an alarm; instead, it will ignore the anomaly and gradually blend it into the background as the situation becomes static. This results in existing traffic anomaly detection systems lacking the ability to detect sudden road collapses and other abnormalities, preventing highway operation and management personnel from automatically obtaining such anomaly information through monitoring systems.

[0007] Meanwhile, due to the scarcity of samples of occasional abnormal conditions such as road collapses, it is difficult to construct a large-scale dataset. As a result, when using deep learning methods for detection, there is insufficient training, leading to low model detection accuracy, serious problems such as false positives and false negatives, and poor practical effect. Summary of the Invention

[0008] The purpose of this invention:

[0009] (1) Based on the existing traffic monitoring system, computer vision technology is used to detect occasional traffic anomalies such as road collapses. Highways have basically achieved full-process monitoring. Existing traffic monitoring systems are generally only equipped with detection functions for common anomalies such as parking, wrong-way driving, pedestrians, littering, and congestion, and these detection functions are achieved through dedicated traffic event detectors. They lack the ability to detect occasional traffic anomalies such as road collapses, water accumulation, and rockfalls. This invention is proposed to address this problem.

[0010] (2) Improve the detection capability and accuracy of existing detection technologies for abnormal road traffic conditions. Existing target detection methods for traffic incident detection divide the detection objects into foreground and background. Abnormal moving targets belong to the foreground, while the background is relatively fixed, and content identified as background can no longer be identified as foreground. To address this problem, this invention improves the algorithm to detect changing content in the background, thereby improving the detection capability of existing detection technologies for abnormal conditions.

[0011] To achieve the above objectives, this invention, based on existing highway traffic monitoring systems, constructs a method and system for detecting occasional traffic anomalies such as road collapses and rockfalls by processing monitoring video images. This invention is primarily applied to traffic monitoring systems in highway monitoring centers. By minimizing equipment investment, it detects videos and images captured by fixed traffic monitoring facilities along the highway, aiming to enhance the monitoring center's automatic identification capabilities for traffic anomalies, rather than for application in intelligent vehicles, vehicle-mounted terminals, or drone terminals.

[0012] Specifically, the technical solution of this invention is as follows:

[0013] A computer vision-based system for detecting sporadic traffic anomalies includes:

[0014] Video acquisition module: Acquires highway traffic conditions through traffic monitoring facilities and exports monitoring videos from the monitoring center management platform;

[0015] Image processing module: used to model the background of surveillance video and analyze the stability of the modeled background; store the stable background, perform a subtraction operation with the modeled background at the current moment to determine the threshold; mark the road boundary of the video frame, dividing the frame into the road surface area and the roadside area; perform anomaly location judgment and road surface reflection calculation;

[0016] Data storage module: Used to store exported videos, background images generated by modeling, video frames extracted when an anomaly occurs, alarm information, etc.

[0017] Target recognition module: Targets are identified and marked using target detection algorithms such as YOLO, and an alarm message is issued to confirm the category.

[0018] A computer vision-based method for detecting sporadic traffic anomalies includes:

[0019] S1. The video acquisition module acquires traffic monitoring video through highway monitoring facilities, and the image processing module performs image processing on the video.

[0020] S2. An adaptive Gaussian mixture model is used for background modeling. By separating normal moving targets such as frequently passing vehicles, a relatively clean background image of the surveillance video is obtained.

[0021] S3. Perform initial modeling on the first 500 frames or so of video to obtain a stable background model under normal traffic conditions. And save it;

[0022] S4. Continue with background modeling and perform background difference calculations. If the difference result is less than the threshold, proceed to the next step.

[0023] S5. Continuously perform Gaussian mixture background modeling and save the background at regular intervals. ;

[0024] S6, Perform background model difference operation If the difference result is greater than the threshold, proceed to the next step.

[0025] S7. Extract the background model saved within a certain time range after the suspected anomaly occurs, and input it into the target detection algorithm YOLOv11 for detection; further judge the suspected anomaly through YOLOv11 detection, mark the detection results and store the detection results; at the same time, send an alarm message to the system to confirm the anomaly.

[0026] Furthermore, the Gaussian mixture modeling algorithm uses the probability distribution of each pixel as the basis for distinguishing between the background and the foreground, and its basic formula is expressed as:

[0027]

[0028] Where X is a d-dimensional data vector of a pixel in the image over a certain period of time; This represents the probability density function of the pixel; m is the number of Gaussian distributions. The weights are the mixture weights of the i-th Gaussian distribution, and the sum of the weights of all Gaussian distributions should be equal to 1. Let represent the probability density function of the i-th Gaussian distribution. The mean, Let it be its covariance matrix.

[0029] Furthermore, an initial model is first performed on the first 500 frames or so of the video to obtain a stable background under normal traffic conditions. The image is then saved; subsequently, Gaussian blending background modeling is continuously performed, and a background image is saved every 30 frames. Each time a background image is saved, a difference operation is performed between it and the stable background saved from the initial modeling, that is, let... , For its difference result, if If the difference is less than the threshold T, the result is discarded and the system does not perform further processing; if If the difference is greater than or equal to a given threshold T, the difference result is saved and awaits further processing. When anomalies such as road collapse occur, the newly built background model will include the road collapse or other anomalies as part of the background. At this time, the difference between the stable background saved under normal conditions and the background saved after the suspected anomaly occurs is calculated. If the difference is greater than the threshold, the difference image is saved, resulting in a page containing only the suspected anomaly information.

[0030] Furthermore, the weights of the Gaussian mixture model Update as follows:

[0031]

[0032] in, The learning rate is used. To incorporate abnormal situations into the background model more promptly, an adaptive dynamic learning rate adjustment mechanism is employed. Regions of Interest (ROIs) are used to mark the road edge lines in the monitoring image, dividing the image into the road surface area and the roadside area. Under normal traffic conditions, the roadside area contains no moving targets and serves only as background content; if moving targets appear in the roadside area, it indicates a possible abnormal situation. Since direct background subtraction is faster than Gaussian mixture modeling, while the system performs the above modeling and subtraction operations between the background model, a standard background subtraction method is used to detect the roadside area, i.e., using the current video frame... and background image Differential operations are performed only on the roadside area, serving as a trigger condition for adjusting the learning rate. Let... The roadside area anomaly threshold is set to Learning rate Adjust as follows:

[0033]

[0034] in, is the initial learning rate, and n is the number of frames required for Gaussian modeling. Adjusting the learning rate according to the above formula can increase the rate at which abnormal situations are incorporated into the model when anomalies occur, and quickly update the background model containing the abnormal situations.

[0035] Furthermore, when an anomaly suddenly appears within the road surface area, it is initially judged as a suspected road collapse. Based on this, road surface reflectivity detection is performed, and reflective areas are identified using a fusion detection method combining HSV and Lab spatial methods. If reflectivity is detected, it is further judged as a suspected water accumulation. When an anomaly suddenly appears within the roadside area, it is initially judged as a suspected rockfall. If this anomaly is continuously detected in subsequent video frames and extends to the road edge, it is determined as a suspected rockfall intrusion into the road; simultaneously, the system issues a suspected rockfall anomaly alarm.

[0036] Furthermore, in step S4, if the difference result is greater than the threshold, the learning rate is adjusted and background modeling continues.

[0037] Furthermore, in step S6, if the difference result is less than the threshold, the background is saved again at certain time intervals. .

[0038] Furthermore, in step S7, YOLOv11 has been pre-trained using the corresponding dataset and has the basic ability to identify abnormal conditions such as road collapse, water accumulation, and rockfall.

[0039] The technical effects and advantages of this invention are as follows:

[0040] (1) The present invention is simple to deploy. It utilizes existing highway monitoring facilities and makes up for the functional deficiencies of existing traffic incident detection facilities without requiring additional equipment investment. This makes the traffic monitoring system more complete and improves the highway's safe operation capability.

[0041] (2) Unlike the traditional background subtraction method, which uses the background and real-time video frames for subtraction, this invention finds anomalies by comparing background images with each other. Compared with traditional target detection methods such as background subtraction, this invention solves the problem that it is not easy to detect background changes and improves its ability to detect anomalies within the scope of the surveillance video.

[0042] (3) Unlike the general method of using deep learning to detect video frames one by one, this invention only detects the extracted background model image. Compared with using deep learning alone for anomaly detection, it greatly reduces the computational complexity and computational load, narrows the target detection range, thereby reducing the requirements for computing facilities, reducing energy consumption, and improving detection accuracy. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the system of the present invention;

[0044] Figure 2 This is a flowchart of the detection method. Detailed Implementation

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1: System Composition

[0047] (1) Video acquisition module. The system acquires highway traffic conditions through traffic monitoring facilities and exports the monitoring video from the monitoring center management platform.

[0048] (2) Image processing module. The background modeling of the surveillance video is performed and the stability of the modeled background is analyzed; the stable background is stored and a threshold judgment is made by subtracting it from the modeled background at the current time; the highway boundary is marked on the video screen and the screen is divided into the highway surface area and the roadside area; abnormal location judgment and road surface reflection calculation are performed.

[0049] (3) Data storage module. This module stores exported videos, background images generated from modeling, video frames extracted when anomalies occur, alarm information, etc.

[0050] (4) Target recognition module. Targets are identified and marked using target detection algorithms such as YOLO, and an alarm message is issued to confirm the category of the target.

[0051] Example 2: Detection Method

[0052] The video acquisition module acquires traffic monitoring videos through highway monitoring facilities, and then the image processing module processes the videos. First, background modeling is performed using an adaptive Gaussian mixture model background modeling algorithm. This involves separating normally moving targets, such as frequently passing vehicles, to obtain a relatively clean (vehicle-free) background image from the monitoring video.

[0053] The Gaussian mixture model (Gaussian mixture model) uses the probability distribution of each pixel as the basis for distinguishing between the background and foreground. Its basic formula is expressed as:

[0054]

[0055] Where X is a d-dimensional data vector of a pixel in the image over a certain period of time; This represents the probability density function of the pixel; m is the number of Gaussian distributions. The weights are the mixture weights of the i-th Gaussian distribution, and the sum of the weights of all Gaussian distributions should be equal to 1. Let represent the probability density function of the i-th Gaussian distribution. The mean, Let it be its covariance matrix.

[0056] First, an initial model was created from the first 500 frames or so of the video to obtain a stable background under normal traffic conditions. The image is then saved; subsequently, Gaussian blending background modeling is continuously performed, and a background image is saved every 30 frames. Each time a background image is saved, a difference operation is performed between it and the stable background saved from the initial modeling, that is, let... , For its difference result, if If the difference is less than the threshold T, the result is discarded and the system does not perform further processing; if If the difference is greater than or equal to a given threshold T, the difference result is saved and awaits further processing. When anomalies such as road collapse occur, the newly built background model will include the road collapse or other anomalies as part of the background. At this time, the difference between the stable background saved under normal conditions and the background saved after the suspected anomaly occurs is calculated. If the difference is greater than the threshold, the difference image is saved, resulting in a page containing only the suspected anomaly information.

[0057] Weights of Gaussian mixture model Update as follows:

[0058]

[0059] in, The learning rate is used. To incorporate abnormal situations into the background model more promptly, an adaptive dynamic learning rate adjustment mechanism is employed. Regions of Interest (ROIs) are used to mark the road edge lines in the monitoring image, dividing the image into the road surface area and the roadside area. Under normal traffic conditions, the roadside area contains no moving targets and serves only as background content; if moving targets appear in the roadside area, it indicates a possible abnormal situation. Since direct background subtraction is faster than Gaussian mixture modeling, while the system performs the above modeling and subtraction operations between the background model, a standard background subtraction method is used to detect the roadside area, i.e., using the current video frame... and background image Differential operations are performed only on the roadside area, serving as a trigger condition for adjusting the learning rate. Let... The roadside area anomaly threshold is set to Learning rate Adjust as follows:

[0060]

[0061] in, is the initial learning rate, and n is the number of frames required for Gaussian modeling. Adjusting the learning rate according to the above formula can increase the rate at which abnormal situations are incorporated into the model when anomalies occur, and quickly update the background model containing the abnormal situations.

[0062] When an anomaly suddenly appears within the road surface area, it is initially judged as a suspected road collapse. Based on this, road surface reflectivity detection is performed, and reflective areas are identified using a fusion detection method combining HSV and Lab spatial methods. If reflectivity is present, it is further judged as a suspected water accumulation. When an anomaly suddenly appears within the roadside area, it is initially judged as a suspected rockfall. If this anomaly is continuously detected in subsequent video frames and extends to the road edge, it is determined as a suspected rockfall intrusion into the road; simultaneously, the system issues a suspected rockfall anomaly alarm.

[0063] The background model, saved within a certain time range after a suspected anomaly occurs, is extracted and input into the YOLOv11 target detection algorithm for detection. YOLOv11 has been pre-trained using a relevant dataset and has basic recognition capabilities for anomalies such as road collapses, water accumulation, and rockfalls. The suspected anomalies are further judged using YOLOv11 detection, the detection results are marked and stored, and an alarm message confirming the anomaly is sent to the system.

[0064] By performing differential operations between the background models, an image containing only road surface anomalies is obtained. By performing differential operations between the background models and real-time video frames, the learning rate of the Gaussian mixture model is adjusted, improving the rate at which anomalies are integrated into the background model. Through the above differential operations and reflection calculations, anomalies are initially detected. Finally, the images are imported into YOLOv11 for further identification and confirmation, and an alarm is issued to alert staff to pay attention and handle the situation.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computer vision-based system for detecting sporadic traffic anomalies, characterized in that, include: Video acquisition module: Acquires highway traffic conditions through traffic monitoring facilities and exports monitoring videos from the monitoring center management platform; Image processing module: used to model the background of surveillance video and analyze the stability of the modeled background; The stable background is stored, and a threshold is determined by subtracting it from the current modeling background. Mark the highway boundaries on the video footage, dividing the image into the highway surface area and the roadside area; perform anomaly location identification and road surface reflectivity calculation; Data storage module: Used to store exported videos, background images generated by modeling, video frames extracted when an anomaly occurs, and alarm information; Target recognition module: The YOLO target detection algorithm is used to identify and label targets, and an alarm message is issued to confirm the category. The detection methods of computer vision-based systems for detecting sporadic traffic anomalies include: S1. The video acquisition module acquires traffic monitoring video through highway monitoring facilities, and the image processing module performs image processing on the video. S2. An adaptive Gaussian mixture model is used for background modeling, and a relatively clean background image of the surveillance video is obtained by separating normal moving targets. S3. Perform initial modeling on the first 500 frames or so of video to obtain a stable background model under normal traffic conditions. And save it; S4. Continue with background modeling and perform background difference calculations. If the difference result is less than the threshold, proceed to the next step. S5. Continuously perform Gaussian mixture background modeling and save the background at regular intervals. ; S6, Perform background model difference operation If the difference result is greater than the threshold, proceed to the next step. S7. Extract the background model saved within a certain time range after the suspected anomaly occurs, and input it into the target detection algorithm YOLOv11 for detection; further judge the suspected anomaly through YOLOv11 detection, mark the detection results and store the detection results; at the same time, send an alarm message to the system to confirm the anomaly.

2. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: The Gaussian mixture modeling algorithm uses the probability distribution of each pixel as the basis for distinguishing between the background and the foreground. Its basic formula is expressed as follows: ， Where X is a d-dimensional data vector of a pixel in the image over a certain period of time; This represents the probability density function of the pixel; m is the number of Gaussian distributions. The weights are the mixture weights of the i-th Gaussian distribution, and the sum of the weights of all Gaussian distributions should be equal to 1. Let represent the probability density function of the i-th Gaussian distribution. The mean, Let it be its covariance matrix.

3. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: Initial modeling was performed on the first 500 frames or so of the video to obtain a stable background under normal traffic conditions. The image is then saved; subsequently, Gaussian blending background modeling is continuously performed, and a background image is saved every 30 frames. Each time a background image is saved, a difference operation is performed between it and the stable background saved from the initial modeling, that is, let... , For its difference result, if If the difference is less than the threshold T, the result is discarded and the system does not perform further processing; if If the result is greater than or equal to the given threshold T, the difference result is saved and awaits further processing.

4. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: When a road collapse anomaly occurs, the newly built background model will include the road collapse anomaly as part of the background. At this time, the difference between the stable background saved under normal conditions and the background saved after the suspected anomaly occurs is calculated. If the difference is greater than the threshold, the difference image is saved to obtain a page containing only the suspected anomaly information.

5. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: Weights of Gaussian mixture model Update as follows: ， in, The learning rate is set as follows: To incorporate abnormal situations into the background model more promptly, an adaptive dynamic learning rate adjustment mechanism is adopted; Regions of Interest (ROIs) are used to mark the road edge lines in the monitoring image, dividing the image into the road surface area and the roadside area; Under normal traffic conditions, the roadside area contains no moving targets and serves only as background content; If moving targets appear in the roadside area, it indicates a possible abnormal situation; Since direct background subtraction is faster than Gaussian mixture modeling, while the system performs the above modeling and subtraction operations between the background model, a standard background subtraction method is used to detect the roadside area, i.e., using the current video frame... and background image Differential operations are performed only on the roadside area as a trigger condition for adjusting the learning rate; let The roadside area anomaly threshold is set to Learning rate Adjust as follows: ， in, The initial learning rate is n, and the number of frames required for Gaussian modeling is n. The learning rate is adjusted according to the above formula to increase the rate at which abnormal situations are incorporated into the model when anomalies occur, and to quickly update the background model containing abnormal situations.

6. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: When an anomaly suddenly appears within the road surface area, it is initially judged as a suspected road collapse. Based on this, road surface reflection detection is performed, and the reflective area is identified by the fusion detection method of HSV space and Lab space. If there is reflection, it is further judged as a suspected water accumulation. When an anomaly suddenly appears within the roadside area, it is initially judged as a suspected rockfall. If the anomaly is continuously detected in subsequent video frames and it touches the edge line of the road, it is judged as a suspected rockfall intrusion into the road. At the same time, the system issues a suspected rockfall anomaly alarm message.

7. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: In step S4, if the difference result is greater than the threshold, the learning rate is adjusted and background modeling continues.

8. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: In step S6, if the difference result is less than the threshold, the background is saved again at certain time intervals. .

9. The computer vision-based system for detecting occasional traffic anomalies according to claim 1, characterized in that: In step S7, YOLOv11 has been pre-trained using the corresponding dataset and has the basic ability to identify abnormal conditions such as road collapse, water accumulation and rockfall.

Images