Traffic violation detection method based on target detection

Abstract

The invention relates to a traffic violation detection method based on target detection. The method comprises the following steps: S1, acquiring image data for training; s2, homography estimation is adopted, and world latitude and longitude coordinates of a real traffic road correspond to image plane points of the image data for training; s3, training the lightweight target detection network based on the training image data to obtain a vehicle and pedestrian detection model; s4, performing image preprocessing on a to-be-detected image, and inputting the to-be-detected image into the vehicle and pedestrian detection model to obtain a detection result; s5, calculating the real movement condition in the detection result based on the corresponding relation between the world latitude and longitude coordinates of the real traffic road and the image plane points of the training image data, and judging traffic violation based on the real movement condition; and S6, displaying the traffic violation judgment result on a visual interface. Compared with the prior art, the method has the advantages of being less in calculation amount, lower in cost, higher in model robustness in various complex environments and the like.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and image processing, and in particular to a traffic violation detection method based on target detection. Background Art

[0002] With advancements in technology and industrialization, vehicles are traveling at ever-increasing speeds, bringing people closer together. Consequently, people's demand for mobility has also increased, with automobiles being a popular mode of transportation. The transportation system is the foundation for sustained economic development, and transportation plays a crucial role in economic and social development. However, with the dramatic increase in transportation demand, traffic congestion and accidents are also a growing problem. Approximately 1.35 million people worldwide are involved in traffic accidents due to speeding, and the negative impacts are becoming increasingly prominent, becoming a global issue in economic and social development. Road traffic accident statistics from various regions in China show extremely high casualty rates and severity of damage, posing a significant threat to the lives and property of drivers involved.

[0003] Therefore, to protect human life, reasonable speed limits are considered one of the most effective means of improving road safety. Both speeding and underspeeding increase the probability of traffic accidents, making the need for accurate speed estimation increasingly important. The growing number of cameras installed worldwide and their coverage areas provide a strong foundation for measuring vehicle speed using video data. However, in real-world traffic, the sheer number of vehicles, complex traffic conditions, and even the reliance on hardware configurations present challenges for measuring vehicle speed using video processing.

[0004] Currently, vehicle speed measurement methods include radar, laser, loop detector, GPS, and photoelectric sensors. However, these methods are expensive, require professional maintenance, and lack full speed accuracy. GPS can be susceptible to signal interference, and photoelectric sensors have strict installation requirements and can be affected by strong sunlight or inclement weather.

[0005] With the continuous development of computer vision technology, object detection has gradually become a research hotspot. Object detection refers to the process of finding and locating objects of interest in images or videos, and is often used for object identification and tracking. In traffic detection, object detection technology can automatically identify objects such as moving vehicles and pedestrians in live road images captured by cameras, thereby determining whether there are any traffic violations.

[0006] Currently, deep learning-based object detection algorithms, such as Faster R-CNN and YOLO, have become mainstream methods in the field. They can accurately detect and locate objects in a short time and are effective in detecting complex scenes and small objects. The core idea of ​​these algorithms is to use deep neural networks to extract features from images, and then use classifiers such as multi-layer perceptrons (MLPs) or convolutional neural networks (CNNs) to classify and locate objects in the images.

[0007] In summary, the detection method based on target detection technology is feasible and practical. It can automatically detect and warn traffic violations of vehicles and pedestrians on site, improve traffic safety and orderliness, and provide effective protection for traffic safety.

[0008] Patent No. 202211717058.9 provides a method and device for measuring the speed of vehicles on highways based on roadside monocular camera calibration. The method of the present invention includes: inputting a fixed monocular camera traffic video and processing it into images by frame to produce a traffic road condition dataset, inputting a trained multi-target detection algorithm to determine the pixel position of the target vehicle, tracking the target vehicle detection results through a multi-target tracking algorithm to obtain the target vehicle's motion trajectory; based on the target vehicle's motion trajectory and the fitted pixel coordinate and world coordinate conversion ratio curve, calculating the actual travel distance of the target vehicle, thereby calculating the target vehicle's speed; obtaining K sections along the road traffic direction by adjusting the height and vertical angle of the fixed camera multiple times. The conversion ratio curve between the pixel coordinate and the world coordinate is relatively complex, and the camera detection effect is poor in bad weather.

[0009] Patent No. 2023107771606 provides a method for capturing vehicle images based on a binocular camera: obtaining vehicle images captured by a binocular camera at a preset frequency; identifying the image and selecting key feature points in the vehicle image; calculating the two-dimensional coordinate values ​​of the key feature points in the vehicle image, and mapping the vehicle image to the same plane based on the intrinsic and extrinsic parameters of the binocular camera, and mapping the pixel coordinates of the key feature points in the vehicle image to the same world coordinate system to obtain the position information of the key feature points in the key space; finally, calculating the average speed of the vehicle based on the position information of the key feature points at a preset time interval; mapping the vehicle image to the same plane based on the intrinsic and extrinsic parameters of the binocular camera, identifying and selecting the key target area of ​​the vehicle, and calculating the vehicle speed based on the position information of the key feature points before and after the preset time interval. The calculation process is relatively complex. Summary of the Invention

[0010] The purpose of this invention is to address the problem of poor detection effect in complex weather environments, to reduce the amount of calculation parameters while ensuring detection accuracy, and to use computer vision technology to more efficiently realize speed measurement and pedestrian detection violations, thereby providing a traffic violation detection method based on target detection.

[0011] The purpose of the present invention can be achieved by the following technical solutions:

[0012] A traffic violation detection method based on target detection, the method comprising the following steps:

[0013] S1. Capturing vehicle and pedestrian images of traffic road scenes using a monocular camera and preprocessing them to obtain training image data.

[0014] S2. Using homography estimation, the world latitude and longitude coordinates of the real traffic road are matched with the image plane points of the training image data;

[0015] S3. Train a lightweight object detection network based on the training image data to obtain vehicle and pedestrian detection models;

[0016] S4, after image preprocessing, the image to be detected is input into the vehicle and pedestrian detection model to obtain the detection result;

[0017] S5, calculating the real movement situation in the detection result based on the correspondence between the world latitude and longitude coordinates of the real traffic road in S2 and the image plane points of the training image data, and judging the traffic violation based on the real movement situation;

[0018] S6. Display the traffic violation judgment result on a visual interface.

[0019] Furthermore, the specific steps of S2 are:

[0020] Define the world latitude and longitude coordinates and image pixel coordinates of four real scenes, obtain the homography matrix based on the defined coordinates, obtain the estimated points of the predicted world latitude and longitude coordinates based on the homography matrix, use the EDA algorithm to optimize the estimated points, and obtain the correspondence between the world latitude and longitude coordinates of the real traffic road and the image plane points of the training image data.

[0021] Furthermore, the lightweight target detection network includes CSPDarknet, FPN and YOLO Head, CSPDarknet is the backbone feature extraction network,

[0022] The specific steps for training a lightweight object detection network using image data are:

[0023] The training image data is input into CSPDarknet for feature extraction to obtain feature layers. Feature layers of different dimensions are input into FPN for sampling and feature fusion. The obtained fused features are input into YOLO Head to output the predicted position, and the loss function is calculated for iterative training.

[0024] Furthermore, the specific steps of S5 are:

[0025] Based on the detection results, the process of the target vehicle driving and the pedestrian walking is obtained, the overlapping detection results are removed from the detection results, and the actual moving distance of the target vehicle is calculated based on the correspondence between the removed detection results and the image plane points of the training image data. The actual moving distance is divided by the vehicle's driving time to obtain the speed of the target vehicle. Based on the speed of the target vehicle, it is determined whether there is a traffic violation.

[0026] Furthermore, the actual moving distance is:

[0027]

[0028] Among them, p and q are the coordinate points before and after the target vehicle travels, p x and p y is the world longitude and latitude coordinates of the target vehicle frame before it travels, q x and q y It is the world longitude and latitude coordinates of the target vehicle frame after it travels.

[0029] Furthermore, the vehicle travel time is:

[0030]

[0031] Where t is the travel time.

[0032] Furthermore, S5 also includes:

[0033] If the lower left coordinate of the pedestrian exceeds the coordinate of the lane line, the pedestrian is judged to have violated traffic regulations.

[0034] Furthermore, the specific steps for determining whether a traffic violation occurs based on the speed of the target vehicle are as follows:

[0035] If the speed of the target vehicle exceeds the threshold, it is a traffic violation; otherwise, it is not a traffic violation.

[0036] Furthermore, preprocessing includes image segmentation and image enhancement.

[0037] Furthermore, if the result obtained by judging the traffic violation based on the actual transfer situation is a violation, an alarm is sent to the visualization interface.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The core innovation of this invention lies in the fact that it only requires a monocular camera and utilizes computer vision to detect traffic violations, thus reducing costs. Image enhancement and a lightweight network are used to improve the robustness of low-quality image detection, and the EDA algorithm is used to optimize the coordinate mapping relationship of homography estimation to improve the accuracy of estimated points. Simultaneously, the lightweight backbone network structure effectively reduces the number of parameters. Because it does not rely on radar, laser speed meters, or binocular cameras, and image enhancement, lightweight networks, and EDA algorithms respectively address the problems of poor detection in complex environments, large number of calculated parameters, and coordinate mapping errors, it can overcome the shortcomings of existing technologies such as poor detection in complex weather environments and high computational costs, achieving more efficient speed measurement and pedestrian violation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a traffic violation detection method based on target detection according to the present invention;

[0041] Figure 2 This is the flowchart of image preprocessing;

[0042] Figure 3 This is a structural diagram of the lightweight object detection network proposed in this invention;

[0043] Figure 4 Flowchart for training lightweight object detection networks;

[0044] Figure 5 Flowchart of post-detection processing;

[0045] Figure 6 Flowchart for visualization of alarm information. DETAILED DESCRIPTION

[0046] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0047] The present invention proposes a traffic violation detection method based on target detection. The method first preprocesses the collected traffic road images, then uses a trained lightweight target detection network to perform target detection, and finally post-processes the detection results and performs alarm visualization. Figure 1 This is a flow chart of the traffic violation detection method based on target detection proposed in the present invention, where the dotted line path is the construction and training process of the convolutional network, and the solid line path is the detection process in actual operation.

[0048] In this embodiment, the implementation of the technical solution of the present invention generally includes the following steps:

[0049] (1) Acquisition and preprocessing of traffic road scene images

[0050] Use a 300dpi high-definition industrial camera to capture traffic road scene images. Before inputting the captured images into the convolutional network for target detection, image preprocessing is required, such as Figure 2 As shown, the specific process of image preprocessing is as follows: the image preprocessing in step S1 specifically includes:

[0051] (1) Image segmentation: Clean the images collected on site, extract the video clips of vehicles driving on traffic roads, remove invalid video clips, and segment the video frames into pictures and put them into the training dataset.

[0052] (2) Image enhancement: Preprocess the image to improve image quality and contrast, eliminate noise and other interference, improve image quality and recognizability, and make the image more conducive to observation or further analysis and processing.

[0053] (3) Dataset Partitioning: The expanded dataset is divided into training, validation, and test sets for training and testing the target detection model. The video dataset for this experiment is a video of actual road traffic conditions captured by a camera. The video frames are cut into 100 images, each with a size of 256×256. 80% of the samples are randomly selected from the total training samples and placed in the training dataset, and the remaining 20% ​​of the samples are placed in the speed measurement dataset.

[0054] (2) Calculate the mapping relationship between world coordinates and image coordinates

[0055] (1) Manually define the world latitude and longitude coordinates and image pixel coordinates of four real scenes to ensure that the world coordinates of the real scene and the image pixel coordinates correspond to the same position.

[0056] (2) The mapping relationship can be found using the coordinates of the image plane and the world's longitude and latitude coordinates. The homography matrix H can be calculated by the correspondence between the world coordinates of the real scene and the image pixel coordinates. The homography matrix H uses the DLT algorithm to establish a direct linear relationship equation between the "coordinate machine coordinates" of the pixel point and the object space coordinates of the corresponding object point, which can be solved. Defining a set of correspondences between the two planes can estimate the homography matrix of the current scene. The matrix H can be used to find the projection of the world coordinates on the image level.

[0057] (3) Based on computer vision, image processing can calculate the actual distance traveled by the vehicle. By giving the initial manually defined coordinate mapping relationship to obtain the homography matrix H, each starting latitude and longitude point on the scene can obtain the image coordinates through the mapping relationship, and find the projection of the world coordinates on the image level. In order to ensure that the mapping relationship between the world coordinates and the image pixel coordinates is more accurate, it can be combined with the EDA algorithm to optimize the accuracy. The EDA algorithm describes the distribution of candidate solutions in space through a probability model, and uses statistical learning methods to establish a probability model that describes the distribution of solutions from a macroscopic perspective of the group. Then, the probability model is randomly sampled to generate a new population. This is repeated to achieve population evolution until the termination condition is reached. The loss function of the density estimation evolution EDA algorithm can be introduced to calibrate the coordinates to reduce misalignment. The actual distance traveled by the vehicle can be mapped through the distance between the image coordinates.

[0058] (3) Building a lightweight target detection network

[0059] The network of the present invention is built using the deep learning framework Pytorch. The entire network can be divided into three parts, namely CSPDarknet, FPN and YOLO Head. CSPDarknet can be called the backbone feature extraction network of the network. The input image will first be feature extracted in CSPDarknet. The extracted features are called feature layers. The feature layers extracted to different dimensions are then input into the FPN network for further sampling and feature fusion. Finally, these features are used by YOLO Head to predict the position of objects appearing in the image. The structural diagram of the lightweight target detection network proposed in the present invention is shown in the figure below. Figure 3 shown.

[0060] FPN is a network structure that combines the multi-resolution scale prediction of SSD and the multi-resolution feature fusion of U-Net. It is a fully convolutional network that takes images of any scale as input. For each convolutional backbone, the scale of the output feature maps of each layer maintains a fixed ratio with the scale of the original image. In the FPN part, feature construction includes three processes: bottom-up (Bottom-Up), top-down (Top-Down), and same-layer connection. At the same time, to eliminate the problem of insufficient fusion caused by directly adding the corresponding elements of the two feature maps, FPN uses a 3×3 convolution to smooth the fused feature map, resulting in a more fully fused feature map.

[0061] YOLO Head is the network's classifier and regressor, and is used to detect objects within the feature pyramid. YOLO Head passes the feature maps output by the backbone network through a Conv module to reduce the number of channels and scale the feature maps. Feature maps from different levels are then fused to obtain richer feature information, thereby improving detection performance. The module's Anchors can be calculated before model training and stored in the model for use in generating detection boxes during prediction. The Regression component is used to regress each detection box to determine its position and size, typically using a fully connected layer for feature regression.

[0062] The network uses the SiLU activation function, which is an improved version of Sigmoid and ReLU. SiLU has the characteristics of no upper bound, lower bound, smoothness, and non-monotonicity. SiLU performs better than ReLU on deep models. It can be regarded as a smoothed ReLU activation function. The activation function formula is:

[0063]

[0064] Where x is the feature map input.

[0065] Table 1 Lightweight target detection network parameters

[0066] level type Input size Kernel size / stride Output size 1 convolution (640,640,3) (1,1,4) / 2 (320,320,12) 2 Pooling (320,320,12) (2,2,12) / 2 (160,160,128) 3 convolution (160,160,128) (3,3,128) / 1 (160,160,128) 4～6 Pooling (160,160,128) (2,2,256) / 2 (80,80,256) 7 convolution (80,80,256) (3,3,256) / 1 (80,80,256) 8～10 Pooling (80,80,256) (2,2,512) / 2 (40,40,512) 11 convolution (40,40,512) (3,3,512) / 1 (40,40,512) 12 Pooling (40,40,1024) (2,2,512) / 2 (20,20,1024) 13～15 convolution (20,20,1024) (3,3,1024) / 1 (20,20,1024) 16 convolution (20,20,1024) (3,3,1024) / 1 (20,20,1024)

[0067] Table 2 Comparison of network parameters and computational complexity between this network and YOLOv3 and YOLOv5

[0068] network Million Params Computational capacity (GFlops) YOLOv3 6.06 606.11 YOLOv4 5.86 569.28 YOLOv5 5.12 506.21 This network 4.45 91.08

[0069] Table 1 is a table of parameters of the lightweight target detection network described in the present invention, and Table 2 is a comparison of the network parameters and computational complexity of this network with those of YOLOv3 and YOLOv5.

[0070] (4) Training lightweight object detection network

[0071] The image pre-processed in step (1) is used as the input of the lightweight object detection network in step (2) to train the model. The training process is as follows Figure 4As shown, the dataset is first read for iterative training. After the number of iterations is reached, the validation set is read for model evaluation. If the evaluation result meets the indicator, the training is terminated and the model is saved. Otherwise, training continues. This network uses a method of pre-defined candidate areas, that is, the image is divided into 7*7 grids, each grid allows the prediction of 2 bounding boxes, a total of 49*2 bounding boxes, which can be understood as 98 candidate areas. They roughly cover the entire area of ​​the image, reducing mAP at the expense of greatly improving time efficiency. Compared with traditional region-based detection methods, the network described in the present invention is faster and less expensive, and has more accurate traffic violation detection results.

[0072] In addition, in order to eliminate the imbalance between positive and negative samples, positive and negative samples are used alternately during training, and the order of samples is disrupted after each iteration to prevent overfitting.

[0073] (5) Use the trained network for target detection and result post-processing

[0074] After the preprocessing in step (1), the traffic scene image to be detected is input into the trained lightweight object detection network. Multiple convolutional layers and pooling layers are used to extract features from the input image. Finally, the bounding box and category probability of the detected object are output. The output resolution is equal to the resolution of the input image.

[0075] The specific process of post-processing is as follows Figure 5 As shown, the steps are as follows:

[0076] (1) Removing low-confidence detection results: A lightweight object detection network is used to detect the locations of target vehicles and pedestrians. Each detection result has a confidence score. The present invention can set a threshold and remove detection results with confidence scores below the threshold, thereby obtaining a rectangular box with the optimal detection results.

[0077] (2) Removing overlapping detection results: Detecting multiple targets at the same location is called overlapping detection, which may lead to false positives or double counting. Therefore, the present invention can use the non-maximum suppression (NMS) algorithm to eliminate overlapping detection results. This algorithm merges detection results with similar shapes and confidence scores into a single detection result.

[0078] (3) Determine whether there is a traffic violation: The mapping relationship between the world coordinates and the image pixel coordinates can be found through the homography matrix. The distance between the image pixel coordinates before and after the vehicle is driven can be calculated through the mapping relationship to obtain the actual distance the vehicle has traveled. The speed of the target vehicle can be calculated by dividing it by the driving time of the vehicle. The speed is compared with the pre-set threshold to determine whether the vehicle is speeding. The second step is to detect whether the pedestrian position has crossed the lane line to determine whether there is a violation.

[0079] (4) Return alarm signal: If a vehicle is found to be speeding or a pedestrian is found to be violating a traffic rule, the post-processing algorithm will return an alarm signal so that further measures can be taken. For example, an alarm can be sent to notify the management personnel, an alarm email can be sent, etc.

[0080] (6) Visualization of alarm information

[0081] In order to directly display the detection results, the alarm information is visualized. Figure 6 The specific process is as follows:

[0082] (1) Visually display the location and bounding boxes of target vehicles and pedestrians, for example, by marking target vehicles and pedestrians with rectangular boxes of different colors.

[0083] (2) Add text descriptions to the graphical interface, such as the current vehicle speed, driving time, whether an alarm is triggered, etc., so that users can understand the content and situation of the alarm information more intuitively.

[0084] (3) To meet different user needs, more visual information can be added, such as real-time video stream display, historical data analysis, etc. This information can help users better understand alarm conditions and trends.

[0085] (4) For alarm information that requires a timely response, you can set up automatic sending of alarm emails or text messages so that users can obtain the alarm information in a timely manner.

[0086] In general, alarm information visualization needs to be intuitive, clear, and easy to understand to help users better understand the alarm situation and take appropriate actions.

[0087] The specific steps of the present invention include:

[0088] S1. Use a high-resolution camera to collect and pre-process images of vehicles and pedestrians in traffic road scenes;

[0089] S2, using homography estimation to match the world coordinates of the real traffic road with the image plane points input by the camera;

[0090] S3. Build and train a lightweight object detection network specifically for road vehicle speed detection and pedestrian detection;

[0091] S4, after image preprocessing, the image to be detected is input into the trained lightweight target detection network for target detection and target tracking;

[0092] S5. Post-process the results detected by the network. The real distance is obtained by mapping the real movement distance captured by the camera with the pixel movement distance. The homography estimation matrix is ​​obtained and the distance is calculated. The distance is then compared with the vehicle's travel time to determine whether the vehicle is speeding. If so, an alarm signal is returned. If a pedestrian enters the lane illegally, an alarm signal is also returned.

[0093] S6. Display the obtained alarm information on a visual interface to facilitate management by the administrator.

[0094] The image preprocessing in step S1 specifically includes:

[0095] A. Image Segmentation: Clean the images collected on-site, extract the video clips of vehicles driving on traffic roads, remove invalid video clips, and cut the video frames into pictures and put them into the training dataset.

[0096] B. Image enhancement: Preprocess the image to improve image quality and contrast, eliminate noise and other interference, improve image quality and recognizability, and make the image more conducive to observation or further analysis and processing.

[0097] C. Dataset partitioning: The amplified dataset is divided into training set, validation set, and test set for training and testing the target detection model.

[0098] The image mapping in step S2 specifically includes:

[0099] (1) Manually define the world latitude and longitude coordinates and image pixel coordinates of four real scenes respectively, ensuring that the world coordinates of the real scene and the image pixel coordinates correspond to the same position.

[0100] (2) The mapping relationship can be found using the coordinates of the image plane and the world's latitude and longitude coordinates. The homography matrix H can be calculated by the correspondence between the world coordinates of the real scene and the image pixel coordinates. Defining a set of correspondences between the two planes can estimate the homography matrix of the current scene. The matrix H can be used to find the projection of the world coordinates on the image level. The calculation expression of the camera calibration mapping relationship is:

[0101]

[0102] Where S is the scale factor, P'=(x'y'1) T and P = (xy 1) T are the coordinate points of the image and the coordinate points of the real world respectively. The H matrix is ​​the homography matrix, which can be calculated by the direct linear transformation DLT algorithm.

[0103] (3) By giving the initial manually defined coordinate mapping relationship to obtain the homography matrix H, each starting latitude and longitude point on the scene can obtain the image coordinates through the mapping relationship, and find the projection of the world coordinates on the image level. In order to ensure that the mapping relationship between the world coordinates and the image pixel coordinates is more accurate, the loss function of the density estimation evolutionary EDA algorithm is introduced to calibrate the coordinates and reduce misalignment. The mapping relationship follows the cost function of the density estimation evolutionary EDA algorithm:

[0104]

[0105] Among them, p i is a manually defined point on the image plane, The EDA algorithm predicts the estimated point of longitude and latitude coordinates through a mapping relationship. The EDA algorithm can randomly generate a series of original points and find the coordinate point with the smallest error within these preset points. Through the error calculation of the loss function, multiple individuals with the smallest projection error are selected each time. The mean and variance of the coordinate point with the smallest error are calculated for each iteration. After multiple rounds of iteration until the error converges, the coordinate point with the smallest error can be found. The loss function of the EDA algorithm can evaluate the accuracy by calculating the distance between the point on the image plane and the estimated point. After continuous iterative calculation of the loss function, the individual with the smallest projection error can be selected, thereby improving the accuracy of the estimated point.

[0106] The lightweight object detection network in step S3 is described as follows:

[0107] The network of the present invention is built using the deep learning framework Pytorch. The entire network can be divided into three parts: CSPDarknet, FPN, and YOLO Head. CSPDarknet can be called the backbone feature extraction network of the network. The input image is first subjected to feature extraction in CSPDarknet. The extracted features are called feature layers. The feature layers extracted in different dimensions are then input into the FPN network for further sampling and feature fusion. Finally, these features are used by YOLO Head to predict the location of objects appearing in the image.

[0108] FPN is a network structure that combines the multi-resolution scale prediction of SSD and the multi-resolution feature fusion of U-Net. It is a fully convolutional network that takes images of any scale as input. For each convolutional backbone, the scale of the output feature maps of each layer maintains a fixed ratio with the scale of the original image. In the FPN part, feature construction includes three processes: bottom-up (Bottom-Up), top-down (Top-Down), and same-layer connection. At the same time, to eliminate the problem of insufficient fusion caused by directly adding the corresponding elements of the two feature maps, FPN uses a 3×3 convolution to smooth the fused feature map, resulting in a more fully fused feature map.

[0109] YOLO Head is the network's classifier and regressor, and is used to detect objects within the feature pyramid. YOLO Head passes the feature maps output by the backbone network through a Conv module to reduce the number of channels and scale the feature maps. Feature maps from different levels are then fused to obtain richer feature information, thereby improving detection performance. The module's Anchors can be calculated before model training and stored in the model for use in generating detection boxes during prediction. The Regression component is used to regress each detection box to determine its position and size, typically using a fully connected layer for feature regression.

[0110] The network uses the SiLU activation function, which is an improved version of Sigmoid and ReLU. SiLU has the characteristics of no upper bound, lower bound, smoothness, and non-monotonicity. SiLU performs better than ReLU on deep models. It can be regarded as a smoothed ReLU activation function. The activation function formula is:

[0111]

[0112] Where x is the feature map input.

[0113] The target detection in step S4 is detailed as follows:

[0114] Target detection refers to the use of multiple convolutional layers and pooling layers in a lightweight target detection network to extract features from the input image, find all targets of interest in the image, and determine their categories and locations. The output resolution is equal to the resolution of the input image. This network uses a method of predefined candidate areas, which means that the image is divided into 7*7 grids. Each grid allows the prediction of 2 bounding boxes, a total of 49*2 bounding boxes, which can be understood as 98 candidate areas. They roughly cover the entire area of ​​the image, greatly improving time efficiency at the cost of reducing mAP. Compared with traditional region-based detection methods, the network described in the present invention is faster and less expensive, and has more accurate results for traffic violation detection.

[0115] The post-processing in step S5 is detailed as follows:

[0116] Post-processing refers to the processing and analysis of the detection results after the target detection is completed to obtain more reliable target information. The post-processing process after the detection of road vehicles can be achieved through the following steps:

[0117] (1) Target tracking: A lightweight target detection network is used to detect the position of the target vehicle. Target tracking can be used to track the target vehicle's movement and pedestrians' walking.

[0118] (2) Removing overlapping detection results: Detecting multiple targets at the same location is called overlapping detection, which may lead to false positives or double counting. Therefore, the present invention can use the non-maximum suppression (NMS) algorithm to eliminate overlapping detection results. This algorithm merges detection results with similar shapes and confidence scores into a single detection result.

[0119] (3) Determine whether there is a traffic violation: The mapping relationship between the world coordinates and the image pixel coordinates can be found through the homography matrix. The actual distance traveled by the vehicle can be obtained by calculating the distance between the image pixel coordinates before and after the vehicle travels through the mapping relationship. The speed of the target vehicle can be calculated by dividing it by the vehicle travel time. The speed can be compared with the pre-set threshold to determine whether it is speeding; the second step is to detect whether the pedestrian position has crossed the lane line to determine whether it is a violation. For the target vehicle, the present invention can calculate the distance based on the coordinate points mapped in step S1. The calculation formula is:

[0120] Where p and q are the coordinate points before and after the target vehicle travels, p x and p y is the latitude and longitude coordinates of the target vehicle frame before it travels, q x and q yIt is the latitude and longitude coordinates of the target vehicle frame after it travels. In order to calculate the distance, the inverse matrix of the homography matrix H must be used to convert the position recorded in the image point to the real world plane, so as to calculate the distance before and after the target vehicle travels.

[0121] The time calculation formula is:

[0122]

[0123] By processing the video frame by frame, the time difference between the previous frame and the next frame can be obtained by dividing the frame difference by the video frame rate. The speed of the target vehicle can be calculated by dividing the distance by the time. The unit is kilometers per hour. The calculation formula is:

[0124]

[0125] where d h (p,q) is the actual driving distance of the target vehicle calculated above, and t is the driving time of the target vehicle.

[0126] Given a speed threshold, if the calculated speed of the target vehicle is greater than the threshold, it is considered speeding and a traffic violation. For pedestrians, the target detection algorithm can be used to identify the pedestrian target and give the coordinates of the target box. If the lower left coordinate of the pedestrian exceeds the coordinate of the lane line, it is considered a traffic violation.

[0127] (4) Return alarm signal: If a vehicle is found to be speeding or a pedestrian is found to be violating a traffic rule, the post-processing algorithm will return an alarm signal so that further measures can be taken. For example, an alarm can be sent to notify management personnel.

[0128] The visualization of the alarm information in step S6 includes:

[0129] (1) Visually display the process of tracking moving target vehicles, such as marking moving vehicles and pedestrians with rectangular boxes of different colors.

[0130] (2) Add text descriptions on the graphical interface, such as vehicle speed, driving time, etc., so that users can understand the content and situation of the alarm information more intuitively.

[0131] (3) To meet different user needs, more visual information can be added, such as real-time video stream display, historical data analysis, etc. This information can help users better understand alarm conditions and trends.

[0132] (4) For alarm information that requires a timely response, you can set up automatic sending of alarm emails or text messages so that users can obtain the alarm information in a timely manner.

[0133] In general, alarm information visualization needs to be intuitive, clear, and easy to understand to help users better understand the alarm situation and take appropriate actions.

[0134] The present invention proposes a traffic violation detection method based on target detection, which aims to overcome the shortcomings of the existing technology and equipment problems, and quickly and accurately judge whether vehicles on the current traffic road are speeding and whether pedestrians are violating traffic laws based on computer vision for video processing.

[0135] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A traffic violation detection method based on target detection, characterized in that: The method comprises the following steps: S1. Capturing vehicle and pedestrian images of traffic road scenes using a monocular camera and preprocessing them to obtain training image data. S2. Using homography estimation, the world latitude and longitude coordinates of the real traffic road are matched with the image plane points of the training image data; S3. Train a lightweight object detection network based on the training image data to obtain vehicle and pedestrian detection models; S4, after image preprocessing, the image to be detected is input into the vehicle and pedestrian detection model to obtain the detection result; S5, calculating the real movement situation in the detection result based on the correspondence between the world latitude and longitude coordinates of the real traffic road in S2 and the image plane points of the training image data, and judging the traffic violation based on the real movement situation; S6. Display the traffic violation judgment result on a visual interface.

2. A traffic violation detection method based on target detection according to claim 1, characterized in that: The specific steps of S2 are: Define the world latitude and longitude coordinates and image pixel coordinates of four real scenes, obtain the homography matrix based on the defined coordinates, obtain the estimated points of the predicted world latitude and longitude coordinates based on the homography matrix, use the EDA algorithm to optimize the estimated points, and obtain the correspondence between the world latitude and longitude coordinates of the real traffic road and the image plane points of the training image data.

3. The traffic violation detection method based on target detection according to claim 1, characterized in that: The lightweight target detection network includes CSPDarknet, FPN and YOLO Head, CSPDarknet is the backbone feature extraction network, The specific steps for training a lightweight object detection network using image data are: The training image data is input into CSPDarknet for feature extraction to obtain feature layers. Feature layers of different dimensions are input into FPN for sampling and feature fusion. The obtained fused features are input into YOLO Head to output the predicted position, and the loss function is calculated for iterative training.

4. The traffic violation detection method based on target detection according to claim 1, characterized in that: The specific steps of S5 are: Based on the detection results, the process of the target vehicle driving and the pedestrian walking is obtained, the overlapping detection results are removed from the detection results, and the actual moving distance of the target vehicle is calculated based on the correspondence between the removed detection results and the image plane points of the training image data. The actual moving distance is divided by the vehicle's driving time to obtain the speed of the target vehicle. Based on the speed of the target vehicle, it is determined whether there is a traffic violation.

5. The traffic violation detection method based on target detection according to claim 4, characterized in that: The actual moving distance is: Among them, p and q are the coordinate points before and after the target vehicle travels, p x and p y is the world longitude and latitude coordinates of the target vehicle frame before it travels, q x and q y It is the world longitude and latitude coordinates of the target vehicle frame after it travels.

6. A traffic violation detection method based on target detection according to claim 5, characterized in that: The vehicle travel time is: Where t is the travel time.

7. A traffic violation detection method based on target detection according to claim 6, characterized in that: The S5 also includes: If the lower left coordinate of the pedestrian exceeds the coordinate of the lane line, the pedestrian is judged to have violated traffic regulations.

8. The method for detecting traffic violations based on target detection according to claim 7, characterized in that: The specific steps for determining whether a traffic violation occurs based on the speed of the target vehicle are: If the speed of the target vehicle exceeds the threshold, it is a traffic violation; otherwise, it is not a traffic violation.

9. The method for detecting traffic violations based on target detection according to claim 1, wherein: Preprocessing includes image segmentation and image enhancement.

10. The traffic violation detection method based on target detection according to claim 1, characterized in that: If the result obtained by judging the traffic violation based on the actual transfer situation is a violation, an alarm is sent to the visualization interface.

Images