A traffic violation detection method and system based on visual target tracking

Abstract

The present invention discloses a traffic violation detection method and system based on visual target tracking. It relates to the field of image recognition technology, uses a visual detection algorithm to detect traffic violation vehicles, and uses a Kalman filter method to track the illegal vehicles when the license plate of the illegal vehicles cannot be clearly photographed, and judges the tracking state. When the target is partially blocked, the vehicle position is fitted to continue vehicle tracking; when the target tracking is lost, a re-tracking operation is further performed to improve the continuity and robustness of the target tracking until the license plate of the traffic violation vehicle can be clearly photographed, and the video of the traffic violation vehicle is recorded throughout the process, and multiple photos such as the violation in progress, the violation is terminated, and the clear license plate are taken at the same time to achieve reliable detection of traffic violations.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition processing, and in particular to a traffic violation detection method and system based on visual target tracking. Background Art

[0002] The traffic violation detection system is based on the video sequence collected by the camera. It uses image processing and computer vision technology to determine whether the vehicle has violated the law and submit it to the traffic management department for investigation and punishment. It plays a vital role in the enforcement of traffic regulations and the maintenance of traffic order.

[0003] At present, traffic violation detection systems have been widely used at home and abroad, and can be mainly divided into three categories: the inductive violation detection system pre-arranges coils on the ground, generates induced current signals when vehicles pass by, and controls the camera to capture, which is mostly used to detect violations of traffic lights at intersections; visual violation detection directly uses algorithms in the image to identify violations and captures them in time; there is also a detection system assisted by radar, which is mainly used for speed measurement and photography. The above methods all require obtaining images of vehicle violations as a basis for punishment, and the vehicle license plate number must be clearly visible. However, it is not necessarily possible to obtain images that meet the above requirements when the violation occurs, such as when the illegal vehicle license plate is blocked by other vehicles, and the distance between the vehicle and the camera is too far to be clearly photographed. In addition, with the continuous improvement of navigation software, the locations of many illegal monitoring cameras have long been known to the public. A few drivers take advantage of the limitations of the imaging clarity of the vehicle license plate by monitoring cameras, stop illegal behaviors at a distance before monitoring, and resume the illegal state after leaving the monitoring range to avoid punishment and seek personal gain from illegal behaviors. This phenomenon is most serious in occupying emergency lanes and bus lanes. Such behavior poses a great challenge to the authority of traffic regulations and also poses a hidden danger to travel safety. Summary of the invention

[0004] The purpose of the present invention is to provide a traffic violation detection method and system based on visual target tracking, which can realize reliable detection of traffic violations.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for detecting traffic violations based on visual target tracking, the method comprising:

[0007] Determine a violation detection area, capture an illegal vehicle in the violation detection area based on a target detection algorithm, obtain a clear license plate of the illegal vehicle, and output an illegal behavior image and license plate information of the illegal vehicle;

[0008] When a clear license plate of the illegal vehicle cannot be obtained, Harris corner points are extracted as feature points of the illegal vehicle based on the captured image of the illegal vehicle, and Kalman filtering is used to track the illegal vehicle;

[0009] When the preset detection frame is reached, the predicted target frame of the illegal vehicle is matched with the feature points of the illegal vehicle, and the tracking state of the illegal vehicle is determined according to the matching result;

[0010] When the tracking state is that the target tracking is normal, a clear license plate of the vehicle is obtained in the vehicle target detection frame of the current detection frame, and the illegal behavior image of the illegal vehicle and the license plate information are output; if the clear license plate of the illegal vehicle cannot be obtained, return to the step of "using Kalman filtering to track the illegal vehicle";

[0011] When the tracking state is that the target is partially blocked, the fitting position of the illegal vehicle in the current detection frame is determined according to the feature points of the illegal vehicle in the current detection frame and the previous detection frame, and the process returns to the step of "target tracking of the illegal vehicle using Kalman filtering";

[0012] When the tracking state is target tracking loss, the illegal vehicle is re-tracked in the next detection frame based on the position and size of the vehicle target detection frame in the last frame before the loss;

[0013] When the re-tracking is successful, return to the step of "using Kalman filtering to track the illegal vehicle".

[0014] Optionally, when detecting illegal road occupation, determine the illegal detection area, including:

[0015] The road area image is obtained by cropping the image taken by the violation detection system;

[0016] Performing color space conversion on the road area image, and performing edge detection on the converted image using a Canny operator;

[0017] According to the edge detection results, Hough transform is used to extract straight line segments, and lane dividing lines are determined based on the extracted straight line segments, and several lane areas are obtained;

[0018] Any one of the lane areas is selected as the violation detection area.

[0019] Optionally, Harris corner points are extracted based on the captured illegal vehicle image as illegal vehicle feature points, specifically including:

[0020] Grayscale the illegal vehicle image, and calculate the gradient in the x direction and the gradient in the y direction within the vehicle target detection frame by using a differential operator;

[0021] Constructing a first matrix based on the gradient in the x direction and the gradient in the y direction;

[0022] Performing Gaussian smoothing filtering on the elements in the first matrix;

[0023] Selecting a window size and a corresponding weight, and calculating a second matrix according to the window size, the weight and the first matrix;

[0024] Calculate the Harris response value corresponding to each pixel point in the illegal vehicle image according to the second matrix;

[0025] Pixel points whose Harris response values ​​are greater than a threshold are selected as the Harris corner points to obtain the illegal vehicle feature points.

[0026] Optionally, matching the predicted target frame and the feature points of the illegal vehicle is performed, specifically including:

[0027] In the current detection frame, the target detection algorithm is used to perform vehicle detection within the road range to obtain a number of vehicle detection frames corresponding to the current detection frame;

[0028] Applying the vehicle target frame offset velocity vector obtained by the Kalman filter to the vehicle target detection frame extracted from the previous detection frame and the illegal vehicle feature point to obtain the predicted target frame of the current detection frame and the predicted position of the illegal vehicle feature point;

[0029] Calculating the intersection-over-union ratio of the predicted target frame and each of the vehicle detection frames;

[0030] Determine whether the intersection-over-union ratio is greater than an intersection-over-union ratio threshold, and obtain a first determination result;

[0031] If the first judgment result is yes, a vehicle target detection frame corresponding to the illegal vehicle is obtained, and feature points of the illegal vehicle in the vehicle target detection frame are extracted;

[0032] Performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients;

[0033] If the first judgment result is no, the current predicted target frame is expanded by a preset expansion coefficient, and the illegal vehicle feature points in the expanded predicted frame are extracted;

[0034] Performing nearest neighbor association matching on the illegal vehicle feature points in the predicted target frame before and after expansion to obtain the feature point matching coefficients;

[0035] When the matching coefficient of the feature point is greater than the coefficient threshold, the matching is successful;

[0036] When the feature point matching coefficient is less than or equal to the coefficient threshold, the matching fails.

[0037] Optionally, performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients specifically includes:

[0038] Calculating the Euclidean distance between each of the illegal vehicle feature points in the vehicle target detection frame and the predicted target frame to construct an association cost matrix;

[0039] Comparing the matrix elements in the association cost matrix with a preset association threshold;

[0040] The two feature points corresponding to the matrix elements exceeding the preset association threshold are regarded as the nearest neighbor association matching point pair;

[0041] The feature point matching coefficient is calculated according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle feature points in the previous detection frame, and the number of illegal vehicle feature points in the current detection frame.

[0042] Optionally, determining the fitting position of the illegal vehicle in the current detection frame according to the feature points of the illegal vehicle in the current detection frame and the previous detection frame specifically includes:

[0043] Calculate the relative position between each matching feature point of the previous detection frame and the center point of the target frame, the position of each matching feature point of the current detection frame in the illegal vehicle image coordinates, and construct an equation group in combination with the coordinates of the center point of the illegal vehicle in the current detection frame; the equation group includes n equations; n is the number of matching feature points; the matching feature points are the nearest neighbor associated matching point pairs obtained after nearest neighbor associated matching;

[0044] The system of equations is solved using the least square method to obtain the fitting position of the illegal vehicle in the current detection frame.

[0045] Optionally, judging the tracking status of the illegal vehicle according to the matching result specifically includes:

[0046] When the first judgment result is yes and the second judgment result is yes, the tracking state is that the target tracking is normal;

[0047] When the first judgment result is no and the second judgment result is yes, when the tracking state is that the target is partially blocked;

[0048] When the first judgment result is no and the second judgment result is no, the tracking state is that the target tracking is lost.

[0049] Optionally, re-tracking the illegal vehicle based on the position and size of the vehicle target detection frame of the last frame before the loss specifically includes:

[0050] Obtain the position and size of the vehicle target detection frame of the last frame before the loss;

[0051] Determine the moving direction of the illegal vehicle by using the velocity vector when the Kalman filter is used for target tracking;

[0052] Determine the target potential appearance domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle;

[0053] Calculate the relative position of each illegal vehicle feature point in the vehicle target detection frame of the last frame before the loss and the center point of the vehicle target detection frame, and record it as the first relative position;

[0054] Calculate the intersection-and-union ratio of each vehicle detection frame and the target potential appearance domain in the next detection frame, and record it as the re-tracking intersection-and-union ratio;

[0055] When there is a vehicle target detection frame whose re-tracking intersection-and-union ratio is greater than the re-tracking intersection-and-union ratio threshold, extract the illegal vehicle feature point in the vehicle target detection frame during re-tracking, and calculate the relative coordinates of the illegal vehicle feature point extracted during re-tracking and the center point of the vehicle target detection frame during re-tracking, which are recorded as the second relative position;

[0056] Using a nearest neighbor association matching method, feature point matching is performed according to the first relative position and the second relative position to obtain a feature point matching coefficient during re-tracking;

[0057] When the feature point matching coefficient during re-tracking is greater than the coefficient threshold during re-tracking, the illegal vehicle is recaptured in the potential target appearance domain;

[0058] The vehicle position is corrected using the vehicle target detection frame that currently captures the illegal vehicle, and a re-tracking mark is added to the re-captured illegal vehicle.

[0059] Optionally, determining the target potential appearance domain according to the vehicle target detection frame of the last frame before the loss and the moving direction of the illegal vehicle specifically includes:

[0060] Calculate the intersection of the reverse ray of the center speed of the vehicle target detection frame of the last frame before the loss and the rear frame line of the detection frame, and draw a perpendicular line to the lane line through the intersection;

[0061] Determine the lane where the illegal vehicle is located according to the coordinates of the detection frame lines on both sides, and extract the lane boundary line;

[0062] The area formed by the lane boundary line, the vertical line of the lane line and the edge of the field of view is regarded as the potential appearance domain of the target.

[0063] The present invention also provides a traffic violation detection system based on visual target tracking, the system comprising:

[0064] The area determination and target detection module is used to determine the illegal detection area, capture the illegal vehicle in the illegal detection area based on the target detection algorithm, obtain the clear license plate of the illegal vehicle, and output the illegal behavior image and license plate information of the illegal vehicle;

[0065] A feature extraction and target tracking module, which is used to extract Harris corner points as feature points of the illegal vehicle based on the captured image of the illegal vehicle when a clear license plate of the illegal vehicle cannot be obtained, and to track the illegal vehicle using Kalman filtering;

[0066] A matching and tracking state judgment module is used to match the predicted target frame and the feature points of the illegal vehicle when a preset detection frame is reached, and to judge the tracking state of the illegal vehicle according to the matching result;

[0067] The vehicle license plate acquisition and target tracking module is used to acquire a clear vehicle license plate in the vehicle target detection frame of the current detection frame when the tracking state is normal target tracking, and output the illegal behavior image of the illegal vehicle and the vehicle license plate information; if the clear vehicle license plate of the illegal vehicle cannot be acquired, the "target tracking of the illegal vehicle by using Kalman filtering" in the matching and tracking state judgment module is executed;

[0068] A position fitting and target tracking module, which is used to determine the fitting position of the illegal vehicle in the current detection frame according to the feature points of the illegal vehicle in the current detection frame and the previous detection frame when the tracking state is that the target is partially blocked, and execute the "target tracking of the illegal vehicle using Kalman filtering" in the matching and tracking state judgment module;

[0069] The re-tracking module is used to re-track the illegal vehicle in the next detection frame when the tracking state is target tracking loss, based on the position and size of the vehicle target detection frame in the last frame before the loss; when the re-tracking is successful, the "using Kalman filtering to track the illegal vehicle" in the matching and tracking state judgment module is executed.

[0070] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0071] The present invention proposes a traffic violation detection method and system based on visual target tracking. A visual detection algorithm is used to detect traffic violating vehicles. When the license plate of the traffic violating vehicle cannot be clearly photographed, the Kalman filter method is used to continuously track the traffic violating vehicle. When the traffic violating vehicle terminates the traffic violation by changing lanes or other behaviors, the tracking of the traffic violating vehicle is also maintained, and the tracking state is judged. When the target tracking is lost, a re-tracking operation is further performed to improve the continuity and robustness of the target tracking until the license plate of the traffic violating vehicle can be clearly photographed. The video of the traffic violating vehicle is recorded throughout the process, and multiple photos such as the violation, the termination of the violation, and the clear license plate are taken at the same time to achieve reliable detection of traffic violations. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0073] Figure 1 A flow chart of a method for detecting traffic violations based on visual target tracking provided in Embodiment 1 of the present invention;

[0074] Figure 2 The Kalman filter tracking process provided in the first embodiment of the present invention;

[0075] Figure 3 The tracking status determination process provided in the first embodiment of the present invention;

[0076] Figure 4 The vehicle target that cannot be accurately detected due to partial occlusion provided in the first embodiment of the present invention;

[0077] Figure 5 This is the execution process of re-tracing provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] The purpose of the present invention is to provide a method and system for detecting traffic violations based on visual target tracking. A traffic violation vehicle in the distance is detected by a visual detection algorithm. When the license plate of the traffic violation vehicle cannot be clearly photographed, the Kalman filter method is used to continuously track the traffic violation vehicle. When the traffic violation vehicle terminates the traffic violation by changing lanes before approaching the violation detection area where the vehicle license plate can be clearly photographed, the tracking of the traffic violation vehicle is also maintained. In order to cope with the situation where the target is partially blocked and the tracking is lost during the tracking period, the nearest neighbor matching based on Harris corner points and the target re-tracking module using the potential occurrence domain are designed respectively to improve the continuity and robustness of the target tracking until the vehicle license plate of the traffic violation vehicle can be clearly photographed. The video of the traffic violation vehicle is recorded throughout the process, and multiple photos such as the violation, the violation termination, and the clear license plate are taken at the same time to achieve reliable detection of traffic violations.

[0080] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0081] Embodiment 1

[0082] like Figure 1 As shown, this embodiment provides a traffic violation detection method based on visual target tracking, the method comprising:

[0083] S1: Determine the illegal detection area, capture the illegal vehicle in the illegal detection area based on the target detection algorithm, obtain a clear license plate of the illegal vehicle, and output the illegal behavior image and license plate information of the illegal vehicle.

[0084] Among them, in step S1, when detecting illegal behaviors such as illegal occupying of the road, Hough transform is used to detect lane lines and define the starting area for illegal detection and tracking (illegal detection area). The tracking starting area is used for the calibration operation of the illegal detection system after changing the camera position, representing the set of positions where the vehicle should not appear.

[0085] The present invention is mainly designed for illegal behaviors such as illegal road occupation, so after each change of the camera position, lane line detection is required to re-delineate the illegal area. In this case, the specific delineation method is as follows:

[0086] First, the image is cropped based on color characteristics to obtain the road surface area, and Gaussian blur is performed to reduce noise. That is, the image captured by the violation detection system is cropped to obtain the road surface area image.

[0087] Perform color space conversion on the image. The lane lines are mainly yellow and white, so the original image RGB color space is converted to YCbCr color space to distinguish the difference between the two lane lines and the road surface. The conversion method is:

[0088]

[0089] The image is converted into a grayscale image and the Canny operator is used for edge detection. It determines whether the pixel is an edge based on the pixel gradient, and only pixels greater than the threshold are accepted.

[0090] Finally, Hough transform is used to extract straight line segments, and straight lines with similar distances in Hough space are fitted into a straight line as the lane boundary. The road is divided into several lane areas and numbered from left to right. The corresponding number is selected as the tracking starting area of ​​the violation detection system according to the road setting.

[0091] For other traffic violations, the method of dividing the violation starting area is different, but the subsequent processing steps are basically the same. For example, when monitoring red light running, the appropriate range in front of the vehicle's stop line when the red light is on can be selected as the detection area.

[0092] Among them, in step S1, the illegal target is captured in the designated area based on the target detection algorithm, and an attempt is made to obtain the vehicle license plate.

[0093] Use target detection algorithms, such as yolov5 and Faster-RCNN, to detect vehicle targets. Calculate the ratio of the intersection of the detection frame and the tracking start area obtained in step 1 to the detection frame, and screen out illegal targets in the tracking start area obtained in step 1. When an illegal vehicle target is detected, take a photo as a photo of the illegal behavior and record the timestamp. Perform vehicle license plate recognition in its target bounding box. If successfully recognized, it will be directly used as illegal photo evidence.

[0094] S2: When a clear license plate of the illegal vehicle cannot be obtained, Harris corner points are extracted as feature points of the illegal vehicle based on the captured image of the illegal vehicle, and Kalman filtering is used to track the illegal vehicle.

[0095] If the vehicle license plate cannot be recognized, the Plessey algorithm is used to extract Harris corner points as vehicle feature points in the acquired target detection frame. The basic idea is to use a fixed window to slide in any direction on the image and compare the degree of change in pixel grayscale in the window before and after sliding. If a large grayscale change occurs when sliding in any direction, it means that there are Harris corner points in the window. The grayscale change of the pixel point of the sliding window is expressed by the following formula:

[0096]

[0097] Among them, [u,v] is the offset of window W; (x,y) is the pixel coordinate position corresponding to window WW; I(x,y) is the image grayscale value at the pixel coordinate position (x,y); I(x+u,y+v) is the image grayscale value after offset at the pixel coordinate position (x,y); w(x,y) is the window weight function.

[0098] The specific process of extracting Harris corner points is as follows:

[0099] a) grayscale the illegal vehicle image, and calculate the gradient I in the x direction within the vehicle target detection frame by using a differential operator x and the gradient in the y direction I y .

[0100] I x =I×[-1,0,1]

[0101] I y =I×[-1,0,1] T

[0102] b) Based on the gradient I in the x direction x and the gradient in the y direction I y Construct the first matrix N.

[0103] Calculate the image directional gradient product to form a matrix

[0104] c) performing Gaussian smoothing filtering on the elements in the first matrix N.

[0105] Gaussian smoothing filtering is performed on the four elements of the first matrix N, and the discrete two-dimensional zero-mean Gaussian function used is:

[0106]

[0107] d) Select the size of the window W and the corresponding weight w(x, y), and calculate the second matrix M based on the window size, weight and the first matrix N.

[0108]

[0109] e) calculating the Harris response value corresponding to each pixel point in the illegal vehicle image according to the second matrix.

[0110] Calculate the Harris response value corresponding to each pixel point (x, y) R = det(M)-k(trace(M)) 2 .

[0111] f) Pixel points whose Harris response values ​​are greater than a threshold are selected as the Harris corner points to obtain the feature points of the illegal vehicle.

[0112] After the feature points are extracted, target tracking is performed:

[0113] For illegal targets whose vehicle license plates cannot be detected immediately, the Kalman filter method is used to predict the movement trend of the vehicle in the next frame and track the target. The tracking process is as follows: Figure 2 shown.

[0114] Kalman filter target position prediction can be divided into two steps: prediction update and measurement update. Prediction update receives the observation input at time t to update the covariance matrix of the state vector and its error; measurement update obtains the prediction of the system state at the next moment based on the state equation and estimates the covariance matrix of the state vector error. The formulas included in this process are:

[0115]

[0116]

[0117] Among them, x, z are state variables and measurement variables respectively, which are four-dimensional vectors composed of the target frame position x, y, target frame speed dx, dy, P is the error covariance matrix, A is the state matrix, B is the control matrix, H is the gain matrix, R represents measurement noise, Q represents observation noise, K k represents the Kalman gain coefficient.

[0118] S3: When the preset detection frame is reached, the predicted target frame of the illegal vehicle is matched with the feature points of the illegal vehicle, and the tracking status of the illegal vehicle is determined according to the matching result.

[0119] Among them, in step S3, matching the predicted target frame and the feature points of the illegal vehicle is performed, specifically including:

[0120] (1) In the current detection frame, the target detection algorithm is used to perform vehicle detection within the road range to obtain a number of vehicle detection frames corresponding to the current detection frame.

[0121] In view of the high computing power requirements of the target detection algorithm, feature point extraction and matching steps, frame-by-frame detection will affect the sampling frequency of the system. The present invention uses Kalman filtering to predict the target position in real time, sets detection frames at a certain interval, and detects and matches illegal vehicles to correct the accumulated error of Kalman filtering.

[0122] When the detection frame is reached, the target detection algorithm is used to detect vehicles within the road range to obtain several vehicle target frames.

[0123] (2) The vehicle target frame offset velocity vector obtained by the Kalman filter is applied to the vehicle target detection frame extracted from the previous detection frame and the illegal vehicle feature point to obtain the predicted target frame of the current detection frame and the predicted position of the illegal vehicle feature point.

[0124] like Figure 3 As shown in the figure, the tracking state judgment process is based on the above prediction and detection results. The nearest neighbor matching method is used to obtain the Harris feature point matching coefficient, and the target frame matching is completed according to the intersection and union ratio of the prediction frame and the detection frame. The state of the target vehicle is comprehensively judged. The states include normal tracking, partial occlusion, and target loss. The specific process is as follows: steps (3) to (9).

[0125] (3) Calculate the IOU of the predicted target frame and each of the vehicle detection frames. Determine whether the IOU is greater than an IOU threshold to obtain a first determination result.

[0126] (4) If the first judgment result is yes, a vehicle target detection frame corresponding to the illegal vehicle is obtained, and feature points of the illegal vehicle within the vehicle target detection frame are extracted.

[0127] (5) Performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients.

[0128] If there is a detection target frame that meets the requirements, the Plessey algorithm is used within its range to extract Harris corner points as feature points, and the nearest neighbor association matching is performed with the feature points in the prediction frame. The Euclidean distance between each point is calculated as the association cost matrix. Starting from the smallest element in the matrix, each point is matched one-to-one with its nearest neighbor. During the matching process, the association threshold is set as the maximum distance limit for adjacent point matching, and the number of successfully matched point pairs n is calculated. Let the number of feature points obtained in the previous frame be X, and the number of feature points detected in the current target frame be Y, and calculate the feature point matching coefficient Among them, the feature point matching coefficient is obtained as follows:

[0129] 1) Calculate the Euclidean distance between each of the illegal vehicle feature points in the vehicle target detection frame and the predicted target frame, and construct an associated cost matrix.

[0130] 2) Comparing the matrix elements in the association cost matrix with a preset association threshold.

[0131] 3) The two feature points corresponding to the matrix elements exceeding the preset association threshold are regarded as the nearest neighbor association matching point pair.

[0132] 4) Calculating the feature point matching coefficient according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle feature points in the previous detection frame, and the number of illegal vehicle feature points in the current detection frame.

[0133] (7) If the first judgment result is no, the current predicted target frame is expanded by a preset expansion coefficient, and the illegal vehicle feature points in the expanded prediction frame are extracted.

[0134] (8) Performing nearest neighbor association matching on the illegal vehicle feature points in the predicted target box before and after expansion to obtain the feature point matching coefficients.

[0135] If there is no target box that meets the IOU requirement, the target box predicted for the current frame is expanded by the coefficient k p The method is expanded to extract Harris corner points, match feature points using the nearest neighbor association matching method in the previous steps, and calculate feature point matching coefficients.

[0136] Different from the aforementioned nearest neighbor association matching method, the feature points matched here are the feature points of illegal vehicles in the predicted target box before and after expansion.

[0137] (9) When the feature point matching coefficient is greater than the coefficient threshold, the matching is successful; when the feature point matching coefficient is less than or equal to the coefficient threshold, the matching fails.

[0138] Determine whether the feature point matching coefficient is greater than the threshold thr match If it is greater than the threshold, it means that the feature point is matched successfully, otherwise the match fails.

[0139] S31: When the tracking state is normal target tracking, a clear vehicle license plate is obtained in the vehicle target detection frame of the current detection frame, and the illegal behavior image of the illegal vehicle and the vehicle license plate information are output; if a clear vehicle license plate of the illegal vehicle cannot be obtained, return to step S2 of "using Kalman filtering to track the illegal vehicle as a target".

[0140] When both the IOU match and the feature point match are greater than the threshold, it means that the illegal target state tracking is normal. Try to obtain the vehicle license plate information in the detection frame. If successful, take a photo as a clear license plate evidence photo and record the timestamp, then stop acquiring video frames, output the saved video and the recorded photos of the illegal behavior, the illegal behavior termination and the license plate cleaning, otherwise, use the detection frame target frame to correct the predicted position. If the detection frame leaves the illegal area for the first time, take a photo as the illegal behavior termination photo, enter step S2 to use Kalman filtering to track the illegal vehicle, and start the next tracking detection cycle.

[0141] S32: When the tracking state is that the target is partially blocked, the fitting position of the illegal vehicle in the current detection frame is determined according to the feature points of the illegal vehicle in the current detection frame and the previous detection frame, and the process returns to step S2 of "target tracking of the illegal vehicle using Kalman filtering".

[0142] When the IOU match is less than the threshold, but the feature point match extracted using the current frame prediction box position is greater than the threshold, it means that the vehicle is at the tracking prediction position, but due to partial occlusion caused by the preceding vehicle or environmental factors, the target detection algorithm cannot identify the tracked target. Figure 4 Vehicle shown obscured on the left by a truck.

[0143] In this case, since the interval between the two detection frames is short, the relative position change of the feature points caused by the change in viewing angle of the road vehicle is ignored, and the position of the feature points relative to the vehicle between the previous and next detection frames can be considered unchanged. Based on this, the position of the target vehicle in the current frame can be reversely deduced through the position of the feature points.

[0144] In the feature point matching pair (nearest neighbor associated matching point pair), calculate the relative position (x) between each feature point of the previous detection frame and the center point of the target frame (the center point of the vehicle target detection frame or the center point of the predicted target frame) i ,y i ), the position of each corresponding feature point in the current frame in the image coordinates (u i ,v i ), let the center point of the vehicle in the current frame be (U, V), and the equation group is obtained from the above geometric relationship:

[0145]

[0146] The system of equations has only two unknowns, which can be solved by the least square method to obtain the fitting position of the target vehicle in the current detection frame, and then return to step S2 to track the illegal vehicle using the Kalman filter to enter the next tracking detection cycle.

[0147] S33: When the tracking state is target tracking loss, the illegal vehicle is re-tracked in the next detection frame based on the position and size of the vehicle target detection frame in the last frame before the loss; when the re-tracking is successful, return to the step "using Kalman filtering to track the illegal vehicle" in step S2.

[0148] When the feature point matching is less than the threshold, it indicates that the system cannot reliably track the target, which may be caused by the target vehicle being severely blocked by other vehicles or the tracking module being abnormal. Regardless of the IOU matching result, the target state is judged as lost, and the loss duration is recorded to perform re-tracking operations.

[0149] For lost targets, the re-tracking module searches and matches them in the next detection frame. Taking into account the constraints of lanes on vehicle driving, in order to reduce the probability of mismatching, the potential appearance domain of the target is obtained according to the position and size of the target frame in the previous frame, and the search range is constrained.

[0150] like Figure 5 As shown, the specific steps for re-tracking illegal vehicles are as follows:

[0151] (1) Obtaining the position and size of the vehicle target detection frame of the last frame before the loss;

[0152] (2) Determine the moving direction of the illegal vehicle by using the velocity vector during target tracking using the Kalman filter.

[0153] (3) Determine the target potential appearance domain based on the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle.

[0154] Among them, the intersection of the reverse direction ray of the center speed of the detection frame and the rear frame line of the detection frame is calculated, and a perpendicular line to the lane line is drawn through the intersection. The lane where the vehicle is located is determined according to the coordinates of the frame lines on both sides, and the corresponding lane boundary line is extracted. When the detection frame spans two lanes, the outer boundary lines of the two lanes are selected. The area formed by the above boundary lines, the perpendicular lines of the lane lines and the edge of the field of view is the potential appearance domain of the target.

[0155] (4) Calculate the relative position of each illegal vehicle feature point in the vehicle target detection frame of the last frame before the loss and the center point of the vehicle target detection frame, and record it as the first relative position.

[0156] (5) Calculate the intersection-and-union ratio between each vehicle detection frame and the target potential appearance domain in the next detection frame, and record it as the re-tracking intersection-and-union ratio.

[0157] (6) When there is a vehicle target detection frame whose re-tracking intersection-and-union ratio is greater than the re-tracking intersection-and-union ratio threshold, the illegal vehicle feature point in the vehicle target detection frame during re-tracking is extracted, and the relative coordinates of the illegal vehicle feature point extracted during re-tracking and the center point of the vehicle target detection frame during re-tracking are calculated and recorded as the second relative position.

[0158] (7) Using a nearest neighbor association matching method, feature point matching is performed according to the first relative position and the second relative position to obtain a feature point matching coefficient during re-tracking.

[0159] (8) When the feature point matching coefficient during re-tracking is greater than the re-tracking coefficient threshold, the illegal vehicle is recaptured in the potential target appearance domain; the vehicle position is corrected using the vehicle target detection frame that currently captures the illegal vehicle, and a re-tracking mark is added to the re-captured illegal vehicle. The process returns to step S2 of "using Kalman filtering to track the illegal vehicle", starts the next tracking detection cycle, and verifies the re-tracking reliability using the vehicle tracking state detection module.

[0160] In this embodiment, when the license plate of the illegal target vehicle is detected, the target vehicle moves to the edge of the camera's field of view, or the target is lost for a period of time exceeding a threshold, the saved video and the photos of the illegal behavior, the end of the illegal behavior, and the clear license plate recorded in steps S2 and S3 are exported and classified according to different target tracking situations.

[0161] 1) For target vehicles that have not been lost and whose license plates have been detected, the tracking results have a high degree of credibility. The video clips and process photos from the start of detection to the detection of the license plate are retained as video evidence of the illegal behavior.

[0162] 2) For vehicles that have lost their targets during tracking, i.e., vehicles that have been marked for re-tracking in step S33, since the re-tracking module in the potential appearance area cannot rule out the possibility of misidentifying unrelated vehicles with extremely similar or identical appearances as target illegal vehicles, such videos and process photos are used as references for illegal behavior and handed over to the traffic management department for manual review to determine whether they can be used as evidence of vehicle violation.

[0163] 3) For vehicles that have never been able to obtain a vehicle license plate, they will be submitted to the traffic management department for manual review and observation through cameras in other locations to see whether there is image or video evidence of vehicle violations.

[0164] Compared with the prior art of handling violations based on photos, the present invention first detects the illegal vehicle, then tracks the illegal vehicle through Kalman filtering and detection frame matching until the vehicle license plate is detected, thereby generating video evidence of the illegal behavior, so that the effective field of view of the illegal monitoring camera is increased to the vehicle detection point. In addition, in order to deal with the situation where the target is continuously occluded, Harris corner points are extracted as vehicle feature points, and for partially occluded targets, the corresponding feature point positions are matched in the current frame, and the vehicle position is fitted using the least squares method for subsequent tracking, so that the tracking system remains robust under the influence of occlusion. In response to the situation where the tracked target is lost, a potential occurrence domain is designed, in which the lost target is searched and re-tracked, which increases the tracking ability of the system, and the generated video judgments are classified according to the tracking process, which provides convenience for traffic management departments to handle violations.

[0165] Embodiment 2

[0166] This embodiment provides a traffic violation detection system based on visual target tracking, the system comprising:

[0167] The area determination and target detection module is used to determine the illegal detection area, capture the illegal vehicle in the illegal detection area based on the target detection algorithm, obtain the clear license plate of the illegal vehicle, and output the illegal behavior image and license plate information of the illegal vehicle.

[0168] The feature extraction and target tracking module is used to extract Harris corner points as feature points of the illegal vehicle based on the captured image of the illegal vehicle when a clear license plate of the illegal vehicle cannot be obtained, and to track the illegal vehicle using Kalman filtering.

[0169] The matching and tracking state judgment module is used to match the predicted target frame and the feature points of the illegal vehicle when reaching a preset detection frame, and judge the tracking state of the illegal vehicle based on the matching result.

[0170] The vehicle license plate acquisition and target tracking module is used to acquire the clear vehicle license plate in the vehicle target detection frame of the current detection frame when the tracking state is normal target tracking, and output the illegal behavior image of the illegal vehicle and the vehicle license plate information; if the clear vehicle license plate of the illegal vehicle cannot be obtained, the "using Kalman filtering to track the illegal vehicle" in the matching and tracking state judgment module is executed.

[0171] The position fitting and target tracking module is used to determine the fitting position of the illegal vehicle in the current detection frame based on the feature points of the illegal vehicle in the current detection frame and the previous detection frame when the tracking state is that the target is partially blocked, and execute the "target tracking of the illegal vehicle using Kalman filtering" in the matching and tracking state judgment module.

[0172] The re-tracking module is used to re-track the illegal vehicle in the next detection frame when the tracking state is target tracking loss, based on the position and size of the vehicle target detection frame in the last frame before the loss; when the re-tracking is successful, the "using Kalman filtering to track the illegal vehicle" in the matching and tracking state judgment module is executed.

[0173] Embodiment 3

[0174] This embodiment provides an electronic device, including a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the traffic violation detection method based on visual target tracking of embodiment 1.

[0175] Optionally, the above-mentioned electronic device may be a server.

[0176] In addition, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the traffic violation detection method based on visual target tracking of embodiment 1.

[0177] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0181] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0182] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A traffic violation detection method based on visual target tracking, characterized in that: The method comprises: Determine a violation detection area, capture an illegal vehicle in the violation detection area based on a target detection algorithm, obtain a clear license plate of the illegal vehicle, and output an illegal behavior image and license plate information of the illegal vehicle; When a clear license plate of the illegal vehicle cannot be obtained, Harris corner points are extracted as feature points of the illegal vehicle based on the captured image of the illegal vehicle, and Kalman filtering is used to track the illegal vehicle; When the preset detection frame is reached, the predicted target frame of the illegal vehicle is matched with the feature points of the illegal vehicle, and the tracking state of the illegal vehicle is determined according to the matching result; When the tracking state is that the target tracking is normal, a clear license plate of the vehicle is obtained in the vehicle target detection frame of the current detection frame, and the illegal behavior image of the illegal vehicle and the license plate information are output; if the clear license plate of the illegal vehicle cannot be obtained, return to the step of "using Kalman filtering to track the illegal vehicle". When the tracking state is that the target is partially blocked, the fitting position of the illegal vehicle in the current detection frame is determined according to the feature points of the illegal vehicle in the current detection frame and the previous detection frame, and the process returns to the step of "target tracking of the illegal vehicle using Kalman filtering"; When the tracking state is target tracking lost, the illegal vehicle is re-tracked in the next detection frame based on the position and size of the vehicle target detection frame in the last frame before the loss; when the re-tracking is successful, the process returns to step "target tracking of the illegal vehicle using Kalman filtering"; The step of matching the predicted target frame and the feature points of the illegal vehicle specifically includes: In the current detection frame, the target detection algorithm is used to perform vehicle detection within the road range to obtain a number of vehicle detection frames corresponding to the current detection frame; Applying the vehicle target frame offset velocity vector obtained by the Kalman filter to the vehicle target detection frame extracted from the previous detection frame and the illegal vehicle feature point to obtain the predicted target frame of the current detection frame and the illegal vehicle feature point in the predicted target frame; Calculating the intersection-over-union ratio of the predicted target frame and each of the vehicle detection frames; Determine whether the intersection-over-union ratio is greater than an intersection-over-union ratio threshold, and obtain a first determination result; If the first judgment result is yes, a vehicle target detection frame corresponding to the illegal vehicle is obtained, and feature points of the illegal vehicle in the vehicle target detection frame are extracted; Performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients; If the first judgment result is no, the current predicted target frame is expanded by a preset expansion coefficient, and the illegal vehicle feature points in the expanded predicted target frame are extracted; Performing nearest neighbor association matching on the illegal vehicle feature points in the predicted target frame before and after expansion to obtain the feature point matching coefficients; When the matching coefficient of the feature point is greater than the coefficient threshold, the matching is successful; When the feature point matching coefficient is less than or equal to the coefficient threshold, the matching fails; Among them, performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients specifically includes: Calculating the Euclidean distance between each of the illegal vehicle feature points in the vehicle target detection frame and the predicted target frame to construct an association cost matrix; Comparing the matrix elements in the association cost matrix with a preset association threshold; The two feature points corresponding to the matrix elements exceeding the preset association threshold are regarded as the nearest neighbor association matching point pair; Calculating the feature point matching coefficient according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle feature points in the previous detection frame, and the number of illegal vehicle feature points in the current detection frame; Wherein, determining the fitting position of the illegal vehicle in the current detection frame according to the illegal vehicle feature points in the current detection frame and the previous detection frame specifically includes: Calculate the relative position between each matching feature point of the previous detection frame and the center point of the target frame, the position of each matching feature point of the current detection frame in the illegal vehicle image coordinates, and construct an equation group in combination with the coordinates of the center point of the illegal vehicle in the current detection frame; the equation group includes n equations; n is the number of matching feature points; the matching feature points are the nearest neighbor associated matching point pairs obtained after nearest neighbor associated matching; Solving the equation group using the least square method to obtain the fitting position of the illegal vehicle in the current detection frame; The tracking status of the illegal vehicle is determined based on the matching result, specifically including: When the first judgment result is yes and the second judgment result is yes, the tracking state is that the target tracking is normal; the second judgment result is a result obtained by judging whether the feature point matching coefficient is greater than a coefficient threshold; When the first judgment result is no and the second judgment result is yes, when the tracking state is that the target is partially blocked; When the first judgment result is no and the second judgment result is no, the tracking state is that the target tracking is lost.

2. The method according to claim 1, characterized in that When detecting illegal behaviors such as illegal road occupation, determine the illegal detection area, including: The road area image is obtained by cropping the image taken by the violation detection system; Performing color space conversion on the road area image, and performing edge detection on the converted image using a Canny operator; According to the edge detection results, Hough transform is used to extract straight line segments, and lane dividing lines are determined based on the extracted straight line segments, and several lane areas are obtained; Any one of the lane areas is selected as the violation detection area.

3. The method according to claim 1, characterized in that Harris corner points are extracted based on the captured illegal vehicle image as the illegal vehicle feature points, including: Grayscale the illegal vehicle image, and calculate the gradient in the x direction and the gradient in the y direction within the vehicle target detection frame by using a differential operator; Constructing a first matrix based on the gradient in the x direction and the gradient in the y direction; Performing Gaussian smoothing filtering on the elements in the first matrix; Selecting a window size and a corresponding weight, and calculating a second matrix according to the window size, the weight and the first matrix; Calculate the Harris response value corresponding to each pixel point in the illegal vehicle image according to the second matrix; Pixel points whose Harris response values ​​are greater than a threshold are selected as the Harris corner points to obtain the illegal vehicle feature points.

4. The method according to claim 1, characterized in that: Re-tracking the illegal vehicle based on the position and size of the vehicle target detection frame of the last frame before the loss, specifically including: Obtain the position and size of the vehicle target detection frame of the last frame before the loss; Determine the moving direction of the illegal vehicle by using the velocity vector when the Kalman filter is used for target tracking; Determine the target potential appearance domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle; Calculate the relative position of each illegal vehicle feature point in the vehicle target detection frame of the last frame before the loss and the center point of the vehicle target detection frame, and record it as the first relative position; Calculate the intersection-and-union ratio of each vehicle detection frame and the target potential appearance domain in the next detection frame, and record it as the re-tracking intersection-and-union ratio; When there is a vehicle target detection frame whose re-tracking intersection-and-union ratio is greater than the re-tracking intersection-and-union ratio threshold, extract the illegal vehicle feature point in the vehicle target detection frame during re-tracking, and calculate the relative coordinates of the illegal vehicle feature point extracted during re-tracking and the center point of the vehicle target detection frame during re-tracking, which are recorded as the second relative position; Using a nearest neighbor association matching method, feature point matching is performed according to the first relative position and the second relative position to obtain a feature point matching coefficient during re-tracking; When the feature point matching coefficient during re-tracking is greater than the coefficient threshold during re-tracking, the illegal vehicle is recaptured in the target potential appearance domain; The vehicle position is corrected using the vehicle target detection frame that currently captures the illegal vehicle, and a re-tracking mark is added to the re-captured illegal vehicle.

5. The method according to claim 4, characterized in that Determining the target potential appearance domain according to the vehicle target detection frame of the last frame before the loss and the movement direction of the illegal vehicle specifically includes: Calculate the intersection of the reverse ray of the center speed of the vehicle target detection frame of the last frame before the loss and the rear frame line of the detection frame, and draw a perpendicular line to the lane line through the intersection; Determine the lane where the illegal vehicle is located according to the coordinates of the detection frame lines on both sides, and extract the lane boundary line; The area formed by the lane boundary line, the vertical line of the lane line and the edge of the field of view is regarded as the potential appearance domain of the target.

6. A traffic violation detection system based on visual target tracking, characterized in that: The system comprises: The area determination and target detection module is used to determine the illegal detection area, capture the illegal vehicle in the illegal detection area based on the target detection algorithm, obtain the clear license plate of the illegal vehicle, and output the illegal behavior image and license plate information of the illegal vehicle; A feature extraction and target tracking module, which is used to extract Harris corner points as feature points of the illegal vehicle based on the captured image of the illegal vehicle when a clear license plate of the illegal vehicle cannot be obtained, and to track the illegal vehicle using Kalman filtering; A matching and tracking state judgment module is used to match the predicted target frame and the feature points of the illegal vehicle when a preset detection frame is reached, and to judge the tracking state of the illegal vehicle according to the matching result; The vehicle license plate acquisition and target tracking module is used to acquire a clear vehicle license plate in the vehicle target detection frame of the current detection frame when the tracking state is normal target tracking, and output the illegal behavior image of the illegal vehicle and the vehicle license plate information; if the clear vehicle license plate of the illegal vehicle cannot be acquired, the "target tracking of the illegal vehicle by using Kalman filtering" in the matching and tracking state judgment module is executed; A position fitting and target tracking module, which is used to determine the fitting position of the illegal vehicle in the current detection frame according to the feature points of the illegal vehicle in the current detection frame and the previous detection frame when the tracking state is that the target is partially blocked, and execute "target tracking of the illegal vehicle using Kalman filtering" in the matching and tracking state judgment module; A re-tracking module, for re-tracking the illegal vehicle in the next detection frame based on the position and size of the vehicle target detection frame in the last frame before the loss when the tracking state is target tracking loss; when the re-tracking is successful, executing "target tracking of the illegal vehicle using Kalman filtering" in the matching and tracking state judgment module; The step of matching the predicted target frame and the feature points of the illegal vehicle specifically includes: In the current detection frame, the target detection algorithm is used to perform vehicle detection within the road range to obtain a number of vehicle detection frames corresponding to the current detection frame; Applying the vehicle target frame offset velocity vector obtained by the Kalman filter to the vehicle target detection frame extracted from the previous detection frame and the illegal vehicle feature point to obtain the predicted target frame of the current detection frame and the illegal vehicle feature point in the predicted target frame; Calculating the intersection-over-union ratio of the predicted target frame and each of the vehicle detection frames; Determine whether the intersection-over-union ratio is greater than an intersection-over-union ratio threshold, and obtain a first determination result; If the first judgment result is yes, a vehicle target detection frame corresponding to the illegal vehicle is obtained, and feature points of the illegal vehicle in the vehicle target detection frame are extracted; Performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients; If the first judgment result is no, the current predicted target frame is expanded by a preset expansion coefficient, and the illegal vehicle feature points in the expanded predicted target frame are extracted; Performing nearest neighbor association matching on the illegal vehicle feature points in the predicted target frame before and after expansion to obtain the feature point matching coefficients; When the matching coefficient of the feature point is greater than the coefficient threshold, the matching is successful; When the feature point matching coefficient is less than or equal to the coefficient threshold, the matching fails; Among them, performing nearest neighbor association matching on the illegal vehicle feature points in the vehicle target detection frame and the illegal vehicle feature points in the predicted target frame to obtain feature point matching coefficients specifically includes: Calculating the Euclidean distance between each of the illegal vehicle feature points in the vehicle target detection frame and the predicted target frame to construct an association cost matrix; Comparing the matrix elements in the association cost matrix with a preset association threshold; The two feature points corresponding to the matrix elements exceeding the preset association threshold are regarded as the nearest neighbor association matching point pair; Calculating the feature point matching coefficient according to the number of nearest neighbor associated matching point pairs, the number of illegal vehicle feature points in the previous detection frame, and the number of illegal vehicle feature points in the current detection frame; Wherein, determining the fitting position of the illegal vehicle in the current detection frame according to the illegal vehicle feature points in the current detection frame and the previous detection frame specifically includes: Calculate the relative position between each matching feature point of the previous detection frame and the center point of the target frame, the position of each matching feature point of the current detection frame in the illegal vehicle image coordinates, and construct an equation group in combination with the coordinates of the center point of the illegal vehicle in the current detection frame; the equation group includes n equations; n is the number of matching feature points; the matching feature points are the nearest neighbor associated matching point pairs obtained after nearest neighbor associated matching; Solving the equation group using the least square method to obtain the fitting position of the illegal vehicle in the current detection frame; The tracking status of the illegal vehicle is determined based on the matching result, specifically including: When the first judgment result is yes and the second judgment result is yes, the tracking state is that the target tracking is normal; the second judgment result is a result obtained by judging whether the feature point matching coefficient is greater than a coefficient threshold; When the first judgment result is no and the second judgment result is yes, when the tracking state is that the target is partially blocked; When the first judgment result is no and the second judgment result is no, the tracking state is that the target tracking is lost.

Images