A method for detecting illegally parked vehicles based on motion vector differences
Through the vehicle illegal parking detection method based on motion vector differences, the yolov5 model and three-step search algorithm are used to solve the problems of low vehicle illegal parking detection efficiency and high false alarm rate in the existing technology, and efficient and accurate judgment of illegal parking is achieved. , suitable for a variety of traffic monitoring scenarios.
Patent Information
- Application Number
- CN202410586944.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-13
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-05-13
AI Technical Summary
The existing computer vision-based vehicle illegal parking detection methods are inefficient and have high false alarm rates, making it difficult to accurately determine whether a vehicle is illegally parked, especially in traffic management.
The vehicle illegal parking detection method based on motion vector differences is adopted, and the vehicle position is detected using the yolov5 model, and the motion vector difference between the vehicle area and the surrounding area is calculated in blocks. The matching of the front and rear frames and the motion vector variance are judged by combining the three-step search algorithm. , determine whether the vehicle is in an illegal parking state.
It realizes fast and accurate vehicle violation detection, improves detection efficiency and accuracy, enhances robustness and real-timeness, and is suitable for traffic surveillance image data collected by drones and fixed cameras.
Smart Images

Figure CN118334603B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision and traffic violation detection, and more specifically, relates to a vehicle parking violation detection method based on motion vector difference. Background Art
[0002] With the acceleration of urbanization, the number of vehicles has increased dramatically, and traffic management faces huge challenges. Among them, the problem of illegal parking of vehicles is particularly prominent, which not only affects traffic order, but may also cause road congestion and safety hazards. Traditional vehicle illegal parking detection mainly relies on manual inspections. However, manual inspections are inefficient and easily affected by human factors. With the development of edge computing and computer vision, many vehicle illegal parking detection methods based on target tracking and target detection have emerged. The vehicle illegal parking method based on target tracking can well distinguish whether the vehicle is illegally parked, but it often requires large computing resources. The vehicle illegal parking detection method based on target detection can efficiently detect the vehicle location information, but there is a lot of room for improvement in how to accurately judge whether the vehicle is illegally parked.
[0003] The present application is different from the prior art as follows:
[0004] Technical comparison with "Community illegal parking warning method based on license plate recognition and image variance algorithm":
[0005] The "community illegal parking alarm method based on license plate recognition and image variance algorithm" collects static scene images without illegally parked vehicles as a comparison background map. After collecting a new current image, the variance of the grayscale difference between the current image and the comparison background map is calculated. If the variance is greater than a pre-set threshold, it is considered that there is illegal parking of the vehicle.
[0006] The present invention firstly detects the vehicle using the yolov5 model, then delineates the judgment area with the vehicle as the center and divides it into blocks, calculates the variance of the motion vector of the sub-block in the judgment area between two consecutive frames, and if the variance is less than the set threshold, it is considered that the vehicle is illegally parked.
[0007] In terms of vehicle illegal parking recognition, the former is based on the variance of grayscale difference, and the latter is based on the variance of motion vector.
[0008] Technical comparison with "A vehicle illegal parking recognition method based on yolov5"
[0009] "A vehicle illegal parking identification method based on yolov5" uses the yolov5 model to detect vehicles and calculate the position difference of the vehicle between the previous and next frames. If the position difference is less than the overlap threshold and it is the same vehicle, it is considered that the vehicle has been illegally parked.
[0010] The present invention also uses the yolov5 model for vehicle detection. Based on the detected vehicle position, the motion vectors of the vehicle area and the area around the vehicle are calculated simultaneously. The difference in motion vectors between the two is used to determine whether the vehicle is stationary. If the vehicle is stationary, it is determined that there is illegal parking.
[0011] In terms of vehicle illegal parking recognition, "A vehicle illegal parking recognition method based on yolov5" uses the overall displacement of the vehicle between the previous and next frames, and the present invention uses the local displacement of the vehicle area sub-block and the vehicle surrounding area sub-block. There are significant differences in the basic principles of the two.
[0012] Therefore, how to achieve efficient and accurate vehicle illegal parking detection has become an urgent problem to be solved in the field of traffic management. Summary of the invention
[0013] In order to solve the problems of low efficiency and false alarms in the existing vehicle illegal parking behavior detection based on computer vision, the present invention proposes a vehicle illegal parking detection method based on motion vector difference, which can quickly and accurately detect vehicle illegal parking behavior and record vehicle illegal parking information to facilitate traffic management.
[0014] To achieve the above object, the technical solution adopted by the present invention is:
[0015] A method for detecting illegal parking of vehicles based on motion vector difference comprises the following steps:
[0016] (1) Collect UAV images of traffic scenes containing vehicles to build a training dataset, and use this dataset to train the yolov5 vehicle detection model;
[0017] (2) For each frame of image captured by the drone, use the trained yolov5 vehicle detection model to detect it and obtain the location information of each vehicle in the image;
[0018] (3) According to the detection results of the yolov5 vehicle detection model, determine the discrimination area for each vehicle for calculating the motion vector, and divide the discrimination area into blocks;
[0019] (4) For each vehicle, a three-step search algorithm is used to match the position of each discriminant region sub-block in the previous frame in the next frame;
[0020] (5) Calculate the motion vector of each sub-block in the judgment area between the previous and next two frames and the variance of the motion vectors of all sub-blocks in the judgment area, and make a judgment on the illegal parking of the vehicle in combination with the vehicle illegal parking judgment rules.
[0021] Further, in step (1), it includes: collecting traffic scene drone images containing vehicles, establishing a training data set, using the labelimg tool to annotate the positions of each vehicle in the images, generating the annotation files required for the yolov5 vehicle detection model, using the training data set to train the yolov5 vehicle detection model, and determining the model parameters.
[0022] Further, in step (2), it includes: calling the yolov5 vehicle detection model obtained in step (1) to detect the traffic scene images collected by the drone, and returning the position information of each vehicle, (id, (x, y), w, h), where id represents the vehicle label, (x, y) represents the coordinates of the vehicle center point, and w and h represent the width and length of the vehicle respectively.
[0023] Further, in step (3), it includes: based on the vehicle position information (id, (x, y), w, h) obtained in step (2), taking the center point (x, y) of each vehicle as the center, and expanding the width (w) and length (h) by c times proportionally. The expanded width and length are denoted as w' and h' respectively, where 1.2 < c < 1.5, so as to determine the discrimination region for calculating the motion vector of the id-th vehicle. Subsequently, the discrimination region is divided into n * n sub-blocks, called "discrimination region sub-blocks", and n * n = N. The size of each discrimination region sub-block is
[0024] Further, in step (4), it includes: based on the discrimination region sub-blocks divided in step (3), denoting the position data of each discrimination region sub-block of the id-th vehicle in the previous frame as P K1 ={(x 11 , y 11 ), (x 12 , y 12 ), …, (x 1i , y 1i ), …, (x 1N , y 1N )}, where (x 1i , y 1i ) represents the center position coordinates of the i-th discrimination region sub-block of the id-th vehicle in the previous frame. Using the three-step search algorithm to match each discrimination region sub-block of the id-th vehicle in the previous frame in the next frame, and obtaining the position data P K2 ={(x 21 , y 21 ), (x 22 , y 22 ), …, (x 2i , y 2i ), …, (x 2N , y 2N )}, where (x2i ,y 2i ) represents the center position coordinate of the ith discrimination region sub-block of the idth vehicle in the next frame, 1 <i<N。
[0025] Furthermore, the step (5) of calculating the motion vector of each sub-block in the determination area between the previous and next frames and the motion vector variance of all sub-blocks in the determination area includes: based on the position data of each sub-block in the determination area of the vehicle in the previous and next frames obtained in step (4), calculating the motion vector of each sub-block in the determination area of the id-th vehicle in the previous and next frames, and the motion vector calculation rule is:
[0026] For the discriminative region sub-block i, the motion change in the x component is Δx i =x 2i -x 1i , the change in motion on the y component is Δy i =y 2i -y 1i , the motion vector of the i-th discriminant area block of the id-th vehicle in the previous and next two frames is expressed as (Δx i ,Δy i ), perform this calculation on each discrimination region sub-block of the id-th vehicle, and obtain the motion vector set V of the discrimination region of the two frames before and after the id-th vehicle id ={(Δx1,Δy1),(Δx2,Δy2),…,(Δx N ,Δy N )}, and then calculate the motion vector variance of all sub-blocks in the judgment area in the x component and the y component. The motion vector variance calculation rule is as follows:
[0027] (a) Calculate the motion vector set V id The average of the x and y components
[0028]
[0029]
[0030] (b) Calculate the motion vector set V id The variance of the motion vector Var of the x and y components x 、Var y :
[0031]
[0032]
[0033] Furthermore, the vehicle illegal parking judgment rule described in step (5) is: take Var x 、Va xyTake the maximum value in it as the variance of the motion vector Var of the vehicle with ID between two consecutive frames. Set the threshold of the variance of the motion vector as threshold. Compare the variance of the motion vector Var of the vehicle with ID between two consecutive frames. When Var < threshold, it is determined that the vehicle is in a stationary state, that is, it is marked as an illegally parked vehicle; otherwise, the vehicle is marked as a moving vehicle.
[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0035] The method for detecting illegally parked vehicles based on the difference in motion vectors of the present application has the following remarkable advantages:
[0036] High efficiency: By using the trained YOLOv5 vehicle detection model, this method can quickly detect vehicles in the image and accurately obtain the position information of each vehicle. In addition, through the three-step search algorithm to quickly match the discriminant region sub-blocks between two consecutive frames, the efficiency of illegally parked vehicle detection is further improved.
[0037] Accuracy: By calculating the motion vectors of each sub-block in the discriminant region between two consecutive frames and the variance of the motion vectors of all sub-blocks in the discriminant region, and combining the discriminant rules for illegally parked vehicles, this method can accurately determine whether the vehicle is in an illegally parked state. In particular, using the variance of the motion vector as an index can more accurately reflect the motion state of the vehicle.
[0038] Robustness: This method not only considers the overall motion of the vehicle, but also improves the robustness of the detection of the vehicle motion state by processing the discriminant region in blocks and calculating the motion vectors of each sub-block. Even if the camera shakes or part of the vehicle is blocked, it will not have too much impact on the overall judgment of illegal parking.
[0039] Easy to implement: This method is mainly based on computer vision and image processing technologies, without the need for complex hardware devices or external sensors, so it is easy to implement and promote in the existing traffic management system.
[0040] Real-time performance: Since this method uses efficient vehicle detection and motion vector calculation technologies, it can realize real-time monitoring of traffic scenes and illegally parked vehicle detection, which helps to discover and handle illegal parking behaviors in a timely manner.
[0041] Scalability: This method is not only applicable to the image data collected by drones, but can also be extended to other types of traffic monitoring image data, such as images taken by fixed high-position cameras. In addition, this method can further combine other traffic management information, such as license plate recognition, traffic flow statistics, etc., to achieve more comprehensive traffic management functions. Description of the Drawings
[0042] Figure 1It is the flowchart of the overall method in the embodiment of the present invention;
[0043] Figure 2 It is the detection graph of the yolov5 vehicle detection model and the motion vector difference discrimination region graph in the embodiment of the present invention;
[0044] Figure 3 It is the schematic diagram of the sub-block division of the motion vector difference discrimination region in the embodiment of the present invention;
[0045] Figure 4 It is the schematic diagram of the block matching in the discrimination region in the embodiment of the present invention;
[0046] Figure 5 It is the motion vector distribution graph of the sub-blocks in the discrimination region in the embodiment of the present invention;
[0047] Figure 6 It is the determination result graph in the embodiment of the present invention. Detailed implementation manners
[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners:
[0049] The present invention provides a vehicle illegal parking detection method based on motion vector difference, and its overall process is as Figure 1 shown.
[0050] (1) Configure the Pytorch deep learning environment, obtain the yolov5 model structure configuration file, construct a training data set of UAV images of traffic scenes containing vehicles, where the images are selected from the KITTI data set jointly released by KIT and TTI-C, with a total of 7480 images. Use the labelimg tool to annotate the position of each vehicle in the images to generate the annotation files required for the yolov5 vehicle detection model, and use this training data set to train the yolov5 vehicle detection model to determine the model parameters.
[0051] (2) Use the trained yolov5 vehicle detection model to detect the image data collected by the UAV, and return the position information of each vehicle, (id, (x, y), w, h), where id represents the vehicle label, (x, y) represents the coordinates of the vehicle center point, and w and h represent the width and length of the vehicle respectively.
[0052] (3) With the center point (x, y) of each vehicle as the center, the width (w) and length (h) are enlarged by c times in proportion. The enlarged width and length are respectively denoted as w' and h', where 1.2 < c < 1.5, so as to obtain the discrimination region for calculating the motion vector of the vehicle with id. The motion vector difference discrimination region is as Figure 2 shown. Subsequently, the discrimination region is divided into n * n sub-blocks, called "discrimination region sub-blocks", and n * n = N is recorded. The size of each discrimination region block is The determination area is divided into Figure 3 shown.
[0053] (4) Based on the discrimination area sub-blocks divided in step (3), the position data of each discrimination area sub-block of the id-th vehicle in the previous frame is recorded as P K1 ={(x 11 ,y 11 ),(x 12 ,y 12 ),…,(x 1i ,y1i),…,(x 1N ,y 1N )}, where (x 1i ,y 1i ) represents the center coordinates of the ith discrimination region sub-block of the id-th vehicle in the previous frame. A three-step search algorithm is used to match each discrimination region sub-block of the id-th vehicle in the previous frame in the next frame. The matching process is as follows: Figure 4 As shown, for the i-th discriminant area sub-block, its center position coordinate (x 1i ,y 1i ) as the starting point, set a search window of size W*H in the next frame, and the search step length is s pixels. The three-step search algorithm steps are as follows:
[0054] (a) First step search
[0055] With (x 1i ,y 1i ) is taken as the center point, and a preliminary search is performed at 8 positions around it in the search window with a step size s. For each search position, the discrimination region sub-block centered at the position is extracted, and the absolute difference (SAD) of the discrimination region sub-block at the current search position and the discrimination region sub-block in the previous frame is calculated. The position with the smallest SAD value is taken as the best matching candidate point M in the first step of the search; the calculation rule of the absolute difference (SAD) is as follows:
[0056]
[0057] Where A represents the ith discrimination region subblock of the id-th vehicle in the previous frame, B represents the discrimination region subblock of the current search position in the next frame, A(p,q) and B(p,q) represent the pixel values of A and B at position (p,q), and the pixel size of each discrimination region subblock is make This formula indicates that the pixel values of all corresponding positions between the discrimination region sub-block A and the discrimination region sub-block B of the current search position are subtracted point by point, and the absolute value is taken, and then all the absolute values are added to obtain the final SAD value. The smaller the SAD value, the more similar the two image blocks are;
[0058] (b) Second step search
[0059] Taking the best matching candidate point M obtained in the first step as the center, the search step length is s / 2, and the SAD value of each search position is calculated. The position with the smallest SAD value is used as the best matching candidate point M' in the second step;
[0060] (c) Step 3 Search
[0061] Taking the best matching candidate point M' obtained in the second step as the center, the search step length is s / 4, and the SAD value of each search position is calculated. The position with the smallest SAD value is used as the best matching candidate point M" in the third step;
[0062] Repeat the above steps (a), (b), and (c) for each discrimination region sub-block of the id-th vehicle, and obtain the position data of each discrimination region block of the id-th vehicle in the next frame as P K2 ={(x 21 ,y 21 ),(x 22 ,y 22 ),…,(x 2i ,y 2i ),…,(x 2N ,y 2N )},(x 2u ,y 2i ) represents the center coordinates of the ith discrimination region block of the idth vehicle in the next frame, 1 <i<N。
[0063] (5) Based on the position data of each discrimination region sub-block of the vehicle in the previous and next frames obtained in step (4), the motion vector of each discrimination region sub-block of the id-th vehicle in the previous and next frames is calculated. The motion vector calculation rule is:
[0064] For the discriminative region sub-block i, the motion change in the x component is Δx i =x 2i -x 1i , the change in motion on the y component is Δy i =y 2i -y 1i , the motion vector of the i-th discriminant area block of the id-th vehicle in the previous and next two frames can be expressed as (Δx i ,Δy i ), perform this calculation on each discrimination region sub-block of the id-th vehicle, and obtain the motion vector set V of the discrimination region of the two frames before and after the id-th vehicle id ={(Δx1,Δy1),(Δx2,Δy2),…,(Δx N ,Δy N )}, where the previous frame is Figure 5 (I), the next frame is as follows Figure 5 (II), the motion vector distribution is as Figure 5 (III). Subsequently, calculate the variances of the motion vectors of all sub - blocks in the discrimination region in the x - component and y - component. The calculation rules of the motion vector variance are as follows:
[0065] (a) Calculate the average values of the x - component and y - component in the motion vector set V id in the x - component and y - component
[0066]
[0067]
[0068] (b) Calculate the variances of the motion vectors of the x - component and y - component, Var id in the motion vector set V x 、Var y :
[0069]
[0070]
[0071] (6) Take the maximum value of Var x 、Var y as the motion vector variance Var of the id - th vehicle between two consecutive frames. Set the threshold of the motion vector variance as threshold. Compare the motion vector variance Var of the id - th vehicle between two consecutive frames. When Var < threshold, it is determined that the vehicle is in a stationary state, that is, marked as an illegally parked vehicle; otherwise, the vehicle is marked as a moving vehicle. Among them, the previous frame is as Figure 6 (I), the next frame is as Figure 6 (II), and the marking result is as Figure 6 (III).
[0072] The above - mentioned are only the preferred embodiments of the present invention, and do not impose any other form of limitation on the present invention. Any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope protected by the present invention.
Claims
1. A vehicle parking violation detection method based on motion vector difference, characterized in that: The steps include: (1) Collect UAV images of traffic scenes containing vehicles to build a training dataset, and use this dataset to train the yolov5 vehicle detection model; Step (1) includes: collecting drone images of traffic scenes containing vehicles, establishing a training data set, using the labelimg tool to mark the position of each vehicle in the image, generating the labeling file required by the yolov5 vehicle detection model, using the training data set to train the yolov5 vehicle detection model, and determining the model parameters; (2) For each frame of image captured by the drone, use the trained yolov5 vehicle detection model to detect it and obtain the location information of each vehicle in the image; Step (2) includes: calling the yolov5 vehicle detection model obtained in step (1) to detect the traffic scene image collected by the drone, and returning the location information of each vehicle, (id, (x, y), w, h), where id represents the vehicle number, (x, y) represents the coordinates of the center point of the vehicle, and w and h represent the width and length of the vehicle respectively; (3) According to the detection results of the yolov5 vehicle detection model, determine the discrimination area for each vehicle for calculating the motion vector, and divide the discrimination area into blocks; In step (3), it includes: based on the vehicle position information (id, (x, y), w, h) obtained in step (2), with the center point (x, y) of each vehicle as the center, the width (w) and length (h) are enlarged by c times proportionally, and the enlarged width and length are denoted as w' and h' respectively, where 1.2 < c < 1.5, so as to determine the discrimination region for calculating the motion vector of the id-th vehicle. Subsequently, the discrimination region is divided into n * n sub-blocks, called "discrimination region sub-blocks", and n * n = N is denoted. The size of each discrimination region sub-block is (4) For each vehicle, a three-step search algorithm is used to match the position of each discriminant region sub-block in the previous frame in the next frame; Step (4) includes: based on the discrimination area sub-blocks divided in step (3), the position data of each discrimination area sub-block of the id-th vehicle in the previous frame is recorded as P K1 ={(x 11 ,y 11 ), (x 12 ,y 12 ),...,(x 1i ,y 1i ),...,(x 1N ,y 1N )}, where (x 1i ,y 1i ) represents the center position coordinates of the ith discrimination region sub-block of the idth vehicle in the previous frame. The three-step search algorithm is used to match each discrimination region sub-block of the idth vehicle in the previous frame in the next frame, and the position data P of each discrimination region sub-block in the next frame is obtained. K2 ={(x 21 ,y 21 ), (x 22 ,y 22 ),...,(x 2i ,y 2i ),...,(x 2N ,y 2N )}, where (x 2i ,y 2i ) represents the center position coordinate of the ith discrimination region sub-block of the idth vehicle in the next frame, 1 <i<N; (5) Calculate the motion vector of each sub-block in the determination area between the previous and next two frames and the variance of the motion vectors of all sub-blocks in the determination area, and make a determination on the illegal parking of the vehicle in combination with the vehicle illegal parking determination rules; The step (5) of calculating the motion vector of each sub-block in the determination area between the previous and next frames and the motion vector variance of all sub-blocks in the determination area includes: based on the position data of each sub-block in the determination area of the vehicle in the previous and next frames obtained in step (4), calculating the motion vector of each sub-block in the determination area of the id-th vehicle in the previous and next frames, and the motion vector calculation rule is: For the discriminative region sub-block i, the motion change in the x component is Δx i =x 2i -x 1i , the change in motion on the y component is Δy i =y 2i -y 1i , the motion vector of the i-th discriminant area block of the id-th vehicle in the previous and next two frames is expressed as (Δx i , Δy i ), perform this calculation on each discrimination region sub-block of the id-th vehicle, and obtain the motion vector set V of the discrimination region of the two frames before and after the id-th vehicle id ={(Δx1, Δy1), (Δx2, Δy2),..., (Δx N , Δy N )}, and then calculate the motion vector variance of all sub-blocks in the judgment area in the x component and the y component. The motion vector variance calculation rule is as follows: (a) Calculate the motion vector set V id The average of the x and y components (b) Calculate the motion vector set V id The variance of the motion vector Var of the x and y components x 、Var y : The vehicle illegal parking discrimination rule described in step (5) is: take the maximum value of Var x and Var y as the motion vector variance Var of the id-th vehicle between the previous and the next frames. Set the threshold of the motion vector variance as threshold, and compare the motion vector variance Var of the id-th vehicle between the previous and the next frames. When Var < threshold, it is determined that the vehicle is in a stationary state, that is, marked as an illegally parked vehicle; otherwise, the vehicle is marked as a moving vehicle.
Citation Information
Patent Citations
Mobile target in complex background automatic testing method and device
CN101022505A
Motor vehicle illegal parking intelligent identification method and system based on deep learning
CN114648748A
Vehicle illegal parking identification method and related equipment
CN114724093A
Rolov5-based vehicle illegal parking identification method
CN115546181A
Method and systems of classifying a vehicle using motion vectors
US20140362230A1
Cited By
Algorithm for judging illegal parking of vehicle through vehicle thermodynamic diagram statistics
CN119541219A
An algorithm for judging illegal parking of vehicles through vehicle heat map statistics
CN119541219B
An algorithm for determining vehicle parking violations by background modeling
CN122888544A
