An algorithm for judging illegal parking of vehicles through vehicle heat map statistics

Abstract

The present application relates to the technical field of video analysis, and discloses an algorithm for judging illegal parking of vehicles through vehicle thermal map statistics, which includes: video frame extraction, road area identification, vehicle identification, thermal area acquisition, candidate target confirmation and illegal parking judgment. In the present application, after the vehicle target is identified, the strategy of vehicle tracking is not directly adopted, but the thermal map of a period of time is statistically analyzed, and the area with high hot spots is obtained through the thermal map statistics, and then the vehicle target is matched in the area with high hot spots, so as to obtain suspicious candidate targets, and then the candidate targets are template matched for a period of time to determine whether their positions have moved, so as to draw a conclusion on whether they are illegally parked. The tracking targets can be effectively reduced through thermal map statistics, and the algorithm calculation amount can be reduced. Then, the movement of the target can be judged by the image matching algorithm without relying on the tracking algorithm, thereby improving the detection rate.

Description

Technical Field

[0001] The present application relates to the field of video analysis technology, and specifically to an algorithm for determining illegal parking of vehicles through vehicle heat map statistics. Background Art

[0002] Illegal parking of vehicles on the road not only affects the appearance of the city, but also brings safety hazards to road traffic. Therefore, legal and compliant handling of illegally parked vehicles is a very effective punishment measure to reduce illegal parking of vehicles on the road. If the judgment of illegal parking depends solely on manual judgment, it requires a lot of manpower and material resources to achieve, and the efficiency is poor. Therefore, a more convenient method is needed.

[0003] For example, a Chinese patent application number CN202410586944.5 discloses a method for detecting illegal parking of vehicles based on motion vector differences, comprising: constructing a drone image dataset of traffic scenes containing vehicles, and using the dataset to train a yolov5 vehicle detection model; using the trained yolov5 vehicle detection model to detect each frame of image collected by the drone to obtain the position information of each vehicle in the image; then determining a discrimination area for calculating the motion vector difference for each vehicle and dividing it into blocks, using a three-step search algorithm to match the position of each discrimination area sub-block in the previous frame in the next frame; calculating the motion vector of each sub-block in the discrimination area between the previous and next frames and the motion vector variance of all sub-blocks in the discrimination area, and combining the vehicle illegal parking discrimination rules to determine whether the vehicle is illegally parked.

[0004] This technology uses a vehicle detection model to detect the collected images to obtain the vehicle position, and then tracks and analyzes the status of each vehicle to determine whether it is stationary to identify illegal parking. This method requires tracking and identifying all vehicles, and there are many targets that need to maintain their status, resulting in a large amount of calculation and high requirements for the tracking algorithm. If the detection effect is not good, tracking errors are likely to occur, resulting in missed judgments. Summary of the invention

[0005] The purpose of this application is to provide an algorithm for determining illegal parking of vehicles through vehicle heat map statistics, so as to solve the technical problems raised in the background technology.

[0006] To achieve the above purpose, the present application discloses the following technical solution: an algorithm for judging illegal parking of vehicles through vehicle heat map statistics, comprising the following steps in sequence:

[0007] Video frame extraction: Get full-frame images in high-frame rate videos and extract one image every F frames;

[0008] Identify road areas: Identify the extracted images to obtain road areas;

[0009] Vehicle identification: Identify vehicle targets in the identified road area;

[0010] Thermal area acquisition: according to The heat map statistics of the vehicle target in the T-th frame image are performed as a period. The image is converted into a binary image by thresholding, and the thermal area is obtained by morphological processing;

[0011] Candidate target confirmation: Match the thermal area with the current detection target list to obtain a list of targets at the thermal area location ; List the targets for two consecutive cycles and target list Perform IOU matching to obtain vehicle targets that continuously meet thermal statistics, and use the vehicle targets as candidate targets ;

[0012] Illegal parking determination: Candidate target The corresponding position in the continuous T frame images is matched to obtain its position P in the Mth frame, and the position P is matched with the candidate target Perform IOU matching. When the number of IOUs corresponding to the matching results > 0.4 is greater than the preset threshold , then the candidate target The corresponding vehicle target is determined as an illegal parking object.

[0013] Preferably, the video frame extraction specifically includes:

[0014] Full-frame images of high-frame rate videos are obtained from road cameras, and one image is extracted every F frames.

[0015] Preferably, the identifying road area specifically includes:

[0016] Identify the ROI regions {[x1, y1], [x2, y2], [x3, y3]...[xn, yn]} where target detection is required in the images extracted from the video frames by manual marking, wherein xn and yn represent the coordinate positions of the center of each region in the image;

[0017] Alternatively, the deep learning image semantic segmentation model DeepLab is used to identify the ROI area {[x1, y1], [x2, y2], [x3, y3] ... [xn, yn]} where target detection is required in the image extracted from the video frame, where xn and yn represent the coordinate position of the center of each area in the image.

[0018] Preferably, the vehicle identification method specifically includes:

[0019] Draw a polygon grayscale image grep according to the ROI area;

[0020] The image extracted from the full-frame image obtained in the high-frame rate video is bitwise ANDed with the polygon grayscale image grep to obtain a preprocessed image;

[0021] The preprocessed image is used to identify the vehicle target in the YOLO target recognition model to obtain the position information of the vehicle target in the image {[x1, x2, y1, y2], [x3, x4, y3, y4], ......}.

[0022] Preferably, the thermal region acquisition specifically includes:

[0023] Create a blank grayscale image frame by frame;

[0024] According to the position information of the detected vehicle target in the image, the assignment operation is performed on the established blank grayscale image to obtain the position mapping heat map of the vehicle target in each frame of the image;

[0025] Each interval Time, the position mapping heat map of the T frame image is added to obtain the final Moment heat map;

[0026] To get Morphological operations are performed on the heat map at each moment to calculate the contours of the hot spot areas [area1, area2,......].

[0027] Preferably, the candidate target confirmation specifically includes:

[0028] Will be in The vehicle targets identified within the time are matched with the contours of the hotspot area by IOU to obtain a target list ;

[0029] Will be in The vehicle targets identified within time t are matched with the contours of the hotspot area by IOU to obtain a target list ;

[0030] When the target list and target list The targets that meet the heat map statistics at the same position in the image are selected as candidate targets. .

[0031] Preferably, the illegal parking determination specifically includes:

[0032] Get candidate targets The corresponding position information P is obtained, and its image in the current frame is selected;

[0033] In the image of M consecutive frames, according to the candidate target The corresponding position information P is recalculated and scaled a certain multiple to obtain a template image;

[0034] The candidate target Perform template matching between the image in the current frame and the template image, and select the image with a matching degree greater than the preset matching threshold. The target and count its number is ,when The candidate target is determined The corresponding vehicle target is the illegally parked object.

[0035] Technical effect: The algorithm of the present application uses vehicle heat map statistics to judge whether a vehicle is illegally parked. After identifying the vehicle target, it does not directly adopt a strategy of tracking the vehicle, but instead counts a heat map for a period of time, obtains areas with high hot spots through heat map statistics, and then matches the vehicle targets in the areas with high hot spots, thereby obtaining suspicious candidate targets, and then performs template matching on the candidate targets for a period of time to determine whether their positions have moved, thereby concluding whether they are illegally parked. The statistics of heat maps can effectively reduce the number of tracked targets and the amount of algorithm calculations. Then, the movement of the target can be determined by an image matching algorithm rather than a tracking algorithm, thereby improving the detection rate. In this way, the amount of calculation of the illegal parking judgment algorithm can be reduced, the speed of algorithm detection can be improved, and the cost of the application machine can be reduced, which has practical landing value. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A flowchart of an algorithm for determining illegal parking of vehicles through vehicle heat map statistics provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0039] In this article, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprising..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0040] This embodiment discloses a Figure 1 The algorithm for judging illegal parking of vehicles by using vehicle heat map statistics shown includes the following steps in sequence:

[0041] 1. Video frame extraction: Get full-frame images of high-frame rate videos from road cameras and extract one picture every F frames.

[0042] 2. Identify the road area: Identify the extracted image to obtain the road area. It is feasible to adopt the Sa1-manual identification or Sa2-intelligent identification method, wherein the manual identification method is specifically: identify the ROI area {[x1, y1], [x2, y2], [x3,y3]...[xn, yn]} where target detection is required in the image extracted from the video frame by manual marking, wherein xn and yn represent the coordinate position of the center of each area in the image. The intelligent identification method is: identify the ROI area {[x1, y1], [x2, y2], [x3, y3]...[xn, yn]} where target detection is required in the image extracted from the video frame by the deep learning image semantic segmentation model DeepLab, wherein xn and yn represent the coordinate position of the center of each area in the image.

[0043] 3. Vehicle identification: Identify vehicle targets in the identified road area. Specifically include:

[0044] Draw a polygon grayscale image grep according to the ROI area;

[0045] The image extracted from the full-frame image obtained in the high-frame rate video is bitwise ANDed with the polygon grayscale image grep to obtain a preprocessed image;

[0046] The preprocessed image is used to identify the vehicle target in the YOLO target recognition model to obtain the position information of the vehicle target in the image {[x1, x2, y1, y2], [x3, x4, y3, y4], ......}.

[0047] 4. Thermal area acquisition: according to The heat map statistics of the vehicle target in the T-th frame image are performed as a period. The threshold is processed into a binary image, and the thermal area is obtained through morphological processing. Specifically including:

[0048] Create a blank grayscale image frame by frame;

[0049] According to the position information of the detected vehicle target in the image, the assignment operation is performed on the established blank grayscale image to obtain the position mapping heat map of the vehicle target in each frame of the image;

[0050] Each interval Time, the position mapping heat map of the T frame image is added to obtain the final Moment heat map;

[0051] To get Morphological operations are performed on the heat map at each moment to calculate the contours of the hot spot areas [area1, area2,......].

[0052] 5. Candidate target confirmation: Match the thermal area with the current detection target list to obtain the target list at the thermal area location ; List the targets for two consecutive cycles and target list Perform IOU matching to obtain vehicle targets that continuously meet thermal statistics, and use the vehicle targets as candidate targets . Specifically include:

[0053] Will be in The vehicle targets identified within the time are matched with the contours of the hotspot area by IOU to obtain a target list ;

[0054] Will be in The vehicle targets identified within time t are matched with the contours of the hotspot area by IOU to obtain a target list ;

[0055] When the target list and target list The targets that meet the heat map statistics at the same position in the image are selected as candidate targets. .

[0056] 6. Illegal parking determination: The corresponding position in the continuous T frame images is matched to obtain its position P in the Mth frame, and the position P is matched with the candidate target Perform IOU matching. When the number of IOUs corresponding to the matching results > 0.4 is greater than the preset threshold , then the candidate target The corresponding vehicle target is determined as an illegal parking object. Specifically including:

[0057] Get candidate targets The corresponding position information P is obtained, and its image in the current frame is selected;

[0058] In the image of M consecutive frames, according to the candidate target The corresponding position information P is recalculated and scaled a certain multiple to obtain a template image;

[0059] The candidate target Perform template matching between the image in the current frame and the template image, and select the image with a matching degree greater than the preset matching threshold. The target and count its number is ,when The candidate target is determined The corresponding vehicle target is the illegally parked object.

[0060] Based on the above, the algorithm of this embodiment that uses vehicle thermal map statistics to judge illegal parking of vehicles does not directly adopt the strategy of vehicle tracking after vehicle target identification, but instead counts the thermal map for a period of time, obtains the area with high hot spots through thermal map statistics, and then matches the vehicle target in the area with high hot spots, thereby obtaining suspicious candidate targets, and then performs template matching on the candidate targets for a period of time to determine whether its position has moved, thereby drawing a conclusion on whether it is illegally parked. The thermal map statistics can effectively reduce the tracking targets and reduce the amount of algorithm calculations. Then, the movement of the target can be determined by the image matching algorithm instead of the tracking algorithm, thereby improving the detection rate. In this way, not only the amount of calculation of the illegal parking judgment algorithm is reduced, but also the detection rate is improved, which has practical application significance, and it can also improve the speed of algorithm detection, reduce the cost of application machines, and have better landing value.

[0061] Finally, it should be noted that the above are only preferred embodiments of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An algorithm for judging illegal parking of vehicles by using vehicle heat map statistics, characterized in that: The following steps are included in sequence: Video frame extraction: Get full-frame images in high-frame rate videos and extract one image every F frames; Identify road areas: Identify the extracted images to obtain road areas; Vehicle identification: Identify vehicle targets in the identified road area; Thermal area acquisition: according to The heat map statistics of the vehicle target in the T-th frame image are performed as a period. The image is converted into a binary image by thresholding, and the thermal area is obtained by morphological processing; Candidate target confirmation: Match the thermal area with the current detection target list to obtain a list of targets at the thermal area location ; List the goals for two consecutive cycles and target list Perform IOU matching to obtain vehicle targets that continuously meet thermal statistics, and use the vehicle targets as candidate targets ; Illegal parking determination: Candidate target The corresponding position in the continuous T frame images is matched to obtain its position P in the Mth frame, and the position P is matched with the candidate target Perform IOU matching. When the number of IOUs corresponding to the matching results > 0.4 is greater than the preset threshold , then the candidate target The corresponding vehicle target is determined as an illegal parking object.

2. The algorithm for judging illegal parking of vehicles through vehicle heat map statistics according to claim 1 is characterized in that: The video frame extraction specifically includes: Full-frame images of high-frame rate videos are obtained from road cameras, and one image is extracted every F frames.

3. The algorithm for judging illegal parking of vehicles through vehicle heat map statistics according to claim 1 is characterized in that: The identifying road area specifically includes: Identify the ROI regions {[x1, y1], [x2, y2], [x3, y3]...[xn, yn]} where target detection is required in the images extracted from the video frames by manual marking, wherein xn and yn represent the coordinate positions of the center of each region in the image; Alternatively, the deep learning image semantic segmentation model DeepLab is used to identify the ROI area {[x1, y1], [x2, y2], [x3, y3] ... [xn, yn]} where target detection is required in the image extracted from the video frame, where xn and yn represent the coordinate position of the center of each area in the image.

4. The algorithm for determining illegal parking of vehicles by using vehicle heat map statistics according to claim 3 is characterized in that: The vehicle identification method specifically includes: Draw a polygon grayscale image grep according to the ROI area; The image extracted from the full-frame image obtained in the high-frame rate video is bitwise ANDed with the polygon grayscale image grep to obtain a preprocessed image; The preprocessed image is used to identify the vehicle target in the YOLO target recognition model to obtain the position information of the vehicle target in the image {[x1, x2, y1, y2], [x3, x4, y3, y4], ......}.

5. The algorithm for judging illegal parking of vehicles through vehicle heat map statistics according to claim 4 is characterized in that: The thermal area acquisition specifically includes: Create a blank grayscale image frame by frame; According to the position information of the detected vehicle target in the image, the assignment operation is performed on the established blank grayscale image to obtain the position mapping heat map of the vehicle target in each frame of the image; Each interval Time, the position mapping heat map of the T frame image is added to obtain the final Moment heat map; To get Morphological operations are performed on the heat map at each moment to calculate the contours of the hot spot areas [area1, area2,......].

6. The algorithm for judging illegal parking of vehicles through vehicle heat map statistics according to claim 5 is characterized in that: The candidate target confirmation specifically includes: Will be in The vehicle targets identified within the time are matched with the contours of the hotspot area by IOU to obtain a target list ; Will be in The vehicle targets identified within time t are matched with the contours of the hotspot area by IOU to obtain a target list ; When the target list and target list The targets that meet the heat map statistics at the same position in the image are selected as candidate targets. .

7. The algorithm for judging illegal parking of vehicles through vehicle heat map statistics according to claim 6 is characterized in that: The illegal parking determination specifically includes: Get candidate targets The corresponding position information P is obtained, and its image in the current frame is selected; In the image of M consecutive frames, according to the candidate target The corresponding position information P is recalculated and scaled a certain multiple to obtain a template image; The candidate target Perform template matching between the image in the current frame and the template image, and select the image with a matching degree greater than the preset matching threshold. The target and count its number is ,when The candidate target is determined The corresponding vehicle target is the illegally parked object.

Images