A road occupation business monitoring method based on target detection and pedestrian tracking

Abstract

The present invention discloses a road-occupying business monitoring method based on target detection and pedestrian tracking, and the steps are as follows: set the target monitoring area position according to the video image obtained by the road monitoring camera, continuously extract the image of the input video stream according to the preset period, and use the trained target detector to detect the road-occupying behavior; when the target detector detects the existence of road-occupying behavior in the video image, it starts to detect the person appearing in the video screen, and when the person is detected, the pedestrian tracking is turned on to record each person appearing in the video screen and their movement trajectory; analyze the pedestrian trajectory data in the video stream to determine whether it meets the characteristics of road-occupying business behavior, if it meets the characteristics, trigger an alarm, if it does not meet the characteristics, ignore the road-occupying behavior. The present invention combines the movement trajectory information of people appearing in the monitoring scene, and automatically and accurately analyzes and judges the road-occupying business behavior, which can effectively distinguish between temporary road-occupying and road-occupying business events and reduce the false alarm rate.

Description

Technical Field

[0001] The invention relates to monitoring of road occupation business, and in particular to a method for monitoring road occupation business based on target detection and pedestrian tracking. Background Art

[0002] Occupying the road for business refers to the act of operators occupying public places such as urban roads, bridges, and city squares to sell goods or services for profit. Such behavior not only seriously affects fair competition, but also damages the surrounding environmental sanitation, thereby damaging the appearance of the city. Traditionally, the rectification of occupying the road for business mainly relies on the daily patrols and camera control of the Municipal Urban Management Law Enforcement Bureau and the City Appearance Bureau. However, the groups occupying the road for business are often highly mobile, the business locations are not fixed, and there is a certain degree of randomness, which has caused many troubles for law enforcement officers.

[0003] The existing technology uses an end-to-end deep learning method to detect illegal stalls in public areas such as specific roads in real time, and analyze and feedback the detection results in real time, which greatly reduces the workload of staff and improves work efficiency. However, this method only warns stalls suspected of occupying the road, and does not identify the nature of their business. Therefore, some stalls such as temporary stalls, temporarily parked agricultural tricycles carrying fruits, vegetables and other goods, pickup trucks, and vehicles being loaded and unloaded may be identified as occupying the road for business. Although these scenes are similar to occupying the road for business, they are essentially different, resulting in an increase in the system's false alarm rate. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for monitoring road occupation based on target detection and pedestrian tracking, which can effectively distinguish between temporary road occupation and road occupation events while increasing the forms and categories of road occupation, thereby reducing the false alarm rate.

[0005] Technical solution: The method for monitoring road occupation based on target detection and pedestrian tracking described in the present invention comprises the following steps:

[0006] S1: Set the monitoring area in the shooting picture for each surveillance camera. If not set, the entire picture content is assumed to be within the monitoring range;

[0007] S2: Set the video image sampling period T1 according to actual needs and hardware conditions, and collect video images regularly;

[0008] S3: Use the trained target detector to detect the road-occupying targets in the collected video images;

[0009] S4: When the target detector detects a suspected road occupation target in the video image in a certain sampling period t, it determines whether the location where the target appears is within the monitoring area preset in S1. If not, it returns to S3; otherwise, it saves the detected target frame diagrams and their position coordinates and goes to S5;

[0010] S5: At sampling period t+1, the target detector is used to detect the road-occupying business targets in the current image. If several road-occupying business targets are detected in the monitoring area, the position coordinates of these target frame diagrams are compared one by one with the target frame diagrams detected at the previous moment. If the Euclidean distance between the center points of the target frame diagrams in the two detection results is less than a preset threshold, the SIFT algorithm is used to extract the feature vectors of the two frame diagrams, and the cosine distance of the two vectors is calculated, or the similarity value of the two images is directly obtained as the distance using a deep learning-based method. The deep learning-based method includes Siamese and 2-channel network. If the distance is less than the preset threshold, it is determined to be the same road-occupying behavior. After all the comparisons are completed, if it is found that there is a road-occupying behavior and the duration of the behavior meets the preset m1 sampling period, then go to S6; otherwise, save the several target frame diagrams detected this time and their position coordinates, and repeat S5;

[0011] S6: Start pedestrian tracking and record the movement trajectory of pedestrians in the current monitoring scenario of road occupation behavior;

[0012] S7: Analyze the coordinate position of the road-occupying target and the moving trajectory of each pedestrian in the monitoring image, and determine whether the average number of pedestrian targets detected in each sampling image exceeds the preset value within the m2 sampling period. If it exceeds the preset value, go to S7.1, otherwise go to S7.2;

[0013] S7.1: Determine whether the current sampling period meets the following conditions:

[0014] Condition 1: Within the m2 sampling period, in the pedestrian target detection results of no less than m3 times, the number of pedestrian targets that appear within the range of the preset value d1 of the Euclidean distance from the center point coordinates of the road-occupying target accounts for a proportion of the total number of pedestrian targets that exceeds the preset value; where m3<=m2;

[0015] Condition 2: The crowd density analysis algorithm is used to obtain the crowd density distribution. In the m2 sampling period, in the sampling images of not less than m4 times, the average density of the crowd within the range of the Euclidean distance from the coordinates of the center point of the road target to the preset value d1 is higher than the average density outside the range; where m4<=m3<=m2;

[0016] If any one of condition 1 and condition 2 is met, it is considered that the road-occupying target has suspected business behavior, and go to S8 in example 1; otherwise, go to S2 in example 1;

[0017] S7.2: Analyze the coordinate position of the road-occupying target and the movement trajectory of each pedestrian in the monitoring image to determine whether the following conditions are met:

[0018] Condition 1: Within the m2 sampling period, there is at least one pedestrian target, the Euclidean distance between the pedestrian target and the center point of the pedestrian target at each sampling is less than the preset value d1, and the relative movement distance within the sampling period is less than the preset value d2;

[0019] Condition 2: Within the m2 sampling period, there is at least one pedestrian target, after entering the monitoring screen,

[0020] Move from a point outside the range whose Euclidean distance from the center point coordinates of the occupied target is a preset value d1 to within the range, and the relative moving distance within the range is less than a preset value d2;

[0021] Condition 3: Within the m2 sampling period, there is at least one pedestrian target, who enters the monitoring screen.

[0022] Move from a point within a range whose Euclidean distance from the center point coordinates of the occupied target is a preset value d1 to outside the range, and the relative moving distance within the range is less than a preset value d2;

[0023] If only condition 1 is met, it is considered that the road-occupying target only has a suspected business operator, and the process returns to S6; if both condition 1 and condition 2 are met, or both condition 1 and condition 3 are met, it is considered that the road-occupying target has a suspected business operator, and the process goes to S8; if condition 1 is not met, the process goes to S2;

[0024] S8: Issue an alarm message for road occupation, store the block diagram and location coordinate information of the road occupation target in the alarm record, add a comparison with the alarmed target in the subsequent monitoring step S5, ignore the alarmed target, avoid repeated alarms, and return to S2.

[0025] The road-occupying target detection model described in S3 is trained by the following steps:

[0026] S3.1: Collect a certain number of pictures of actual road scenes. The pictures must contain sample data of pedestrians and road-occupying business targets. Road-occupying business targets include roadside department store stalls, snack food stalls, and agricultural tricycles and pickup trucks carrying fruits, vegetables and other goods. The types of targets can be adjusted according to actual needs and road scene characteristics;

[0027] S3.2: Label the collected sample data, mark the specific location of the target in each sample image, and divide the labeled data into a training set, a test set, and a validation set in a ratio of 6:2:2;

[0028] S3.3: Select the yolov5 target detection framework, and after multiple rounds of testing and verification, obtain the optimal target detection model as the target detector of this solution.

[0029] The steps of pedestrian tracking described in S6 are:

[0030] S6.1: Adjust the video sampling period to the preset value T2, increase the sampling frequency, and use the object detector in S3 to detect the position coordinates of pedestrians in the sampled image;

[0031] S6.2: Use the deepsort algorithm to determine whether the pedestrian target appearing in the two sampling images is the same person or a new pedestrian target;

[0032] S6.3: After the preset m2 sampling cycles, the analysis algorithm results are recorded to obtain the pedestrians and their movement trajectories that appear in the monitoring scene during this monitoring time.

[0033] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned road occupation business monitoring method based on target detection and pedestrian tracking.

[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a method for monitoring road occupation based on target detection and pedestrian tracking as described above is implemented.

[0035] Beneficial effects: Compared with the prior art, the present invention has the following advantages: 1. The ability to extract image features by using deep learning algorithms can further increase the accuracy of the method in judging whether the road-occupying targets detected by two consecutive samplings are the same; 2. The difficulty of trajectory tracking caused by large-scale crowds can be effectively solved by using crowd counting and crowd density algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0038] Embodiment 1:

[0039] like Figure 1As shown, a method for monitoring road occupation based on target detection and pedestrian tracking includes the following steps:

[0040] S1: Set the monitoring area in the shooting picture for each surveillance camera. If not set, the entire picture content is assumed to be within the monitoring range;

[0041] S2: Set the video image sampling period T1 according to actual needs and hardware conditions, and collect video images regularly;

[0042] S3: Use the trained target detector to detect the road-occupying targets in the collected video images;

[0043] S4: When the target detector detects a suspected road occupation target in the video image in a certain sampling period t, it determines whether the location where the target appears is within the monitoring area preset in S1. If not, it returns to S3; otherwise, it saves the detected target frame diagrams and their position coordinates and goes to S5;

[0044] S5: At sampling period t+1, the target detector is used to detect the road-occupying business targets in the current image. If several road-occupying business targets are detected in the monitoring area, the position coordinates of these target frame diagrams are compared one by one with the target frame diagrams detected at the previous moment. If the Euclidean distance between the center points of the target frame diagrams in the two detection results is less than a preset threshold, the SIFT algorithm is used to extract the feature vectors of the two frame diagrams, and the cosine distance of the two vectors is calculated, or the similarity value of the two images is directly obtained as the distance using a deep learning-based method. The deep learning-based method includes Siamese and 2-channel network. If the distance is less than the preset threshold, it is determined to be the same road-occupying behavior. After all the comparisons are completed, if it is found that there is a road-occupying behavior and the duration of the behavior meets the preset m1 sampling period, then go to S6; otherwise, save the several target frame diagrams detected this time and their position coordinates, and repeat S5;

[0045] S6: Start pedestrian tracking and record the movement trajectory of pedestrians in the current monitoring scenario of road occupation behavior;

[0046] S7: Analyze the coordinate position of the road-occupying target and the moving trajectory of each pedestrian in the monitoring image, and determine whether the average number of pedestrian targets detected in each sampling image exceeds the preset value within the m2 sampling period. If it exceeds the preset value, go to S7.1, otherwise go to S7.2;

[0047] S7.1: Determine whether the current sampling period meets the following conditions:

[0048] Condition 1: Within the m2 sampling period, in the pedestrian target detection results of no less than m3 times, the number of pedestrian targets that appear within the range of the preset value d1 of the Euclidean distance from the center point coordinates of the road-occupying target accounts for a proportion of the total number of pedestrian targets that exceeds the preset value; where m3<=m2;

[0049] Condition 2: The crowd density analysis algorithm is used to obtain the crowd density distribution. In the m2 sampling period, in the sampling images of not less than m4 times, the average density of the crowd within the range of the Euclidean distance from the coordinates of the center point of the road target to the preset value d1 is higher than the average density outside the range; where m4<=m3<=m2;

[0050] If any one of condition 1 and condition 2 is met, it is considered that the road-occupying target has suspected business behavior, and go to S8 in example 1; otherwise, go to S2 in example 1;

[0051] S7.2: Analyze the coordinate position of the road-occupying target and the movement trajectory of each pedestrian in the monitoring image to determine whether the following conditions are met:

[0052] Condition 1: Within the m2 sampling period, there is at least one pedestrian target, the Euclidean distance between the pedestrian target and the center point of the pedestrian target at each sampling is less than the preset value d1, and the relative movement distance within the sampling period is less than the preset value d2;

[0053] Condition 2: Within the m2 sampling period, there is at least one pedestrian target, after entering the monitoring screen,

[0054] Move from a point outside the range whose Euclidean distance from the center point coordinates of the occupied target is a preset value d1 to within the range, and the relative moving distance within the range is less than a preset value d2;

[0055] Condition 3: Within the m2 sampling period, there is at least one pedestrian target, who enters the monitoring screen.

[0056] Move from a point within a range whose Euclidean distance from the center point coordinates of the occupied target is a preset value d1 to outside the range, and the relative moving distance within the range is less than a preset value d2;

[0057] If only condition 1 is met, it is considered that the road-occupying target only has a suspected business operator, and the process returns to S6; if both condition 1 and condition 2 are met, or both condition 1 and condition 3 are met, it is considered that the road-occupying target has a suspected business operator, and the process goes to S8; if condition 1 is not met, the process goes to S2;

[0058] S8: Issue an alarm message for road occupation, store the block diagram and location coordinate information of the road occupation target in the alarm record, add a comparison with the alarmed target in the subsequent monitoring step S5, ignore the alarmed target, avoid repeated alarms, and return to S2.

[0059] The road-occupying target detection model described in S3 is trained by the following steps:

[0060] S3.1: Collect a certain number of pictures of actual road scenes. The pictures must contain sample data of pedestrians and road-occupying business targets. Road-occupying business targets include roadside department store stalls, snack food stalls, and agricultural tricycles and pickup trucks carrying fruits, vegetables and other goods. The types of targets can be adjusted according to actual needs and road scene characteristics;

[0061] S3.2: Label the collected sample data, mark the specific location of the target in each sample image, and divide the labeled data into a training set, a test set, and a validation set in a ratio of 6:2:2;

[0062] S3.3: Select the yolov5 target detection framework, and after multiple rounds of testing and verification, obtain the optimal target detection model as the target detector of this solution.

[0063] The steps of pedestrian tracking described in S6 are:

[0064] S6.1: Adjust the video sampling period to the preset value T2, increase the sampling frequency, and use the object detector in S3 to detect the position coordinates of pedestrians in the sampled image;

[0065] S6.2: Use the deepsort algorithm to determine whether the pedestrian target appearing in the two sampling images is the same person or a new pedestrian target;

[0066] S6.3: After the preset m2 sampling cycles, the analysis algorithm results are recorded to obtain the pedestrians and their movement trajectories that appear in the monitoring scene during this monitoring time.

[0067] Embodiment 2:

[0068] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned road occupation business monitoring method based on target detection and pedestrian tracking.

[0069] Embodiment 3:

[0070] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a method for monitoring road occupation based on target detection and pedestrian tracking as described above is implemented.

Claims

1. A road occupation business monitoring method based on target detection and pedestrian tracking, characterized in that: The following steps are involved: S1: Set the monitoring area in the shooting picture for each surveillance camera. If not set, the entire picture content is assumed to be within the monitoring range; S2: Set the video image sampling period T1 according to actual needs and hardware conditions, and collect video images regularly; S3: Use the trained target detector to detect the road-occupying targets in the collected video images; S4: When the target detector detects a suspected road occupation target in the video image in a certain sampling period t, it determines whether the location where the target appears is within the monitoring area preset in S1. If not, it returns to S3; otherwise, it saves the detected target frame diagrams and their position coordinates and goes to S5; S5: At sampling period t+1, the target detector is used to detect the road-occupying business targets in the current image. If several road-occupying business targets are detected in the monitoring area, the position coordinates of these target frame diagrams are compared one by one with the target frame diagrams detected at the previous moment. If the Euclidean distance between the center points of the target frame diagrams in the two detection results is less than a preset threshold, the SIFT algorithm is used to extract the feature vectors of the two frame diagrams, and the cosine distance of the two vectors is calculated, or the similarity value of the two images is directly obtained as the distance using a deep learning-based method. The deep learning-based method includes Siamese and 2-channel network. If the distance is less than the preset threshold, it is determined to be the same road-occupying behavior. After all the comparisons are completed, if it is found that there is a road-occupying behavior and the duration of the behavior meets the preset m1 sampling period, then go to S6; otherwise, save the several target frame diagrams detected this time and their position coordinates, and repeat S5; S6: Start pedestrian tracking and record the movement trajectory of pedestrians in the current monitoring scenario of road occupation behavior; S7: Analyze the coordinate position of the road-occupying target and the moving trajectory of each pedestrian in the monitoring image, and determine whether the average number of pedestrian targets detected in each sampling image exceeds the preset value within the m2 sampling period. If it exceeds the preset value, go to S7.1, otherwise go to S7.2; S7.1: Determine whether the current sampling period meets the following conditions: Condition 1: Within the m2 sampling period, in the pedestrian target detection results of no less than m3 times, the number of pedestrian targets that appear within the range of the preset value d1 of the Euclidean distance from the center point coordinates of the road-occupying target accounts for a proportion of the total number of pedestrian targets that exceeds the preset value; where m3<=m2; Condition 2: The crowd density analysis algorithm is used to obtain the crowd density distribution. In the m2 sampling period, in the sampling images of not less than m4 times, the average density of the crowd within the range of the Euclidean distance from the coordinates of the center point of the road target to the preset value d1 is higher than the average density outside the range; where m4<=m3<=m2; If any one of conditions 1 and 2 is met, it is considered that the road-occupying target has suspected business behavior and go to S8, otherwise go to S2; S7.2: Analyze the coordinate position of the road-occupying target and the movement trajectory of each pedestrian in the monitoring image to determine whether the following conditions are met: Condition 1: Within the m2 sampling period, there is at least one pedestrian target, the Euclidean distance between the pedestrian target and the center point of the pedestrian target at each sampling is less than the preset value d1, and the relative movement distance within the sampling period is less than the preset value d2; Condition 2: Within the m2 sampling period, there is at least one pedestrian target, after entering the monitoring screen, Move from a point outside the range whose Euclidean distance from the center point coordinates of the occupied target is a preset value d1 to within the range, and the relative moving distance within the range is less than a preset value d2; Condition 3: Within the m2 sampling period, there is at least one pedestrian target, who enters the monitoring screen. Move from a point within a range whose Euclidean distance from the center point coordinates of the occupied target is a preset value d1 to outside the range, and the relative moving distance within the range is less than a preset value d2; If only condition 1 is met, it is considered that the road-occupying target only has a suspected business operator, and the process returns to S6; if both condition 1 and condition 2 are met, or both condition 1 and condition 3 are met, it is considered that the road-occupying target has a suspected business operator, and the process goes to S8; if condition 1 is not met, the process goes to S2; S8: Issue an alarm message for road occupation, store the block diagram and location coordinate information of the road occupation target in the alarm record, add a comparison with the alarmed target in the subsequent monitoring step S5, ignore the alarmed target, avoid repeated alarms, and return to S2.

2. The method for monitoring road occupation based on target detection and pedestrian tracking according to claim 1, characterized in that: The road-occupying target detection model described in S3 is trained by the following steps: S3.1: Collect a certain number of pictures of actual road scenes. The pictures must contain sample data of pedestrians and road-occupying business targets. Road-occupying business targets include roadside department store stalls, snack food stalls, and agricultural tricycles and pickup trucks carrying fruits, vegetables and other goods. The types of targets can be adjusted according to actual needs and road scene characteristics; S3.2: Label the collected sample data, mark the specific location of the target in each sample image, and divide the labeled data into a training set, a test set, and a validation set in a ratio of 6:2:2; S3.3: Select the yolov5 target detection framework, and after multiple rounds of testing and verification, obtain the optimal target detection model as the target detector of this solution.

3. The method for monitoring road occupation based on target detection and pedestrian tracking according to claim 1, characterized in that: The steps of pedestrian tracking described in S6 are: S6.1: Adjust the video sampling period to the preset value T2, increase the sampling frequency, and use the object detector in S3 to detect the position coordinates of pedestrians in the sampled image; S6.2: Use the deepsort algorithm to determine whether the pedestrian target appearing in the two sampling images is the same person or a new pedestrian target; S6.3: After the preset m2 sampling cycles, the analysis algorithm results are recorded to obtain the pedestrians and their movement trajectories that appear in the monitoring scene during this monitoring time.

4. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a method for monitoring road occupation based on target detection and pedestrian tracking as described in any one of claims 1 to 3 is implemented.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements a road occupation business monitoring method based on target detection and pedestrian tracking as described in any one of claims 1-3.

Images