Adaptive monitoring video crowd counting method in complex scene

Abstract

The invention discloses a self-adaptive monitoring video crowd counting method in a complex scene, which mainly comprises the following steps of: framing a monitoring video, and extracting a video frame as input at a fixed time interval according to an application requirement; detecting the video frame by using a target detection network, and selecting a target frame in which the detection result is a pedestrian and the confidence coefficient is not less than a threshold value; counting the number of target frames as a counting result of nearby large-target pedestrians; detecting the frame image by using a crowd density detection network to obtain a density map with the same size as the frame image; setting far pixel values of the density map in the coordinate range of all the target frames as 0 to obtain a new density map; summing the density map and rounding down to obtain a crowd counting result obtained by the density detection network; adding the results of the two parts to obtain a crowd counting result of the current frame image; repeating the above steps on each extracted video frame so as to carry out automatic real-time crowd counting; according to the method, crowd counting in a complex scene based on the monitoring video can be realized.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep learning, and in particular to an adaptive surveillance video crowd counting method in complex scenes. Background Art

[0002] As people's living standards continue to improve, the flow of people in public places such as shopping malls, theaters, and stadiums continues to grow, and potential risks are also increasing. Rapidly counting the number of people in the above key places through surveillance video helps management departments quickly grasp the crowd gathering situation, formulate relevant evacuation management strategies, and maintain public safety.

[0003] Most of the current automatic crowd counting methods are implemented using a deep learning framework. The camera equipment is used to shoot images of key places, and the deep network is used to detect and count pedestrians in the images. This type of method can obtain relatively accurate results when the number of pedestrians is small and the scene is relatively simple. However, in actual application scenarios, the number of pedestrians in various key places is often very large, the crowd density is very high, and the on-site environment is also very complex and diverse. The size of pedestrian targets in the same scene may vary greatly. A single deep network model often misses or misdetects pedestrians, resulting in poor results of existing crowd counting methods and difficulty in accurate counting. This also greatly reduces the practicality of existing crowd counting methods.

[0004] In order to accurately monitor the number of crowds, the most commonly used method is still for managers to make manual judgments after watching on-site images and videos, which often requires a lot of manpower and time, and is also prone to various errors, affecting the subsequent crowd management and evacuation strategy formulation. Therefore, it is of great practical significance to study accurate and automated crowd counting methods in complex scenarios. Summary of the invention

[0005] In order to solve the problem of crowd counting in surveillance videos under current complex scenes, the present invention provides an adaptive crowd counting method for surveillance videos under complex scenes. The method uses surveillance videos as input, adaptively applies different deep network models to detect and count people at different positions in the frame image, and then fuses the counting results of various deep network models to obtain the total number of people in the current frame image, thereby realizing automatic and accurate crowd counting.

[0006] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0007] An adaptive surveillance video crowd counting method in complex scenes, characterized by comprising the following steps:

[0008] S1. Perform frame processing on the surveillance video and extract one video frame every time t according to application requirements as the input image for crowd counting;

[0009] S2. Using target detection network N 1 Detect the frame image and select the detection result as a pedestrian, and the confidence level is not less than the threshold T 1 The target box;

[0010] S3. Count the number of all target frames detected and use it as the counting result of the nearby large target population, recorded as n 1 ;

[0011] S4. Using crowd density detection network N 2 Detect the frame image to obtain a density map P, the size of which is consistent with the frame image, with its width denoted as W and its height denoted as H;

[0012] S5. For the first pedestrian target frame, obtain its leftmost coordinate l 1 , rightmost coordinate r 1 , the top coordinate u 1 , the bottom coordinate b 1 , set the density map element value within its range to 0, that is, P(x,y)=0,l 1 ≤x≤r 1 ,u 1 ≤y≤b 1

[0013] Where P(x,y) represents the element value of the density map P at the (x,y) coordinate;

[0014] S6, repeat the process of S5 for other pedestrian target frames, and 1 The density map element values ​​within the range of the pedestrian target box are set to 0, and the new density map is finally obtained and recorded as P';

[0015] S7. Sum the new density map to obtain the crowd counting result n obtained by the density detection network. 2 , that is,

[0016]

[0017] in Indicates rounding down;

[0018] S8. Add the counting results obtained by the two parts of network detection to obtain the crowd counting result n=n of the current frame image. 1 +n 2 ;

[0019] S9. Repeat steps S1 to S8 for each video frame extracted from the surveillance video to achieve automatic real-time crowd counting.

[0020] As a preferred technical solution, in the step S1, the interval time for extracting video frames is t=1 second.

[0021] As a preferred technical solution, in the step S2, the target detection network N 1 It is a FasterRCNN network model with ResNet50 network as the backbone, and its weights are the default pre-trained weights provided by torchvision.

[0022] As a preferred technical solution, in step S2, the threshold T 1 =0.5.

[0023] As a preferred technical solution, in the step S4, the crowd density detection network N 2 It is a Cascaded Multi-task Learning (CMTL) network model, and its weights are open source weights pre-trained in the ShanghaiTech database.

[0024] Compared with the prior art, the present invention has the following advantages and effects:

[0025] 1. The present invention provides an adaptive surveillance video crowd counting method in complex scenarios, which can realize automatic crowd counting based on surveillance videos, assist management personnel to timely grasp the crowd gathering situation in key public places, and improve the effectiveness of management and control work.

[0026] 2. The present invention utilizes the target detection network and the crowd density detection network to respectively process the detection and counting problems of nearby large-target pedestrians and distant small-target pedestrians in the same video frame, and utilizes the density map mask to avoid repeated counting of the two, thereby improving the accuracy of crowd counting in complex environments.

[0027] 3. The present invention extracts frames from the surveillance video at certain time intervals according to actual needs, and uses a relatively lightweight deep network model to perform detection tasks, effectively improving the practical application efficiency of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an overall flow chart of the adaptive surveillance video crowd counting method in complex scenes disclosed by the present invention;

[0029] Figure 2 is a structural diagram of a target detection network model used in an embodiment of the present invention;

[0030] Figure 3 It is a structural diagram of a crowd density detection network model used in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] This embodiment discloses an adaptive surveillance video crowd counting method in complex scenarios. The following takes the surveillance video of a shopping mall as an example as a preferred embodiment to introduce the implementation process of the present invention in detail. The overall implementation process is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0033] The surveillance video to be processed is divided into frames, and one video frame is extracted every 1 second according to application requirements as the input image for crowd counting.

[0034] Apply the FasterRCNN network model with ResNet50 as the backbone, download its default pre-trained weights from torchvision, detect the first frame image, select the target box classified as a pedestrian in the detection result with a confidence level of not less than 0.5, and save it. The structure of the FasterRCNN network model used is as follows: Figure 2 shown.

[0035] Count the total number of saved target frames and use it as the nearby large target crowd counting result n 1 In this embodiment, n 1 =37.

[0036] The open source CMTL crowd density detection network is used to detect the frame image and obtain the density map P. The size of P is consistent with the video frame, which is 1920×1080.

[0037] For the 37 pedestrian target frames, the density map element values ​​within their range are set to 0. Taking the first pedestrian target frame as an example, its leftmost coordinate l is obtained. 1 =45, rightmost coordinate r 1 =62, top coordinate u 1 =766, bottom coordinate b 1 =826, then we have

[0038] P(x,y)=0,45≤x≤62,766≤y≤826

[0039] Where P(x,y) represents the element value of the density map P at the (x,y) coordinate. After processing all 37 pedestrian target frames in this way, the new density map is recorded as P ’ .

[0040] Sum the new density map to obtain the crowd counting result n obtained by the density detection network 2 , that is,

[0041]

[0042] in In this embodiment, n 2 =613.

[0043] Add the counting results obtained by the two parts of the network detection to get the crowd counting result of the current frame image n = n 1 +n 2 =37+613=650 people. Repeat the above steps for each frame extracted thereafter, thereby achieving automatic real-time crowd counting.

[0044] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An adaptive surveillance video crowd counting method in complex scenes, characterized in that: The following steps are involved: S1. Perform frame processing on the surveillance video and extract one video frame every time t according to application requirements as the input image for crowd counting; S2, use the target detection network N1 to detect the frame image, and select the target box whose detection result is a pedestrian and whose confidence is not less than the threshold T1; S3, counting the number of all target frames detected, taking it as the counting result of the nearby large target population, recorded as n1; S4, using the crowd density detection network N2 to detect the frame image, and obtain a density map P, the size of which is consistent with the frame image, and its width is recorded as W, and its height is recorded as H; S5. For the first pedestrian target frame, obtain its leftmost coordinate l1, rightmost coordinate r1, topmost coordinate u1, and bottommost coordinate b1, and set the density map element value within its range to 0, that is, P(x, y) = 0, l1 ≤ x ≤ r1, u1 ≤ y ≤ b1 Where P(x,y) represents the element value of the density map P at the (x,y) coordinate; S6, repeat the process of S5 for other pedestrian target frames until the density map element values ​​within the range of n1 pedestrian target frames are set to 0, and finally a new density map is obtained, which is recorded as P'; S7. Sum the new density map to obtain the crowd counting result n2 obtained by the density detection network, that is, in Indicates rounding down; S8, adding the counting results obtained by the two parts of network detection to obtain the crowd counting result n=n1+n2 of the current frame image; S9. Repeat steps S1 to S8 for each video frame extracted from the surveillance video to achieve automatic real-time crowd counting.

2. According to the method for adaptive crowd counting of surveillance videos in complex scenes in claim 1, it is characterized in that: In the step S1, the interval time for extracting video frames is t=1 second.

3. The adaptive surveillance video crowd counting method in complex scenes according to claim 1 is characterized in that: In the step S2, the target detection network N1 is a FasterRCNN network model with a ResNet50 network as the backbone, and its weights are the default pre-trained weights provided by torchvision.

4. The adaptive surveillance video crowd counting method in complex scenes according to claim 1 is characterized in that: In the step S2, the threshold value T1=0.

5.

5. The adaptive surveillance video crowd counting method in complex scenes according to claim 1 is characterized in that: In the step S4, the crowd density detection network N2 is a Cascaded Multi-task Learning (CMTL) network model, and its weights are open source weights pre-trained in the ShanghaiTech database.

Images