A crowd counting and positioning method based on adaptive analysis of dense areas

Abstract

This invention discloses a crowd counting and localization method based on adaptive analysis of dense regions, belonging to the field of target detection. The method first constructs a dense crowd localization dataset for training. Secondly, a deep target detection network model is designed to learn the mapping relationship between input images and crowd locations, implementing an end-to-end prediction model. The method then roughly estimates the crowd density of the image under test, and then performs a secondary detection of the dense crowd image area to achieve rapid and accurate counting and localization of dense crowds. This method is simple and easy to implement, and can achieve efficient and accurate crowd counting and localization.

Description

Technical Field

[0001] The present invention relates to the fields of image processing, target detection and artificial intelligence, and in particular to a crowd counting and positioning method based on dense area adaptive analysis. Background Art

[0002] In order to effectively ensure the safety of public activities and prevent the spread of the epidemic, public places need to monitor the degree of crowd gathering in real time. How to effectively detect and locate crowded areas to achieve crowd counting and gathering warning has become a hot topic in scientific research.

[0003] Currently, there are three main methods for locating dense crowds: target detection-based methods, density map-based methods, and point supervision-based methods. Target detection-based methods directly use target detection models such as Faster RCNN and YOLO to predict head positions, which may miss small targets in dense crowd areas. Density map-based methods directly regress the density heat map of the crowded crowd from the image, only obtaining an overall crowding index for the scene, but not the specific locations of individuals in the crowd. Point supervision-based methods directly predict detection boxes based on point markers, but this method is computationally intensive, making it difficult to implement. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a crowd counting and positioning method based on dense area adaptive analysis. The method is simple and easy to implement and can achieve crowd counting and positioning efficiently and accurately.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A crowd counting and positioning method based on dense area adaptive analysis includes the following steps:

[0007] Step 1: Build a training sample library and annotate the head bounding box of the training samples;

[0008] Step 2: Construct a deep object detection network, which includes a high-resolution network HRNet and a binary segmentation network. The high-resolution network HRNet is used to extract shallow feature vectors of the image and input the shallow feature vectors of the image into the binary segmentation network. The binary segmentation network includes a threshold feature map generation network and a density feature map generation network, which are used to generate a threshold feature map T and a density feature map P of the same size as the original input image. In addition, the deep object detection network compares the threshold feature map T with the density feature map P to generate a binary map B, and performs a 4-neighborhood segmentation operation on the binary map B to obtain independent head detection frame position information.

[0009] Step 3: Use the training sample library to train the deep object detection network model to obtain a trained deep object detection model;

[0010] Step 4: Obtain a real scene image, scale the real scene image, and then use the trained deep object detection model to make a preliminary estimate of the crowd density to obtain a global heat map of the real scene image; then perform sliding frame detection on the global heat map, count the density of the crowd in each sliding frame, and perform a secondary detection on the unscaled area of ​​the real scene image corresponding to the sliding frame with high density to obtain the final crowd counting and positioning results.

[0011] Furthermore, the threshold feature map generation network generates a first feature vector with the same size as the original input image by sequentially connecting a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a first average pooling layer, a fifth convolutional layer, a second average pooling layer, and a first deconvolutional layer, and then performs a Sigmoid function operation on the first feature vector to obtain a threshold feature map with a value range of (0, 1);

[0012] The density feature map generation network generates a second eigenvector with the same size as the original input image by sequentially connecting the sixth convolutional layer, the second deconvolutional layer, and the third deconvolutional layer, and then performs a Sigmoid function operation on the second eigenvector to obtain a density feature map with a value range of (0, 1);

[0013] The binary image B has the same size as the original input image. For each position of the binary image B, if the value of the same position in the density feature map P is greater than the value of the same position in the threshold feature map T, then the position of the binary image B is 1, otherwise it is 0.

[0014] Furthermore, the specific method of step 4 is:

[0015] Step 401: For images with a large resolution after scaling, they are cropped into small tiles. The trained deep object detection model is then used to perform preliminary estimation on each tile, and finally the tiles are stitched together to obtain a global heat map of the entire image. For real scene images with medium resolution, they are directly scaled by 2 / 3, and then the trained deep object detection model is used to perform preliminary estimation to obtain a global heat map.

[0016] In step 402, the global heat map is detected using square sliding boxes, and the density level of each sliding box is calculated using the following formula:

[0017]

[0018] Where (i, j) is the position of the sliding frame, D is the calculated value of the crowd density level in the sliding frame, ∑block_heatMap is the cumulative value of the human heads in the sliding frame detected by the deep object detection model, and area(img) is the area of ​​the sliding frame;

[0019] Step 403: Set the density threshold to r. If the density level of the slider exceeds r, use the deep object detection model to perform a secondary detection on the unscaled original image area corresponding to the slider.

[0020] Step 404: Combine the results of the first detection and the second detection to obtain the final people counting and positioning results.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The present invention first performs scaling processing on the image, which can quickly obtain a rough estimate of the density of the crowd in the image.

[0023] 2. The present invention uses a sliding detection frame to count the density of the crowd in each image block and performs secondary detection on the dense crowd area. This adaptive detection process ensures the running speed of the method while also effectively ensuring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0025] Figure 2 Schematic diagram of the binary segmentation network structure of an embodiment of the present invention;

[0026] Figure 3 Schematic diagram of the effect of an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0028] A crowd counting and positioning method based on adaptive analysis of dense areas, such as Figure 1 As shown, the specific process is:

[0029] S1: Obtain training samples. Create a sample library using public datasets and surveillance videos from stadiums, subway stations, intersections, etc. Use LabelImg software to annotate the bounding boxes of human heads.

[0030] The specific method is:

[0031] (1) Collect open source datasets, including NWPU-Crowd and ShanghaiTech-Dataset;

[0032] (2) Collect surveillance videos of stadiums, subway stations, intersections, etc., as well as video images of crowds gathered on the Internet;

[0033] (3) Annotate the image. The annotation coordinate information is the bounding box of the human head in the form of (x, y, w, h), where x and y are the coordinate information of the upper left corner of the annotation box, and w and h are the width and height of the annotation box respectively.

[0034] S2: Build a deep object detection network and establish a head detection model for dense crowd scenes.

[0035] The specific method is:

[0036] (1) The basic skeleton network uses a high-resolution network HRNet to extract shallow features of the image; the input size of HRNet is fixed to 512*512, and the final output is a 4x downsampled 720*128*128-dimensional feature vector.

[0037] (2) Design a binary segmentation network, learn the segmentation threshold and prediction heat map, and calculate the binary segmentation map.

[0038] The input of the binary segmentation network is the output feature vector of HRNet. The binary segmentation network includes two parts: a threshold feature map generation network and a density feature map generation network. They are used to generate a threshold feature map T and a density feature map P of the same size as the original input image. The threshold feature map generation network includes network structures such as Conv, Relu, AvgPool, and ConvTranspose deconvolution layers. These network structures generate feature vectors of the same size as the original image. Finally, the Sigmoid function is performed on the feature vector to obtain a threshold feature map with a value range of (0, 1). The density feature map generation network includes Conv, Relu, and ConvTranspose. Finally, the Sigmoid function is also performed to obtain a density feature map with a value range of (0, 1).

[0039] The binary segmentation network structure is as follows Figure 2As shown. Taking 1X1conv,720, / 8 as an example, 1X1conv means that the filter of the convolution layer adopts a size of 1X1, 720 represents the pooling channel, and / 8 means that the generated feature map is downsampled by 8 relative to the original image. The same applies to other convolution layer parameters. For 15x15AvgPool, 15x15AvgPool means that the filter of the average pooling layer adopts a size of 15X15, and / 8 means that the generated feature map is downsampled by 8 relative to the original image. For ConvTranspose,1, / 1, it means that the filter of the deconvolution layer adopts a size of 1X1, and / 1 means that the generated feature map is downsampled by 1 relative to the original image.

[0040] (3) Compare the threshold feature map T with the density feature map P. Set the values ​​in the density feature map P at the corresponding position that are larger than the threshold feature map T to 1, and set the rest to 0. The binary map B is obtained as the output of the binary segmentation network. The formula is as follows:

[0041]

[0042] Finally, the binary image B is subjected to a 4-neighborhood segmentation algorithm to obtain the independent head detection frame position information.

[0043] S3: Use the sample library and the constructed deep object detection network to train the model and obtain a trained parameter model.

[0044] The specific method is:

[0045] (1) Build an algorithm training environment. The hardware requirements are: a server equipped with an NVIDIA GeForce GTX 2080Ti GPU, 32GB of memory or more, and CUDA 10.0, CUDNN 7.6.0, Python 3.8 and related dependency libraries, OpenCV 4.5.3, Pytorch 1.9.1, etc.

[0046] (2) Set the parameters as follows:

[0047] Batch size: 32

[0048] The optimization algorithm is Adam;

[0049] Initial learning rate: 1e-5;

[0050] Iterative training times: 200.

[0051] S4: Detect real scenes. Specifically, first acquire data, scale the image, and use a deep object detection network to obtain a rough estimate of the crowd density in the image to obtain a global heat map. Then, perform sliding frame detection on the global heat map, count the crowd density in each detection frame image block, perform a second detection on the densely populated areas, and fuse the two detection results to obtain the final crowd counting and positioning results.

[0052] Specifically:

[0053] 1) For high-resolution images, first crop the original image into small tiles, then perform detection on each tile separately, and finally stitch them together to obtain a global heatmap of the original image. For medium-resolution images, directly scale them by a factor of 2 / 3 to obtain global detection results. This approach can ensure accuracy while saving computing power when hardware resources are limited.

[0054] 2) Use a w*w square sliding frame to detect the global heat map, and average the density maps of each detection frame image block to evaluate the density level of each detection frame image block. The calculation formula is as follows:

[0055]

[0056] Where (i, j) is the detection frame image block index, D is the crowd density estimation value of the detection frame image block, ∑block_heatMap is the accumulation of the values ​​of the density feature map output by the deep object detection network in the corresponding area of ​​the detection frame image block, and area(img) is the area of ​​the detection frame image block;

[0057] 3) The density threshold is set to r. If the density level of the detection frame image block exceeds r, the deep object detection network is used to perform secondary detection on the unscaled original image area corresponding to the detection frame.

[0058] 4) The two detection results are integrated to obtain the final people counting and positioning results.

[0059] In summary, this paper constructs a dense crowd location dataset for training, designs a deep object detection network model, learns the mapping relationship between input images and crowd locations, and implements an end-to-end prediction model. By first roughly estimating the crowd density of the image under test and then performing a secondary detection of the dense crowd image area, this method achieves rapid and accurate counting and location of dense crowds. This method is simple and easy to implement, and can achieve efficient and accurate crowd counting and location.

Claims

1. A crowd counting and positioning method based on adaptive analysis of dense areas, characterized by: The following steps are involved: Step 1: Build a training sample library and annotate the head bounding box of the training samples; Step 2: Construct a deep object detection network, which includes a high-resolution network HRNet and a binary segmentation network. The high-resolution network HRNet is used to extract shallow feature vectors of the image and input the shallow feature vectors of the image into the binary segmentation network. The binary segmentation network includes a threshold feature map generation network and a density feature map generation network, which are used to generate a threshold feature map T and a density feature map P of the same size as the original input image. In addition, the deep object detection network compares the threshold feature map T with the density feature map P to generate a binary map B, and performs a 4-neighborhood segmentation operation on the binary map B to obtain independent head detection frame position information. Step 3: Use the training sample library to train the deep object detection network model to obtain a trained deep object detection model; Step 4: Obtain a real scene image, scale the real scene image, and then use the trained deep object detection model to make a preliminary estimate of the crowd density to obtain a global heat map of the real scene image; then perform sliding frame detection on the global heat map, count the density of the crowd in each sliding frame, and perform a secondary detection on the unscaled area of ​​the real scene image corresponding to the sliding frame with high density to obtain the final crowd counting and positioning results.

2. The method for crowd counting and positioning based on dense area adaptive analysis according to claim 1, characterized in that: The threshold feature map generation network generates a first eigenvector with the same size as the original input image through the sequentially connected first convolutional layer, second convolutional layer, third convolutional layer, fourth convolutional layer, first average pooling layer, fifth convolutional layer, second average pooling layer, and first deconvolutional layer, and then performs a Sigmoid function operation on the first eigenvector to obtain a threshold feature map with a value range of (0, 1); The density feature map generation network generates a second eigenvector with the same size as the original input image by sequentially connecting the sixth convolutional layer, the second deconvolutional layer, and the third deconvolutional layer, and then performs a Sigmoid function operation on the second eigenvector to obtain a density feature map with a value range of (0, 1); The binary image B has the same size as the original input image. For each position of the binary image B, if the value of the same position in the density feature map P is greater than the value of the same position in the threshold feature map T, then the position of the binary image B is 1, otherwise it is 0.

3. The method for crowd counting and positioning based on dense area adaptive analysis according to claim 1, characterized in that: The specific method of step 4 is: Step 401: For images with a large resolution after scaling, they are cropped into small tiles. The trained deep object detection model is then used to perform preliminary estimation on each tile, and finally the tiles are stitched together to obtain a global heat map of the entire image. For real scene images with medium resolution, they are directly scaled by 2 / 3, and then the trained deep object detection model is used to perform preliminary estimation to obtain a global heat map. In step 402, the global heat map is detected using square sliding boxes, and the density level of each sliding box is calculated using the following formula: Where (i, j) is the position of the sliding frame, D is the calculated value of the crowd density level in the sliding frame, ∑block_heatMap is the cumulative value of the human heads in the sliding frame detected by the deep object detection model, and area(img) is the area of ​​the sliding frame; Step 403: Set the density threshold to r. If the density level of the slider exceeds r, use the deep object detection model to perform a secondary detection on the unscaled original image area corresponding to the slider. Step 404: Combine the results of the first detection and the second detection to obtain the final people counting and positioning results.

Images