Dense pedestrian detection and identification method based on Crowd-YOLO

Abstract

A Crowd-YOLO-based dense pedestrian detection and recognition method belongs to the technical field of image recognition and detection, and comprises the following steps: searching and acquiring a data set for dense pedestrian detection; constructing a Crowd-YOLO model specially used for dense pedestrian detection, and enabling the Crowd-YOLO model to be capable of performing recognition in dense crowds; carrying out the training of the constructed Crowd-YOLO model; and testing by using the trained network. When the method is applied to a dense scene, a new possibility is brought to pedestrian detection in the whole dense scene; the crowd detection in a dense scene can be efficiently realized, and the requirements of practical application are better met; the prospective technical application brings a brand-new management and operation mode for the security industry, improves the safety guarantee benefit, reduces the risk, and provides a more reliable and intelligent solution for large public places with dense crowds.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition and detection, and in particular relates to a dense pedestrian detection and recognition method based on Crowd-YOLO. Background Art

[0002] In today's society, with the acceleration of urbanization and the frequent holding of large-scale events, dense pedestrian scenes are becoming more and more common. For example, busy commercial streets, crowded subway stations, large-scale sports events, and tourist attractions during holidays. In these scenes, accurate detection of pedestrians is of vital importance to ensuring public safety, optimizing traffic management, and providing better services.

[0003] Dense pedestrian detection technology has gradually become a research hotspot in the field of computer vision. New methods are committed to improving the robustness to occlusion and scale changes, while improving computational efficiency to meet the needs of practical applications.

[0004] There are currently a variety of algorithms for pedestrian detection. Among traditional machine learning algorithms, the HOG + SVM algorithm uses HOG features and SVM classifiers to distinguish pedestrians from non-pedestrians, and the Haar-like feature + Adaboost algorithm uses Haar-like features and Adaboost algorithms to improve detection accuracy. In terms of deep learning algorithms, Faster R-CNN uses RPN to generate candidate regions and then performs classification and position regression. The YOLO series of algorithms converts detection into a regression problem. SSD uses multi-scale feature maps for detection, and the DeepSORT algorithm is often combined with a pedestrian detection network for tracking. These algorithms each have their own characteristics and advantages, and play an important role in pedestrian detection. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a dense pedestrian detection and recognition method based on Crowd-YOLO, using open source multi-scene and dense scene crowd human data sets to train the network, so as to supplement the deficiencies of dense pedestrian detection under existing conditions and promote the implementation of pedestrian detection technology in dense scenes.

[0006] A dense pedestrian detection and recognition method based on Crowd-YOLO includes the following steps: Step 1: Obtain a dense pedestrian data set and preprocess the data; Step 2: Build a Crowd-YOLO network model for dense pedestrian detection, including feature extraction network, feature fusion network, and classification and regression network; Step 3: Train the constructed Crowd-YOLO network model; Step 4: Use the trained network model to perform dense pedestrian detection.

[0007] The data described in step 1 is preprocessed to annotate the collected dense pedestrian dataset Crowd Human image data in the open source dataset, and the Crowd Human image size is cropped and scaled to 640*640 px.

[0008] The feature extraction network in step 2 includes a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1, a convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0, and a Repc3 operation with a convolution kernel of 3, a step size of 1, and a padding of 1. In step 2, the feature fusion network performs average pooling operation on the feature size map with larger size and bilinear interpolation on the feature size map with smaller size, including a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1, an information injection module that first passes through a convolution operation with a convolution kernel of 3, a step size of 1, and a padding of 3, and then passes through a convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0, and then an information fusion module that performs information fusion; The classification and regression network described in step 2 inputs the feature extraction network and the feature information processed by the feature fusion network, and then undergoes two convolution operations with a convolution kernel of 3, a step size of 2, and a padding of 1, and one convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0 for classification and regression tasks.

[0009] The convolution operation uses the SILU activation function.

[0010] The information injection module of the feature fusion network uses a multi-head convolutional attention mechanism to inject global information into each layer. The specific process is: simultaneously input the local information of the current layer and the global information generated by the feature alignment module In the fusion process, the inconsistent sizes are processed by average pooling or bilinear interpolation, and finally further extracted and fused by RepBlock processing. The formula is: ; Where: is the global information input, To extract features from global information, Represents the input feature map After the convolution operation, it is processed by the Sigmoid activation function, and finally the scale is adjusted by the Resize operation to adapt to the subsequent feature fusion process to obtain the global activation feature map; It is the whole base layer responsible for extracting global information. For resizing operations, It means that after the convolution operation, the obtained global embedding feature will be resized through Resize to align it with the input local information feature for subsequent operations; is the input of the network and is the key part used to multiply with the global attention feature map. It is a convolution operation on the input. ; It represents the feature map obtained after convolution, multiplication and addition operations, which integrates global information, local information and attention mechanism; It means that the final fused information will be further extracted and fused using RepBlock for fusion between different layers.

[0011] The bilinear interpolation generates the feature images of the same size and then fuses them. The formula is:

[0012]

[0013]

[0014] in, , , , Represents the four points closest to point P. , Indicates that in the horizontal direction , The result of linear interpolation between Indicates that in the vertical direction , The result of linear interpolation between ; is the horizontal coordinate, is the vertical coordinate.

[0015] The loss function of the classification and regression network includes two parts. The first part is to use the binary cross entropy loss function BCE Loss in the classification loss. The formula is:

[0016] Where y is a binary label, The output is the label probability, and N is the total number; The second part is to reuse the feature point loss in the regression task. and , the formula is as follows:

[0017] In the formula, , is the score associated with the distribution predicted by the model, y represents the true value, , Represents two adjacent values ​​obtained after discretization of the true value;

[0018] Where A and B are the true box and verification box respectively.

[0019] Through the above design scheme, the present invention can bring the following beneficial effects: a dense pedestrian detection and recognition method based on Crowd-YOLO successfully applies the target detection technology to dense scenes, bringing new possibilities for pedestrian detection in the entire dense scene; it can efficiently realize the detection of crowds in dense scenes and better meet the needs of practical applications; this forward-looking technical application brings a new management and operation mode to the security industry, improves the security benefits, reduces risks, and provides a more reliable and intelligent solution for large public places with dense crowds. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention is further described below with reference to the accompanying drawings and specific embodiments: Figure 1 The figure is a flow chart of a dense pedestrian detection and recognition method based on Crowd-YOLO of the present invention.

[0021] Figure 2 This is a feature extraction network framework diagram of a dense pedestrian detection and recognition method based on Crowd-YOLO in the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of a multi-head convolutional attention mechanism model used in a feature fusion network of a dense pedestrian detection and recognition method based on Crowd-YOLO in the present invention.

[0023] Figure 4 This is a schematic diagram of the detection accuracy of a model of a dense pedestrian detection and recognition method based on Crowd-YOLO in the present invention. DETAILED DESCRIPTION

[0024] In order to better understand the purpose, structure and function of the present invention, a dense pedestrian detection and recognition method based on Crowd-YOLO of the present invention is further described in detail below with reference to the accompanying drawings.

[0025] like Figure 1-Figure 4 As shown, Figure 2 The number below each figure indicates the size of the feature map, and the number next to the figure indicates the number of channels; the present invention proposes to use the existing Crowd Human image data to perform model training for a dense pedestrian detection network. Example

[0026] As shown in the accompanying figure, a dense pedestrian detection and recognition method based on Crowd-YOLO, the specific steps of the method are as follows: S1-1: Obtain labeled Crowd Human image data, where the Crowd Human image data is a labeled Crowd Human image dataset obtained from an open source dataset; S1-2: Adjust the size of the Crowd Human image and crop and scale the original image to 640*640 px.

[0027] S2-1: Use the preprocessed data set and input it into the Crowd-YOLO model to train the detection model.

[0028] S2-2: The Crowd-YOLO network mainly consists of three parts: feature extraction network, feature fusion network, and classification and regression network.

[0029] S2-2-1: Feature extraction network, including convolution operation with kernel 3, stride 2, padding 1, two convolution operations with kernel 1, stride 1, padding 0, and three convolution operations with kernel 3, stride 1, padding 1. The specific steps are as follows: input the preprocessed crowd human image, first undergo the above convolution operation, then repeat the convolution and Repc3 operation four times, and finally pass through the spatial pyramid pooling module to prepare for the feature fusion network.

[0030] S2-2-2: Feature fusion network. The feature lifting network is composed of a feature alignment module that performs average pooling operations on large feature maps and bilinear interpolation on small feature maps, including a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1. An information injection module that first undergoes a convolution operation with a convolution kernel of 3, a step size of 1, and a padding of 3, and then another convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0, and then an information fusion module for information fusion. The specific process is as follows: feature information obtained from the feature extraction network is aligned, and then the aligned modules are feature fused. The fused feature information undergoes a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1. Then, the feature information of different layers is aligned and fused. Then, a convolution operation including a convolution kernel of 1, a step size of 1, and a padding of 0 is performed, and the feature map processed by the convolution operation is divided into blocks. A Bottleneck operation with a convolution kernel of 3, a step size of 1, and a padding of 1 is performed, and then a C2f module with a convolution kernel of 1, a step size of 1, and a padding of 0 is performed. Then, a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1 is performed, and the feature information of different layers is aligned and fused. After passing through the C2f module, the feature information fusion operation of different layers is performed, after passing through the C2f module, an information injection module is passed, and finally, the C2f module is passed to realize multi-scale feature fusion.

[0031] S2-2-3: Classification and regression network, the input is the feature extraction network and the feature information processed by the feature fusion network, and then it undergoes two convolution operations with a convolution kernel of 3, a step size of 2, and a padding of 1, and one convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0 for classification and regression tasks.

[0032] S3-1: During the training process, Python 3.8.11 was used on the Pytorch 1.10.1 framework, using the SGD optimizer, setting the learning rate to 0.01, the kinetic energy to 0.937, and the batch-size to 16 for training.

[0033] S3-2: The convolution operation in the Crowd-YOLO model uses the SILU activation function. The SILU activation function is a smooth non-monotonic function. This smoothness helps the model better adapt to the data during the optimization process and improves the convergence of the model. The function expression is:

[0034] S3-3: The information injection module in the feature fusion network uses a multi-head convolutional attention mechanism to more effectively inject global information into different layers. The specific process is: simultaneously input the local information of the current layer and the global information generated by the feature alignment module In the fusion process, if the sizes are inconsistent, average pooling or bilinear interpolation is performed, and finally RepBlock is used for further extraction and fusion. The formula is as follows: , , , .

[0035] Where: is the global information input, To extract features from global information, Represents the input feature map After the convolution operation, it is processed by the Sigmoid activation function, and finally the scale is adjusted by the Resize operation to adapt to the subsequent feature fusion process to obtain the global activation feature map; It is the whole base layer responsible for extracting global information. For resizing operations, It means that after the convolution operation, the obtained global embedding feature will be resized through Resize to align it with the input local information feature for subsequent operations; is the input of the network and is the key part used to multiply with the global attention feature map. It is a convolution operation on the input. ; It represents the feature map obtained after convolution, multiplication and addition operations, which integrates global information, local information and attention mechanism; It means that the final fused information will be further extracted and fused using RepBlock for fusion between different layers.

[0036] S3-4: In the feature fusion network, bilinear interpolation is performed on the feature maps to the same scale, and then the feature maps are fused. The bilinear interpolation formula is as follows:

[0037]

[0038]

[0039] in , , , Represents the four points closest to point P. , Indicates that in the horizontal direction , The result of linear interpolation between Indicates that in the vertical direction , The result of linear interpolation between . is the horizontal coordinate, is the vertical coordinate.

[0040] S3-5: The loss function in the classification and regression network is divided into two parts. The first part is to use the binary cross entropy loss function in the classification loss. The formula of the binary cross entropy loss function Binary CrossEntropy Loss (BCE Loss) is as follows:

[0041] in is a binary label, The output is the label probability, is the total number.

[0042] The second part is to use feature point loss in the regression task ( ) and Generalized Intersectionover Union ( ), the formula is as follows:

[0043] in is the score associated with the distribution predicted by the model, y represents the true value, , Represents two adjacent values ​​obtained after discretization of the true value.

[0044]

[0045] Among them, A and B are the true box and the verification box respectively.

[0046] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A dense pedestrian detection and recognition method based on Crowd-YOLO, characterized by: The following steps are included: Step 1: Obtain a dense pedestrian data set and preprocess the data; Step 2: Build a Crowd-YOLO network model for dense pedestrian detection, including feature extraction network, feature fusion network, and classification and regression network; Step 3: Train the constructed Crowd-YOLO network model; Step 4: Use the trained network model to perform dense pedestrian detection.

2. According to the method of claim 1, the method comprises the following steps: The data is preprocessed by annotating the collected dense pedestrian dataset Crowd Human image data in the open source dataset, and cropping and scaling the Crowd Human image size to 640*640 px.

3. The method for dense pedestrian detection and recognition based on Crowd-YOLO according to claim 1, characterized in that: The feature extraction network in step 2 includes a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1, a convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0, and a Repc3 operation with a convolution kernel of 3, a step size of 1, and a padding of 1. In step 2, the feature fusion network performs average pooling operation on the feature size map with larger size and bilinear interpolation on the feature size map with smaller size, including a convolution operation with a convolution kernel of 3, a step size of 2, and a padding of 1, an information injection module that first passes through a convolution operation with a convolution kernel of 3, a step size of 1, and a padding of 3, and then passes through a convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0, and then an information fusion module that performs information fusion; The classification and regression network described in step 2 inputs the feature extraction network and the feature information processed by the feature fusion network, and then undergoes two convolution operations with a convolution kernel of 3, a step size of 2, and a padding of 1, and one convolution operation with a convolution kernel of 1, a step size of 1, and a padding of 0 for classification and regression tasks.

4. The method for dense pedestrian detection and recognition based on Crowd-YOLO according to claim 3 is characterized by: The convolution operation uses the SILU activation function.

5. The method for dense pedestrian detection and recognition based on Crowd-YOLO according to claim 3, characterized in that: The information injection module of the feature fusion network uses a multi-head convolutional attention mechanism to inject global information into each layer. The specific process is: simultaneously input the local information of the current layer and the global information generated by the feature alignment module In the fusion process, the inconsistent sizes are processed by average pooling or bilinear interpolation, and finally further extracted and fused by RepBlock processing. The formula is: ； Where: is the global information input, To extract features from global information, Represents the input feature map After the convolution operation, it is processed by the Sigmoid activation function, and finally the scale is adjusted by the Resize operation to adapt to the subsequent feature fusion process to obtain the global activation feature map; It is the whole base layer responsible for extracting global information. For resizing operations, It means that after the convolution operation, the obtained global embedding feature will be resized through Resize to align it with the input local information feature for subsequent operations; is the input of the network and is the key part used to multiply with the global attention feature map. It is a convolution operation on the input. ; It represents the feature map obtained after convolution, multiplication and addition operations, which integrates global information, local information and attention mechanism; It means that the final fused information will be further extracted and fused using RepBlock for fusion between different layers.

6. The method for dense pedestrian detection and recognition based on Crowd-YOLO according to claim 5, characterized in that: The bilinear interpolation generates the feature images of the same size and then fuses them. The formula is: ； in, , , , Represents the four points closest to point P. , Indicates that in the horizontal direction , The result of linear interpolation between Indicates that in the vertical direction , The result of linear interpolation between ; is the horizontal coordinate, is the vertical coordinate.

7. The method for dense pedestrian detection and recognition based on Crowd-YOLO according to claim 3, characterized in that: The loss function of the classification and regression network includes two parts. The first part is to use the binary cross entropy loss function BCE Loss in the classification loss. The formula is: ； Where y is a binary label, The output is the label probability, and N is the total number; The second part is to reuse the feature point loss in the regression task. and , the formula is as follows: ； In the formula, , is the score associated with the distribution predicted by the model, y represents the true value, , Represents two adjacent values ​​obtained after discretization of the true value; ; Where A and B are the true frame and verification frame respectively.

Images