A pedestrian attribute recognition method based on deep learning

Abstract

The application relates to a pedestrian attribute recognition method based on deep learning and belongs to the technical field of pedestrian recognition. The recognition method comprises the following steps: constructing a pedestrian attribute recognition network; the pedestrian attribute recognition network comprises a backbone network, a first pooling layer and a full connection layer, and an attention module is introduced between the backbone network and the first pooling layer; training the pedestrian attribute recognition network; and performing pedestrian attribute recognition according to the trained pedestrian attribute recognition network. The pedestrian attribute recognition network for performing pedestrian attribute recognition introduces an attention module between the backbone network and the first pooling layer, the importance of high-level semantic information is distinguished through the attention module, and then the high-level semantic information is purposefully selected and processed by the first pooling layer, the global features corresponding to the high-level semantic information are emphasized, and the recognition accuracy is improved. The pedestrian attribute recognition network structure is simple, the complexity of the network structure is avoided from being greatly increased, and the recognition efficiency is improved.

Description

TECHNICAL FIELD

[0001] The application relates to a pedestrian attribute recognition method based on deep learning and belongs to the technical field of pedestrian recognition. BACKGROUND

[0002] With the development of computer technology and the Internet, video monitoring systems have become more intelligent and efficient. In a monitoring scene, cameras are usually installed at a high place to obtain a wide field of view, resulting in that pedestrians are presented in images at a low resolution. In the case of missing detailed information such as faces, the pedestrian attribute features and high-level semantic information can be obtained according to the full-body image of a pedestrian at a long distance, so that personnel retrieval, re-identification and other tasks can be implemented.

[0003] The classical pedestrian attribute recognition is a method using a multi-label classification task, and an end-to-end attribute recognition network is designed. The task of the network is to predict a 1x n-dimensional attribute vector given a pedestrian image. The vector corresponds to a pre-defined attribute name list AN={an1, an2, an3,..ann} in sequence to describe the pedestrian features.

[0004] With the development of technology, Deep Learning Based Multiple Attributes Recognition (DeepMAR) first takes pedestrian attribute recognition as a multi-label classification task. The algorithm fully considers the relationship between attributes, such as the length of hair and gender, and designs a weighted cross-entropy loss to alleviate the imbalance of attribute samples. Compared with the classical pedestrian attribute recognition, better recognition accuracy is achieved.

[0005] However, pedestrian attribute recognition is not the same as common image classification. Generally, dozens of attributes of pedestrians need to be analyzed and recognized, and the picture sources are complex and changeable: the posture of pedestrians is variable, the body of pedestrians is blocked or missing, the light changes, the resolution is low, and various different pictures result in low recognition accuracy. Therefore, someone proposes to introduce an attention mechanism into the recognition network. The attention mechanism in the neural network originates from the visual attention mechanism of human beings, and based on a multi-directional and attention-based network (HP-Net), multi-layer attention feature maps are mapped to different feature layers to provide more optional feature expressions for fine-grained pedestrian analysis. The ALM algorithm independently uses an attribute localization module (ALM) for each type of attribute, learns the local area corresponding to the attribute in a weakly supervised manner, and then judges the attribute according to the located area.

[0006] In recent years, some of the algorithms proposed by researchers increase a large number of attention modules, increasing the complexity of the network structure. For example: a master's degree thesis of the University of Electronic Science and Technology, the author is Shi Fangyan, the thesis discloses a human detection and appearance attribute recognition integrated algorithm research, proposes a new pedestrian attribute recognition network, introduces the attention mechanism into the residual module of the backbone network ResNet-50, and achieves high recognition accuracy. However, the residual module of ResNet-50 network is composed of two 1x1 convolution and one 3x3 convolution, including 4 large residual module groups and 16 small standard residual modules, and the attention module is added in each residual module, so that the ResNet-50 network increases a large number of attention modules, increases the complexity of the network structure, and reduces the recognition efficiency. SUMMARY

[0007] The purpose of the present application is to provide a pedestrian attribute recognition method based on deep learning, to solve the problem of complex recognition network structure and low efficiency in the existing recognition method.

[0008] To achieve the above purpose, the present application provides a technical scheme of a pedestrian attribute recognition method based on deep learning, including the following steps:

[0009] 1) Construct a pedestrian attribute recognition network; the pedestrian attribute recognition network includes a backbone network, a first pooling layer, and a full connection layer, and an attention module is introduced between the backbone network and the first pooling layer; the backbone network is used to receive picture information and output high-level semantic features of the picture information; the attention module is used to distinguish the importance of the high-level semantic features; the first pooling layer is used to select and process the high-level semantic features output by the backbone network and the high-level semantic features after importance distinction;

[0010] 2) Train the pedestrian attribute recognition network;

[0011] 3) Recognize the pedestrian attribute according to the trained pedestrian attribute recognition network.

[0012] The technical scheme of the pedestrian attribute recognition method based on deep learning has the beneficial effects that: the pedestrian attribute recognition network for pedestrian attribute recognition in the present application introduces an attention module between the backbone network and the first pooling layer, distinguishes the importance of high-level semantic information through the attention module, and then selects and processes the high-level semantic information by the first pooling layer, pays attention to the global features corresponding to the high-level semantic information, and improves the recognition accuracy. The pedestrian attribute recognition network structure of the present application is simple, avoids large-scale increase of network structure complexity, and improves the recognition efficiency.

[0013] Further, the attention module sequentially includes a second pooling layer, a first convolution layer, a first activation layer, a second convolution layer, and a second activation layer in the order from input to output.

[0014] Further, the first pooling layer and the second pooling layer are both Global pool.

[0015] Further, the first convolution layer is 1x1 convolution.

[0016] Further, the second convolution layer is 1x1 convolution.

[0017] Further, the activation function adopted by the first activation layer is ReLU.

[0018] Further, the activation function adopted by the second activation layer is sigmoid.

[0019] Further, the backbone network is ResNet50.

[0020] Further, in order to suppress the class difference caused by data imbalance, balance each task in training, the loss function Loss used in the step 2) of training the pedestrian attribute recognition network is:

[0021]

[0022]

[0023] wherein N is the number of pedestrian attributes; w i is a weight parameter; y i is the attribute label corresponding to the picture in the training set; σ out-i is the probability of the i-th attribute feedforward; a i is the proportion of the i-th attribute positive sample in the training set, which is the weight part of the class imbalance. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is the structural schematic diagram of the pedestrian attribute recognition network of the present application;

[0025] Figure 2 is the recognition comparison diagram of the pedestrian attribute recognition method of the present application and the DeepMAR algorithm. DETAILED DESCRIPTION

[0026] Pedestrian attribute recognition method based on deep learning embodiment:

[0027] The main idea of the present application is that, based on the problem of increasing the complexity of the network structure in the prior art by adding a large number of attention modules in the backbone network, the attention module is added between the backbone network and the pooling layer, which can distinguish the importance of high-level semantic information so as to be selected and processed by the pooling layer purposefully, not only paying attention to the global features corresponding to the high-level semantic information, but also simplifying the network structure.

[0028] Specifically, the pedestrian attribute recognition method based on deep learning comprises the following steps:

[0029] 1) Construct a pedestrian attribute recognition network.

[0030] The pedestrian attribute recognition network constructed by the present application, as shown in the figure, comprises a backbone network ResNet50, a first pooling layer Global pool, a fully connected layer FC, and an attention module introduced between the backbone network ResNet50 and the first pooling layer Global pool. Figure 1

[0031] The attention module is from the ALM algorithm, and the attention module comprises a second pooling layer Global pool, a first convolution layer 1x1 convolution, a first activation layer ReLU, a second convolution layer 1x1 convolution and a second activation layer sigmoid in sequence from input to output.

[0032] The backbone network ResNet50 is used for receiving picture information (InputImg), extracting and outputting high-level semantic features of the picture information; the attention module distinguishes the importance of the high-level semantic features in the form of channel weighting; the first pooling layer Global pool is used for selecting and processing the high-level semantic features output by the backbone network and the high-level semantic features after importance distinction, and plays a role of fusing features; and the fully connected layer FC is used for integrating and classifying the selected features to obtain various attributes: Att0, Att1, Att2, …, Attn.

[0033] As other embodiments, the attention module can also adopt the attention module of the SENet, SKNet and other algorithms, wherein the difference between the attention module based on the SENet algorithm and the attention module based on the ALM algorithm is that the convolution layer in the attention module based on the ALM algorithm is a fully connected layer in the attention module based on the SENet algorithm.

[0034] 2) Train the pedestrian attribute recognition network.

[0035] ​In multi-task learning, the global loss function is linearly related to a single task, and different attribute tasks have different contributions to the loss function due to their different scales, data quantities, and locations. The linear superposition method is often used to calculate the loss of the global task. The existing pedestrian attribute dataset has the problem of imbalance between attributes, which often leads the model to focus on the class with relatively more samples, reducing the generalization ability of the model on the test data. This problem greatly affects the overall recognition effect of the model. The solution is usually to assign different weights to the loss of different attributes in the loss function according to the proportion of positive samples of the attribute. This method, which increases the penalty weight of small sample misclassification for all samples of a certain class and reflects it in the loss function, has been widely used in attribute recognition and has achieved good results.

[0036] In addition, due to the difference in sample quantity and scale size, there are difficult and easy training samples. To improve the overall recognition accuracy, the model should pay more attention to difficult classification samples and assign higher weights to them. Focal Loss loss function considers the imbalance of sample quantity and the difference in learning difficulty to handle the imbalance problem of multi-task categories. For the weighting of training difficulty, Focal Loss believes that the loss weight of each task is negatively related to the probability of the task, which can balance the convergence speed of each task during training, so that each task can be fully trained to achieve better recognition accuracy.

[0037] In the training of the pedestrian attribute recognition network, in order to suppress the inter-class difference caused by data imbalance, the training difficulty is introduced into the weight based on the Focal Loss loss function idea, and the loss function Loss used is:

[0038]

[0039]

[0040] wherein N is the number of pedestrian attributes; w i is the overall weight parameter, including the class imbalance weight and the weight of learning difficulty; y i is the attribute label corresponding to the picture in the training set; sigma out-i is the probability of the i-th attribute feedforward; a i is the proportion of the i-th attribute positive sample in the training set, which belongs to the weight part of class quantity imbalance.

[0041] 3) Recognize the pedestrian attribute according to the trained pedestrian attribute recognition network.

[0042] The accuracy of the pedestrian attribute recognition method of the present application is verified by specific data as follows.

[0043] The data set used in the experiments of the present application is PETA and PA100k. The PETA data set contains a total of 19,000 pictures, annotates 35 pedestrian attributes, and the resolution varies from 17x39 to 169x365. The data is divided into three parts, the training set contains 9,500 images, the verification set contains 1,900 images, and the test set contains 7,600 images. The PA100k data set contains a total of 100,000 pictures, which is the largest pedestrian attribute data set at present, annotates 26 pedestrian attributes, and the resolution varies from 50x100 to 754x454, and is divided into training set, verification set and test set in the ratio of 8:1:1.

[0044] In order to verify the performance of the algorithm in this paper, the DeepMAR algorithm and the ALM algorithm in the prior art are used as the control group on the basis of the above two data sets, and five indexes of average accuracy mA, accuracy Accu, precision Prec, recall Recall and F1 value are listed, and the comparison results are shown in Table 1.

[0045] Table 1 Performance comparison of different methods on PETA data set and PA100k data set (%)

[0046]

[0047] As can be seen from Table 1, the present method exceeds the DeepMAR and ALM methods in the two indexes of mA and Accu, wherein mA is the average value calculated after the accuracy of each attribute is calculated, which is a key evaluation index of attribute recognition. It can be seen that the algorithm proposed in this paper is better than the above two algorithms.

[0048] By using the recognition method of the present application and the DeepMAR algorithm in the prior art to recognize the image, the recognition comparison chart as shown in Figure 2 is obtained, Figure 2 in which the performance of the DeepMAR algorithm before and after adding the channel attention module in the PETA data set is shown, and in order to verify the algorithm of the present application and highlight the performance of the attention mechanism, the accuracy of recognition of 20 macro attributes is listed. Through calculation, the average accuracy mA of the algorithm with increased channel attention on these macro attributes is improved by 0.51%.

[0049] Finally, 1,000 images in the PETA data set are selected for inference on a test platform with an RTX2070 graphics card. In this case, time is used as an index to evaluate the performance of the algorithm, and the average inference time per image is taken as the final result. The inference time of the recognition method proposed in the present application and the DeepMAR algorithm is 18.61ms and 18.25ms respectively, so the inference time of the algorithm in this paper does not increase significantly, and it has application value.

[0050] The network structure of the application improves the DeepMAR algorithm framework, introduces an attention module Attention between the backbone network ResNet50 and the first pooling layer Global pool, and when performing pedestrian attribute recognition, the recognition accuracy is improved, and the recognition rate is not obviously increased.

Claims

1.A pedestrian attribute recognition method based on deep learning, characterized in that, The method comprises the following steps: 1) constructing a pedestrian attribute recognition network; the pedestrian attribute recognition network adopts a DeepMAR algorithm framework, comprises a backbone network, a first pooling layer and a full connection layer, and an attention module is introduced between the backbone network and the first pooling layer; the backbone network is used for receiving picture information and outputting high-level semantic features of the picture information; the attention module is used for distinguishing the importance of the high-level semantic features; the first pooling layer is used for selecting and processing the high-level semantic features output by the backbone network and the high-level semantic features after the importance distinction; a loss function Loss of the pedestrian attribute recognition network is as follows: where N is the number of pedestrian attributes; w i is a weight parameter; y i is the attribute label corresponding to the picture in the training set; σ out-i is the probability of the i-th attribute feedforward; a i is the proportion of the i-th attribute positive sample in the training set, which is the weight part of the class imbalance. 2) training the pedestrian attribute recognition network; 3) identifying pedestrian attributes according to the trained pedestrian attribute recognition network. 2.The deep learning-based pedestrian attribute recognition method of claim 1, wherein, The attention module comprises a second pooling layer, a first convolution layer, a first activation layer, a second convolution layer and a second activation layer in turn from input to output. 3.The pedestrian attribute recognition method based on deep learning according to claim 2, characterized in that, Both the first pooling layer and the second pooling layer are Global pool. 4.The pedestrian attribute recognition method based on deep learning according to claim 2, characterized in that, The first convolution layer is 1x1 convolution. 5.The pedestrian attribute recognition method based on deep learning according to claim 2, characterized in that, The second convolution layer is 1x1 convolution. 6.The pedestrian attribute recognition method based on deep learning according to claim 2, characterized in that, The activation function adopted by the first activation layer is ReLU. 7.The pedestrian attribute recognition method based on deep learning according to claim 2, characterized in that, The activation function adopted by the second activation layer is sigmoid. 8.The pedestrian attribute recognition method based on deep learning according to claim 1, characterized in that, The backbone network is ResNet50.

Images