A method and device for identifying pedestrian attributes

By dividing the feature map of the pedestrian attribute recognition picture into multiple feature blocks and using spatial position relationship and attention mechanism, combined with Resnet-50 feature extraction and loss function training, the problem of low pedestrian attribute recognition accuracy in the existing technology is solved. Accurate recognition of global and local features is achieved, significantly improving the recognition accuracy.

CN115147860BActive Publication Date: 2024-11-12CHINA MOBILE COMM LTD RES INST +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110338880.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-30
Publication Date
2024-11-12
Estimated Expiration
2041-03-30

AI Technical Summary

Technical Problem

The existing pedestrian attribute recognition methods have low recognition accuracy, especially in the recognition of local attributes such as hats, glasses and hair colors, which are difficult to achieve accurate recognition.

Method used

By dividing the feature map of the picture to be identified into multiple feature blocks, using the spatial position relationship and attention mechanism, we automatically learn the relationship between the spatial position information of the attributes and the feature blocks, and identify them in combination with the initial recognition results and the probability of existence. Resnet-50 performs feature extraction and trains the model based on the loss function to improve recognition accuracy.

Benefits of technology

实现了对行人属性特征的全局和局部特征的精确提取与识别,提高了行人属性识别的精度,特别是在细粒度属性的学习上显著提升了识别效果。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115147860B_ABST
    Figure CN115147860B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for pedestrian attribute recognition, which belongs to the field of pattern recognition technology. The method comprises: respectively obtaining the initial recognition results of n attributes in each feature block, and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship; the feature block is obtained by dividing the feature map of the image to be recognized into multiple areas; according to the initial recognition result and the existence probability of each attribute in each feature block, determining the recognition result of each attribute in the image to be recognized. The present application divides the global features of the attributes into blocks, utilizes the spatial position relationship between the feature blocks, and assists in the recognition of the attributes, so as to not only accurately extract and recognize the global features in the pedestrian attribute features, but also accurately extract and recognize the local features in the pedestrian attribute features, thereby realizing the refined recognition of pedestrian attributes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of pattern recognition technology, and in particular to a method and device for pedestrian attribute recognition. Background Art

[0002] Pedestrian attribute recognition is used to identify local feature information of pedestrian attributes in images (such as whether they are wearing hats or glasses). It is useful in many real-life scenarios, such as pedestrian tracking and pedestrian re-identification (it is necessary to determine whether pedestrians photographed at different angles, in different lighting conditions, and in different postures are the same person). However, the current pedestrian attribute recognition methods have limited recognition accuracy, especially for local attributes such as hats, glasses, and hair color, which cannot be accurately recognized. Summary of the invention

[0003] In view of this, the present application provides a pedestrian attribute recognition method and device, which are used to solve the problem of low recognition accuracy of the current pedestrian attribute recognition method.

[0004] In order to solve the above technical problems, in a first aspect, the present application provides a method for pedestrian attribute recognition, comprising:

[0005] Obtaining initial recognition results of n attributes in each feature block respectively, and obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship respectively; the feature block is obtained by dividing the feature map of the image to be identified into multiple areas;

[0006] According to the initial recognition result and the existence probability of each attribute in each of the feature blocks, a recognition result of each attribute in the to-be-recognized picture is determined.

[0007] Optionally, determining the recognition result of each attribute in the to-be-recognized picture according to the initial recognition result and the existence probability of each attribute in each feature block includes:

[0008] For each of the attributes in each of the feature blocks, multiply the initial recognition result by the existence probability to obtain a recognition result for each of the attributes in each of the feature blocks;

[0009] For each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks is determined as the recognition result of the attribute in the picture to be recognized.

[0010] Optionally, before respectively obtaining the initial recognition results of the n attributes in each feature block and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship, the method further includes:

[0011] The basic network Resnet-50 with the last fully connected layer removed is used to extract features of the image to be identified, thereby obtaining a feature map of the image to be identified.

[0012] Optionally, the obtaining of initial recognition results of n attributes in each feature block respectively includes:

[0013] For each of the feature blocks, a first operation is performed to obtain first feature information; the first operation includes a pooling operation;

[0014] The initial recognition results of the n attributes are obtained according to the first feature information.

[0015] Optionally, the first operation also includes a convolution operation.

[0016] Optionally, performing a first operation on each of the feature blocks to obtain first feature information includes:

[0017] For each of the feature blocks, a pooling operation is performed according to the following formula:

[0018]

[0019] Wherein, O is the feature information obtained by pooling the feature block, h and w are the pixel values ​​of the height and width of the feature block, and f xy is the characteristic value of the pixel point at the height x and width y in the feature block.

[0020] Optionally, respectively obtaining initial recognition results of n attributes in each feature block, and respectively obtaining existence probabilities of each attribute in each feature block determined based on spatial position relationships, includes:

[0021] Based on the loss function, a pedestrian attribute recognition model is trained, wherein each attribute in the loss function corresponds to a weight value, and the weight value corresponding to each attribute is determined according to the number of positive examples and / or negative examples of the attribute in the training sample;

[0022] The pedestrian attribute recognition model is used to obtain initial recognition results of the n attributes in each feature block, and the existence probability of each attribute in each feature block determined based on the spatial position relationship is obtained.

[0023] Optionally, the loss function is:

[0024]

[0025] Where Q is the number of training samples, n is the number of attributes to be identified, and l ks is the true label of the sth attribute of the kth sample, ωs is the weight value of the sth attribute, m ks The probability that the sth attribute of the kth sample is predicted to be a positive example.

[0026] Optionally, the weight value corresponding to each attribute in the loss function is determined according to the following formula:

[0027]

[0028] Among them, ω s is the weight value of the sth attribute, l s is the true label of the sth attribute, p s is the ratio of negative examples to positive examples of the sth attribute in the training sample.

[0029] Optionally, the training of the pedestrian attribute recognition model based on the loss function includes:

[0030] Calculating the loss value of each feature block recognition result respectively;

[0031] The sum of the loss values ​​of each feature block recognition result is added together to obtain the final loss value.

[0032] In a second aspect, the present application further provides a pedestrian attribute recognition device, comprising:

[0033] The first recognition module is used to obtain the initial recognition results of n attributes in each feature block, and obtain the existence probability of each attribute in each feature block determined based on the spatial position relationship; the feature block is obtained by dividing the feature map of the image to be recognized into multiple areas;

[0034] The second recognition module is used to determine the recognition result of each attribute in the to-be-recognized picture according to the initial recognition result and the existence probability of each attribute in each feature block.

[0035] Optionally, the second identification module includes:

[0036] A first calculation unit, configured to multiply the initial recognition result by the existence probability for each attribute in each feature block to obtain a recognition result for each attribute in each feature block;

[0037] The second calculation unit is used to determine, for each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks as the recognition result of the attribute in the to-be-recognized picture.

[0038] Optionally, the pedestrian attribute recognition device further includes:

[0039] The feature extraction module is used to extract features of the image to be identified by using the basic network Resnet-50 with the last fully connected layer removed, so as to obtain a feature map of the image to be identified.

[0040] Optionally, the first identification module includes:

[0041] A feature processing unit, configured to perform a first operation on each of the feature blocks to obtain first feature information; the first operation includes a pooling operation;

[0042] A classification unit is used to obtain the initial recognition results of the n attributes according to the first feature information.

[0043] Optionally, the first operation also includes a convolution operation.

[0044] Optionally, the feature processing unit is used to perform a pooling operation on each feature block according to the following formula:

[0045]

[0046] Wherein, O is the feature information obtained by pooling the feature block, h and w are the pixel values ​​of the height and width of the feature block, and f xy is the characteristic value of the pixel point at the height x and width y in the feature block.

[0047] Optionally, the pedestrian attribute recognition device further includes:

[0048] A model training module, used for training a pedestrian attribute recognition model based on a loss function, wherein each attribute in the loss function corresponds to a weight value, and the weight value corresponding to each attribute is determined according to the number of positive examples and / or negative examples of the attribute in the training sample;

[0049] The first recognition module is used to use the pedestrian attribute recognition model to obtain initial recognition results of n attributes in each feature block, and to obtain the existence probability of each attribute in each feature block determined based on the spatial position relationship.

[0050] Optionally, the loss function is:

[0051]

[0052] Where Q is the number of training samples, n is the number of attributes to be identified, and l ks is the true label of the sth attribute of the kth sample, ω s is the weight value of the sth attribute, m ks The probability that the sth attribute of the kth sample is predicted to be a positive example.

[0053] Optionally, the weight value corresponding to each attribute in the loss function is determined according to the following formula:

[0054]

[0055] Among them, ω s is the weight value of the sth attribute, l s is the true label of the sth attribute, p s is the ratio of negative examples to positive examples of the sth attribute in the training sample.

[0056] Optionally, the model training module includes:

[0057] A first loss value calculation unit, used to calculate the loss value of each feature block recognition result respectively;

[0058] The second loss value calculation unit is used to add the loss values ​​of each feature block recognition result to obtain a sum as the final loss value.

[0059] In a third aspect, the present application also provides a pedestrian attribute recognition device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the steps in any one of the above-mentioned pedestrian attribute recognition methods are implemented.

[0060] In a fourth aspect, the present application also provides a readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in any of the above-mentioned pedestrian attribute recognition methods.

[0061] The beneficial effects of the above technical solution of the present application are as follows:

[0062] In the embodiment of the present application, by dividing the global features of the attributes into blocks and utilizing the spatial position relationship between the feature blocks to assist in the identification of the attributes, not only the global features of the pedestrian attribute features (such as gender, age, etc.) can be accurately extracted and identified, but also the local features of the pedestrian attribute features (such as hats, glasses, and hair color, etc.) can be accurately extracted and identified, thereby achieving refined identification of pedestrian attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of a pedestrian attribute recognition method in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of the framework structure of a method for identifying pedestrian attributes based on spatial position relationship in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of the structure of a pedestrian attribute recognition device in an embodiment of the present application;

[0066] Figure 4 Schematic diagram of the structure of a pedestrian attribute recognition device in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme of the embodiment of the present application will be clearly and completely described below in conjunction with the drawings of the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all of the embodiments. Based on the described embodiment of the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0068] See also Figure 1 , Figure 1 A flowchart of a pedestrian attribute recognition method provided in Example 1 of the present application includes the following steps:

[0069] Step 11: respectively obtain the initial recognition results of n attributes in each feature block, and respectively obtain the existence probability of each attribute in each feature block determined based on the spatial position relationship; the feature block is obtained by dividing the feature map of the image to be recognized into multiple regions;

[0070] When dividing the image to be identified, for example, it can be divided into multiple areas from top to bottom, and it can be divided manually, and each feature block is equal or substantially equal in size. In other words, the feature blocks are divided evenly or substantially evenly. n is a positive integer.

[0071] Optionally, the pedestrian attribute recognition model can be used to obtain the initial recognition results of n attributes in each feature block, and the existence probability of each attribute in each feature block determined based on the spatial position relationship can be obtained. Figure 2 , the pedestrian attribute recognition model includes an initial attribute branch 21 and an attention mechanism module 22 based on spatial position relationship, the initial attribute branch 21 is used to classify n attributes in multiple feature blocks (Ψ(I1), Ψ(I2), Ψ(I3)...), and multiple feature blocks (Ψ(I1), Ψ(I2), Ψ(I3)...) will obtain initial recognition results of n attributes. The attention mechanism module 22 based on spatial position relationship is used to determine the existence probability of each attribute in each feature block (Ψ(I1), Ψ(I2), Ψ(I3)...), respectively. Specifically, in the embodiment of the present application, the attention mechanism is used to automatically learn the spatial position information of the attribute and the spatial position relationship between the feature blocks, and learn the importance of different feature blocks for different attributes, and obtain the weight of the spatial position relationship to better identify each attribute.

[0072] Step 12: Determine the recognition result of each attribute in the image to be recognized based on the initial recognition result and the existence probability of each attribute in each feature block. The recognition result of each attribute in the image to be recognized is the probability score of the image to be recognized having this attribute. A threshold can be set. If the probability score is greater than the threshold, the image to be recognized has this attribute, otherwise it does not have this attribute.

[0073] Most of the current pedestrian attribute recognition methods first divide the image evenly, then extract features for each divided image area, and then perform subsequent multi-classification to obtain the final pedestrian attribute label set. In the task of pedestrian attribute recognition, the label only has information about whether the corresponding attribute exists, and lacks the spatial position information of the attribute. However, in the actual application of pedestrian attribute recognition, pedestrian attribute features include global features (such as gender, age, etc.) and local features (such as hats, hair color and glasses, etc.). A network cannot simultaneously recognize these fine-grained and coarse-grained attributes well. For example, attributes such as glasses often have poor recognition effects due to their small number of samples and small feature scales. The pedestrian attribute recognition method provided in the embodiment of the present application further explores and utilizes the spatial position relationship between attributes on the basis of multi-label classification. For example, glasses are usually located on the head of pedestrians, that is, the features of glasses exist in the upper part of the pedestrian features. Therefore, when identifying the attribute of glasses, enhancing the utilization of the feature blocks in the upper part can improve the recognition effect. Specifically, the embodiment of the present application utilizes the attention mechanism to automatically learn a weight vector for the pedestrian attribute label set for each divided feature block, which represents the possibility of all pedestrian attributes appearing in the image area corresponding to the feature block (for example, glasses are more likely to appear in the upper part of the image), thereby more deeply exploring the spatial position relationship between pedestrian attributes and feature representation, and then effectively solving the learning of fine-grained attributes in pedestrian attributes and improving recognition accuracy.

[0074] The pedestrian attribute recognition method provided in the embodiment of the present application divides the global features of the attributes into blocks and utilizes the spatial position relationship between the feature blocks to assist in the recognition of the attributes. It can not only accurately extract and recognize the global features of the pedestrian attribute features (such as gender, age, etc.), but also accurately extract and recognize the local features of the pedestrian attribute features (such as hats, glasses, and hair color, etc.), thereby realizing refined recognition of pedestrian attributes.

[0075] Compared with the solution of automatically learning pedestrian attribute features by improving the deep learning model framework and then inputting the attribute features into the classifier to obtain the attribute category of the pedestrian sample, the pedestrian attribute recognition method provided in the embodiment of the present application has stronger feature extraction performance and can greatly improve the probability of attribute recognition.

[0076] The following example illustrates the above pedestrian attribute recognition method.

[0077] In an embodiment of the present application, the global features are divided into blocks through feature blocks to obtain feature blocks at different spatial positions. The spatial position relationship of each attribute is explored and learned in combination with the spatial position information of the feature blocks. The relationship between each attribute and the spatial position is converted into the relationship between the attribute and the feature block. The different attributes of different feature blocks are weighted using the attention mechanism, and the utilization of feature blocks at important positions is strengthened, so that the recognition results can be more efficiently fused and the accuracy of attribute recognition can be improved.

[0078] Optionally, the spatial position relationship-based attention mechanism module may include different multiple (eg, two) fully connected layers to better learn the spatial features of each attribute, thereby learning each attribute more accurately.

[0079] Optionally, determining the recognition result of each attribute in the to-be-recognized picture according to the initial recognition result and the existence probability of each attribute in each feature block includes:

[0080] For each of the attributes in each of the feature blocks, multiply the initial recognition result by the existence probability to obtain a recognition result for each of the attributes in each of the feature blocks;

[0081] For each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks is determined as the recognition result of the attribute in the picture to be recognized.

[0082] For example, when the feature map Ψ(I) of the image to be identified is divided from top to bottom into three feature blocks Ψ(I1), Ψ(I2) and Ψ(I3), in the initial attribute recognition branch, the three feature blocks will obtain initial recognition results of n attributes. In the attention mechanism module based on spatial position relationship, the three feature blocks are respectively input into different fully connected layers, and three different groups of weights (w11, w21, ..., wn1), (w12, w22, ..., wn2), (w13, w23, ..., wn3) are automatically learned. Each weight in each group corresponds to the existence probability of different attributes in the feature block. That is, in the weights (w11, w21, ..., wn1) learned by the feature block Ψ(I1), w11 represents the probability that the feature of attribute 1 exists in the feature block Ψ(I1), w21 represents the probability that the feature of attribute 2 exists in the feature block Ψ(I1), and wn1 represents the probability that the feature of attribute n exists in the feature block Ψ(I1). Among the weights (w12, w22, …, wn2) learned by the feature block Ψ(I2), w12 represents the probability that the feature of attribute 1 exists in the feature block Ψ(I2), w22 represents the probability that the feature of attribute 2 exists in the feature block Ψ(I2), and wn2 represents the probability that the feature of attribute n exists in the feature block Ψ(I2). Then, the output result of the initial attribute recognition branch and the attention mechanism module based on spatial position relationship are fused, that is, the weights learned in the attention mechanism module based on spatial position relationship are multiplied with the initial recognition result of the attribute of the corresponding feature block in the initial attribute recognition branch as the recognition result of each pedestrian attribute of the feature block. In other words, for the feature block Ψ(I1), the initial attribute recognition branch learns the preliminary attribute recognition results (Attribute_11, Attribute_21, …, Attribute_n1); the attention mechanism module based on the spatial position relationship learns the weights (w11, w21, …, wn1), and the final recognition result learned by the feature block is (w11*Attribute_11, w21*Attribute_21, …, wn1*Attribute_n1). The feature blocks Ψ(I2) and Ψ(I3) are the same as Ψ(I1), and they will all get a set of weighted recognition results. The maximum value of these three sets of recognition results is taken as the final output result of the network, that is, (max(w11*Attribute_11,w12*Attribute_12,w13*Attribute_13),max(w21*Attribute_21,w22*Attribute_22,w23*Attribute_23)…,max(wn1*Attribute_n1,wn2*Attribute_n2,wn3*Attribute_n3)).

[0083] In an embodiment of the present application, weights are automatically learned through an attention mechanism based on spatial position relationships and integrated into the entire network framework, so that the model can pay more attention to the information of the correspondence between pedestrian attributes and spatial positions, thereby greatly improving the recognition accuracy of the model.

[0084] Optionally, before respectively obtaining the initial recognition results of the n attributes in each feature block and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship, the method further includes:

[0085] The basic network Resnet-50 with the last fully connected layer removed is used to extract features of the image to be identified, thereby obtaining a feature map of the image to be identified.

[0086] The features of an image can be represented by x=[N,C,H,W], where N is the current batch size of the image, C is the number of channels of the image, and H and W are the pixel values ​​of the image.

[0087] In the embodiment of the present application, the feature map of the image to be identified can be recorded as Ψ(I). When the feature map Ψ(I) is divided into feature blocks, it can be divided into three feature blocks Ψ(I1), Ψ(I2) and Ψ(I3) from top to bottom.

[0088] Optionally, the basic network Resnet-50 is a part of the pedestrian attribute recognition model, that is, the pedestrian attribute recognition model includes the basic network Resnet-50.

[0089] In an embodiment of the present application, a lower-level network that shares all attributes before branching learns visual attribute features (i.e., feature maps of images), and then is divided into multiple feature blocks. Each feature block uses deep convolution to learn high-level features of the attributes, and weights the preliminary recognition results (i.e., initial recognition results) according to the weights automatically learned by the network.

[0090] Optionally, the obtaining of initial recognition results of n attributes in each feature block respectively includes:

[0091] For each of the feature blocks, a first operation is performed to obtain first feature information; the first operation includes a pooling operation; the pooling operation may be, for example, an average pooling operation, and may further be a global average pooling operation;

[0092] The initial recognition results of the n attributes are obtained according to the first feature information. For example, the first feature information of each feature block may be input into a fully connected layer to obtain the initial recognition result.

[0093] In an embodiment of the present application, a pooling layer is used to replace the traditional fully connected layer in the feature extraction network (i.e., the basic network Resnet-50), which can reduce the number of parameters and prevent overfitting.

[0094] Further optionally, the first operation further includes a convolution operation. The convolution operation may be, for example, a 1×1 convolution operation. In the embodiment of the present application, after performing a pooling operation on each feature block, a convolution operation may be performed to better represent the extracted features.

[0095] In the embodiment of the present application, in the initial attribute branch, each feature block is subjected to pooling and convolution operations, and n attributes are classified respectively according to the obtained features, and each feature block will obtain initial recognition results of n attributes.

[0096] When classifying attributes, the softmax function can be used. The softmax function can be found in the following formula:

[0097]

[0098] Among them, θ is the parameter to be determined. For each output, the probabilities of all categories are output, and the one with the highest probability is assigned to that category.

[0099] Optionally, performing a first operation on each of the feature blocks to obtain first feature information includes:

[0100] For each of the feature blocks, a pooling operation is performed according to the following formula:

[0101]

[0102] Wherein, O is the feature information obtained by pooling the feature block, h and w are the pixel values ​​of the height and width of the feature block, and f xy is the characteristic value of the pixel point at the height x and width y in the feature block.

[0103] In the embodiment of the present application, a 2D image can be converted into a numerical value through an average pooling operation.

[0104] Optionally, respectively obtaining initial recognition results of n attributes in each feature block, and respectively obtaining existence probabilities of each attribute in each feature block determined based on spatial position relationships, includes:

[0105] Based on the loss function, a pedestrian attribute recognition model is trained, wherein each attribute in the loss function corresponds to a weight value, and the weight value corresponding to each attribute is determined according to the number of positive examples and / or negative examples of the attribute in the training sample;

[0106] The pedestrian attribute recognition model is used to obtain initial recognition results of the n attributes in each feature block, and the existence probability of each attribute in each feature block determined based on the spatial position relationship is obtained.

[0107] In the embodiment of the present application, the weight value corresponding to each attribute is used to solve the problem of uneven distribution of sample attributes (i.e., there are a large number of small samples (such as samples labeled as eyes)) and improve the recognition accuracy.

[0108] Optionally, the loss function is:

[0109]

[0110] Where Q is the number of training samples, n is the number of attributes to be identified, and l ks is the true label of the sth attribute of the kth sample, ω s is the weight value of the sth attribute, m ks The probability that the sth attribute of the kth sample is predicted to be a positive example.

[0111] Optionally, the weight value corresponding to each attribute in the loss function is determined according to the following formula:

[0112]

[0113] Among them, ω s is the weight value of the sth attribute, l s is the true label of the sth attribute, p s is the ratio of negative examples to positive examples of the sth attribute in the training sample.

[0114] Optionally, the training of the pedestrian attribute recognition model based on the loss function includes:

[0115] Calculating the loss value of each feature block recognition result respectively;

[0116] The sum of the loss values ​​of each feature block recognition result is added together to obtain the final loss value (or total loss value).

[0117] In the embodiment of the present application, although the final output result of the network is the maximum value of the recognition results of multiple feature blocks, a loss function is required for the recognition result of each feature block (the recognition result of each feature block is calculated according to the formula of the above loss function), that is, the loss between the recognition results of multiple feature blocks and the true annotation is calculated separately. Compared with the loss value of only taking the maximum value of the recognition results of multiple feature blocks, the convergence can be faster, thereby allowing the network to converge faster and better.

[0118] In addition, the pedestrian attribute recognition module trained in the embodiment of the present application also includes an attention mechanism module based on spatial position relationship, and its convergence effect is better than that of the model that only includes the initial attribute recognition module.

[0119] The following example illustrates the training process of the pedestrian attribute recognition model.

[0120] First, the data set is divided into a training set (train), a test set (val) during the training process, and a test set (test) used to evaluate the model results after the training model is completed. The original image is preprocessed, including resizing the image, image enhancement (or image enhancement, i.e. random erasing), geometric transformation (translation, flipping, rotation, etc.) and / or random shearing. The images in the data set are labeled, and the label is the value of the corresponding attribute, generally 0 or 1, 0 represents the absence of this attribute, and 1 represents the presence of this attribute. If there is an attribute that is a multi-classification problem, it is processed into multiple binary classification problems, i.e. multiple 0s or 1s.

[0121] Then, the pedestrian attribute recognition model is trained using the pictures in the training set. During the training process, the pedestrian attribute recognition model is tested using the pictures in the val set, and the trained pedestrian attribute recognition model is output.

[0122] During the training process, the features of the labeled images are first extracted. For details, please refer to the feature extraction process in the above attribute recognition process, which will not be repeated here. The features of the image can be represented by x = [N, C, H, W], where N is the current batch size of the image, C is the number of channels of the image, and H and W are the pixel values ​​of the image. When the labeled image is an RGB image, before the feature extraction is performed using the basic network Resnet-50 with the last fully connected layer removed, the value of C is 3 (for the three RGB channels). After feature extraction, the value of C can be 256, for example.

[0123] When training the network model, in order to make the model converge better and learn the weights in the attention mechanism based on the spatial position relationship, it is assumed here that the recognition result X of the feature block is (w11*Attribute_11, w21*Attribute_21, ..., wn1*Attribute_n1). During the training process, in order to make X closer to the true annotation (Ground Truth) in the data set, the parameters of the convolutional layer in the initial attribute recognition branch of this network model will make reasonable changes as the gradient of the network is updated, (Attribute_11, Attribute_21, ..., Attribute_n1) will approach the correct result with the back propagation and forward propagation of the network, and the parameters of the fully connected layer in the attention mechanism module based on the spatial position relationship will also make reasonable changes with the gradient update, and then learn more reasonable weights (w11, w21, ..., wn1), thereby accelerating the recognition result to approach the correct result. In other words, in the embodiment of the present application, the pedestrian attribute recognition model is obtained through multiple iterative training.

[0124] During the training process, each feature block will output a recognition result and take the loss function respectively. Finally, the loss values ​​corresponding to multiple feature blocks are added together as the final loss value, and backpropagation is continuously performed to update the parameters of the network model.

[0125] During the training process, the output of the network model is the probability score of each attribute having a positive sample. A threshold can be set. If it is greater than the threshold, the attribute exists, otherwise it does not exist.

[0126] In the pedestrian attribute recognition model, attributes can be classified by the softmax function. The loss function of the model can be expressed as cross entropy:

[0127] Among them, p represents the true distribution and q represents the predicted distribution of the trained model.

[0128] This network model utilizes label information and does not calculate the similarity between images, which reduces a lot of calculations and speeds up training.

[0129] Finally, the images in the test set are input into the trained pedestrian attribute recognition model for attribute recognition, and the evaluation indicators are used for testing. The test results are output and the results are analyzed.

[0130] Due to sufficient training data, the pedestrian attribute recognition method based on spatial position relationship provided in the embodiment of the present application is much better than the traditional method.

[0131] In the embodiment of the present application, by considering the correspondence between the spatial position of the pedestrian attribute and the feature representation, an attention mechanism based on the spatial position relationship is proposed. Using the attention mechanism and the form of feature segmentation, the correlation between the spatial positions of the attributes is fully explored, and the attributes are learned separately and fused. Feature extraction solves the effective learning of fine-grained attributes and coarse-grained attributes in pedestrian attributes to a certain extent. In addition, the embodiment of the present application also handles the previous class imbalance of each attribute and pays attention to the positive samples that are misclassified. Therefore, the feature extraction of the embodiment of the present application is more accurate and the effect is better.

[0132] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a pedestrian attribute recognition device provided in the second embodiment of the present application. The pedestrian attribute recognition device 30 includes:

[0133] The first recognition module 31 is used to obtain the initial recognition results of n attributes in each feature block, and obtain the existence probability of each attribute in each feature block determined based on the spatial position relationship; the feature block is obtained by dividing the feature map of the image to be recognized into multiple areas;

[0134] The second recognition module 32 is used to determine the recognition result of each attribute in the to-be-recognized picture according to the initial recognition result and the existence probability of each attribute in each feature block.

[0135] Optionally, the second identification module 32 includes:

[0136] A first calculation unit, configured to multiply the initial recognition result by the existence probability for each attribute in each feature block to obtain a recognition result for each attribute in each feature block;

[0137] The second calculation unit is used to determine, for each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks as the recognition result of the attribute in the to-be-recognized picture.

[0138] Optionally, the pedestrian attribute recognition device 30 further includes:

[0139] The feature extraction module is used to extract features of the image to be identified by using the basic network Resnet-50 with the last fully connected layer removed, so as to obtain a feature map of the image to be identified.

[0140] Optionally, the first identification module 31 includes:

[0141] A feature processing unit, configured to perform a first operation on each of the feature blocks to obtain first feature information; the first operation includes a pooling operation;

[0142] A classification unit is used to obtain the initial recognition results of the n attributes according to the first feature information.

[0143] Optionally, the first operation also includes a convolution operation.

[0144] Optionally, the feature processing unit is used to perform a pooling operation on each feature block according to the following formula:

[0145]

[0146] Wherein, O is the feature information obtained by pooling the feature block, h and w are the pixel values ​​of the height and width of the feature block, and f xy is the characteristic value of the pixel point at the height x and width y in the feature block.

[0147] Optionally, the pedestrian attribute recognition device 30 further includes:

[0148] A model training module, used for training a pedestrian attribute recognition model based on a loss function, wherein each attribute in the loss function corresponds to a weight value, and the weight value corresponding to each attribute is determined according to the number of positive examples and / or negative examples of the attribute in the training sample;

[0149] The first recognition module 31 is used to use the pedestrian attribute recognition model to obtain initial recognition results of n attributes in each feature block, and to obtain the existence probability of each attribute in each feature block determined based on the spatial position relationship.

[0150] Optionally, the loss function is:

[0151]

[0152] Where Q is the number of training samples, n is the number of attributes to be identified, and l ks is the true label of the sth attribute of the kth sample, ω s is the weight value of the sth attribute, m ks The probability that the sth attribute of the kth sample is predicted to be a positive example.

[0153] Optionally, the weight value corresponding to each attribute in the loss function is determined according to the following formula:

[0154]

[0155] Among them, ω s is the weight value of the sth attribute, l s is the true label of the sth attribute, p sis the ratio of negative examples to positive examples of the sth attribute in the training sample.

[0156] Optionally, the model training module includes:

[0157] A first loss value calculation unit, used to calculate the loss value of each feature block recognition result respectively;

[0158] The second loss value calculation unit is used to add the loss values ​​of each feature block recognition result to obtain a sum as the final loss value.

[0159] The embodiment of the present application is a product embodiment corresponding to the above-mentioned method embodiment 1, so it will not be repeated here. Please refer to the above-mentioned embodiment 1 for details.

[0160] See also Figure 4 , Figure 4 : is a schematic diagram of the structure of a pedestrian attribute recognition device provided in Embodiment 3 of the present application. The pedestrian attribute recognition device 40 includes a processor 41, a memory 42, and a program stored in the memory 42 and executable on the processor 41; when the processor 41 executes the program, the following steps are implemented:

[0161] Obtaining initial recognition results of n attributes in each feature block respectively, and obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship respectively; the feature block is obtained by dividing the feature map of the image to be identified into multiple areas;

[0162] According to the initial recognition result and the existence probability of each attribute in each of the feature blocks, a recognition result of each attribute in the to-be-recognized picture is determined.

[0163] Optionally, the processor 41 may further implement the following steps when executing the program:

[0164] Determining the recognition result of each attribute in the to-be-recognized picture according to the initial recognition result and the existence probability of each attribute in each feature block includes:

[0165] For each of the attributes in each of the feature blocks, multiply the initial recognition result by the existence probability to obtain a recognition result for each of the attributes in each of the feature blocks;

[0166] For each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks is determined as the recognition result of the attribute in the picture to be recognized.

[0167] Optionally, the processor 41 may further implement the following steps when executing the program:

[0168] Before respectively obtaining the initial recognition results of the n attributes in each feature block and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship, the method further includes:

[0169] The basic network Resnet-50 with the last fully connected layer removed is used to extract features of the image to be identified, thereby obtaining a feature map of the image to be identified.

[0170] Optionally, the processor 41 may further implement the following steps when executing the program:

[0171] The step of respectively obtaining the initial recognition results of the n attributes in each feature block includes:

[0172] For each of the feature blocks, a first operation is performed to obtain first feature information; the first operation includes a pooling operation;

[0173] The initial recognition results of the n attributes are obtained according to the first feature information.

[0174] Optionally, the first operation also includes a convolution operation.

[0175] Optionally, the processor 41 may further implement the following steps when executing the program:

[0176] The step of performing a first operation on each of the feature blocks to obtain first feature information includes:

[0177] For each of the feature blocks, a pooling operation is performed according to the following formula:

[0178]

[0179] Wherein, O is the feature information obtained by pooling the feature block, h and w are the pixel values ​​of the height and width of the feature block, and f xy is the characteristic value of the pixel point at the height x and width y in the feature block.

[0180] Optionally, the processor 41 may further implement the following steps when executing the program:

[0181] The obtaining of the initial recognition results of the n attributes in each feature block respectively, and obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship respectively, includes:

[0182] Based on the loss function, a pedestrian attribute recognition model is trained, wherein each attribute in the loss function corresponds to a weight value, and the weight value corresponding to each attribute is determined according to the number of positive examples and / or negative examples of the attribute in the training sample;

[0183] The pedestrian attribute recognition model is used to obtain initial recognition results of the n attributes in each feature block, and the existence probability of each attribute in each feature block determined based on the spatial position relationship is obtained.

[0184] Optionally, the loss function is:

[0185]

[0186] Where Q is the number of training samples, n is the number of attributes to be identified, and l ks is the true label of the sth attribute of the kth sample, ω s is the weight value of the sth attribute, m ks The probability that the sth attribute of the kth sample is predicted to be a positive example.

[0187] Optionally, the weight value corresponding to each attribute in the loss function is determined according to the following formula:

[0188]

[0189] Among them, ω s is the weight value of the sth attribute, l s is the true label of the sth attribute, p s is the ratio of negative examples to positive examples of the sth attribute in the training sample.

[0190] Optionally, the processor 41 may further implement the following steps when executing the program:

[0191] The training of the pedestrian attribute recognition model based on the loss function includes:

[0192] Calculating the loss value of each feature block recognition result respectively;

[0193] The sum of the loss values ​​of each feature block recognition result is added together to obtain the final loss value.

[0194] The specific working process of the embodiment of the present application is consistent with that in the above-mentioned method embodiment 1, so it will not be repeated here. Please refer to the description of the method steps in the above-mentioned embodiment 1 for details.

[0195] Embodiment 4 of the present application provides a readable storage medium on which a program is stored, and when the program is executed by a processor, the steps in any one of the pedestrian attribute recognition methods in the above embodiment 1 are implemented. For details, please refer to the description of the method steps in the above corresponding embodiments.

[0196] The above-mentioned readable storage medium includes computer-readable storage medium. Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0197] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A pedestrian attribute recognition method, characterized in that: include: Training a pedestrian attribute recognition model, using the pedestrian attribute recognition model, respectively obtaining initial recognition results of n attributes in each feature block, and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship; The feature block is obtained by dividing the feature map of the image to be identified into multiple regions; Determining a recognition result of each attribute in the to-be-recognized picture according to the initial recognition result and the existence probability of each attribute in each feature block; These include: For each of the attributes in each of the feature blocks, multiply the initial recognition result by the existence probability to obtain a recognition result for each of the attributes in each of the feature blocks; For each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks is determined as the recognition result of the attribute in the picture to be recognized.

2. The method according to claim 1, characterized in that Before respectively obtaining the initial recognition results of the n attributes in each feature block and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship, the method further includes: The basic network Resnet-50 with the last fully connected layer removed is used to extract features of the image to be identified, thereby obtaining a feature map of the image to be identified.

3. The method according to claim 1, characterized in that: The step of respectively obtaining the initial recognition results of the n attributes in each feature block includes: For each of the feature blocks, a first operation is performed to obtain first feature information; the first operation includes a pooling operation; The initial recognition results of the n attributes are obtained according to the first feature information.

4. The method according to claim 3, characterized in that The first operation also includes a convolution operation.

5. The method according to claim 3, characterized in that: The step of performing a first operation on each of the feature blocks to obtain first feature information includes: For each of the feature blocks, a pooling operation is performed according to the following formula: Wherein, O is the feature information obtained by pooling the feature block, h and w are the pixel values ​​of the height and width of the feature block, and f xy is the characteristic value of the pixel point at the height x and width y in the feature block.

6. The method according to any one of claims 1 to 5, characterized in that Training a pedestrian attribute recognition model, using the pedestrian attribute recognition model, respectively obtaining initial recognition results of n attributes in each feature block, and respectively obtaining the existence probability of each attribute in each feature block determined based on the spatial position relationship, including: Based on the loss function, a pedestrian attribute recognition model is trained, wherein each attribute in the loss function corresponds to a weight value, and the weight value corresponding to each attribute is determined according to the number of positive examples and / or negative examples of the attribute in the training sample; The pedestrian attribute recognition model is used to obtain initial recognition results of the n attributes in each feature block, and the existence probability of each attribute in each feature block determined based on the spatial position relationship is obtained.

7. The method according to claim 6, characterized in that The loss function is: Where Q is the number of training samples, n is the number of attributes to be identified, and l ks is the true label of the sth attribute of the kth sample, ω s is the weight value of the sth attribute, m ks The probability that the sth attribute of the kth sample is predicted to be a positive example.

8. The method according to claim 6, characterized in that The weight value corresponding to each attribute in the loss function is determined according to the following formula: Among them, ω s is the weight value of the sth attribute, l s is the true label of the sth attribute, p s is the ratio of negative examples to positive examples of the sth attribute in the training sample.

9. The method according to claim 6, characterized in that The training of the pedestrian attribute recognition model based on the loss function includes: Calculating the loss value of each feature block recognition result respectively; The sum of the loss values ​​of each feature block recognition result is added together to obtain the final loss value.

10. A pedestrian attribute recognition device, characterized in that: include: Model training module, used to train pedestrian attribute recognition model; A first recognition module, used to obtain initial recognition results of n attributes in each feature block by using the pedestrian attribute recognition model, and to obtain the existence probability of each attribute in each feature block determined based on the spatial position relationship; The feature block is obtained by dividing the feature map of the image to be identified into multiple regions; A second recognition module, configured to determine a recognition result of each attribute in the to-be-recognized image according to the initial recognition result and the existence probability of each attribute in each feature block; The second recognition module comprises: a first calculation unit, configured to multiply the initial recognition result by the existence probability for each attribute in each feature block to obtain a recognition result for each attribute in each feature block; The second calculation unit is used to determine, for each of the attributes, a maximum value among the recognition results of the attributes of the plurality of feature blocks as the recognition result of the attribute in the to-be-recognized picture.

11. A pedestrian attribute recognition device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the steps in the pedestrian attribute recognition method according to any one of claims 1 to 9 are implemented.

12. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the pedestrian attribute recognition method as described in any one of claims 1 to 9 are implemented.