A pose estimation based occluded pedestrian re-identification method

Abstract

The application provides an occluded pedestrian re-identification method based on pose estimation, which comprises the following steps: constructing and training an occluded pedestrian re-identification model, the model having a ResNet50 network and a human body pose estimation network; inputting a standard image obtained by pre-processing a to-be-detected image into the occluded pedestrian re-identification model; extracting pedestrian features from the ResNet50 network, and performing horizontal block feature and global average pooling on the pedestrian features to obtain horizontal block features and global features respectively; dividing the pedestrian features into head-shoulder region features and other region features according to human body visible key point features extracted by the human body pose estimation network; mapping the horizontal block features and the human body visible key point features to obtain pedestrian visible region bar features based on the head-shoulder region features and the other region features; calculating cosine distances between the pedestrian visible region bar features and the to-be-detected image, and arranging the cosine distances in ascending order according to absolute values, and taking the arrangement result as an output of the occluded pedestrian re-identification model.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of target detection and target classification in computer vision, and particularly relates to a method for occluded pedestrian re-identification based on pose estimation. BACKGROUND

[0002] Pedestrian re-identification is a technology for judging whether a specific pedestrian exists in an image or a video by using computer vision technology, that is, given a monitoring pedestrian image, the pedestrian image under a cross-device is retrieved. Many scenes in life can be applied to this technology, such as smart city, etc.

[0003] In a real scene, a pedestrian will be photographed by multiple different cameras. The images of the same pedestrian will be greatly different due to the pedestrian pose, lighting conditions, and object occlusion. It is necessary to design a pedestrian re-identification method considering all situations, which will inevitably lead to a complex model and low practicability. It is necessary to design a simple and efficient pedestrian re-identification method for occlusion.

[0004] In recent years, convolutional neural networks (CNN) have achieved remarkable results in various computer vision tasks, and pedestrian re-identification has also made great progress, especially in the ideal campus pedestrian re-identification dataset, which has achieved very high indicators. However, the efficiency of the current pedestrian re-identification method for occlusion is still relatively low.

[0005] Patent No. 202111255008.9 discloses an occluded pedestrian re-identification method based on multi-granularity occlusion perception. The method acquires pedestrian image data to be re-identified in real time; inputs the data into a trained occluded pedestrian re-identification model based on multi-granularity occlusion perception, and outputs an occluded pedestrian re-identification result. Compared with the fixed segmentation and single-level method, the method enhances feature diversity. Compared with the method of using only horizontal segmentation along the height dimension, vertical division is added to cope with arbitrarily distributed semantic information. Compared with the one-step prediction method, multi-step prediction is performed at multiple levels to improve accuracy and achieve more accurate occluded pedestrian re-identification. The method focuses on considering from the multi-granularity perception direction, and adds vertical division to cope with arbitrarily distributed semantic information along the height dimension. However, the method actually considers from the block angle and is not specifically designed for occlusion, and the model is relatively complex, the calculation amount is large, and the time cost is high.

[0006] The patent with application number 202110123640.1 discloses a posture-driven occluded pedestrian re-identification method. The method inputs an image to be identified, adjusts the standard input image after preprocessing; uses a pedestrian re-identification network to extract input image features to obtain global features; uses a human pose estimation network to extract input image features to obtain human pose features; based on human pose features, respectively uses a local visibility predictor module to obtain a visibility score, and uses a local attention generator module to obtain an attention mask; finally, according to the visibility score, the local features of the visible part of the input image and the corresponding features in the image library are selected to calculate the Euclidean distance and sorted in ascending order, and the sorting result is taken as the output of the pedestrian re-identification model. This method helps the pedestrian re-identification network to locate the occluded area, reduces the feature difference between the same identity of the occluded pedestrian and the complete pedestrian, and improves the generalization ability of the pedestrian re-identification model. However, the human pose features are not fully utilized, the attention is relatively simple and cannot handle the occluded image well, and the time cost is high.

[0007] The patent with application number 202011116582.1 discloses a global and local feature fusion occluded robust pedestrian re-identification method, including the following steps: 1) preparation of training data; 2) model design and training; the model includes a ResNet-50 skeleton network, a global branch, a local branch, and a semantic branch, the global branch uses SPC loss to extract global features, the local branch extracts local features, and the semantic branch predicts human semantic labels, the three branches can be jointly trained end-to-end. 3) Use the trained model to extract the global features, local features and predicted semantic labels of all pedestrian images in the pedestrian re-identification data training set and test set, and calculate the non-occluded area indicator. 4) Calculate the similarity between each pedestrian image in the query set and all pedestrian images in the database, and sort them in descending order of similarity, thereby completing pedestrian re-identification. Although this method uses multiple branches to solve the problem, it is not specifically designed for occlusion, and is relatively complex and time-consuming.

[0008] After summarizing, it is found that the existing identification technology has the defects of complex model, large amount of calculation and high time cost, and cannot well handle the occluded picture and the problem of distinguishing features. SUMMARY

[0009] To solve the above problems, a method for identifying pedestrians based on posture estimation network processing occluded pictures is provided, and the technical scheme adopted by the present application is as follows:

[0010] The application provides a pose estimation-based occluded pedestrian re-identification method, characterized by comprising the following steps: step S1, constructing and training an occluded pedestrian re-identification model, which has a ResNet50 network and a human pose estimation network; step S2, inputting a to-be-detected image as a standard image into the occluded pedestrian re-identification model after pretreatment; step S3, extracting pedestrian features from the ResNet50 network, and horizontally pooling and globally averaging the pedestrian features to obtain horizontal block features and global features respectively; step S4, extracting human visible key point features from the human pose estimation network, and dividing the pedestrian features into head-shoulder region features and other region features based on the key point features; step S5, mapping the horizontal block features and the human visible key point features to obtain pedestrian visible region bar features based on the head-shoulder region features and the other region features; and step S6, calculating the cosine distance between the pedestrian visible region bar features and corresponding features in the to-be-detected image, and arranging the cosine distance in ascending order according to the absolute value, and taking the arrangement result as the output of the occluded pedestrian re-identification model.

[0011] In the pose estimation-based occluded pedestrian re-identification method provided by the application, the human pose estimation network can be an HRNet network.

[0012] In the pose estimation-based occluded pedestrian re-identification method provided by the application, the horizontal block features obtained based on the horizontal pooling of the pedestrian features can be 4 blocks.

[0013] In the pose estimation-based occluded pedestrian re-identification method provided by the application, the pedestrian visible bar region features can be obtained by calculating the height proportion of the position of the human visible key point features in the to-be-detected image, and mapping each of the 4 horizontal block features and the head-shoulder region features and the other region features based on the height proportion.

[0014] In the pose estimation-based occluded pedestrian re-identification method provided by the application, the mapping formula can be:

[0015]

[0016] In the formula, cy represents the vertical coordinate of the human visible key point features, label represents the i-th local feature block of the pedestrian features, j represents the serial number of the key point coordinate, and H represents the height of the image. j i

[0017] ​​In the pose estimation-based occluded pedestrian re-identification method provided by the application, the training process of the occluded pedestrian re-identification model can further have the following technical features: in step S1-1, a pedestrian re-identification dataset is obtained, and the dataset is divided into a training set and a test set, and input images are obtained based on the training set; in step S1-2, pedestrian features of the input images are extracted through a ResNet50 network; in step S1-3, the pedestrian features are horizontally pooled and globally averaged to obtain horizontal block features and global features, respectively; in step S1-4, human visible key point features of the input images are extracted through a human pose estimation network, and the pedestrian features are divided into head-shoulder region features and other region features; in step S1-5, the human visible key point features and the horizontal block features are mapped to obtain pedestrian visible bar region features; in step S1-6, the human visible key point features, the global features, and the pedestrian visible bar region features are batch normalized to calculate a classification loss; in step S1-7, parameters of the pedestrian re-identification network are optimized based on the classification loss; and in step S1-8, steps S1-1 to S1-7 are repeated until the value of the classification loss no longer changes, thereby completing the training.

[0018] In the pose estimation-based occluded pedestrian re-identification method provided by the application, the input images can be P pedestrians randomly selected from the training set each time in the training process, and K images of each pedestrian are randomly selected as the input images.

[0019] In the pose estimation-based occluded pedestrian re-identification method provided by the application, the calculation formula of the classification loss can be as follows:

[0020]

[0021] In the formula, N represents the number of pedestrian images in a batch, C represents the total number of pedestrian identities in the training set, p represents the true identity, and q represents the predicted identity. i,j i,j

[0022] Invention function and effect

[0023] According to the pose estimation-based occluded pedestrian re-identification method, the ResNet50 network is used to extract pedestrian features, and the pose estimation network is used to capture human visible key point features in the occluded image, thereby solving the problem that the prior art cannot process occluded images with high performance. Secondly, the human visible key point features and the horizontal block features of the pedestrian features are mapped to generate pedestrian visible bar region features, thereby effectively improving the prediction accuracy. And the bar region features are used to train the occluded pedestrian re-identification model, so that the model can accurately and efficiently re-identify and predict pedestrians in the occluded image.​​ BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart of a method for occluded pedestrian re-identification based on pose estimation in an embodiment of the present application; and

[0025] Figure 2 is a structural diagram of a method for occluded pedestrian re-identification in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the technical means, creative features, purposes and effects achieved by the present application easy to understand, the following describes a method for occluded pedestrian re-identification based on pose estimation in combination with embodiments and drawings.

[0027] <EMBODIMENT>

[0028] Figure 1 is a flowchart of a method for occluded pedestrian re-identification based on pose estimation in an embodiment of the present application, and Figure 2 is a structural diagram of a method for occluded pedestrian re-identification in an embodiment of the present application.

[0029] As shown in Figure 1 and Figure 2 , the specific process of the method for occluded pedestrian re-identification based on pose estimation in the embodiment is as follows:

[0030] Step S1, a model for occluded pedestrian re-identification is constructed and trained, which has a ResNet50 network and an HRNet human pose estimation network.

[0031] In the embodiment, the training process of the model for occluded pedestrian re-identification is as follows:

[0032] Step S1-1, a pedestrian re-identification dataset is obtained, and the dataset is divided into a training set and a test set. In the training process, P pedestrians are randomly selected from the training set each time, and K images of each pedestrian are randomly selected as input images for model training.

[0033] Step S1-2, the pedestrian features of the input images are extracted through the ResNet50 network.

[0034] Step S1-3, the pedestrian features are horizontally pooled and globally averaged to obtain 4 blocks of horizontal partition features and global features, respectively.

[0035] Step S1-4, the human visual key point features of the input images are extracted through the HRNet network, and the pedestrian features are divided into head-shoulder region features and other region features based on the key point features.

[0036] In this embodiment, the division into head-shoulder region features and other region features means that all key point features of the head-shoulder region are fixed as a large feature for overall training, while the key point features of other regions are still trained as individual key point features.

[0037] In step S1-5, the position proportion of the human visible key point feature in the height of the image to be detected is calculated, and based on the height proportion, each of the four horizontal block features is mapped with the head-shoulder region feature and the other region feature, so as to obtain the pedestrian visible bar region feature.

[0038] The mapping formula is:

[0039]

[0040] In the formula, cy j represents the vertical coordinate of the human visible key point feature, label i represents the i-th local feature block of the pedestrian feature, j represents the serial number of the key point coordinate, and H represents the height of the picture.

[0041] In step S1-6, the human visible key point feature, the global feature, and the pedestrian visible bar region feature are batch normalized to calculate the classification loss, and the calculation formula of the classification loss is:

[0042]

[0043] In the formula, N represents the number of pedestrian pictures in a batch, C represents the total number of pedestrian identities in the training set, p i,j represents the true identity, q i,j represents the predicted identity.

[0044] In step S1-7, the parameters of the pedestrian re-identification network are optimized based on the classification loss.

[0045] In step S1-8, the above steps are repeated until the value of the classification loss no longer changes, thereby completing the training.

[0046] In step S2, the image to be detected is preprocessed and input to the trained occluded pedestrian re-identification model as a standard image.

[0047] In step S3, the pedestrian feature is extracted by the ResNet50 network, and the pedestrian feature is horizontally pooled and globally averaged to obtain four horizontal block features and a global feature.

[0048] In step S4, the human visible key point feature is extracted by the HRNet human pose estimation network, and based on the key point feature, the pedestrian feature is divided into head-shoulder region features and other region features.

[0049] In this embodiment, the HRNet algorithm first extracts the pedestrian key point heat map, and then obtains the position coordinates of the human body key points and the belief score of each position coordinate after Gaussian function processing and the like based on the heat map. If the belief score is too low, the position coordinate is discarded and set to 0, and the human body position is considered to be occluded. The following formula can be used to represent it:

[0050]

[0051] In the formula, L j represents the vertical coordinate of the jth key point, cy j represents the vertical coordinate of the jth key point, cy represents the belief score, and μ represents the threshold value.

[0052] Step S5: Based on the head-shoulder region feature and other region features, the horizontal block feature is mapped with the human body visible key point feature to obtain the pedestrian visible region bar feature.

[0053] Step S6: The cosine distance between the to-be-measured image and the pedestrian visible region bar feature in the image library is calculated, and the cosine distance is sorted in ascending order according to the absolute value. The sorted result is taken as the output of the occluded pedestrian re-identification model.

[0054] In this embodiment, the occluded pedestrian re-identification model achieves good results on the pedestrian re-identification dataset, has a short training time, significantly improves the performance of the occluded situation, and also obtains average performance on the general pedestrian re-identification dataset.

[0055] Effects of the embodiments

[0056] According to the pose estimation-based occluded pedestrian re-identification method provided in this embodiment, the ResNet50 network is used to extract the pedestrian feature, and the pose estimation network is used to capture the visible key point feature of the pedestrian in the occluded image, thereby solving the problem that the prior art cannot process the occluded image with high performance. Secondly, the visible key point feature of the pedestrian and the horizontal block feature of the pedestrian feature are mapped to generate the pedestrian visible bar region feature, thereby effectively improving the prediction accuracy. And the bar region feature is used to train the occluded pedestrian re-identification model, so that the model can accurately and efficiently predict the re-identification of the pedestrian in the occluded image.

[0057] The above embodiments are only used to illustrate the specific implementation of the present application, and the present application is not limited to the description range of the above embodiments.

Claims

1. A method for occluded pedestrian re-identification based on pose estimation, characterized in that, The method comprises the following steps: Step S1, constructing and training an occluded pedestrian re-identification model having a ResNet50 network and a human pose estimation network; Step S2, inputting a to-be-tested image as a standard image into the occluded pedestrian re-identification model after pre-processing; Step S3, extracting pedestrian features from the ResNet50 network, and horizontally pooling and globally averaging the pedestrian features to obtain horizontal block features and global features respectively; Step S4, extracting human visible key point features from the human pose estimation network, and dividing the pedestrian features into head-shoulder region features and other region features based on the key point features; Step S5, mapping the horizontal block features and the human visible key point features based on the head-shoulder region features and the other region features to obtain pedestrian visible region bar features; wherein the pedestrian features based on the horizontally pooled horizontal block features are 4 blocks, and the pedestrian visible bar region features are: calculating the height proportion of the position of the human visible key point features in the to-be-tested image, and mapping each of the 4 horizontal block features and the head-shoulder region features and other region features based on the height proportion, thereby obtaining the pedestrian visible bar region features; wherein the mapping formula is: In the formula, cy j represents the longitudinal coordinate of the human visual key point feature, label i represents the i-th local feature block of the pedestrian feature, j represents the serial number of the key point coordinate, and H represents the height of the picture; Step S6, calculating the cosine distance between the to-be-tested image and the pedestrian visible region bar features, and arranging the cosine distances in ascending order according to the absolute values, and taking the sorting result as the output of the occluded pedestrian re-identification model.

2. The pose estimation-based occluded pedestrian re-identification method according to claim 1, wherein: wherein the human pose estimation network is an HRNet network.

3. The pose-estimation-based occluded pedestrian re-identification method of claim 1, characterized in that: wherein the training process of the occluded pedestrian re-identification model is: Step S1-1, obtaining a pedestrian re-identification dataset, dividing the dataset into a training set and a test set, and obtaining input images for training based on the training set; Step S1-2, extracting pedestrian features of the input images through the ResNet50 network; Step S1-3, horizontally pooling and globally averaging the pedestrian features to obtain horizontal block features and global features respectively; Step S1-4, extracting human visible key point features of the input images through the human pose estimation network, and dividing the pedestrian features into head-shoulder region features and other region features based on the human visible key point features; Step S1-5, mapping the human visible key point features and the horizontal block features to obtain pedestrian visible bar region features; Step S1-6, performing batch normalization on the human visible key point features, the global features, and the pedestrian visible bar region features, and then calculating a classification loss; Step S1-7, optimizing parameters of the pedestrian re-identification network based on the classification loss; Step S1-8, repeating steps S1-1 to S1-7 until the value of the classification loss no longer changes, thereby completing the training.

4. The pose estimation-based occluded pedestrian re-identification method according to claim 3, characterized in that: wherein, The input image is that P pedestrian rows are randomly selected in the training set each time in the training process, and K images are randomly selected for each pedestrian as input images.

5. The pose estimation based occluded pedestrian re-identification method according to claim 3, characterized in that: wherein The calculation formula of the classification loss is: In the formula, N represents the number of batches of pedestrian pictures, C represents the total number of pedestrian identities in the training set, p i,j represents the real identity, and q i,j represents the predicted identity.

Images