Occluded pedestrian re-identification method, device and equipment and computer storage medium

Abstract

The present application belongs to the technical field of computer vision and pattern recognition, and relates to a kind of occluded pedestrian re-identification method, device, equipment and computer storage medium.The specific steps of the occluded pedestrian re-identification method are as follows:(1) data preparation is carried out:(2) introducing occlusion information and carrying out image preprocessing;(3) network model building;(4) key point-local semantic feature similarity matching;(5) network objective function construction;(6) occluded pedestrian re-identification based on occlusion information auxiliary and key point enhancement.In addition, the supporting device, equipment and computer storage medium are also proposed.The present application can efficiently identify and retrieve the existing public occluded pedestrian re-identification dataset, effectively solve the problems of pedestrian information loss and feature alignment difficulty caused by occlusion and posture change in the target dataset, and the method converges quickly.The present application is especially suitable for public safety field.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and pattern recognition, and relates to a method, device and equipment for occluded pedestrian re-identification and a computer storage medium; in particular, it relates to an occluded pedestrian re-identification method based on occlusion information assistance and key point enhancement, and also relates to a device, equipment and computer storage medium matched with the occluded pedestrian re-identification method based on occlusion information assistance and key point enhancement. BACKGROUND

[0002] In recent years, researchers have made a lot of achievements in the field of pedestrian re-identification. Network models have various categories and structures, or are based on convolutional neural networks or on Transformers, and each has its own advantages. However, the image samples used in the traditional pedestrian re-identification task are complete pedestrian images, which does not match the actual situation. In real-world scenarios, it is difficult to capture complete pedestrian images. Especially in crowded places such as shopping malls, campuses and stations, part of the body of a pedestrian is often blocked by objects such as billboards, cars, trash cans or other pedestrians. Due to the existence of occlusion, the target pedestrian is difficult to be accurately identified. Therefore, the occluded pedestrian re-identification task was born to solve the occlusion problem in pedestrian re-identification.

[0003] The occluded pedestrian re-identification task mainly faces the following two challenges:

[0004] 1. The occlusion of a pedestrian image results in the loss of information about the occluded body parts. In addition, in the process of extracting image features, the occlusion may be treated as part of the human body and may be introduced as interference information, making it difficult to re-identify the identity of the pedestrian. In the occluded pedestrian re-identification task, there are mainly two types of occlusion: one is occlusion by an occluded object, and the other is occlusion by other pedestrians.

[0005] 2. The change in the posture of each pedestrian or the change in the camera angle causes the displacement of key information in the image, making it difficult for the network to perceive and align the key information in the image.

[0006] In recent years, in order to cope with the above two challenges, researchers have proposed many effective methods. Some methods use the idea of dividing features and training them separately: the part-based convolutional baseline network (PCB), which horizontally divides the feature map into multiple parts and directly learns local features; the region-based method (VPM), which perceives each visible region through a self-supervised mechanism. However, these feature divisions are often divided by simple and crude methods, or are not accurate due to technical immaturity.

[0007] In addition, some methods use the method of introducing external auxiliary information: pose-guided visible part matching (PVPM) model is used to learn distinguishing part features through pose-guided attention. The HOReID method extracts the features of 14 human key points by using a human pose detector, inputs the features into a graph convolution network for a series of processing, and adopts a graph matching strategy in feature matching in the test stage, that is, high-order relationships and human topological information are introduced to learn robust features. Although the method using the idea of introducing external auxiliary information can achieve good results, such methods have high requirements for the quality of external auxiliary information and have too great dependence on external auxiliary information.

[0008] The two ideas above have their own advantages and disadvantages when used alone, therefore, we have organically combined the two ideas and proposed a method of occluded pedestrian re-identification based on occlusion information assistance and key point enhancement for solving the problem of occluded pedestrian re-identification. SUMMARY

[0009] The present application provides an occluded pedestrian re-identification method, device, equipment and computer storable medium, and a device, equipment and computer storable medium matched with the method. The present application is particularly suitable for the field of public security.

[0010] The occluded pedestrian re-identification method of the present application comprises the following steps:

[0011] 1) Select a number of occluded images in the data set and manually crop a number of occlusion images of different sizes. The occlusion images with a height greater than twice the width are used as vertical occlusion, and the occlusion images with a width greater than twice the height are used as horizontal occlusion; randomly select images in the training set of the data set and crop them to generate local pedestrian images, the size of which is 1 / 4-1 / 2 of the original size of the selected training set images; in addition, in order to make the features extracted by the network model more consistent with the human semantic, we use the existing human parsing model to parse the training set images, divide the images into 20 semantic parts, and further classify and merge them into 6 human semantic labels.

[0012] 2) Preprocess the training set images, perform data enhancement, adjust the image size to 256x128, and randomly crop, horizontally flip and randomly erase the images with a probability of 50%; and use the occlusion image and the local pedestrian image to perform occlusion data enhancement on the training set images with a probability of 50%, wherein the probability of occlusion image enhancement and local pedestrian image enhancement is 25% respectively;

[0013] 3) Constructing HRNet-32 as a baseline human semantic feature parsing model and training; the human semantic feature parsing model extracts 1 local semantic feature containing 5 human semantic part information, 1 foreground semantic feature and 1 global semantic feature for each input image, a total of 3 features;

[0014] 4) Extracting 13 human key point features using a human key point feature extraction module based on HRNet-48, and feeding them into a convolutional neural network (GCN) for processing to obtain 13 GCN key point features. The GCN key point features are matched and fused with the human semantic features using a key point-local semantic feature similarity matching method to enhance the human key point information, and a local semantic feature containing human key point information is obtained;

[0015] 5) Using the local semantic feature containing human key point information, the foreground semantic feature and the global semantic feature to participate in the calculation of the loss function;

[0016] 6) The trained network extracts 1 local semantic feature containing 5 human semantic part information and 1 foreground semantic feature, and then processes the two features through batch normalization layer respectively, and then splices the two features processed by batch normalization along the channel direction to obtain the final feature. The Euclidean distance of the features of each sample in the gallery set and the query set is calculated using the final feature, and the cmc and mAP of the samples in the gallery set are calculated according to the distance of the samples in the query set, and finally the re-identification of the pedestrian samples is realized.

[0017] Use the existing available images in the dataset to generate occlusion images and local pedestrian images before training to enhance the occlusion of the training set images; In order to make the network model more robust to occlusion and extract accurate and effective human semantic features, we need to prepare the data before training the network model;

[0018] In the occluded pedestrian re-identification task, there are mainly two kinds of occlusion in the pedestrian image: being occluded by billboards, cars, garbage cans and other occlusion objects, and being occluded by other pedestrians besides the target pedestrian; Therefore, in order to simulate these two kinds of occlusion, we need to manually generate two kinds of occlusion images; In addition, in order to make the network model more accurately locate and extract features of human semantic parts, we need to generate human semantic labels to participate in the loss function calculation and model optimization later;

[0019] Preferably, the data preparation stage needs to prepare occlusion images and human semantic labels, and the specific steps are as follows:

[0020] 1-1) Randomly select a number of images with occlusion in the dataset, and manually crop the image part of the occlusion. In order to further simulate the occlusion in the actual scene, we generate two kinds of occlusion images, and the occlusion image with a height greater than twice the width is used as the vertical direction occlusion, and the occlusion image with a width greater than twice the height is used as the horizontal direction occlusion.

[0021] 1-2) Randomly select images in the training set and crop to generate local pedestrian images, the size is 1 / 4-1 / 2 of the original size of the selected training set images. Since the training set does not contain occlusion, the cropped part should be a part of the pedestrian body, and there is a small probability of containing background information.

[0022] 1-3) After the existing human parsing model SCHP is pre-trained on the human parsing dataset human LIP, each image is segmented into 20 semantic parts, and we group and classify the 20 semantic parts to finally obtain 6 semantic parts, which correspond to "background", "head", "upper body", "accessories", "legs", and "feet".

[0023] Preferably, the training set images are preprocessed and data augmentation is performed. The specific method is as follows:

[0024] 2-1) Adjust the image size to 256x128, and randomly crop, horizontally flip and randomly erase the image with a probability of 50%;

[0025] 2-2) Use occlusion images and local pedestrian images to enhance the occlusion data of the training set images with a probability of 50%, wherein the probability of occlusion image enhancement and local pedestrian image enhancement is 25% respectively, and the same probability is selected in one of the four directions of the image to be occluded.

[0026] Preferably, the main steps of training the human semantic feature parsing model based on HRNet-32 are pixel-level part segmentation and alignment, and the human part is represented by a set of pixel-level features weighted by a set of confidence maps. The specific steps are as follows:

[0027] 3-1) Input n training set images and the identity label of each image to the human semantic feature parsing model to classify the pixel-level features of the pedestrian image and extract 5 human body local semantic features. Input the image x i to the human semantic feature parsing model, and use the mapping function of the backbone network f to obtain the global feature map M i of the image x g . The formula is as follows:

[0028]

[0029] where backbone network f is a pre-trained HRNet-32 network model on ImageNet dataset, θ is the parameter of the backbone network, and M g is the size of the global feature map, which is 64x32; c, h, w are the channel, height and width of the global feature map; and M g (x, y) represents the feature at the spatial position (x, y).

[0030] 3-2) The M g (x, y) is processed through a linear layer to obtain six confidence maps of different semantic parts and one foreground confidence map, respectively P0, P1, P2, P3, P4, P5 and P fore ; the calculation formula is as follows:

[0031]

[0032] where P k (x, y) represents the confidence of the pixel (x, y) belonging to the semantic part k, and it is worth noting that P0 is the confidence of the background part. The foreground semantic confidence map P f is obtained by adding five human body semantic confidence maps. The calculation formula is as follows:

[0033]

[0034] The total confidence map is denoted as , which is obtained by concatenating the six semantic confidence maps along the channel direction.

[0035] 3-3) The confidence map is passed through a softmax layer to obtain five local semantic masks m1, m2, m3, m4, m5 and one foreground semantic mask m fore . The calculation formula of the local semantic mask is as follows:

[0036]

[0037] Similarly, the foreground mask m fore can be calculated using the foreground confidence map P fore .

[0038] The semantic features of the five parts are calculated by the following formula:

[0039]

[0040] The foreground semantic feature is calculated by the following formula:

[0041]

[0042] One global semantic feature is calculated by the following formula

[0043] f global = GAP (M g )

[0044] P k (x, y) represents the confidence of the pixel (x, y) belonging to the semantic part k, k ∈ {0,..., K-1} and represents element-wise multiplication, GAP represents the global average pooling operation, f k is the semantic feature of the k-th part. The human body local semantic feature f p =[f1, f2, f3, f4, f5] is the semantic feature of 5 parts spliced along the channel direction.

[0045] Preferably, a human key point feature extraction module based on HRNet-48 is used to extract 13 human key point features, and after processing by a convolutional neural network, 13 GCN key point features are obtained, which are matched with the 5 local human semantic features; the specific steps are as follows:

[0046] 4-1) input an image x i with a size of HxW into the human key point feature extraction module to extract 13 key point heat maps Each heat map has a size of (H / 4)x(W / 4), and the maximum response point of each heat map corresponds to a key point. After processing by a fully connected layer, the key point heat map H obtains a heat map H', which has the same size as the global feature f.

[0047] After multiplying H' and f, the pose guided feature , i.e., the key point feature, can be obtained.

[0048] 4-2) At this time, the weights of the 13 key point features are the same, but due to the existence of the occluded part, we want the key point feature weight of the occluded part to be as small as possible, and the key point feature weight of the non-occluded part to be as large as possible. In order to achieve this purpose, we use a convolutional neural network (GCN) to update the weight. The 13 key point features 1 foreground semantic feature f fore and 1 adjacency matrix A representing the adjacency relationship of the reaction key points are input into the convolutional neural network for processing. In theory, the similarity between the unoccluded key point feature and the foreground feature is higher than that between the occluded key point feature and the foreground feature. Based on this principle, we can update the weight according to the similarity between the key point feature and the foreground feature. The weight is updated according to the similarity between the key point feature and the foreground feature, and an updated weight matrix A g is obtained. Finally, 13 GCN key point features The calculation formula is as follows:

[0049] K g =[(f c1 (A g K)+(f c2 (K), f fore ]

[0050] Where f c1 with f c2 Both are fully connected layers. This is an element-wise multiplication operation.

[0051] 4-3) To enhance keypoint features within local semantic features, we utilize a keypoint-local semantic feature similarity matching method. After the input image is processed by the human semantic feature parsing model, a local semantic feature containing five parts of human semantic information can be obtained. The keypoint-local semantic feature similarity matching method matches and fuses 13 keypoint features with the five parts of human semantic features.

[0052] The specific method involves calculating the features of 13 GCN keypoints. Human semantic features in 5 parts The cosine similarity between the features is used to select the parts with the highest cosine similarity for integration, so that the human semantic features contain relevant key point information.

[0053] First, the matching of GCN keypoint information and human semantic features needs to be completed using the following formula:

[0054]

[0055] in For the j-th individual's key feature, f i Let i be the semantic features of the human body in part i.

[0056] Then, the integration of GCN keypoint information and human semantic features is completed using the following formula:

[0057] Where F i Then, it is the i-th local human feature containing key point information generated after matching, where k is the number of human part features f. i The most similar human keypoint features are numbered. The final local semantic features of the human body containing keypoint information are:

[0058] The occluded pedestrian re-identification method, the calculation process of the loss function in step 5) is as follows: the triplet loss function and the cross-entropy loss function are used to guide the training and optimization of the network model; the human body local semantic feature containing key point information, the foreground semantic feature and the global semantic feature are also constrained by the cross-entropy loss function and the triplet loss function; in addition, in order to make the network model more accurately locate each human body semantic part and extract human body semantic features, a human body analysis loss function is set, the human body semantic label generated in the data preparation stage and the global confidence map output by the network model are used to participate in the calculation of the loss function, so that the optimization of the network model is realized in the training.

[0059] The application further provides a pedestrian re-identification device, characterized in that the device comprises:

[0060] A human body semantic feature extraction module extracts human body semantic features from each image by using the network model to obtain the local semantic feature, the foreground semantic feature and the global semantic feature.

[0061] A human body key point feature extraction module extracts posture guiding features, i.e., key point features, of a human body by using the network model to obtain 13 key point features.

[0062] A graph convolution module updates the key point feature weight by using the graph convolution network to obtain 13 GCN key point features.

[0063] A key point-local semantic feature matching module matches the GCN key point feature with the local semantic feature by using the key point-local semantic feature similarity matching method to obtain the human body local semantic feature containing key point information.

[0064] The application further provides a pedestrian re-identification device, characterized in that the device comprises:

[0065] A memory for storing executable computer programs;

[0066] A processor for executing the executable computer programs stored in the memory to realize the algorithm of any one of the application.

[0067] The application further provides a computer readable storage medium storing a computer program for being executed by a processor to realize the occluded pedestrian re-identification method based on occlusion information assistance and key point enhancement.

[0068] The application has the following advantages and beneficial effects:

[0069] 1) Introduce two categories of occlusion auxiliary information in a feature enhancement manner, simulate the situation that pedestrians are occluded by occlusion or other pedestrians in real scenes, and enhance the robustness of the network model to occlusion;

[0070] 2) Update the key point feature weight using a convolutional neural network (GCN), reduce the key point feature weight of the occluded part, and use a "key point-human local semantic feature similarity matching module" to match and fuse the key point information with the human semantic information, realize the strengthening of the posture key point information, and enhance the robustness of the network model to posture or shooting angle changes;

[0071] 3) The sample is trained by the model to obtain discriminative features, and by combining the posture key point information, robust image features can be obtained, and the model converges quickly and can achieve efficient recognition performance on multiple pedestrian re-identification datasets with or without occlusion. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 The method model structure diagram of the application.

[0073] Figure 2 The human semantic label generation module schematic diagram.

[0074] Figure 3 The human semantic label classification and combination effect schematic diagram. DETAILED DESCRIPTION

[0075] The application will be further described below in conjunction with the drawings;

[0076] Example 1:

[0077] As shown in the figure, it is the method model structure diagram of the occluded pedestrian re-identification method based on occlusion information auxiliary and key point enhancement in the application, and the operation steps of the method include: Figure 1

[0078] Step 1) Data preparation

[0079] In the occluded pedestrian re-identification task, there are mainly two occlusion situations in the pedestrian image: being occluded by billboards, cars, garbage cans and other occlusion objects, and being occluded by other pedestrians. Therefore, in order to simulate these two occlusion situations, we need to manually generate two types of occlusion images. In addition, in order to make the network model more accurately locate and extract features of the human semantic part, we need to generate human semantic labels to participate in loss function calculation and model optimization later. The specific method is as follows:

[0080] ​First, randomly select several images with occlusion in the dataset, and manually crop the image part of the occlusion. In order to further simulate the occlusion in the actual scene, we generate two kinds of occlusion images, and the occlusion image with a height greater than twice the width is used as the vertical direction occlusion, and the occlusion image with a width greater than twice the height is used as the horizontal direction occlusion; randomly select images in the training set and crop to generate local pedestrian images, the size is 1 / 4-1 / 2 of the original size of the selected training set images. Since the training set does not contain occlusion, the cropped part should be part of the pedestrian body, and there is a small probability of containing background information.

[0081] After the existing human parsing model SCHP is pre-trained on the human parsing dataset human LIP, each image is segmented into 20 semantic parts, and we group and classify the 20 semantic parts to finally obtain 6 semantic parts, which correspond to "background", "head", "upper body", "accessories", "legs", and "feet", respectively.

[0082] Step 2) Image preprocessing

[0083] Adjust the image size to 256x128, and perform random cropping, horizontal flipping, and random erasing on the image with a probability of 50%; use the occlusion image and the local pedestrian image to perform occlusion data augmentation on the training set image with a probability of 50%, wherein the probability of occlusion image augmentation and local pedestrian image augmentation is 25% respectively, and the same probability is selected in one of the four directions of the image to be occluded.

[0084] Step 3) Network model building

[0085] After completing the preprocessing of the training set images, the human semantic feature parsing model based on HRNet-32 can be trained using the training set images. n training set images and the identity label of each image are input into the human semantic feature parsing model to classify the pixel-level features of the pedestrian image and realize the extraction of 5 human body local semantic features. The image x i is input into the human semantic feature parsing model, and the mapping function of the backbone network f is used to obtain the global feature map M i of the image x g . The formula is as follows:

[0086]

[0087] Where the backbone network f is an HRNet-32 network model pre-trained on the ImageNet dataset, θ is the parameter of the backbone network, and the global feature map M g64x32; c, h, w are global feature map channel, height and width; M g (x, y) denotes the feature at spatial location (x, y).

[0088] M g (x, y) is processed by a linear layer to get 6 confidence maps of different semantic parts and one foreground confidence map, denoted as P0, P1, P2, P3, P4, P5 and P for e, respectively. The calculation formula is as follows:

[0089]

[0090] where P k (x, y) denotes the confidence of pixel (x, y) belonging to semantic part k. It is worth noting that P0 is the confidence of the background part. The foreground semantic confidence map P f is the sum of the 5 human body semantic confidence maps. The calculation formula is as follows:

[0091]

[0092] The total confidence map is denoted as which is the concatenation of the 6 semantic confidence maps along the channel direction.

[0093] The confidence map is passed through a softmax layer to get 5 local semantic masks m1, m2, m3, m4, m5 and one foreground semantic mask m fore . The calculation formula of the local semantic mask is as follows:

[0094]

[0095] Similarly, the foreground mask m fore can be calculated using the foreground confidence map P fore .

[0096] The semantic feature of the 5 parts is calculated by the following formula:

[0097]

[0098] The foreground semantic feature is calculated by the following formula:

[0099]

[0100] The global semantic feature is calculated by the following formula

[0101] f global = GAP(M g )

[0102] In the formula, P k(x, y) represents the confidence of the pixel (x, y) belonging to the semantic part k, k e {0,..., K-1} and represents element-wise multiplication, GAP represents the global average pooling operation, f k is the semantic feature of the k-th part. The human body local semantic feature f p =[f1, f2, f3, f4, f5] is the semantic feature of 5 parts spliced along the channel direction.

[0103] Step 30 generates GCN key point features

[0104] An image x of size HxW is input i An HRNet-48-based human key point feature extraction module is input to extract 13 key point heat maps The size of each heat map is (H / 4)x(W / 4), and the maximum response point of each heat map corresponds to a key point. After processing the key point heat map H through a fully connected layer, a heat map H' is obtained, which has the same size as the global feature f. After multiplying H' and f, the pose-guided feature , i.e., the key point feature, is obtained.

[0105] At this time, the weights of the 13 key point features are the same, but due to the existence of the occluded part, we hope that the key point feature weight of the occluded part is as small as possible, and the key point feature weight of the non-occluded part is as large as possible. In order to achieve this purpose, we use a convolutional neural network (GCN) to update the weight. The 13 key point features 1 foreground semantic feature f fore and 1 adjacency matrix A representing the adjacency relationship of the key points are input into the convolutional neural network for processing. In theory, the similarity between the unoccluded key point feature and the foreground feature is higher than that between the occluded key point feature and the foreground feature. Based on this principle, we can update the weight according to the similarity between the key point feature and the foreground feature. The weight is updated according to the similarity between the key point feature and the foreground feature, and the updated weight matrix A g is obtained. Finally, 13 GCN key point features are obtained.

[0106] K g = [(f c1 (A g K) + (f c2 (K), f fore ]

[0107] where f c1 and f c2 are fully connected layers. is an element-wise multiplication operation.

[0108] Step 4) Key point-local semantic feature similarity matching

[0109] To enhance the key point features in the local semantic features, we use the key point-local semantic feature similarity matching method. After the input image is processed by the human semantic feature analysis model, we can obtain a human local semantic feature containing 5 human semantic part information. The key point-local semantic feature similarity matching method matches and fuses the 13 key point features and the 5 human semantic features.

[0110] The specific method is to calculate the cosine similarity between the 13 GCN key point features and the 5 human semantic features Select the highest cosine similarity part for integration, so that the human semantic feature contains relevant key point information:

[0111] First, we need to complete the matching between the GCN key point information and the human semantic feature by the following formula:

[0112]

[0113] Where is the jth human key point feature, f i is the ith human semantic feature.

[0114] Then complete the integration of the GCN key point information and the human semantic feature by the following formula:

[0115]

[0116] Where F i is the ith human local feature containing key point information generated after matching, and k is the number of human key point features most similar to the human part feature f i The final human local semantic feature containing key point information is

[0117] Step 5) Network objective function construction

[0118] The pedestrian re-identification problem is regarded as a metric problem, and the distance between different image features of the same person should be smaller than the distance between image features of different people. For the occluded pedestrian re-identification method based on occlusion information assistance and key point enhancement, its final output is the global semantic feature, the foreground semantic feature, the local semantic feature, and the prediction results of the above features through the classifier respectively.

[0119] In the objective function of the network, the triplet loss makes the distance between samples of the same class smaller and the distance between samples of different classes larger, so that the learned features have better robustness. The triplet loss formula is as follows:

[0120] L Triplet = max(d(a, p) - d(a, n) + margin, 0)

[0121] Where margin is the maximum distance boundary of the current batch. d(a, p) represents the distance between the fusion features of the anchor sample and the corresponding positive sample. Similarly, d(a, n) represents the distance between the fusion features of the anchor sample and the corresponding negative sample.

[0122] In addition, the cross-entropy loss function as a classification loss makes the learned features have better distinguishability. The classification loss formula is as follows:

[0123]

[0124] Where CE is the cross-entropy loss, is the prediction of the i-th part classifier, y i is the true id.

[0125] In order to enhance the accuracy of the network model in perceiving the semantic parts of the human body, we introduce a human parsing loss function, the specific calculation method is as follows:

[0126]

[0127] Where CE is the cross-entropy loss, P(x, y) is the confidence map at pixel (x, y), and label(x, y) is the human semantic label at pixel (x, y).

[0128] Step 6) Occluded pedestrian re-identification based on occlusion information assistance and key point enhancement

[0129] The human local semantic features F containing key point information and the human foreground semantic features f fore output by the network model are processed through the batch normalization layer, then spliced along the channel direction to obtain the final features, and the cosine similarity of the final features of each sample in the gallery set and the query set is calculated. According to the cosine similarity, cmc and mAP are calculated, and finally the re-identification of the pedestrian samples is realized.

Claims

1. A method for re-identifying occluded pedestrians, characterized in that, Includes the following steps: 1) Select several occluded images from the dataset and manually crop several occluder images of different sizes; use occluder images with a height greater than twice the width as vertical occluders, and use occluders with a width greater than twice the height as horizontal occluders; randomly select images from the training set of the dataset and crop them to generate local pedestrian images with a size of 1 / 4 to 1 / 2 of the original size of the selected training set images; in addition, in order to make the features extracted by the network model more consistent with human semantics, we use the existing human parsing model to perform human parsing on the training set images, divide the images into 20 semantic parts, and then further classify and merge them into 6 human semantic labels; 2) Preprocess the training set images and perform data augmentation. Resize the images to 256×128 and randomly crop, flip, and erase them, each with a probability of 50%. Then, use occluded object images and local pedestrian images to perform occlusion data augmentation on the training set images, with a probability of 50%. The probability of augmenting the occluded object image and the local pedestrian image is 25% each. 3) Construct a human semantic feature parsing model based on HRNet-32 and train it; the human semantic feature parsing model extracts three features for each input image: one local semantic feature containing five human semantic parts, one foreground semantic feature, and one global semantic feature. 4) Use an HRNet-48-based human key point feature extraction module to extract 13 human key point features, feed them into a convolutional neural network GCN for processing to obtain 13 GCN key point features; use the key point-local semantic feature similarity matching method to match and fuse the GCN key point features with human semantic features to enhance human key point information and obtain a local semantic feature containing human key point information. 5) The loss function is calculated using three features: local semantic features containing human keypoint information, foreground semantic features, and global semantic features. 6) After training, the network extracts one local semantic feature containing five human semantic information and one foreground semantic feature. The two features are processed by a batch normalization layer, and then the two features after batch normalization are concatenated along the channel direction to obtain the final feature. The Euclidean distance between the features of each sample in the gallery set and the query set is calculated using the final features. CMC and mAP are calculated based on the distance between the samples in the gallery set and the samples in the query set, thus achieving the re-identification of pedestrian samples.

2. The method for re-identifying occluded pedestrians according to claim 1, characterized in that: Step 1) involves selecting several occluded images from the dataset and manually cropping several occluded images of different sizes. The specific steps are as follows: 1-1) Randomly select several images with occlusions from the dataset and manually crop the images of the occlusion parts; In order to further simulate the occlusion situation in the actual scene, we generate two types of occlusion images, using the occlusion image with a height greater than twice the width as the vertical occlusion, and the occlusion image with a width greater than twice the height as the horizontal occlusion. 1-2) Randomly select and crop images from the training set to generate local pedestrian images, with a size of 1 / 4 to 1 / 2 of the original size of the selected training set images; since the training set does not contain occlusions, the cropped part should be part of the pedestrian's body, with a small probability of containing background information; 1-3) After the existing human body parsing model SCHP is pre-trained on the human body parsing dataset LIP, each image is segmented into 20 semantic parts. We then group and classify these 20 semantic parts to obtain 6 semantic parts, which correspond to "background", "head", "upper body", "accessories", "legs" and "feet".

3. The method for re-identifying occluded pedestrians according to claim 1, characterized in that: Step 2) involves preprocessing the training set images and performing data augmentation. The specific steps are as follows: 2-1) Resize the image to 256×128, and randomly crop, flip horizontally, and erase the image, with a probability of 50% for each step. 2-2) Use occlusion images and local pedestrian images to perform occlusion data augmentation on the training set images with a probability of 50%. The probability of augmenting the occluded image and the local pedestrian image is 25% each. Occlusion is performed in one of the four directions of the image (up, down, left, right) with the same probability.

4. The method for re-identifying occluded pedestrians according to claim 1, characterized in that: Step 3) involves constructing and training a human semantic feature parsing model based on HRNet-32; the specific steps are as follows: 3-1) Take n training set images And the personal identification tags of each image. The input human semantic feature parsing model classifies pixel-level features of pedestrian images to extract local semantic features of five individuals; the image... Input the human semantic feature parsing model and utilize the backbone network The mapping function obtains the image global feature map The formula is as follows: ； Among them, the backbone network This is an HRNet-32 network model pre-trained on the ImageNet dataset. These are the parameters of the backbone network and the global feature map. The dimensions are 64×32; c, h, and w are the global feature map channels, height, and width; using Indicates spatial location Features of the location; 3-2) After processing through a linear layer, we obtain confidence maps for six different semantic parts and a foreground confidence map, which are respectively , , , , , and The calculation formula is as follows: ； in Indicates belonging to the semantic part pixels The confidence level is noteworthy. Confidence of the background portion; Foreground semantic confidence plot It is derived by summing the semantic confidence maps of 5 individuals; the calculation formula is as follows: ； The total confidence graph is denoted as It is composed of six semantic confidence maps spliced ​​along the channel direction; 3-3) Passing the confidence map through a softmax layer yields five local semantic masks. , , , and 1 foreground semantic mask The formula for calculating the local semantic mask is as follows: ； Similarly, foreground confidence plots can be used. Computational foreground mask ; The semantic features of the five parts are calculated using the following formula: ； One foreground semantic feature is obtained by calculating the following formula: ； One global semantic feature is obtained by calculating using the following formula. ； In the formula, Represents the pixels belonging to semantic part k. confidence level and This indicates element-wise multiplication. This indicates a global average pooling operation. For the first Partial semantic features; local human semantic features The semantic features of the five parts are spliced ​​together along the channel direction.

5. The method for re-identifying occluded pedestrians according to claim 1, characterized in that: Step 4) involves using an HRNet-48-based human keypoint feature extraction module to extract 13 keypoint features from the human body. These features are then fed into a Generative Convolutional Neural Network (GCN) for processing, resulting in 13 GCN keypoint features. A keypoint-local semantic feature similarity matching method is then used to match and fuse the GCN keypoint features with the human body semantic features, thereby enhancing the human body keypoint information and obtaining a single local semantic feature containing human body keypoint information. The specific steps are as follows: 4-1) Take a sheet of size Image Input a human keypoint feature extraction module based on HRNet-48 and extract 13 keypoint heatmaps. The size of each heatmap is Each heatmap's maximum response point corresponds to a key point; key point heatmap The heatmap is obtained after processing through a fully connected layer. This heatmap and global features Same size; Will and After performing the multiplication operation, 13 pose guidance features can be obtained. Key features; 4-2) At this point, the weights of the 13 keypoint features are the same. However, due to the presence of occlusion, we want the weights of the keypoint features in the occluded areas to be as small as possible, and the weights of the keypoint features in the unoccluded areas to be as large as possible. To achieve this, we use a convolutional neural network (GCN) to update the weights; the 13 keypoint features... 1 foreground semantic feature An adjacency matrix that reflects the adjacency relationships of one key point. The data is fed into a convolutional neural network for processing. Theoretically, the similarity between unoccluded keypoint features and foreground features is higher than that between occluded keypoint features and foreground features. Using this principle, we can update the weights based on the similarity between keypoint features and foreground features. The weights are updated based on the similarity between keypoint features and foreground features, resulting in an updated weight matrix. Ultimately, 13 GCN keypoint features were obtained. The calculation formula is as follows: ； in and Both are fully connected layers; This is an element-wise multiplication operation; 4-3) To enhance keypoint features within local semantic features, we utilize a keypoint-local semantic feature similarity matching method. After the input image is processed by the human semantic feature parsing model, a local semantic feature containing five parts of human semantic information can be obtained; keypoints - The local semantic feature similarity matching method matches and fuses 13 GCN keypoint features and 5 parts of human semantic features; The specific method involves calculating the features of 13 GCN keypoints. Human semantic features in 5 parts The cosine similarity between the components is used to select the components with the highest cosine similarity and integrate them, so that the human semantic features contain relevant key point information. First, the matching of GCN keypoint information and human semantic features needs to be completed using the following formula: ； in For the first Key features of an individual's body For the first Partial semantic features of the human body; Then, the integration of GCN keypoint information and human semantic features is completed using the following formula: ； in Then it is the first matching generated. A local human body feature containing key point information. To match some human body characteristics The most similar human keypoint features are numbered; the final local semantic features of the human body containing keypoint information are: .

6. The method for re-identifying occluded pedestrians according to claim 1, characterized in that: Step 5) The calculation process of the loss function is as follows: the triplet loss function and the cross-entropy loss function are used to guide the training and optimization of the network model; the local semantic features, foreground semantic features, and global semantic features containing key point information are also constrained by the cross-entropy loss function and the triplet loss function, respectively; in addition, in order to enable the network model to more accurately locate each semantic part of the human body and extract human semantic features, a human body parsing loss function is set, and the human body semantic labels generated in the data preparation stage and the global confidence map output by the network model are used to participate in the calculation of the loss function, thereby optimizing the network model during training.

7. A pedestrian re-identification device, characterized in that, The device includes: Memory is used to store executable computer programs; A processor is configured to execute an executable computer program stored in the memory to implement the occluded pedestrian re-identification method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, A computer program is stored for execution by a processor to implement the occluded pedestrian re-identification method as described in any one of claims 1-6.

Images