A person re-identification method based on multi-scale batch feature discarding network

Abstract

The present invention discloses a pedestrian re-identification method based on a multi-scale batch feature discarding network, which relates to the field of pedestrian recognition technology. The method specifically comprises the following steps: S1, extracting features using an IBN-Net50-a backbone network; S2, fusing a batch feature discarding method on the last two convolutional layers; S3, splicing features of different dimensions to include more useful information at shallow and deep levels. IBN includes IN and BN. The robust features extracted by IN do not change with some appearance changes, such as color, style, and virtual / real, while BN retains information related to content. The present invention uses an improved network model with IBN-Net50-a as the backbone network, and fuses a batch feature discarding method on the last two convolutional layers to enhance focused feature learning in local areas. A multi-scale feature fusion strategy is adopted, and different pooling methods are used for the feature maps output by different branches to improve the robustness, learning efficiency, and recognition rate of the network.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian recognition, and in particular to a pedestrian re-identification method using a multi-scale batch feature discarding network. Background Art

[0002] Person Re-identification (Person ReID) aims to retrieve individuals of interest using multiple, non-overlapping camera feeds. Person Re-ID has numerous applications in intelligent video surveillance systems, smart security, and human-computer interaction, such as searching for suspects, cross-border tracking, and trajectory analysis. It is becoming an increasingly important tool for maintaining public safety and social stability. Therefore, research on Person Re-ID is of great significance, yet also carries challenges.

[0003] In order to eliminate the distraction of irrelevant parts in pedestrian images and focus more attention on the main part of the pedestrian, an end-to-end network (MGN) is designed. By reducing the area of the representation area and using it as a classification task to learn local features, and combining global features, a powerful pedestrian feature representation is obtained, which has achieved good results. A convolutional layer regularization method (DropBlock) is proposed. By randomly discarding units in continuous areas of the feature map, the accuracy and robustness of the model are improved. A batch deletion block (BDB) network is proposed. This method randomly discards a part of a batch feature map, covers the semantic part of the input feature map, and obtains local features in a forced manner. The IBN-Net50-a network architecture is proposed, which can greatly improve the domain adaptation ability of the network and has a significant effect on improving the recognition rate of pedestrian re-identification. However, the simple IBN-Net50-a network cannot solve the problems of pedestrian occlusion and posture changes. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the shortcomings of the existing technology, the present invention provides a pedestrian re-identification method based on a multi-scale batch feature discarding network, which solves the problems of occlusion and posture change in pedestrian re-identification and the defect of low recognition rate of the current network.

[0006] (2) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a pedestrian re-identification method based on a multi-scale batch feature discarding network.

[0008] A person re-identification method based on a multi-scale batch feature discarding network specifically includes the following steps:

[0009] S1. Extract features using IBN-Net50-a as the basic backbone network;

[0010] S2, fusion batch feature discarding method for the last two convolutional layers;

[0011] S3. Splicing features of different dimensions can contain more shallow and deep useful information.

[0012] Preferably, in step S1, IBN includes IN and BN, the robust features extracted by IN do not change with some appearance changes, such as color, style, virtual / real, and BN retains content-related information.

[0013] Preferably, in step S1, IBN-Net50-a only adds IN to the first three groups (conv2_x to conv_4-x) of ResNet50, and the other groups remain unchanged.

[0014] Preferably, in step S2, given the feature maps calculated by the basic backbone network from a batch of input images, the batch feature dropout layer randomly drops the same feature map area, and all units in the dropped area are zero.

[0015] Preferably, in step S2, IBN-Net50-a is used as the basic backbone network, and the batch feature dropout method is integrated into the last two convolutional layers to enhance the focused feature learning of the local area;

[0016] The global maximum pooling is used for the fusion batch feature discarding method, and the global average pooling is used for the others. Convolution dimensionality reduction processing is used to improve the learning efficiency of the network. The improved pedestrian re-identification network structure mainly consists of backbone network, convolution layer, FD module, pooling layer and fully connected layer.

[0017] Preferably, the IBN-Net50-a network consists of three parts:

[0018] 1) It is mainly responsible for extracting feature maps of different scales from different depths of the model, denoted as f1, f2 and f3 respectively. The scale of f1 is 48×16×512, the scale of f2 is 24×8×1024, the stride of the last convolution is set to 1, and the scale of f3 is 24×8×2048;

[0019] 2) The extracted feature maps are processed and divided into four parts. The batch feature discarding method is integrated for f2 and f3 respectively. The global maximum pooling layer is used to obtain the feature vector dimension of f2 is 1024 and the feature vector dimension of f3 is 2048, forcing the network to obtain local detail features. No other processing is done on f3. The global average pooling layer is used to obtain the feature vector dimension of f3 is 2048. Finally, the convolution kernel is 1×1 for dimensionality reduction, and its dimension is further reduced to 512. Finally, the normalized features are spliced to obtain the final feature containing information at different levels;

[0020] 3) Classify and metrically learn the processed feature vectors using a joint loss function of label smoothing loss and triplet loss. The features after the pooling layer of feature maps of different scales use the triplet loss function, and the final fully connected layer uses the label smoothing loss function. When testing inference, all output features are concatenated and then sorted by calculating the Euclidean distance.

[0021] Preferably, in the person re-identification task, the triplet loss optimization process is to optimize the distance between the target sample and the positive sample and the negative sample, so that the distance between the same pedestrian (within the class) is closer and the distance between different pedestrians (between classes) is farther.

[0022] Preferably, a triplet is described as (a, p, n), and the idea of triplet loss is formalized using Euclidean distance as formula (1), where P is the number of pedestrian IDs, K is the number of each pedestrian image, and margin represents a mandatory interval;

[0023]

[0024] The cross entropy loss function is a commonly used loss function for classification tasks, as shown in formula (2), where N is the number of pedestrian IDs in the dataset, n is the pedestrian label, and p i is the probability that the network predicts that the pedestrian belongs to the pedestrian with label i;

[0025]

[0026] The label smoothing loss function is shown in formula (3), where ε is the error rate, 1-ε is the true label, and ε=0.1 is set;

[0027]

[0028] The fully connected layer is optimized using a label smoothing function, as shown in Equation (4):

[0029]

[0030] (3) Beneficial effects

[0031] The present invention provides a person re-identification method using a multi-scale batch feature discarding network. Compared with the existing technology, it has the following advantages:

[0032] This improved network model, using IBN-Net50-a as the backbone, incorporates batch feature dropout into the last two convolutional layers to enhance focused feature learning in local regions. A multi-scale feature fusion strategy employs different pooling strategies for feature maps output by different branches, improving network robustness, learning efficiency, and recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1This is a schematic diagram of the IBN module structure of the present invention;

[0034] Figure 2 This is a structural diagram of the batch feature discarding feature processing process of the present invention;

[0035] Figure 3 This is a schematic diagram showing the structure of the comparison of different discarding methods of the same-batsh in the present invention;

[0036] Figure 4 This is a schematic diagram of the structure of a person re-identification network using a multi-scale batch feature discarding network according to the present invention;

[0037] Figure 5 Schematic diagram of the triplet loss learning structure of the present invention. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field without creative work within the scope of protection of the present invention are within the scope of protection of the present invention.

[0039] See also Figure 1-5 The embodiment of the present invention provides a technical solution: a pedestrian re-identification method using a multi-scale batch feature discarding network, which specifically includes the following steps:

[0040] S1. Extract features using IBN-Net50-a as the basic backbone network;

[0041] Specifically, in step S1, IBN is a careful combination of IN (Instance Normalization) and BN (Batch Normalization). IN extracts robust features that are invariant to appearance changes, such as color, style, and virtual / real, while BN preserves content-related information. Both have their limitations: BN uses a mini-batch mean and variance to normalize each feature channel, while IN uses single-sample statistics. Clearly, BN has a significant advantage in speeding up training. IBN rationally combines IN and BN, improving both learning and generalization capabilities.

[0042] IBN-Net50-a only adds IN to the first three groups (conv2_x to conv_4-x) of ResNet50, leaving the other groups unchanged. For the residual block, the first half of the channels of the first convolution output are added with IN, and the second half of the channels are added with BN. The careful combination of IN and BN gives IBN-Net50-a better generalization and more robust recognition capabilities.

[0043] S2, fusion batch feature discarding method for the last two convolutional layers;

[0044] Specifically, in step S2, given the feature maps calculated by the basic backbone network from a batch of input images, the batch feature drop layer randomly drops the same feature map area, and all units in the dropped area are zero. The feature map processing process is as follows Figure 2 As shown;

[0045] The height and width of the dropout region vary depending on the task. Generally speaking, a rectangular region is the dropout region, and the entire width in person re-identification is the width of the rectangle. Unlike DropBlock, the dropout probability hyperparameter does not need to be changed during the training of the batch feature dropout layer.

[0046] Taking IBN-Net50-a as the backbone network, the batch feature dropout method is integrated into the last two convolutional layers to enhance the focused feature learning of local areas. In order to improve the robustness of the network and the effectiveness of the batch feature dropout method, global maximum pooling is used for the fused batch feature dropout method, and global average pooling is used for the others. Convolutional dimensionality reduction processing is used to improve the learning efficiency of the network. The improved pedestrian re-identification network structure mainly consists of the backbone network, convolutional layer, FD module, pooling layer and fully connected layer, where FD stands for Feature Dropping.

[0047] The IBN-Net50-a network consists of three parts:

[0048] 1) It is mainly responsible for extracting feature maps of different scales from different depths of the model, denoted as f1, f2 and f3 respectively. The scale of f1 is 48×16×512, the scale of f2 is 24×8×1024, the stride of the last convolution is set to 1, and the scale of f3 is 24×8×2048;

[0049] 2) The extracted feature maps are processed and divided into 4 parts. The batch feature discarding method is integrated with f2 and f3 respectively. The global maximum pooling layer is used to obtain the feature vector dimension of f2 is 1024 and the feature vector dimension of f3 is 2048, forcing the network to obtain local detail features. In addition, in order to help the training of these branches integrated with the FD module, no other processing is done on f3. The global average pooling layer is used to obtain the feature vector dimension of f3 is 2048. Finally, the convolution kernel is 1×1 for dimensionality reduction, and its dimension is further reduced to 512. Finally, the normalized features are spliced to obtain the final features containing information at different levels.

[0050] 3) Classify and metrically learn the processed feature vectors using a joint loss function of label smoothing loss and triplet loss. The features after the pooling layer of feature maps of different scales use the triplet loss function, and the final fully connected layer uses the label smoothing loss function. When testing inference, all output features are spliced together and then sorted by calculating the Euclidean distance.

[0051] The triplet loss function was originally designed for face recognition. Since metric learning is also widely used in person re-identification tasks, the triplet loss optimization process in the person re-identification task is to optimize the distance between the target sample and the positive and negative samples, so that the distance between the same pedestrian (within the class) is closer and the distance between different pedestrians (between classes) is farther.

[0052] A triplet is described as (a, p, n), and the idea of triplet loss is formalized using Euclidean distance as formula (1), where P is the number of pedestrian IDs, K is the number of pedestrian images, and margin represents a mandatory interval;

[0053]

[0054] The cross entropy loss function is a commonly used loss function for classification tasks, as shown in formula (2), where N is the number of pedestrian IDs in the dataset, n is the pedestrian label, and p i is the probability that the network predicts that the pedestrian belongs to the pedestrian with label i;

[0055]

[0056] However, the cross entropy loss function assumes that all pedestrian labels are correct. Some pedestrians may have incorrect labels, which can easily cause overfitting in training. To avoid this, label smoothing is used. It is hoped that the network can accept a small number of incorrect labels. The classification loss function in this paper uses the label smoothing loss function, as shown in formula (3). ε is the error rate, and 1-ε is the true label. Set ε = 0.1;

[0057]

[0058] To extract features with stronger generalization capabilities, this paper uses a triplet loss function and a label smoothing loss function to optimize the network. Different loss functions can lead to significant differences in the feature information extracted by the network.

[0059] The person re-identification task involves both measurement and classification, so triplet loss and label smoothing loss are applied in this paper. However, the mechanisms of the two loss functions are different. The triplet loss function reduces the Euclidean distance between the same pedestrians and increases the Euclidean distance between different pedestrians. The label smoothing loss function measures the difference between two probability distributions for classification tasks. Therefore, if they act on the same target at the same time, the loss function will not decrease.

[0060] Therefore, this paper optimizes the features after the feature map pooling layer with the triplet loss function, and finally optimizes the fully connected layer with the label smoothing function. Therefore, the loss function of formula (4) is applied to this paper;

[0061]

[0062] S3. Splicing features of different dimensions can contain more shallow and deep useful information.

[0063] Table 2 shows that replacing the base network with a ResNet50 deep network with IBN and applying the batch feature discarding module to the base ResNet50 deep network improves accuracy. IBN represents replacing the base ResNet50 network with a deep network with IBN-Net50-a, and FD represents embedding the batch feature discarding module in the network.

[0064] For the Market1501 dataset, adding a batch feature discarding module to the base network improved person re-identification accuracy. Replacing the base network ResNet50 with an IBN-Net50-a deep network also improved performance, with Rank1 accuracy increasing by 1.3% and mAP increasing by 2.1%. Replacing the base network with an IBN-Net50-a deep network and embedding a batch feature discarding module also significantly improved performance, with Rank1 accuracy increasing by 1.5% and mAP increasing by 2.8%.

[0065] For the DuKeMTMC-reID dataset, by replacing the base network with a deep network with IBN-Net50-a and embedding a batch feature discarding module, it can be found that the Rank1 accuracy is improved by 1.8% and the mAP is improved by 2.8%.

[0066] The visualization examples selected samples with occlusion and posture changes. The samples given in the DukeMTMC-reID dataset are severely occluded by pedestrians, vehicles and obstacles. However, this paper uses the batch feature discarding method to force the network to extract not only global features but also more detailed local features such as backpacks and hairstyles to correctly match the same person. The visualization results show that the method in this paper has good adaptability to the occlusion and posture change problems in pedestrian re-identification, as well as the low recognition rate of the current network.

[0067] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A person re-identification method based on a multi-scale batch feature discarding network, characterized by: The specific steps include: S1. Extract features using IBN-Net50-a as the basic backbone network; S2, fusion batch feature discarding method for the last two convolutional layers; S3. Splicing features of different dimensions can contain more shallow and deep useful information; In step S1, IBN includes IN and BN. IN extracts robust features that do not change with changes in color, style, virtual or real appearance, while BN retains content-related information. In step S1, IBN-Net50-a only adds IN to the first three groups of ResNet50, namely conv2_x to conv_4-x, and the other groups remain unchanged; In step S2, given the feature maps computed by the basic backbone network from a batch of input images, the batch feature dropout layer randomly drops the same feature map region, and all units in the dropped region are zero; In step S2, IBN-Net50-a is used as the backbone network, and the batch feature dropout method is integrated into the last two convolutional layers to enhance the focused feature learning of local areas; The global maximum pooling is used for the fusion batch feature discarding method, and the global average pooling is used for the others. Convolutional dimensionality reduction processing is used to improve the learning efficiency of the network. The improved pedestrian re-identification network structure consists of a backbone network, a convolutional layer, an FD module, a pooling layer, and a fully connected layer. IBN-Net50-a overall network is used for: 1) Responsible for extracting feature maps of different scales from different depths of the model, denoted as f1, f2 and f3 respectively. The scale of f1 is 48×16×512, the scale of f2 is 24×8×1024, the stride of the last convolution is set to 1, and the scale of f3 is 24×8×2048; 2) The extracted feature maps are processed and divided into four parts. The batch feature discarding method is integrated for f2 and f3 respectively. The global maximum pooling layer is used to obtain the feature vector dimension of f2 is 1024 and the feature vector dimension of f3 is 2048, forcing the network to obtain local detail features. No other processing is done on f3. The global average pooling layer is used to obtain the feature vector dimension of f3 is 2048. Finally, the convolution kernel is 1×1 for dimensionality reduction, and its dimension is further reduced to 512. Finally, the normalized features are spliced to obtain the final feature containing information at different levels; 3) Classify and metrically learn the processed feature vectors using a joint loss function of label smoothing loss and triplet loss. The features after the pooling layer of feature maps of different scales use the triplet loss function, and the final fully connected layer uses the label smoothing loss function. When testing inference, all output features are spliced together and then sorted by calculating the Euclidean distance. In the person re-identification task, the triplet loss optimization process is to optimize the distance between the target sample and the positive and negative samples, so that the distance within the same person group is closer and the distance between different people groups is farther; A triplet is described as (a, p, n), and the idea of triplet loss is formalized using Euclidean distance as formula (1), where P is the number of pedestrian IDs, K is the number of pedestrian images, and margin represents a mandatory interval; The cross entropy loss function is a commonly used loss function for classification tasks, as shown in formula (2), where N is the number of pedestrian IDs in the dataset, n is the pedestrian label, and p i is the probability that the network predicts that the pedestrian belongs to the pedestrian with label i; The label smoothing loss function is shown in formula (3), where ε is the error rate, 1-ε is the true label, and ε=0.1 is set; The fully connected layer is optimized using a label smoothing function, as shown in Equation (4):

Images