A Deep Learning-Based Method for Re-identifying Pedestrians Changing Clothes

Abstract

This invention discloses a deep learning-based method for re-identifying pedestrians changing clothes, comprising the following steps: preparing a dataset for re-identifying pedestrians changing clothes; preprocessing the dataset using pedestrian parsing to extract distinctive identity features of pedestrians unrelated to clothing; constructing a deep learning network model with a gated spatial attention mechanism, the input of which is a preprocessed pedestrian image, using the pedestrian image set and pedestrian identity information as supervision signals to perform supervised training on the gated spatial attention network, during which two key loss functions, identity loss and triplet loss, are calculated, and the network parameters are updated using backpropagation; training the model using a two-stage loss optimization strategy; inputting query pedestrian images into the trained model as input to the network to obtain a list of pedestrian targets with high similarity to the query target. This invention significantly improves the accuracy of pedestrian re-identification tasks changing clothes and has good generalization performance.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically to a method for re-identifying pedestrians changing clothes based on deep learning. Background Technology

[0002] Traditional pedestrian re-identification (ReID) [1] technology aims to identify the same pedestrian at different times or from different camera angles, providing technical support for intelligent monitoring, people flow statistics and other fields. These technologies mainly focus on the appearance features of pedestrians, such as clothing color, style and body features, in order to achieve cross-scene personnel identification. Clothing-changing pedestrian re-identification (CC-ReID) [2] is a challenge in computer vision technology, aiming to identify the same pedestrian who has changed clothes at different times or in different places. This technology has important application value in video surveillance, criminal investigation, personal identity verification and other fields. Compared with traditional pedestrian re-identification technology, clothing-changing pedestrian re-identification is more challenging because it requires the model to identify the same person after significant changes in visual appearance.

[0003] Currently, in the field of pedestrian re-identification, the mainstream methods are mainly divided into two categories. One category learns clothing-independent features by introducing multimodal information (such as contour sketches [3], gait information [4], and skeletal information [5]). However, this method may ignore important internal information of the human body, such as facial expressions, arm postures, and leg shapes. These details are crucial in pedestrian re-identification tasks because they can provide richer and more discriminative features. At the same time, introducing these additional information will also increase the computational burden during training. The other category is based on generative adversarial networks to learn the appearance changes of pedestrians across time periods. However, this method inevitably damages the original RGB image and introduces unrealistic or distorted details. This may lead to model instability and training failure, and requires long training time and fine parameter tuning.

[0004] References:

[0005] [1]Gheissari N,Sebastian TB,Hartley R.Person reidentification using spatiotemporal appearance[C] / / 2006IEEE computer society conference oncomputer vision and pattern recognition(CVPR'06).IEEE,2006,2:1528-1535.

[0006] [2]Zhang P, Wu Q, Xu J, et al. Long-term person re-identification using true motion from videos [C] / / 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018: 494-502.

[0007] [3]Yang Q,Wu A,Zheng W S.Person re-identification by contour sketchunder moderate clothing change[J].IEEE transactions on pattern analysis and machine intelligence,2019,43(6):2029-2046.

[0008] [4]Hong P, Wu T, Wu A, et al. Fine-grained shape-appearance mutual learning for cloth-changing person re-identification[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2021:10513-10522.

[0009] [5] Qian X, Wang W, Zhang L, et al. Long-term cloth-changing person re-identification[C] / / Proceedings of the Asian Conference on ComputerVision.2020. Summary of the Invention

[0010] The purpose of this invention is to provide a deep learning-based method for re-identifying pedestrians changing clothes, in order to solve the problem of high computational cost in existing general benchmark schemes.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] A deep learning-based method for re-identifying pedestrians changing clothes includes the following steps:

[0013] Step 1: Prepare the pedestrian re-identification dataset for changing clothes. The pedestrian re-identification dataset for changing clothes contains a set of pedestrian images and their corresponding pedestrian identity information and the camera ID information of the camera that took the image.

[0014] Step 2: Preprocess the pedestrian re-identification dataset from Step 1 by analyzing pedestrians, indirectly erasing the clothing information of pedestrians in the re-identification dataset and extracting pedestrian distinguishable features unrelated to clothing.

[0015] Step 3: Build a deep learning network model with a gated spatial attention mechanism. The input is the pedestrian image preprocessed in Step 2. Use the pedestrian image set and pedestrian identity information as supervision signals to perform supervised training on the gated spatial attention network. During the training, calculate two key loss functions: identity loss and triplet loss, and update the network parameters through backpropagation.

[0016] Step 4: Use a two-stage loss optimization strategy to train the model obtained in Step 3;

[0017] Step 5: Input the query pedestrian images into the model trained in Step 4 as input to the network to obtain a list of pedestrian targets with high similarity to the query target.

[0018] In step 1, the pedestrian image set includes images of pedestrians' clothing appearance and body posture.

[0019] In step 1, the pedestrian re-identification dataset for changing clothes is the PRCC real dataset.

[0020] In step 2, the SCHP network, pre-trained on the pedestrian parsing LIP dataset, is used to preprocess the pedestrian re-identification dataset. The SCHP network can accurately distinguish and parse the various parts of the human body by deeply analyzing pedestrian images, thereby filtering out clothing-related information and retaining semantic information unrelated to clothing. Then, it is fused with RGB features to indirectly remove clothing features.

[0021] In step 3, ResNet50, pre-trained on the ImageNet dataset, is selected as the baseline model. ResNet50 has four stages, with a gated channel attention mechanism connected after stage 1-stage 3. This mechanism first uses the channel attention mechanism to learn and analyze the importance of each feature channel and assigns different weight coefficients to each channel, which can effectively enhance key features and suppress unimportant features. Combined with the gating mechanism, the model can selectively focus on and utilize key information in the input features while filtering out noise or secondary information.

[0022] In step 4, two loss functions are used to optimize the model: identity loss and triplet loss. The two-stage optimization strategy refers to using only identity loss in the first n rounds of training. The core goal of this stage is to enable the model to effectively identify pedestrians wearing the same clothes and to master some basic skills for identifying pedestrians changing clothes. After the nth round, triplet loss is added for training.

[0023] In step 5, the pedestrian target list is sorted by similarity, which makes it easier to identify pedestrians with the same identity based on the pedestrian target list.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) This invention proposes a controllable deentanglement feature extraction method, which extracts pedestrian distinguishable identity features unrelated to clothing, aiming to reduce the interference of clothing changes on pedestrian re-identification performance. At the same time, this feature is also used for final identity prediction in the inference stage.

[0026] (2) This invention incorporates a gated channel attention mechanism into the baseline model to enhance the representation of important features and suppress unimportant features, thereby improving the model's expressive power.

[0027] (3) This invention employs an innovative two-stage optimization strategy during the training phase. This enhances the model's basic learning capabilities and improves its ability to handle complex scenarios.

[0028] (4) The present invention has conducted extensive experiments on two standard pedestrian re-identification datasets for changing clothes, demonstrating that it outperforms the current state-of-the-art methods in pedestrian re-identification tasks for changing clothes. Attached Figure Description

[0029] Figure 1 This invention presents a complete network structure with a controllable unentanglement feature extraction method and an embedded gated channel attention mechanism. Detailed Implementation

[0030] The invention will now be further explained with reference to the accompanying drawings.

[0031] The present invention provides a deep learning-based method for re-identifying pedestrians changing clothes, comprising the following steps:

[0032] Step 1, Dataset Preparation: The raw data for pedestrian re-identification is usually surveillance video, which requires manual or algorithmic extraction of key pedestrian information. For example, pedestrian images can be cropped from surveillance video frames. This invention uses two pre-cropped datasets, PRCC and VC-Clothes, as experimental datasets.

[0033] Step 2, Controlled Deentanglement Feature Extraction: The SCHP model pre-trained on the LIP dataset is used to perform pedestrian parsing processing on the changing-clothes pedestrian re-identification dataset. Each image in the LIP dataset contains the main body parts, clothing information, and background, represented by pixel-level annotations. By applying the SCHP model to process the changing-clothes pedestrian re-identification dataset, a semantic segmentation annotation map corresponding to each image can be obtained, containing 20 body part category information. To obtain feature maps unrelated to clothing, the pixel values ​​of the following parts are extracted from the semantic segmentation map: ['Background', 'Hair', 'Face', 'Left-arm', 'Right-arm', 'Left-leg', 'Right-leg', 'Left-shoe', 'Right-shoe']. Subsequently, the semantic segmentation map is converted into a double-precision grayscale image and then fused with the corresponding RGB image, as shown below. Figure 1 As shown. This indirectly erases clothing-related features, achieving controlled de-entanglement.

[0034] Step 3, Gated Channel Attention: This invention proposes a gated channel attention mechanism to improve the performance of the ResNet50 model in person re-identification tasks. The core idea of ​​this mechanism is to finely control the importance of feature channels, enabling the model to more effectively focus on the key features needed to identify pedestrians. This mechanism is implemented at different stages of the ResNet network to adapt to the characteristics of the features at each stage, such as... Figure 1 As shown.

[0035] In the first three stages of ResNet (stages 1 to 3), channel attention and gating mechanisms are integrated. Specifically, the channel attention part employs an efficient channel attention model that dynamically adjusts channel weights by considering local interactions between channels. This allows the network to focus more on features helpful to the current task, such as facial, body, and arm features. Specifically, the channel attention module first performs global average pooling on the input feature map X, then obtains the channel weights ω through one-dimensional convolution and the sigmoid activation function. Then, the original input feature map is multiplied element-wise by the weights to obtain the final output feature map A, as shown in the following expression:

[0036] ω=σ(Conv 1D (Avgpool(X))),

[0037] A=X⊙ω,

[0038] Meanwhile, the gating mechanism achieves self-regulation based on the features themselves. Specifically, feature A first passes through a sigmoid activation function, obtaining a weight between 0 and 1. This weight is then multiplied by the original feature A, achieving feature self-regulation. This combination strategy allows the model to emphasize key channels while maintaining an effective balance between features when extracting pedestrian features. The formula is expressed as follows, where G represents the feature input to the next stage:

[0039] G = A⊙σ(A).

[0040] For ResNet's fourth stage, only gating was chosen. This decision was based on a deep understanding of the feature representations at different stages of the network. In this deeper stage, features have become more abstract and complex. Direct channel recalibration could lead to excessive feature distortion. Therefore, gating was used to individually adjust features, thus maintaining the integrity of higher-level features. This design gives the model enhanced robustness and accuracy in complex tasks such as recognizing changes in pedestrian clothing.

[0041] Step 4, Two-Stage Loss Optimization Strategy: A two-stage optimization strategy is used to train the model to re-identify pedestrians undergoing clothing changes. This approach not only emphasizes the importance of learning basic features but also aims to enhance the model's ability to handle complex situations. In the first 'n' epochs of model training, the model focuses on training using identity loss. The core of this stage is to enable the model to effectively identify pedestrians wearing the same clothing. Identity loss helps learn the basic features of individuals, such as body shape, posture, and facial features. The purpose of this stage is to build a solid foundation so that the model can more effectively adapt to complex tasks in subsequent learning stages. The identity loss is represented as follows:

[0042]

[0043] Where x i y represents the i-th element of the real label. i This represents the probability that x, as predicted by the model, belongs to the (i-th)th category.

[0044] After entering the nth epoch, a triplet loss is introduced, which works in conjunction with the identity loss on the model. The purpose of introducing the triplet loss is to enhance the model's ability to distinguish pedestrians wearing different clothes. The formula for the triplet loss is as follows:

[0045]

[0046] Where P(a) and N(a) represent the sets of samples with the same identity as anchor a and the sets of samples with different identities, respectively, and d(·) is the distance metric function between feature vectors. and These represent the feature vectors of the anchor point, positive sample, and negative sample, respectively. Margin is a hyperparameter used to ensure that there is a certain spatial interval between positive and negative samples.

[0047] This two-stage training method enables the model to exhibit superior performance in the pedestrian re-identification task involving changing clothes. This method not only improves the model's basic recognition ability but also enhances its robustness and accuracy in handling complex and diverse scenarios. The overall loss function of the two-stage method is as follows:

[0048]

[0049]

[0050] in Indicates loss of identity. Let represent the triplet loss, with λ1 and λ2 used as hyperparameters to balance these two losses. Ultimately, this is achieved by minimizing... and To optimize the network model.

[0051] In this invention, the model is trained using pre-processed pedestrian sample data for 120 training epochs. The learning rate is set to 3.5e-4, gradually decreasing to 0.1 every 20 epochs. After the entire training period, the trained parameters are saved for subsequent detection processing. The hyperparameter margin is set to 0.3 based on previous experience. The optimal values ​​for the weight hyperparameters λ1 and λ2, used to balance the contributions of different loss functions, are found on different datasets through grid search. In the PRCC and VC-Clothes datasets, values ​​of λ1 and λ2 are optimally set to 0.1 and 0.9, respectively.

[0052] Step 5: Organize the pedestrian target images to be detected, select query images with more features and input them into the trained model, use all pedestrian sample images generated from surveillance videos as the detection set for detection and matching, the model will output pedestrian image samples with high matching degree, arranged from high to low similarity, the operator finds the same pedestrian target in the pedestrian target list and performs pedestrian tracking.

[0053] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based method for re-identifying pedestrians changing clothes, characterized in that: Includes the following steps: Step 1: Prepare the pedestrian re-identification dataset for changing clothes. The pedestrian re-identification dataset for changing clothes contains a set of pedestrian images and their corresponding pedestrian identity information and the camera ID information of the camera that took the image. Step 2: Preprocess the pedestrian re-identification dataset from Step 1 by analyzing pedestrians, indirectly erasing the clothing information of pedestrians in the re-identification dataset and extracting pedestrian distinguishable features unrelated to clothing. Step 3: Build a deep learning network model with a gated spatial attention mechanism. The input is the pedestrian image preprocessed in Step 2. Use the pedestrian image set and pedestrian identity information as supervision signals to perform supervised training on the gated spatial attention network. During the training, calculate two key loss functions: identity loss and triplet loss, and update the network parameters through backpropagation. The steps for building a deep learning network model with a gated spatial attention mechanism are as follows: A ResNet50 pre-trained on the ImageNet dataset is selected as the baseline model. ResNet50 has four stages, with a gated channel attention mechanism added after stage 1-stage 3. This mechanism first uses the channel attention mechanism to learn and analyze the importance of each feature channel, and assigns different weight coefficients to each channel, effectively enhancing key features and suppressing unimportant features. Combined with the gating mechanism, the model can selectively focus on and utilize key information in the input features while filtering out noise or secondary information. Step 4: Use a two-stage loss optimization strategy to train the model obtained in Step 3; Step 5: Input the query pedestrian images into the model trained in Step 4 as input to the network to obtain a list of pedestrian targets with high similarity to the query target.

2. The method for re-identifying pedestrians changing clothes based on deep learning according to claim 1, characterized in that: In step 1, the pedestrian image set includes images of pedestrians' clothing appearance and body posture.

3. The method for re-identifying pedestrians changing clothes based on deep learning according to claim 2, characterized in that: In step 1, the pedestrian re-identification dataset for changing clothes is the PRCC real dataset.

4. The method for re-identifying pedestrians changing clothes based on deep learning according to claim 1, characterized in that: In step 2, the SCHP network, pre-trained on the LIP (Low Income Pedestrian) dataset for pedestrian re-identification, is used to preprocess the changing-clothes pedestrian re-identification dataset. The SCHP network, through in-depth analysis of pedestrian images, can accurately distinguish and parse different parts of the human body, thereby filtering out clothing-related information and retaining semantic information unrelated to clothing. This information is then fused with RGB features. Indirectly remove clothing features.

5. The method for re-identifying pedestrians changing clothes based on deep learning according to claim 1, characterized in that: In step 4, two loss functions are used to optimize the model: identity loss and triplet loss. The two-stage optimization strategy refers to using only identity loss in the first n rounds of training. The core goal of this stage is to enable the model to effectively identify pedestrians wearing the same clothes and to master some basic skills for identifying pedestrians changing clothes. After the nth round, triplet loss is added for training.

6. The method for re-identifying pedestrians changing clothes based on deep learning according to claim 1, characterized in that: In step 5, the pedestrian target list is sorted by similarity, which makes it easier to identify pedestrians with the same identity based on the pedestrian target list.

Images