A cross-domain pedestrian re-identification method based on feature enhancement
By constructing a feature-enhanced deep convolutional neural network model and adjusting parameters using instance and batch normalized Gaussian processes, the problem of domain generalization of pedestrian re-identification models across different datasets is solved, thereby improving cross-domain recognition performance and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2022-11-15
- Publication Date
- 2026-06-09
AI Technical Summary
Existing pedestrian re-identification models lack sufficient domain generalization ability across different datasets, failing to effectively reduce explicit and implicit differences between datasets, leading to a decline in recognition performance.
By constructing a feature-enhanced deep convolutional neural network model, and utilizing instance normalization and batch normalization Gaussian processes, the instance and batch normalization parameters of the model are adjusted to reduce inter-domain differences and improve the cross-domain recognition performance of the model.
This improves the cross-domain recognition generalization ability and recognition performance of the pedestrian re-identification model, reduces the bias of batch normalization in domain generalization, and enhances the robustness of the model.
Smart Images
Figure CN115909398B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and pattern recognition technology, and in particular to a cross-domain pedestrian re-identification method based on feature enhancement. Background Technology
[0002] In recent years, with the advancement of computer hardware and software and the expansion of data volume, computer vision technology has flourished and is widely used in fields such as face recognition and recommendation algorithms. Pedestrian re-identification, as a sub-problem of image retrieval, has significant applications in intelligent security and smart city construction. Pedestrian re-identification refers to the process of identifying the same pedestrian under camera surveillance from different perspectives and geographical areas. Pedestrian re-identification can be combined with pedestrian detection and tracking technologies to achieve a complete pedestrian search framework. However, due to issues such as low dataset resolution, diverse pedestrian poses, building occlusion, and different camera angles, pedestrian re-identification research remains challenging. Therefore, pedestrian re-identification technology has significant theoretical and practical implications.
[0003] Supervised learning research in person re-identification is quite mature, achieving good retrieval results on existing datasets. However, the generalization technique for person re-identification still needs improvement. Person re-identification domain generalization refers to training a model on one dataset and applying it directly to another dataset without fine-tuning. Because domain generalization does not require collecting datasets and labels for the target domain, once trained, it can be widely applied to all datasets. Therefore, compared to person re-identification within the same domain, person re-identification domain generalization has higher practical application value, but also faces greater challenges.
[0004] Due to the significant differences between different pedestrian re-identification datasets, a model trained on one dataset and then tested on other datasets will exhibit a marked performance drop. Therefore, minimizing the impact of dataset differences on the model is crucial. The differences in pedestrian re-identification datasets can be mainly categorized into explicit differences in pedestrian appearance and style, and implicit differences in dataset statistics. Explicit differences in pedestrian appearance and style can be encoded by the statistics μ and σ of Instance Normalization, while implicit differences in dataset statistics can be encoded by the parameters μ, σ, β, and γ of Batch Normalization. Existing methods for generalizing from pedestrian re-identification often focus on eliminating explicit differences or aligning implicit differences, failing to generalize well to unknown domains. Furthermore, most pedestrian re-identification generalization methods do not consider the cross-domain bias of Batch Normalization, relying excessively on the source domain, thus causing significant recognition bias. Summary of the Invention
[0005] The purpose of this invention is to provide a cross-domain pedestrian re-identification method with strong domain generalization ability and high recognition performance.
[0006] The technical solution to achieve the objective of this invention is: a cross-domain pedestrian re-identification method based on feature enhancement, comprising the following steps:
[0007] Step 1: Establish a deep convolutional neural network model for cross-domain pedestrian re-identification;
[0008] Step 2: Obtain the pedestrian re-identification dataset and input it into the deep convolutional neural network model;
[0009] Step 3: Establish an instance normalization Gaussian process for each camera, collect the instance normalization statistics of images from different cameras, and add them to the instance normalization Gaussian processes for different cameras.
[0010] Step 4: Sample new instance normalization statistics from the instance normalization Gaussian process, and use them as the re-translation and re-scaling parameters for instance normalization in this training.
[0011] Step 5: Extract the features of the image, input them into the classifier for classification, calculate the cross-entropy loss, perform backpropagation, and update the parameters of the deep convolutional neural network model and the hyperparameters of the instance normalization Gaussian process.
[0012] Step 6: Establish batch normalized Gaussian process for all cameras, collect batch normalization parameters of images from different cameras, add them to the batch normalized Gaussian process, and perform batch normalization operation at the same time.
[0013] Step 7: Sample new parameters from the batch normalized Gaussian process and use them as the batch normalized parameters for this training.
[0014] Step 8: Extract the features of the image again, input them into the classifier for classification, calculate the cross-entropy loss, and perform backpropagation to update the parameters of the deep convolutional neural network model and the hyperparameters of the batch normalized Gaussian process.
[0015] Step 9: Iterate through steps 2 to 8 until the model converges, obtaining the final deep convolutional neural network model for cross-domain person re-identification;
[0016] Step 10: Use the final deep convolutional neural network model for cross-domain pedestrian re-identification to perform cross-domain pedestrian re-identification.
[0017] Furthermore, the deep convolutional neural network model for cross-domain pedestrian re-identification described in step 1 is as follows:
[0018] Using ResNet50 as the network base, the original adaptive average pooling was replaced with GeM pooling layers. At the same time, a one-dimensional Batch Normalization layer was added after the pooling layer and before the classifier to establish a deep convolutional neural network model for cross-domain pedestrian re-identification.
[0019] Furthermore, the instance normalization described in step 3 is a layer called Instance Normalization; the batch normalization described in step 4 is a layer called Batch Normalization.
[0020] Furthermore, the classifier described in steps 5 and 8 is a fully connected layer with an input channel of 2048 and an output channel containing the number of pedestrian IDs in the training set of the dataset.
[0021] Furthermore, step 3 involves establishing an instance normalization Gaussian process for each camera, collecting instance normalization statistics for images from different cameras, and adding these statistics to the instance normalization Gaussian processes for different cameras, as detailed below:
[0022] Step 3.1: Create a learnable, self-optimizing instance normalized Gaussian process for each camera. The instance normalized Gaussian process is generated by a. mu a rho b mu b rho Four learnable parameters control the system;
[0023] Step 3.2: Group the current batch of images by camera, and calculate the instance normalization statistics μ of the images within each group. cti and Calculate μ within each group cti and The mean and standard deviation are used as parameters of the instance-normalized Gaussian process for this camera, and are calculated as follows:
[0024]
[0025]
[0026]
[0027]
[0028]
[0029]
[0030] Where c represents the camera ID, T cLet x represent the number of samples from the c-th camera within the batch, W represent the width of the feature, H represent the height of the feature, and x represent the height of the feature. ctilm Let c represent a feature of size [C, H, W].
[0031] Furthermore, in step 4, new instance normalization statistics are sampled from the instance normalization Gaussian process and used as the re-translation and re-scaling parameters for instance normalization in this training, as follows:
[0032] Step 4.1: For each image in the current batch, randomly select m cameras from [1-n], where n is the number of cameras in the current batch;
[0033] Step 4.2: Normalize the Gaussian process cam for each of the m camera instances. c In INGauss, μ and σ are sampled to obtain a set of μ and a set of σ.
[0034] Step 4.3: Calculate the mean of this set of μ values. and the mean of this set of σ
[0035] Step 4.4: Take the result obtained in step 4.3 and As the retranslation and rescaling parameters for instance normalization, their calculation form is as follows:
[0036]
[0037]
[0038] Furthermore, step 6 involves establishing a batch normalized Gaussian process for all cameras, collecting batch normalization parameters from images of different cameras, adding them to the batch normalized Gaussian process, and simultaneously performing batch normalization operations, as detailed below:
[0039] Step 6.1: Create a learnable, self-optimizing batch normalized Gaussian process for all cameras. The batch normalized Gaussian process is generated by a. mu a rho b mu b rho These four learnable parameters control;
[0040] Step 6.2: Create a Batch Normalization layer for each camera;
[0041] Step 6.3: Input the image from a specific camera within the current batch into the BatchNormalization layer corresponding to that camera to obtain the μ of the Batch Normalization layer. B , The parameters β and γ are added to the batch normalized Gaussian process, and the calculation is as follows:
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049] Where c represents the camera ID and T represents the number of samples in the batch.
[0050] Furthermore, in step 7, new parameters are sampled from the batch-normalized Gaussian process and used as the batch-normalized parameters for this training. The calculation form is as follows:
[0051] Step 7.1: Sample a′ and b′ from the batch normalized Gaussian process BNGauss;
[0052] Step 7.2: Perform statistically enhanced batch normalization using a′ and b′ obtained in Step 7.1. The calculation form is as follows:
[0053]
[0054] Furthermore, step 8 involves extracting image features again, inputting them into a classifier for classification, calculating cross-entropy loss, performing backpropagation, and updating the parameters of the deep convolutional neural network model and the hyperparameters of the batch normalized Gaussian process, as detailed below:
[0055] The hyperparameters of the batch normalized Gaussian process refer to the hyperparameter a of the batch normalized Gaussian process. mu a rho b mu b rho ;
[0056] The formula for the cross-entropy loss is as follows:
[0057]
[0058] Where T is the number of samples in the current batch, and N is the number of categories.
[0059] Compared with the prior art, the present invention has the following significant advantages: (1) From the perspective of feature enhancement, the self-learning Gaussian process is constructed by using the parameters of instance normalization and batch normalization respectively, so that the model is robust to different styles and statistics; (2) In view of the bias problem of batch normalization in domain generalization, the Gaussian process is used to construct diverse statistics to avoid the overfitting of batch normalization statistics to the source domain, and the average path of the Gaussian process is used during inference, which reduces the bias problem of batch normalization in domain generalization; (3) The cross-domain recognition generalization ability and recognition performance of the pedestrian re-identification model are improved. Attached Figure Description
[0060] Figure 1 This is a flowchart illustrating a cross-domain pedestrian re-identification method based on feature enhancement according to the present invention. Detailed Implementation
[0061] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0062] Combination Figure 1 The present invention provides a cross-domain pedestrian re-identification method based on feature enhancement, comprising the following steps:
[0063] Step 1: Establish a deep convolutional neural network model for cross-domain person re-identification, as detailed below:
[0064] Using ResNet50 as the network base, the original adaptive average pooling was replaced with GeM pooling layers. At the same time, a one-dimensional Batch Normalization layer was added after the pooling layer and before the classifier to establish a deep convolutional neural network model for cross-domain pedestrian re-identification.
[0065] Step 2: Obtain the pedestrian re-identification dataset and input it into the deep convolutional neural network model.
[0066] Step 3: Establish an instance normalization Gaussian process for each camera, collect instance normalization statistics for images from different cameras, and add them to the instance normalization Gaussian processes for different cameras, as detailed below:
[0067] Step 3.1: Create a learnable, self-optimizing instance normalized Gaussian process for each camera. The instance normalized Gaussian process is generated by a. mu a rho b mu b rho Four learnable parameters control the system;
[0068] Step 3.2: Group the current batch of images by camera, and calculate the instance normalization statistics μ of the images within each group. cti and Calculate μ within each group cti and The mean and standard deviation are used as parameters of the instance-normalized Gaussian process for this camera, and are calculated as follows:
[0069]
[0070]
[0071]
[0072]
[0073]
[0074]
[0075] Where c represents the camera ID, T c Let x represent the number of samples from the c-th camera within the batch, W represent the width of the feature, H represent the height of the feature, and x represent the height of the feature. ctilm Let c represent a feature of size [C, H, W].
[0076] Step 4: From the instance normalization Gaussian process, sample new instance normalization statistics as the re-translation and re-scaling parameters for instance normalization in this training, as follows:
[0077] Step 4.1: For each image in the current batch, randomly select m cameras from [1-n], where n is the number of cameras in the current batch;
[0078] Step 4.2: Normalize the Gaussian process cam for each of the m camera instances. c In INGauss, μ and σ are sampled to obtain a set of μ and a set of σ.
[0079] Step 4.3: Calculate the mean of this set of μ values. and the mean of this set of σ
[0080] Step 4.4: Take the result obtained in step 4.3 and As the retranslation and rescaling parameters for instance normalization, their calculation form is as follows:
[0081]
[0082]
[0083] Step 5: Extract the features of the image, input them into the classifier for classification, calculate the cross-entropy loss, perform backpropagation, and update the parameters of the deep convolutional neural network model.
[0084] Step 6: Establish a batch normalized Gaussian process for all cameras, collect batch normalization parameters of images from different cameras, add them to the batch normalized Gaussian process, and perform batch normalization operations simultaneously, as follows:
[0085] Step 6.1: Create a learnable, self-optimizing batch normalized Gaussian process for all cameras. The batch normalized Gaussian process is generated by a. mu a rho b mu b rho These four learnable parameters control;
[0086] Step 6.2: Create a Batch Normalization layer for each camera;
[0087] Step 6.3: Input the image from a specific camera within the current batch into the BatchNormalization layer corresponding to that camera to obtain the μ of the Batch Normalization layer. B , The parameters β and γ are added to the batch normalized Gaussian process, and the calculation is as follows:
[0088]
[0089]
[0090]
[0091]
[0092]
[0093]
[0094]
[0095] Where c represents the camera ID and T represents the number of samples in the batch.
[0096] Step 7: Sample new parameters from the batch-normalized Gaussian process as the batch-normalized parameters for this training. The calculation form is as follows:
[0097] Step 7.1: Sample a′ and b′ from the batch normalized Gaussian process BNGauss;
[0098] Step 7.2: Perform statistically enhanced batch normalization using a′ and b′ obtained in Step 7.1. The calculation form is as follows:
[0099]
[0100] Step 8: Extract the image features again, input them into the classifier for classification, calculate the cross-entropy loss, perform backpropagation, and update the parameters of the deep convolutional neural network model and the hyperparameters of the batch normalized Gaussian process, as follows:
[0101] The hyperparameters of the batch normalized Gaussian process refer to the hyperparameter a of the batch normalized Gaussian process. mu a rho b mu b rho ;
[0102] The formula for the cross-entropy loss is as follows:
[0103]
[0104] Where T is the number of samples in the current batch, and N is the number of categories.
[0105] Step 9: Iterate through steps 2 to 8 until the model converges, obtaining the final deep convolutional neural network model for cross-domain person re-identification;
[0106] Step 10: Use the final deep convolutional neural network model for cross-domain pedestrian re-identification to perform cross-domain pedestrian re-identification.
[0107] Furthermore, the instance normalization described in step 3 is a layer called Instance Normalization; the batch normalization described in step 4 is a layer called Batch Normalization.
[0108] Furthermore, the classifier described in steps 5 and 8 is a fully connected layer with an input channel of 2048 and an output channel equal to the number of pedestrian IDs in the training set of the dataset.
Claims
1. A feature enhancement based cross-domain pedestrian re-identification method, characterized in that, Includes the following steps: Step 1: Establish a deep convolutional neural network model for cross-domain pedestrian re-identification; Step 2: Obtain the pedestrian re-identification dataset and input it into the deep convolutional neural network model; Step 3: Establish an instance normalization Gaussian process for each camera, collect the instance normalization statistics of images from different cameras, and add them to the instance normalization Gaussian processes for different cameras. Step 4: Sample new instance normalization statistics from the instance normalization Gaussian process, and use them as the re-translation and re-scaling parameters for instance normalization in this training. Step 5: Extract the features of the image, input them into the classifier for classification, calculate the cross-entropy loss, perform backpropagation, and update the parameters of the deep convolutional neural network model and the hyperparameters of the instance normalization Gaussian process. Step 6: Establish batch normalized Gaussian process for all cameras, collect batch normalization parameters of images from different cameras, add them to the batch normalized Gaussian process, and perform batch normalization operation at the same time. Step 7: Sample new parameters from the batch normalized Gaussian process and use them as the batch normalized parameters for this training. Step 8: Extract the features of the image again, input them into the classifier for classification, calculate the cross-entropy loss, and perform backpropagation to update the parameters of the deep convolutional neural network model and the hyperparameters of the batch normalized Gaussian process. Step 9: Iterate through steps 2 to 8 until the model converges, and obtain the final deep convolutional neural network model for cross-domain pedestrian re-identification. Step 10: Use the final deep convolutional neural network model for cross-domain pedestrian re-identification to perform cross-domain pedestrian re-identification.
2. The feature enhancement based cross-domain pedestrian re-identification method according to claim 1, characterized in that, The deep convolutional neural network model for cross-domain person re-identification described in step 1 is as follows: Using ResNet50 as the network base, the original adaptive average pooling was replaced with GeM pooling layers. At the same time, a one-dimensional Batch Normalization layer was added after the pooling layer and before the classifier to establish a deep convolutional neural network model for cross-domain pedestrian re-identification.
3. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, The instance normalization described in step 3 is called the Instance Normalization layer.
4. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, The batch normalization described in step 6 is a Batch Normalization layer.
5. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, The classifier described in steps 5 and 8 is a fully connected layer with an input channel of 2048 and an output channel containing the number of pedestrian IDs in the training set of the dataset.
6. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, Step 3 describes establishing an instance normalization Gaussian process for each camera, collecting instance normalization statistics for images from different cameras, and adding them to the instance normalization Gaussian processes for different cameras, as detailed below: Step 3.1: Create a learnable, self-optimizing instance-normalized Gaussian process for each camera. The instance-normalized Gaussian process is... , , , Four learnable parameters control the system; Step 3.2: Group the current batch of images by camera, and calculate the instance normalization statistics of the images within each group. and Calculate the results within each group. and The mean and standard deviation are used as parameters of the instance-normalized Gaussian process for this camera, and are calculated as follows: Where 'c' represents the camera ID, Let W represent the number of samples from the c-th camera within the batch, W represent the width of the feature, and H represent the height of the feature. Let c represent a feature of size [C, H, W].
7. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, In step 4, during the instance normalization Gaussian process, new instance normalization statistics are sampled and used as the re-translation and re-scaling parameters for instance normalization in this training, as detailed below: Step 4.1: For each image in the current batch, randomly select m cameras from [1-n], where n is the number of cameras in the current batch; Step 4.2: Normalize the Gaussian process for each of the m camera instances. Sample μ and σ to obtain a set of μ and a set of σ; Step 4.3: Calculate the mean of this set of μ values. and the mean of this set of σ ; Step 4.4: Take the result obtained in step 4.3 and As the retranslation and rescaling parameters for instance normalization, their calculation form is as follows: 。 8. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, Step 6 describes establishing a batch normalized Gaussian process for all cameras, collecting batch normalization parameters from images of different cameras, adding them to the batch normalized Gaussian process, and performing batch normalization operations simultaneously, as detailed below: Step 6.1: Create a learnable, self-optimizing batch normalized Gaussian process for all cameras. The batch normalized Gaussian process is... , , , These four learnable parameters control; Step 6.2: Create a Batch Normalization layer for each camera; Step 6.3: Input the images from a specific camera within the current batch into the BatchNormalization layer corresponding to that camera to obtain the Batch Normalization layer. , , as well as The parameters are added to the batch normalized Gaussian process, and the calculation is as follows: Where c represents the camera ID and T represents the number of samples in the batch.
9. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, In step 7, new parameters are sampled from the batch-normalized Gaussian process and used as the parameters for batch normalization in this training. The calculation form is as follows: Step 7.1: From the batch normalized Gaussian process Mid-sampling ; Step 7.2: Using the results obtained in Step 7.1 The statistically enhanced batch normalization is calculated as follows: 。 10. The cross-domain pedestrian re-identification method based on feature enhancement according to claim 1, characterized in that, Step 8 involves extracting image features again, inputting them into a classifier for classification, calculating cross-entropy loss, performing backpropagation, and updating the parameters of the deep convolutional neural network model and the hyperparameters of the batch normalized Gaussian process, as detailed below: The hyperparameters of the batch normalized Gaussian process refer to the hyperparameters of the batch normalized Gaussian process. , , , ; The formula for the cross-entropy loss is as follows: Where T is the number of samples in the current batch, and N is the number of categories.