A pedestrian re-identification method suitable for domain generalization

Abstract

The application discloses a pedestrian re-identification method suitable for field generalization and belongs to the technical field of computer vision and machine learning. The application constructs a field normalization module DN by combining the distribution information of IN layers and BN layers, replaces the normalization module BN of a deep network structure in a deep learning network model by using the field normalization module DN, and constructs a strong baseline model. In addition, the application constructs an implicit distribution space by using the distribution information of all source domains, realizes distribution alignment by projecting the distribution of the source domain and the target domain to the implicit distribution space, and designs an anti-normalization module AN. Each source domain is regarded as a "simulated target domain" of other source domains, the target domain uses the distribution information of the source domain as an anti-normalization parameter on the basis of removing style information and identity discriminative information by using an IN operation, and the target domain is projected to the implicit distribution space, so that the recognition accuracy of the model in the target domain is further improved.

Description

Technical Field

[0001] This invention belongs to the technical fields of computer vision and machine learning, and specifically relates to an unsupervised, domain-generalized pedestrian re-identification method based on deep learning. Background Technology

[0002] Pedestrian re-identification is an important and challenging research problem in computer vision. It involves associating specific pedestrians captured by cameras at different times and physical locations to quickly and accurately obtain pedestrian information from different perspectives. This lays the foundation for subsequent high-level applications such as pedestrian tracking, pedestrian attribute recognition, and pedestrian behavior analysis. Currently, pedestrian re-identification technology has shown significant development prospects and economic benefits in various fields, including intelligent video surveillance, smart city transportation, smart photo albums, and smart retail.

[0003] Based on whether or not pedestrian identification tags are used, deep learning-based person re-identification can be broadly categorized into supervised and unsupervised methods. Currently, supervised learning methods utilizing pedestrian identification tags have achieved very high performance. However, due to the time-consuming and laborious process of obtaining identification tags and the resulting privacy concerns, unsupervised person re-identification methods have gained more attention. These methods typically employ clustering algorithms or similarity metrics to obtain pseudo-labels for the input images, and then train the model in a supervised manner.

[0004] While unsupervised person re-identification methods have achieved remarkable success, most rely on the assumption that training and testing data come from the same dataset (domain), meaning they have similar distributions. However, in real-world scenarios, different person re-identification datasets exhibit significant differences (domain differences), such as the shooting scene, lighting intensity at the shooting point, camera resolution, and pedestrian clothing styles. This means that these unsupervised methods are only applicable to test sets with similar distributions to the current training set, failing to generalize to other datasets and resulting in a significant performance degradation.

[0005] To address the domain disparity problem, many researchers in recent years have focused on training a model with good generalization capabilities using only labeled source domain data. This model can then achieve good recognition performance in unlabeled target domains, i.e., unsupervised cross-domain person re-identification. Based on whether target domain data can be used for training, unsupervised cross-domain person re-identification methods can be mainly divided into two categories: domain adaptation methods and domain generalization methods. Domain adaptation methods aim to adaptively transfer knowledge learned from the source domain to a target domain with different domain styles and known data. Since target domain data can be used for training, these methods focus on exploring the relationship between source and target domain images, or directly using target domain data for unsupervised learning. While domain adaptation methods alleviate the domain disparity problem to some extent, they are highly dependent on the target domain training data and cannot be directly tested in the target domain. Domain generalization methods, on the other hand, effectively solve this problem. These methods aim to obtain generalized representations of pedestrians using only a limited source domain, thereby achieving good recognition performance in target domains with different domain styles and unknown data. The biggest difference between domain adaptation and domain generalization is that target domain data cannot be used for training.

[0006] Currently, unsupervised domain-generalized pedestrian re-identification methods can be mainly divided into four categories: representation learning-based, distribution alignment-based, meta-learning-based, and data augmentation-based methods. (1) Representation learning-based methods aim to learn domain-invariant pedestrian representations to improve the robustness of the model. For example, Jin et al. proposed a style normalization and recovery model to filter out interference factors that are irrelevant to pedestrian identity while decoupling discriminative pedestrian identity features. (2) Distribution alignment-based methods assume that the data distributions of the source and target domains both follow a multivariate Gaussian distribution and aim to use the distribution information of each source domain dataset, i.e., mean and variance, to fit the data distribution of the target domain. For example, Xu et al. proposed a simulated embedding method based on adaptive aggregation, which considers the correlation between unknown target samples and source domain datasets and designs an aggregation module to adaptively integrate the distribution information of multiple source domains to simulate the distribution of the target domain. (3) Meta-learning-based methods focus on formulating effective learning strategies to improve the robustness of the algorithm. These methods enable the model to learn to eliminate domain differences by simulating the domain differences between the source and target domains during training. (4) Data augmentation-based methods typically perform data augmentation at the feature level, aiming to generate more diverse features and thus improve the model's generalization ability to the target domain. For example, Ang et al. proposed a domain representation augmentation method, which improves the domain manifold coverage by implicitly projecting feature points in the direction of the source domain distribution, thereby obtaining augmented features.

[0007] In recent years, influenced by style transfer networks, distribution alignment methods that incorporate instance normalization (IN) layers into the network structure have gained increasing popularity. These methods utilize IN layers to normalize each instance sample, thus removing style information from the source domain. Simultaneously, Batch Normalization (BN) layers in the network normalize the entire source domain dataset to a multivariate Gaussian distribution with a mean of 0 and a variance of 1, thereby extracting discriminative pedestrian identification information from the source domain that is unaffected by domain differences. However, while IN layers can remove some style information that causes domain differences, they inevitably lose discriminative identification information from pedestrian images. BN layers, while able to extract distribution information from the source domain and achieve good recognition performance within that domain, are affected by unique domain information in the source domain when directly tested in a target domain with a completely different distribution, since the target domain data cannot be used for training. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention proposes a pedestrian re-identification method applicable to domain generalization. A Domain Normalization (DN) module is constructed by combining the distribution information of the IN layer and the BN layer. This DN module replaces the BN normalization module in the deep network structure of the deep learning network model, thus building a strong baseline model. This DN module balances the functions of the IN layer and the BN layer, mitigating the impact of domain differences while extracting high-level semantic information that enables pedestrian identification. Since the Batch Normalization (BN) layer still extracts the distribution information of the source domain, simply combining different distribution information using linear interpolation cannot completely eliminate the influence of domain differences. Therefore, this invention constructs a latent distribution space based on the distribution information of all source domains on the basis of a strong baseline model. Distribution alignment is achieved by projecting the distributions of both the source and target domains into a latent distribution space. At the same time, an anti-normalization (AN) module is designed to treat each source domain as a "simulated target domain" of other source domains. This allows the target domain to be constrained to project into the latent distribution space by using the distribution information of the source domain as an anti-normalization parameter after removing style information and identity discrimination information using the IN operation. This further improves the model's recognition accuracy in the target domain.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A pedestrian re-identification method applicable to various domains, characterized by comprising the following steps:

[0011] S1. Establish a strong baseline model.

[0012] S2. Train the strong baseline model using at least two source domains to obtain a trained strong baseline model.

[0013] S3. Input the target domain image to be identified into the trained strong baseline model to complete the re-identification task.

[0014] Specifically, in step S1, the strong baseline model uses a deep learning network model as the backbone network, and replaces the normalization module of the deep network structure in the deep learning network model with the domain normalization module DN to form a strong baseline model.

[0015] The formula for calculating the Domain Normalization Module (DN) is as follows:

[0016] (1)

[0017] in, This represents the feature map result after the i-th DN for the input feature map x. , These represent two learnable parameters used to adaptively learn the functionality of balancing the IN and BN layers; and These represent the learnable scaling and translation parameters, respectively. It is a constant to prevent the denominator from being 0; , Let these represent the mean and variance of the i-th source domain IN layer, respectively. , Let these represent the mean and variance of the i-th source domain BN layer, respectively, and they are calculated as follows:

[0018] (2)

[0019] (3)

[0020] in, N, C, H, and W represent the input feature maps, respectively. The total number of samples, channels, height, and width in the batch.

[0021] Preferably, based on the strong baseline model, a second deep network structure that is parallel to and identical to the deep network structure is added, and the domain normalization module DN in it is replaced with the inverse normalization module AN; at the same time, a storage module is added, and the mean and variance of the source domain stored in the storage module are averaged to obtain the mean and variance of the latent distribution, thereby constructing the latent distribution space.

[0022] The calculation formula for the inverse normalization module AN is as follows:

[0023] (4)

[0024] in, This represents the feature map result after the i-th AN for the input feature map x; for the input feature map x... Source domain ,Will The "simulated target domain" is considered as the remaining source domain. It is the first source domain The learnable parameters, k is the number of source domains; the average mean of the remaining source domains in the deep network structure. and mean variance As an inverse normalization parameter, the source domain after inverse normalization is constrained by the second deep network structure. The distribution is aligned with the implicit distribution space constructed by the storage modules; the specific alignment method is as follows:

[0025] (5)

[0026] in, and Let represent the mean and variance of the latent distribution, respectively. express The square of the norm.

[0027] average mean and mean variance The calculation method is as follows:

[0028] (6)

[0029] (7)

[0030] Where T represents the number of iterations. This is the hyperparameter of the exponential moving average.

[0031] Preferably, KL divergence loss is used. Constraints are applied to each sample in the "simulated target domain," calculated as follows:

[0032] (8)

[0033] in, This represents the mean of the last layer feature map of the nth sample in a batch, after normalization of height and width. This represents the variance of the last layer feature map of the nth sample in a batch after normalization of height and width. and The size is .

[0034] The beneficial effects of this invention are as follows:

[0035] 1. In the strong baseline model established in this invention, a domain normalization module (DN) is established by combining the distribution information of the IN layer and the BN layer. This balances the functions of the IN layer and the BN layer, thereby mitigating the impact of domain differences while extracting discriminative pedestrian identity information.

[0036] 2. Based on the strong baseline model, this invention further proposes a latent distribution space; it calculates the mean and variance of the latent distribution using the distribution information of the source domain, and projects the distribution of all source domain data onto this latent distribution space; at the same time, the designed inverse normalization module AN uses the distribution information of the source domain as the inverse normalization parameter to constrain the projection of the target domain onto the latent distribution space, thus indirectly aligning the distributions of the source domain and the target domain, thereby reducing the domain difference and further enhancing the accuracy of the domain-generalized person re-identification. Attached Figure Description

[0037] Figure 1 A schematic diagram of a strong baseline model suitable for domain-generalized pedestrian re-identification tasks.

[0038] Figure 2 A schematic diagram of a domain-generalized pedestrian re-identification method based on latent distribution alignment. Detailed Implementation

[0039] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0040] Example 1:

[0041] This embodiment provides a domain-generalized person re-identification method based on a strong baseline model, including the following steps:

[0042] S1. Establish a strong baseline model.

[0043] In this embodiment, a ResNet50 network structure with Batch Normalization (BN) layers is used as the backbone network, and it is divided into four stages according to the characteristics of its bottleneck structure. In stages 1 and 2, since these two stages are in the shallow layers of the network, they extract mostly shallow semantic information of pedestrian images; therefore, there is no need to change the structure in these two stages. For the deep network structures stages 3 and 4, all Batch Normalization (BN) layers are replaced with Domain Normalization (DN) modules to form a strong baseline model, such as... Figure 1 As shown.

[0044] For the input feature map Where N, C, H, and W represent the number of batch samples, number of channels, height, and width of the input feature map, respectively. The calculation method of the neighborhood normalization module DN is as follows:

[0045] (1)

[0046] in , These represent two learnable parameters used to adaptively learn the functionality of balancing the IN and BN layers, and each DN module contains... , It is unique to it and is not shared with other DN modules; and These represent the learnable scaling and translation parameters, respectively. It is a constant to prevent the denominator from being 0; , and , Let these represent the mean and variance of the IN layer and BN layer in the i-th source domain, respectively, and they are calculated as follows:

[0047] (2)

[0048] (3)

[0049] In the strong baseline model, only the BN layer positions in the deep network structure stage3 and stage4 are replaced with DN modules for each corresponding source domain, while all other structures share the same structure. This greatly reduces the parameters of the network structure, enabling the model to extract the commonalities of pedestrian identification information in all source domains. At the same time, the use of different DN modules can enhance the generalization of the network to a certain extent.

[0050] S2. Train the strong baseline model using at least two source domains to obtain a trained strong baseline model.

[0051] During training, only one source domain is randomly selected as input each time to prevent the model from being biased towards the source domain that is more easily fitted due to multiple source domains updating the model together. For the i-th source domain of the input... k represents the number of source domains. After the data passes through the shallow network structure shared by the network and the deep network structure containing its unique DN modules, the output features are finally obtained. and the predicted probabilities after each source domain's respective classification layer The training loss function includes classification loss. and three-dimensional loss The calculation method is as follows:

[0052] (4)

[0053] in, ID is the number of pedestrian identities in the i-th source domain. It is the number of samples in the i-th source domain. These are real pedestrian identification tags. This indicates that the model predicts the sample to belong to the pedestrian identity label. The probability, It is a constant used for label smoothing. It is a "soft label" after label smoothing.

[0054] (5)

[0055] in express Norm, and Representing the features of the input samples respectively Positive and negative samples in a batch It is a boundary constant, in this embodiment Take 0.35.

[0056] For the input source domain, the strong baseline model is trained using the above formulas (4) and (5) to obtain the trained strong baseline model.

[0057] S3. Input the target domain image into the trained strong baseline model to complete the re-identification task.

[0058] During the testing process, the target domain samples are input into the trained strong baseline model and pass through the DN module unique to all source domains to obtain k output features. The average value of these k output features is calculated to obtain the final features used for similarity measurement and feature matching to complete the re-identification task.

[0059] In each DN module, the average mean is used. and mean variance Replace the formula (1) and variance The strong baseline model was tested, and its calculation method is as follows:

[0060] (6)

[0061] (7)

[0062] Where T represents the number of iterations. This is the hyperparameter of the exponential moving average.

[0063] Example 2:

[0064] This embodiment provides a neighborhood-generalized person re-identification method based on latent distribution alignment. Building upon the strong baseline model of Embodiment 1, a second deep network structure, parallel to and identical in structure to the deep network structure, is added. The neighborhood normalization module DN is replaced with the inverse normalization module AN. Simultaneously, a storage module is added. The mean and variance of the source domain stored in the storage module are averaged to obtain the mean and variance of the latent distribution, thereby constructing the latent distribution space. Specifically:

[0065] Since the Batch Normalization (BN) layer in the strong baseline model still extracts distribution information from the source domain, simply combining the distribution information from the In and BN layers using linear interpolation cannot completely eliminate the influence of domain differences. Therefore, based on the strong baseline model, a latent distribution space is constructed using the distribution information of all source domains, and the distribution of all source domain data is projected onto this space. To address the issue of unknown target domain data, an inverse normalization (AN) module is designed, treating each source domain as a "simulated target domain" for the others. This allows the target domain, after removing style and identity-discriminating information using the IN operation, to use the distribution information of the source domains as inverse normalization parameters, constraining the projection of the target domain onto the latent distribution space.

[0066] like Figure 2 As shown, the domain-generalized pedestrian re-identification method based on implicit distribution alignment has two main characteristics.

[0067] First, add a storage module (Memory, M) to store the mean and variance of each source domain; its size is [size missing]. Here, k represents the number of source domains, and C represents the number of channels in the last layer feature map. During training, for a batch of data from the i-th source domain, M is updated as follows:

[0068] (8)

[0069] (9)

[0070] in This is a hyperparameter, set to 0.2 in this embodiment. The mean of the latent distribution space composed of k source domains is obtained by averaging the storage modules along the first dimension. and variance .

[0071] Second, stage5 and stage6 network structures were added. The structures of stage5 and stage6 are consistent with those of stage3 and stage4, except that the domain normalization module DN is replaced with the inverse normalization module AN. The inverse normalization module AN is used to project the target domain samples to the latent distribution space. The calculation method of the inverse normalization module AN is as follows:

[0072] (10)

[0073] in, These are learnable parameters; for the i-th source domain of the input... AN first uses the IN operation to remove style information, and then... The remaining source domain is considered as the "simulated target domain". The corresponding mean values ​​in stage 3 and stage 4 and variance As an inverse normalization parameter, using The distribution information is initialized and then denormalized using stage5 and stage6 constraints. The distribution is aligned with the implicit distribution constructed by the storage modules. Because... Since the distribution information of the target domain is not used during the denormalization process, this method can generalize well to the real target domain.

[0074] During training, one source domain is still randomly selected from all source domains as input. For the i-th source domain of the input... A batch of data first passes through stages 1 and 2, which are shared by all source domains, and then through stages 3 and 4 to update the mean and variance of that source domain in the storage module. The classification loss is then calculated according to the method described in Example 1. and three-dimensional loss Meanwhile, in order to constrain the distribution projection of this batch of data to the latent distribution space, this embodiment utilizes mean squared error loss. Constrain the mean of this batch of data and variance To maintain consistency with the mean and variance of the latent distribution space, the calculation method is as follows:

[0075] (11)

[0076] Next, the shared network structure is frozen, and the source domain data is input again, passing through stage 5 and stage 6, to obtain the mean of the normalized feature map of the last layer for each sample in this batch of data. and variance Furthermore, the distribution of each sample is constrained to align with the latent distribution. Since some samples possess unique properties, forcing each sample to conform to the latent distribution would prevent the network from converging; therefore, KL divergence loss is employed. The constraints are applied, and the calculation method is as follows:

[0077] (12)

[0078] Finally, the total loss function during training for:

[0079] (13)

[0080] During the test, the target domain data will pass through stage3 and stage4 branches and stage5 and stage6 branches respectively, and the average feature of the obtained k output features will be calculated. Then, the average features of the two branches will be concatenated in the channel dimension as the final feature used for similarity measurement to perform feature matching and complete the re-identification task.

[0081] Cross-domain training was performed on a large dataset, and its effectiveness was verified through experiments.

[0082] Cross-domain experiments were conducted using four large pedestrian re-identification datasets. The specific experimental setup included: M+MS+CSC3, M+CS+C3MS, and MS+CS+C3M, where “” represents the source domain on the left and the target domain on the right, and only the training sets of all source domains were used as the training sets for the cross-domain experiments.

[0083] Table 1. Comparison with other unsupervised domain generalization pedestrian re-identification methods

[0084]

[0085] As shown in Table 1, bolded numbers represent the highest number of points, and underlined numbers represent the second highest number of points. It can be seen that the distribution-aligned domain-generalized person re-identification methods have achieved good performance. For example, the MetaBIN method combines the features of source domain data after passing through the IN and BN layers, and simultaneously optimizes the model update method using a meta-learning training strategy; the META method also uses the source domain as a "simulated target domain," calculates the distance between the distribution of each sample in the target domain and the distributions of all source domains, and uses this as the weight of the network output features, aggregating the output features of all source domain networks. Unlike these methods, the method of this invention combines the distribution information in the IN and BN layers, allowing the network to adaptively balance the functions of the IN and BN layers, and constructs a strong baseline model, corresponding to the Strong Baseline in the table. It can be seen that the Strong Baseline model outperforms all existing state-of-the-art methods in both mAP and Rank-1 metrics in the cross-domain experiment of M+MS+CSC3, demonstrating the effectiveness of this invention. Furthermore, this method also designs an inverse normalization module to construct a domain-generalized person re-identification method based on latent distribution alignment. This method aligns the distributions of both the source and target domains to a latent distribution, achieving better re-identification performance within this distribution space. Ultimately, the method of this invention achieves the best mAP metric in all cross-domain experimental settings, with average performance improvements of 0.6% and 1.0% over the META method in mAP and Rank-1 metrics, respectively.

[0086] In summary, the present invention has the following characteristics and advantages: (1) By combining the mean and variance of the IN layer and the BN layer, a domain normalization module is established. This module can balance the functions of the IN layer and the BN layer, thereby mitigating the influence of domain differences while extracting high-level semantic information with identity recognition capabilities for pedestrians. This module is added to the network structure and a strong baseline model suitable for domain-generalized pedestrian re-identification is constructed. (2) Based on the strong baseline model, a domain-generalized pedestrian re-identification method based on implicit distribution alignment is proposed. An implicit distribution space is constructed using the distribution information of all source domains. The distribution of all source domain data is projected onto this space. At the same time, an anti-normalization module is designed so that the target domain data uses the distribution information of the source domain as an anti-normalization parameter to constrain the target domain to be projected onto the implicit distribution space. By projecting both the source domain and the target domain onto the implicit distribution space, the distribution alignment of the source domain and the target domain is indirectly achieved, reducing the domain differences between them. (3) The results of the domain generalization experiment on four large pedestrian re-identification datasets show that the method proposed in this invention has reached an advanced level in terms of average performance.

Claims

1. A pedestrian re-identification method applicable to a wide range of domains, characterized in that, Includes the following steps: S1. Establish a strong baseline model; S2. Train the strong baseline model using at least two source domains to obtain a trained strong baseline model; S3. Input the target domain image to be identified into the trained strong baseline model to complete the re-identification task; Specifically, in step S1, the strong baseline model uses a deep learning network model as the backbone network and replaces the normalization module of the deep network structure in the deep learning network model with the domain normalization module DN to form a strong baseline model. The formula for calculating the Domain Normalization Module (DN) is as follows: ； （1） in, This represents the feature map result after the i-th DN for the input feature map x. , These represent two learnable parameters used to adaptively learn the functionality of balancing the IN and BN layers; and These represent the learnable scaling and translation parameters, respectively. It is a constant to prevent the denominator from being 0; , Let these represent the mean and variance of the i-th source domain IN layer, respectively. , Let these represent the mean and variance of the i-th source domain BN layer, respectively, and they are calculated as follows: ； （2） ； （3） in, N, C, H, and W represent the input feature maps, respectively. The total number of samples, channels, height, and width in the batch.

2. The pedestrian re-identification method applicable to a generalized domain as described in claim 1, characterized in that, Based on the strong baseline model, a second deep network structure that is parallel to and identical to the deep network structure is added, and the domain normalization module DN is replaced with the inverse normalization module AN. At the same time, a storage module is added, and the mean and variance of the source domain stored in the storage module are averaged to obtain the mean and variance of the latent distribution, thereby constructing the latent distribution space. The calculation formula for the inverse normalization module AN is as follows: ； （4） in, This represents the feature map result after the i-th AN for the input feature map x; for the input feature map x... Source domain ,Will The "simulated target domain" is considered as the remaining source domain. It is the first source domain The learnable parameters, k is the number of source domains; the average mean of the remaining source domains in the deep network structure. and mean variance As an inverse normalization parameter, the source domain after inverse normalization is constrained by the second deep network structure. The distribution is aligned with the implicit distribution space constructed by the storage modules; the specific alignment method is as follows: ； （5） in, and Let represent the mean and variance of the latent distribution, respectively. express The square of the norm; average mean and mean variance The calculation method is as follows: ； （6） ； （7） Where T represents the number of iterations. This is the hyperparameter of the exponential moving average.

3. The pedestrian re-identification method applicable to a generalized domain as described in claim 2, characterized in that, Using KL divergence loss Constraints are applied to each sample in the "simulated target domain," calculated as follows: ； （8） in, This represents the mean of the last layer feature map of the nth sample in a batch, after normalization of height and width. This represents the variance of the last layer feature map of the nth sample in a batch after normalization of height and width. and The size is .

Images