An unsupervised domain adaptation pedestrian re-identification method based on camera perception and deep learning

Abstract

The present application relates to a kind of based on camera perception and deep learning's unsupervised domain adaptation pedestrian re-identification method, feature in target domain dataset is screened based on the camera number of picture sample self-provided, then clustering is carried out to obtain the camera internal pseudo-label of multiple cameras, and the camera internal pseudo-label of multiple cameras is reorganized based on consistency coefficient matrix and global pseudo-label across camera, so that the class agent number of the pseudo-label of the same class agent in multiple cameras is consistent, and the global pseudo-label across camera after correction overcomes the difference of the same pedestrian on picture caused by global clustering across view angle, and by calculating the contrast loss of source domain dataset and target domain dataset, further alleviate the camera domain difference in target domain dataset, improve the robustness to noise label.

Description

Technical Field

[0001] This invention relates to the field of pedestrian re-identification technology, and more specifically, to an unsupervised domain-adaptive pedestrian re-identification method based on camera perception and deep learning. Background Technology

[0002] Pedestrian re-identification is one of the more popular research directions in the field of computer vision in recent years. It is a problem in image retrieval, which uses computer vision technology to determine whether a specific pedestrian exists in an image or video. That is, given a surveillance pedestrian image, it retrieves images of that pedestrian across different devices.

[0003] With the development of technology, pedestrian re-identification technology has been widely applied in fields such as intelligent security and video surveillance. Currently, pedestrian re-identification has achieved significant breakthroughs in labeled supervised datasets and demonstrated superior performance. However, labeled datasets require high manual labor costs, while unlabeled data is often very inexpensive and large in scale. How to extend to large unlabeled datasets and new data domains is receiving increasing attention, leading to the emergence of unsupervised domain-adaptive pedestrian re-identification.

[0004] Most advanced unsupervised adaptive person re-identification methods currently employ pseudo-labeling. First, a network model is pre-trained using labeled source domain data. Then, pseudo-labels are estimated for unlabeled target domain samples for training. Pseudo-labeling methods significantly improve the performance of unsupervised adaptive person re-identification models. However, unsupervised methods still lag significantly behind supervised methods in performance. Because images of the same pedestrian vary considerably depending on the camera's perspective, clustering algorithms cannot guarantee that images of the same pedestrian will be assigned the same pseudo-label, inevitably introducing noisy labels. Model training is susceptible to label noise, leading to a decline in model performance. Summary of the Invention

[0005] The problem addressed by this invention is how to reduce the differences in images of the same pedestrian caused by cross-camera perspectives, thereby improving robustness to noise labels.

[0006] To address the aforementioned problems, this invention provides an unsupervised, domain-adaptive person re-identification method based on camera perception and deep learning, comprising:

[0007] Step 1: Obtain a labeled source domain dataset and an unlabeled target domain dataset Where, N s N represents the number of image samples in the source domain dataset. t The number of image samples in the target domain dataset and the number of source domain images. Corresponding real tags Source domain dataset and target domain dataset Each image sample in the dataset undergoes preprocessing;

[0008] Step 2: Construct a camera-aware person re-identification network model and process the preprocessed source domain dataset. and target domain dataset Input the pedestrian re-identification network model for feature extraction to obtain the feature set. in, For source domain dataset Image sample x s Features For the target domain dataset Image sample x t Features;

[0009] Step 3: Based on the camera tags attached to the images, respectively... Global clustering is performed on features within the same camera to obtain intra-camera pseudo-labels for multiple cameras. The camera's internal fake label It contains multiple sets of fine-grained class samples; the feature set of pairs. In Perform global clustering to obtain global pseudo-labels across cameras. The cross-camera global pseudo-label It contains multiple sets of coarse-class samples;

[0010] Step 4: Construct camera-agnostic memory modules and camera-aware memory modules based on clustering results, and then use global pseudo-labels across cameras. and camera internal fake labels Calculate the consistency coefficient matrix; then, based on the consistency coefficient matrix and the global pseudo-labels across cameras... Internal camera tags for multiple cameras After editing, the adjusted global pseudo-labels across cameras are obtained. And the repaired fake label inside the camera This leads to the target domain dataset with pseudo-labels.

[0011] Step 5: Employ a joint training approach by computing the source domain dataset. and target domain dataset The training loss is used to update the parameters of the person re-identification network model through gradient backpropagation, dynamically update the parameters of the camera-agnostic memory module and the camera-perceived memory module, and save the optimal parameters of the person re-identification network model.

[0012] The beneficial effects of this invention are as follows: Features in the target domain dataset are screened based on the camera IDs inherent in the image samples. Then, clustering is performed to obtain intra-camera pseudo-labels for multiple cameras. These intra-camera pseudo-labels are then reorganized based on the consistency coefficient matrix and cross-camera global pseudo-labels, ensuring that pseudo-labels belonging to the same agent class within multiple cameras have consistent agent class IDs. The corrected cross-camera global pseudo-labels overcome the differences in images of the same pedestrian caused by cross-viewpoints in global clustering. Furthermore, the source domain dataset is calculated... and target domain dataset The contrast loss further mitigates camera domain discrepancies in the target domain dataset and improves robustness to noisy labels.

[0013] Preferably, in step 1, the source domain dataset is... and target domain dataset The preprocessing of each image sample includes: first, adjusting the size of each image sample to the same height and width; then performing horizontal flipping, zero padding, standardization, and random erasure operations.

[0014] Preferably, the person re-identification network model constructed in step 2 includes a feature extractor for feature extraction. The feature extractor comprises five stages, the first four of which are the first four stages of the ResNet50 network framework, namely:

[0015] The first stage involves sequentially performing convolution, normalization, ReLU activation, and max pooling operations on the image samples. The convolution kernel has 64 data points, a size of 7×7, a zero-padding parameter of 3, and a compensation of 2. The max pooling kernel has a size of 3×3, a zero-padding parameter of 1, and a stride of 2.

[0016] The second phase includes three bottleneck layers;

[0017] The third stage includes four bottleneck layers;

[0018] Phase Four: Includes six bottleneck layers;

[0019] The fifth stage of the feature extractor includes three bottleneck layers, used for pooling the feature maps obtained from the first four stages of the feature extractor to obtain the source domain dataset. and target domain dataset The features of each image sample are defined as follows: in the first bottleneck layer, all convolutional strides are 1, and the convolutional stride in the downsampling operation is 1; the convolutional stride of the other two bottleneck layers is 1, and the downsampling operation is canceled.

[0020] Preferably, step 3 includes:

[0021] Step 301, Calculation The Jaccard distance between each pair of features yields an N×N dimensional distance matrix;

[0022] Step 302: Combining the obtained distance matrix, the DBSCAN density clustering algorithm is used for clustering. Features within the same cluster are assigned the same pseudo-label, and outliers generated by clustering are assigned the pseudo-label of the cluster closest to them; thus obtaining global pseudo-labels across cameras.

[0023] Step 303: Based on the camera labels attached to the image samples, filter the feature sets of the same camera and calculate the Jaccard distance between each pair of features in the feature set to obtain the distance matrix within the camera.

[0024] Step 304: Combining the distance matrix within the camera, an adaptive hierarchical clustering method is used to perform intra-camera clustering operations. Outliers are assigned pseudo-labels of the nearest clusters to obtain the intra-camera pseudo-labels.

[0025] Preferably, step 4 includes:

[0026] Step 401: Construct a camera-agnostic memory module and a camera-perceived memory module based on the clustering results;

[0027] Step 402: Based on the global pseudo-labels across cameras and camera internal fake labels The consistency coefficient matrix is ​​calculated as follows:

[0028]

[0029] In the formula, For camera internal false labels The fine-grained sample set within, Z c For camera internal false labels The total number of class proxies; Global pseudo-tags The set of rough class samples within, l∈{1,…,L};

[0030] Step 403: Global pseudo-labels based on consistency coefficient matrix and cross-camera setup From the internal camera tags of each camera The refined set of samples belonging to the same agent class is selected and reorganized to obtain the corrected global pseudo-labels. The method is as follows:

[0031]

[0032] The reorganized internal pseudo-labels of each camera are as follows:

[0033] Step 404: Based on the corrected global pseudo-tags And the internal pseudo-labels of each camera are Obtain the target domain dataset with pseudo-labels in, This represents the dataset containing labeled samples under camera c. It is the target domain dataset Chinese image sample Global pseudo-tags, This is the total number of class proxies after sorting. It is the target domain dataset Chinese image sample A fake label inside the camera (camera C). This represents the number of agents in the class of camera c. Represents the target domain dataset Chinese image sample Camera serial number, This indicates the number of image samples taken from camera c.

[0034] Preferably, step 5 includes:

[0035] Step 501, Source Domain Dataset The formula for calculating the training loss is:

[0036]

[0037]

[0038]

[0039] In the formula, For source domain dataset Total loss, For source domain dataset Cross-entropy loss, For source domain dataset The triplet loss; Represents the source domain dataset Chinese image sample belong The probability of a class proxy, ||·|| represents L 2 Norm distance, where i,p and i,n represent the distance between mini-batch image samples. The indexes of difficult positive samples and difficult negative samples are defined, where m = 0.3 represents the distance within the triples;

[0040] Step 502, Target Domain Dataset The formula for calculating training loss is:

[0041]

[0042] In the formula, Represents the target domain dataset Global contrastive loss with a self-punishing mechanism; Represents the target domain dataset The camera-perceptual contrast loss is used to balance the proportion of the camera-perceptual contrast loss in the total loss of the target domain. Represents the target domain dataset The triplet loss; where the global contrastive loss, which incorporates a self-punishing mechanism, is calculated as follows:

[0043]

[0044] In the formula, τ a τ represents the temperature coefficient. a =0.05; Represents the target domain dataset The total number of image samples in the dataset;

[0045] The formula for calculating camera-perceived contrast loss is:

[0046]

[0047] In the formula, For in-camera contrast loss, The inter-camera contrast loss is represented by λ² = 0.5, a hyperparameter used to balance the weights of intra-camera and inter-camera contrast losses. The formula for calculating the intra-camera contrast loss is as follows:

[0048]

[0049] In the formula, Indicates from the first camera to the... The total number of class proxies within each camera; This represents the total number of image samples in the c-th camera;

[0050] The formula for inter-camera contrast loss is as follows:

[0051]

[0052] In the formula, and These represent image samples respectively. The set of proxy indices for positive and negative cameras, where |·| represents the number of elements in the proxy index set.

[0053] The loss of the triplet is:

[0054]

[0055] In the formula, ||·|| represents L 2 Norm distance, where i,p and i,n represent the distance between mini-batch image samples. The indices of the hard positive samples and the hard negative samples, where m = 0.3 represents the distance in the triples;

[0056] Step 503: Calculate the source domain dataset Training loss and target domain dataset Total training loss:

[0057]

[0058] Step 504: Update the camera's unknown memory module d represents the feature dimension. This represents the total number of categories in the camera's unknown memory module. Store the average feature momentum representation of all image samples with global pseudo-label l, i.e., the global class proxy of class l;

[0059] Update camera perception memory module This indicates the total number of categories in the camera's perception memory module. Store the average features of all image samples in a certain camera class, i.e., the camera class proxy;

[0060] The update methods for the camera-agnostic memory module and the camera-perceived memory module are the same: both use difficult positive samples of class j to perform momentum updates on class j agents, and the formula is as follows:

[0061]

[0062] Where μ is the momentum renewal coefficient, μ = 0.2;

[0063] Step 505: The number of updates to the camera-agnostic memory module and the camera-perceived memory module is equal to the number of times the person re-identification network model updates the source domain dataset. and target domain dataset The training count is set; it is then determined whether the current training count is within the preset training count range. If so, the process returns to step 2 for feature extraction; otherwise, training is stopped. Detailed Implementation

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0065] An unsupervised, domain-adaptive person re-identification method based on camera perception and deep learning includes:

[0066] Step 1: Obtain a labeled source domain dataset and an unlabeled target domain dataset Where, N s N represents the number of image samples in the source domain dataset. t The number of image samples in the target domain dataset and the number of source domain images. Corresponding real tags Source domain dataset and target domain dataset Each image sample in the dataset undergoes preprocessing; the preprocessing in this embodiment includes preprocessing the source domain dataset. and target domain dataset The preprocessing of each image sample includes: first, adjusting the size of each image sample to the same height and width; then performing horizontal flipping, zero padding, normalization, and random erasure operations.

[0067] Step 2: Construct a camera-aware person re-identification network model and process the preprocessed source domain dataset. and target domain dataset Input the pedestrian re-identification network model for feature extraction to obtain the feature set. in, For source domain dataset Image sample x s Features For the target domain dataset Image sample x t In this embodiment, the characteristics of the present invention are as follows.

[0068] The pedestrian re-identification network model in this embodiment includes a feature extractor for feature extraction. The feature extractor comprises five stages, the first four of which are the first four stages of the ResNet50 network framework, namely:

[0069] The first stage involves sequentially performing convolution, normalization, ReLU activation, and max pooling operations on the image samples. The convolution kernel has 64 data points, a size of 7×7, a zero-padding parameter of 3, and a compensation of 2. The max pooling kernel has a size of 3×3, a zero-padding parameter of 1, and a stride of 2.

[0070] The second phase includes three bottleneck layers;

[0071] The third stage includes four bottleneck layers;

[0072] Phase Four: Includes six bottleneck layers;

[0073] The fifth stage of the feature extractor includes three bottleneck layers, used for pooling the feature maps obtained from the first four stages of the feature extractor to obtain the source domain dataset. and target domain dataset The features of each image sample are defined as follows: in the first bottleneck layer, all convolutional strides are 1, and the convolutional stride in the downsampling operation is 1; the convolutional stride of the other two bottleneck layers is 1, and the downsampling operation is canceled.

[0074] This embodiment uses the parameters of a feature extractor pre-trained on the ImageNet dataset. DSBN is used to replace BN, and generalized average pooling is used to replace global average pooling in the original model. The ResNet50 network framework is existing technology, and each layer of the ResNet50 network framework will not be described in detail.

[0075] Step 3: Based on the camera tags attached to the images, respectively... Global clustering is performed on features within the same camera to obtain intra-camera pseudo-labels for multiple cameras. The camera's internal fake label It contains multiple sets of fine-grained class samples; the feature set of pairs. In Perform global clustering to obtain global pseudo-labels across cameras. The cross-camera global pseudo-label It contains multiple sets of coarse-class samples; specifically including:

[0076] Step 301, Calculation The Jaccard distance between each pair of features yields an N×N dimensional distance matrix;

[0077] Step 302: Combining the obtained distance matrix, the DBSCAN density clustering algorithm is used for clustering. Features within the same cluster are assigned the same pseudo-label, and outliers generated by clustering are assigned the pseudo-label of the cluster closest to them; thus obtaining global pseudo-labels across cameras.

[0078] Step 303: Based on the camera labels attached to the image samples, filter the feature sets of the same camera and calculate the Jaccard distance between each pair of features in the feature set to obtain the distance matrix within the camera.

[0079] Step 304: Combining the distance matrix within the camera, an adaptive hierarchical clustering method is used to perform intra-camera clustering operations. Outliers are assigned pseudo-labels of the nearest clusters to obtain the intra-camera pseudo-labels.

[0080] Step 4: Construct camera-agnostic memory modules and camera-aware memory modules based on clustering results, and then use global pseudo-labels across cameras. and camera internal fake labels Calculate the consistency coefficient matrix; then, based on the consistency coefficient matrix and the global pseudo-labels across cameras... Internal camera tags for multiple cameras After editing, the adjusted global pseudo-labels across cameras are obtained. And the repaired fake label inside the camera This leads to the target domain dataset with pseudo-labels. Specifically, it includes:

[0081] Step 401: Construct a camera-agnostic memory module and a camera-perceived memory module based on the clustering results;

[0082] Step 402: Based on the global pseudo-labels across cameras and camera internal fake labels The consistency coefficient matrix is ​​calculated as follows:

[0083]

[0084] In the formula, For camera internal false labels The fine-grained sample set within, Z c For camera internal false labels The total number of class proxies; Global pseudo-tags The set of rough class samples within, l∈{1,…,L};

[0085] Step 403: Global pseudo-labels based on consistency coefficient matrix and cross-camera setup From the internal camera tags of each camera The refined set of samples belonging to the same agent class is selected and reorganized to obtain the corrected global pseudo-labels. The method is as follows:

[0086]

[0087] The reorganized internal pseudo-labels of each camera are as follows:

[0088] Step 404: Based on the corrected global pseudo-tags And the internal pseudo-labels of each camera are Obtain the target domain dataset with pseudo-labels in, This represents the dataset containing labeled samples under camera c. It is the target domain dataset Chinese image sample Global pseudo-tags, This is the total number of class proxies after sorting. It is the target domain dataset Chinese image sample A fake label inside the camera (camera C). This represents the number of agents in the class of camera c. Represents the target domain dataset Chinese image sample Camera serial number, This indicates the number of image samples taken from camera c.

[0089] Step 5: Employ a joint training approach by computing the source domain dataset. and target domain dataset The training loss is used to update the parameters of the person re-identification network model via gradient backpropagation, dynamically update the parameters of the camera-agnostic memory module and the camera-perceived memory module, and save the optimal parameters of the person re-identification network model; specifically including:

[0090] Step 501, Source Domain Dataset The formula for calculating the training loss is:

[0091]

[0092]

[0093]

[0094] In the formula, For source domain dataset Total loss, For source domain dataset Cross-entropy loss, For source domain dataset The triplet loss; Represents the source domain dataset Chinese image sample belong The probability of a class proxy, ||·|| represents L 2 Norm distance, where i,p and i,n represent the distance between mini-batch image samples. The indexes for difficult positive samples and difficult negative samples are defined, with m = 0.3 representing the distance in the triplet. Thus, supervised learning enables the pedestrian re-identification network model to achieve good performance on the source domain dataset.

[0095] Step 502, Target Domain Dataset The formula for calculating training loss is:

[0096]

[0097] In the formula, Represents the target domain dataset Global contrastive loss with a self-punishing mechanism; Represents the target domain dataset The camera-perceptual contrast loss is used to balance the proportion of the camera-perceptual contrast loss in the total loss of the target domain. Represents the target domain dataset The triplet loss;

[0098] This embodiment includes a self-penalization (SP) mechanism for low-confidence samples. Low-confidence samples are those whose highest probability value, obtained after Softmax normalization and similarity calculation between their features and all global class proxies in the camera-agnostic memory module, is below a threshold μ. Since the pseudo-labels of low-confidence samples are highly unlikely to be correct, it is necessary to increase the similarity between the sample and other pseudo-label class proxies and decrease the similarity between the sample and its corresponding pseudo-label class proxies. The self-penalty mechanism is defined as follows:

[0099]

[0100] In the formula, μ represents the confidence threshold. In this embodiment, μ is fixed as a hyperparameter. However, network prediction often starts from a low confidence level. The confidence value gradually increases with training. A fixed threshold cannot reflect the constantly changing confidence level during training. Therefore, this embodiment adopts an adaptive threshold method. The confidence threshold μ = mean - 2.3 × std, which adapts to the mean and standard deviation of the small batch of samples.

[0101] Therefore, the formula for calculating the global contrastive loss in this embodiment, combined with the self-punishment mechanism, is as follows:

[0102]

[0103] In the formula, τ a τ represents the temperature coefficient. a =0.05; Represents the target domain dataset The total number of image samples in the cluster; global contrastive loss is used to narrow the distance between the corresponding class proxies of image samples in the global cluster and to move them away from other class proxies in the global cluster, thereby achieving classification;

[0104] However, since the image samples in global clustering are greatly affected by the camera, the differences between class proxies are large. Therefore, camera-perceptual contrast loss is added to alleviate the differences caused by global clustering. The camera-perceptual contrast loss constrains the relationship between samples and each camera agent, thereby improving the model's recognition ability.

[0105] The formula for calculating camera-perceived contrast loss is:

[0106]

[0107] In the formula, For in-camera contrast loss, The inter-camera contrast loss is represented by λ² = 0.5, a hyperparameter used to balance the weights of intra-camera and inter-camera contrast losses. The formula for calculating the intra-camera contrast loss is as follows:

[0108]

[0109] In the formula, Indicates from the first camera to the... The total number of class proxies within each camera; This represents the total number of image samples in the c-th camera, thereby alleviating the problem of uneven image sample numbers in each camera and improving the recognition ability of the pedestrian re-identification network model in the camera.

[0110] The formula for inter-camera contrast loss is as follows:

[0111]

[0112] In the formula, and These represent image samples respectively. The set of proxy indices for positive and negative cameras, where |·| represents the number of elements in the proxy index set. This embodiment adds an inter-camera contrast loss to bring the image sample closer to the positive camera class agents in other cameras and away from the mined negative camera class agents, thereby improving the pedestrian re-identification network model's recognition ability across cameras.

[0113] Since camera-perception contrast loss functions similarly to cross-entropy loss, it can only achieve inter-class separation but not intra-class compactness. However, unsupervised and supervised person re-identification tasks, like supervised tasks, require both inter-class separation and intra-class compactness. Therefore, this embodiment adds a triplet loss to the camera-perception contrast loss. The formula for calculating the triplet loss is as follows:

[0114]

[0115] In the formula, ||·|| represents L 2 Norm distance, where i,p and i,n represent the distance between mini-batch image samples. The indices of the hard positive samples and the hard negative samples, where m = 0.3 represents the distance in the triples;

[0116] Step 503: Calculate the source domain dataset Training loss and target domain dataset Total training loss:

[0117]

[0118] Step 504: Update the camera's unknown memory module d represents the feature dimension. This represents the total number of categories in the camera's unknown memory module. Store the average feature momentum representation of all image samples with global pseudo-label l, i.e., the global class proxy of class l;

[0119] Update camera perception memory module This indicates the total number of categories in the camera's perception memory module. Store the average features of all image samples in a certain camera class, i.e., the camera class proxy;

[0120] The update methods for the camera-agnostic memory module and the camera-perceived memory module are the same: both use difficult positive samples of class j to perform momentum updates on class j agents, and the formula is as follows:

[0121]

[0122] Where μ is the momentum renewal coefficient, μ = 0.2;

[0123] Step 505: The number of updates to the camera-agnostic memory module and the camera-perceived memory module is equal to the number of times the person re-identification network model updates the source domain dataset. and target domain dataset The number of training iterations is preset to 50 in this embodiment. It is determined whether the current number of training iterations is within the preset range. If yes, the process returns to step 2 for feature extraction; otherwise, training is stopped.

[0124] Furthermore, in this embodiment, the source domain dataset will be used. and target domain dataset The training sample data is randomly divided into multiple groups, each group containing P class proxies, and each class proxies containing K image samples, so one group has P×K image samples; image samples of one target domain. Includes a feature It also includes two tags, namely a global pseudo-tag across cameras. Samples inside the camera pseudo-label The global pseudo-labels of the samples are corrected to obtain the corrected global pseudo-labels. The camera's internal pseudo-label is

[0125] experiment:

[0126] This application runs on a Quadro P5000 with the Adam optimizer, a learning rate of 0.00035, and 50 training iterations. Before training, the image samples are resized to 256×128, with a batch size of 32, containing 8 pseudo-classes and 4 samples per class. This invention measures the recognition performance using the metrics mAP and Rank-1 / 5 / 10.

[0127] Ablation experiments: To evaluate the superiority of this invention, the proposed camera-aware clustering module (CA), self-penalty mechanism (SP), and camera-aware contrast loss were evaluated. Experiments were conducted on the Market-1501→DukeMTMC-reID task. See Table 1 for details.

[0128]

[0129] Table 1

[0130] Table 1 shows the results in the first row for the base network. The model is trained using only a general clustering algorithm to generate pseudo-labels for the target domain, and the target domain loss function consists only of global contrastive loss and triplet loss. The second row shows the results after adding a camera-aware clustering module (CA) to the base network, demonstrating that the addition of this module improves the network model's performance. The third row shows that adding a self-penalty mechanism (SP) to the base network also results in a stable improvement in model performance. The fourth row shows that using both camera-aware clustering and the self-penalty mechanism on the baseline model further improves its performance, achieving 74.3% mAP and 85.1% Rank-1. The last row shows that after using camera-aware contrastive loss, the model achieves optimal performance on the Market-1501→ DukeMTMC-reID task, reaching 74.7% mAP and 85.8% Rank-1.

[0131] Comparative Experiment: To evaluate the effectiveness of the camera-aware method in this invention, it was compared with state-of-the-art unsupervised adaptive person re-identification methods. The results of the comparative experiment are shown in Table 2:

[0132]

[0133] Table 2

[0134] As shown in Table 2, existing unsupervised domain adaptation pedestrian re-identification methods typically evaluate four domain adaptation tasks from real pedestrian datasets to real pedestrian datasets: DukeMTMC-reID→Market-1501, Market-1501→DukeMTMC-reID, Market-1501→MSMT17, and DukeMTMC-reID→MSMT17. The method proposed in this invention achieves a certain leading advantage in all three tasks, and its performance on the DukeMTMC-reID→Market-1501 task is comparable to state-of-the-art methods.

[0135]

[0136] Table 3

[0137] Table 3 shows the experimental comparison of domain adaptation tasks from synthetic pedestrian datasets to real pedestrian datasets. Because the domain differences between synthetic pedestrian datasets and real-person datasets are often greater than the domain differences between real-person datasets, the domain adaptation task from synthetic pedestrian datasets to real-person datasets is more challenging than the domain adaptation task between real-person datasets. When tested on the DukeMTMC-reID and MSMT17 datasets, our method significantly outperforms current state-of-the-art methods. Specifically, in terms of mAP metrics, our method surpasses the second-place method by 5.1% and 1.4%, respectively. On the Marekt-1501 dataset, the IDM method, in a sense, employs a mix-up strategy to expand the dataset, achieving optimal performance.

[0138] This invention addresses the impact of camera factors on the clustering results of target domain samples, specifically the difficulty in clustering pedestrian images of the same identity captured by different cameras together, and the difficulty in distinguishing different pedestrians captured by the same camera. A camera-aware clustering method is designed to overcome the impact of cross-viewpoint issues in previous global clustering by using a divide-and-conquer approach based on the camera IDs inherent in the image samples. Furthermore, considering the significant intra-class differences among pedestrians of the same identity, this invention adds a camera-aware memory module and a camera-aware contrast loss to mitigate camera-domain differences in the target domain. Finally, this invention designs a self-punishing mechanism to learn by penalizing unreliable pseudo-labels, thereby improving the network's robustness to noisy labels.

[0139] While the disclosure is as stated above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of this disclosure, and all such changes and modifications will fall within the protection scope of this invention.

Claims

1. An unsupervised, domain-adaptive pedestrian re-identification method based on camera perception and deep learning, characterized in that, include: Step 1: Obtain a labeled source domain dataset and an unlabeled target domain dataset ;in, The number of image samples in the source domain dataset. The number of image samples in the target domain dataset and the number of source domain images. Corresponding real tags ; Source domain dataset and target domain dataset Each image sample in the dataset undergoes preprocessing; Step 2: Construct a camera-aware person re-identification network model and process the preprocessed source domain dataset. and target domain dataset Input the pedestrian re-identification network model for feature extraction to obtain the feature set. ,in, For source domain dataset Chinese image sample Features For the target domain dataset Chinese image sample Features; Step 3: Based on the camera tags attached to the images, respectively... Global clustering is performed on features within the same camera to obtain intra-camera pseudo-labels for multiple cameras. The camera's internal pseudo-label It contains multiple sets of fine-grained class samples; for feature sets In Perform global clustering to obtain global pseudo-labels across cameras. The cross-camera global pseudo-label It contains multiple sets of coarse-class samples; Step 4: Construct camera-agnostic memory modules and camera-aware memory modules based on clustering results, and then use global pseudo-labels across cameras. and camera internal fake labels Calculate the consistency coefficient matrix; then, based on the consistency coefficient matrix and the global pseudo-labels across cameras... Internal camera tags for multiple cameras After editing, the adjusted global pseudo-labels across cameras are obtained. And the repaired fake label inside the camera This leads to the acquisition of a target domain dataset with pseudo-labels. ;include: Step 401: Construct a camera-agnostic memory module and a camera-perceived memory module based on the clustering results; Step 402: Based on the global pseudo-labels across cameras and camera internal fake labels The consistency coefficient matrix is ​​calculated as follows: ； In the formula, For camera internal false labels The fine-grained sample set within, , For camera internal false labels The total number of class proxies; Global pseudo-tags The set of rough class samples within, ; Step 403: Global pseudo-labels based on consistency coefficient matrix and cross-camera setup Pseudo-labels from within each camera The refined set of samples belonging to the same agent class is selected and reorganized to obtain the corrected global pseudo-labels. The method is as follows: ； The reorganized internal pseudo-labels of each camera are as follows: ; Step 404: Based on the corrected global pseudo-tags And the internal pseudo-labels of each camera are Obtain the target domain dataset with pseudo-labels ,in, ,express The dataset contains labeled samples from camera number [number]. It is the target domain dataset Chinese image sample Global pseudo-tags, This is the total number of class proxies after sorting. It is the target domain dataset Chinese image sample exist The fake label inside the camera is located under the camera. express Number of agents for camera category # Represents the target domain dataset Chinese image sample Camera serial number, express Number of image samples under camera number 1; Step 5: Employ a joint training approach by computing the source domain dataset. and target domain dataset The total loss is used to update the parameters of the person re-identification network model through gradient backpropagation, dynamically update the parameters of the camera-agnostic memory module and the camera-perceived memory module, and save the optimal parameters of the person re-identification network model.

2. The unsupervised adaptive pedestrian re-identification method based on camera perception and deep learning according to claim 1, characterized in that, In step 1, the source domain dataset and target domain dataset The preprocessing of each image sample includes: first, adjusting the size of each image sample to the same height and width; then performing horizontal flipping, zero padding, standardization, and random erasure operations.

3. The unsupervised adaptive pedestrian re-identification method based on camera perception and deep learning according to claim 2, characterized in that, The person re-identification network model constructed in step 2 includes a feature extractor for feature extraction. The feature extractor consists of five stages, the first four of which are the first four stages of the ResNet50 network framework, namely: The first stage involves performing convolution, normalization, ReLU activation, and max pooling operations sequentially on the image samples. The convolution kernel has 64 data points, a size of 7×7, a zero-padding parameter of 3, and a stride of 2. The max pooling kernel has a size of 3×3, a zero-padding parameter of 1, and a stride of 2. The second phase includes three bottleneck layers; The third stage includes four bottleneck layers; Phase Four: Includes six bottleneck layers; The fifth stage of the feature extractor includes three bottleneck layers, which are used to pool the feature maps obtained from the first four stages of the feature extractor to obtain the source domain dataset. and target domain dataset The features of each image sample are defined as follows: in the first bottleneck layer, all convolutional strides are 1, and the convolutional stride in the downsampling operation is 1; the convolutional stride of the other two bottleneck layers is 1, and the downsampling operation is canceled.

4. The unsupervised adaptive pedestrian re-identification method based on camera perception and deep learning according to claim 3, characterized in that, Step 3 includes: Step 301, Calculation The Jaccard distance between each pair of features yields an N×N dimensional distance matrix; Step 302: Combining the obtained distance matrix, the DBSCAN density clustering algorithm is used for clustering. Features within the same cluster are assigned the same pseudo-label, and outliers generated by clustering are assigned the pseudo-label of their nearest cluster; thus obtaining global pseudo-labels across cameras. ; Step 303: Based on the camera labels attached to the image samples, filter the feature sets of the same camera and calculate the Jaccard distance between each pair of features in the feature set to obtain the distance matrix within the camera. Step 304: Combining the distance matrix within the camera, an adaptive hierarchical clustering method is used to perform intra-camera clustering operations. Outliers are assigned pseudo-labels of the nearest clusters to obtain the intra-camera pseudo-labels. .

5. The unsupervised adaptive pedestrian re-identification method based on camera perception and deep learning according to claim 1, characterized in that, Step 5 includes: Step 501, Source Domain Dataset The formula for calculating the training loss is: ； ； ； In the formula, For source domain dataset Total loss, For source domain dataset Cross-entropy loss, For source domain dataset The triplet loss; Represents the source domain dataset Chinese image sample belong The probability of class proxy, express Norm distance and This indicates a small batch of image samples. Medium-difficulty positive sample index and difficult negative sample index, Represents the distance in a triplet; Step 502, Target Domain Dataset The formula for calculating training loss is: ； In the formula, Represents the target domain dataset Global contrastive loss with a self-punishing mechanism; Represents the target domain dataset Camera-perceived contrast loss, Used to balance the proportion of camera-perceptual contrast loss in the total loss of the target domain; Represents the target domain dataset The triplet loss; where the global contrastive loss, which incorporates a self-punishing mechanism, is calculated as follows: ； In the formula, Indicates the temperature coefficient. ; Represents the target domain dataset The total number of image samples in the dataset; Indicates the confidence threshold; The formula for calculating camera-perceived contrast loss is: ； In the formula, For in-camera contrast loss, For the contrast loss between cameras, These are hyperparameters used to balance the weights of intra-camera contrast loss and inter-camera contrast loss; the formula for calculating intra-camera contrast loss is as follows: ； In the formula, Indicates from the first camera to the... The total number of class proxies within each camera; Indicates the first The total number of image samples in each camera; The formula for inter-camera contrast loss is as follows: ； In the formula, and These represent image samples respectively. The set of proxy indices for positive camera class and the set of proxy indices for negative camera class. This indicates the number of elements in the class proxy index set. ; The loss of the triplet is: ； In the formula, express Norm distance and This indicates a small batch of image samples. The indices of difficult positive samples and difficult negative samples. Represents the distance in a triplet; Step 503: Calculate the source domain dataset Training loss and target domain dataset Total training loss: ； Step 504: Update the camera's unknown memory module , Representing feature dimension, This represents the total number of categories in the camera's unknown memory module. Store global pseudo-tags as The average feature momentum representation of all image samples, i.e., class Global class proxy; Update camera perception memory module , This indicates the total number of categories in the camera's perception memory module. Store the average features of all image samples in a certain camera class, i.e., the camera class proxy; The update methods for the camera-agnostic memory module and the camera-sensing memory module are the same, both utilizing... Difficult positive samples of the class The momentum update formula for class proxies is: ； in, For momentum update coefficients, ; Step 505: The number of updates to the camera-agnostic memory module and the camera-perceived memory module is equal to the number of times the person re-identification network model updates the source domain dataset. and target domain dataset The training count is set; it is then determined whether the current training count is within the preset training count range. If so, the process returns to step 2 for feature extraction; otherwise, training is stopped.

Images