Comparative learning unsupervised pedestrian re-identification method based on multi-view similarity aggregation

Abstract

The invention relates to the technical field of computer vision, in particular to a contrast learning unsupervised pedestrian re-identification method based on multi-view similarity aggregation, which uses ResNet50 as a backbone network to extract global features of all pedestrian training samples, and then uses a DBSCAN clustering algorithm to distribute pseudo labels for all global training examples. Constructing a locally refined multi-branch network to obtain diversified feature representations, enriching high-level semantic information contained in multi-view pedestrian features, and improving a re-recognition effect; the problems of insufficient fine-grained local feature information and low algorithm accuracy of an unsupervised pedestrian re-recognition algorithm in a complex shielding scene are reduced, and the method can be widely applied to the technical field of pedestrian re-recognition.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to an unsupervised pedestrian re-identification method based on multi-view similarity aggregation and contrastive learning. Background Technology

[0002] Unsupervised person re-identification technology utilizes computer vision to identify specific pedestrians in uncertain scenes based on features such as clothing, posture, and hairstyle. Its core lies in achieving automatic pedestrian identification and matching without manual data annotation. This technology relies on efficient feature matching algorithms, comparing pedestrian features across different frames to continuously confirm pedestrian identities across frames. It then extracts key information such as walking speed, movement trajectory, and direction of travel. This technology aims to overcome the visual limitations of fixed cameras and can be combined with pedestrian detection and tracking technologies, finding wide application in intelligent video surveillance and intelligent security. However, unsupervised person re-identification faces numerous challenges, such as mutual occlusion between pedestrians, confusion due to similar appearance features, and image blurring caused by excessively fast pedestrian movement. In practical applications, these complex factors often lead to reduced re-identification performance, resulting in misidentification and identity confusion. Therefore, developing a robust unsupervised person re-identification technology that can operate stably in complex environments is crucial. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose an unsupervised pedestrian re-identification method based on multi-view similarity aggregation and contrastive learning, so as to solve the problem of insufficient recognition accuracy of leisure travel methods.

[0004] To achieve the above objectives, this invention provides a contrastive learning-based unsupervised person re-identification method based on multi-view similarity aggregation, comprising the following steps:

[0005] S1. Acquire pedestrian video, track pedestrians and crop images to obtain multi-angle images of pedestrians;

[0006] S2. Using ResNet50 as the backbone network, extract global features from all pedestrian multi-angle images;

[0007] S3. Use the DBSCAN clustering algorithm to assign pseudo-labels to all training instances globally;

[0008] S4. Use the multi-level contrastive learning module to obtain the relationships between instances;

[0009] S5. Calculate the contrast loss, supervise the learning of the network, and obtain the pedestrian re-identification result.

[0010] Preferably, step S1 specifically includes:

[0011] The unlabeled pedestrian dataset is denoted as Where xi is the i-th image and N is the number of images; the pedestrian samples, after passing through layer 4 of ResNet50, yield coarse shared global features, denoted as F. θ (x i )∈R C×H×W , where C, H, and W represent the channel size, feature map height, and width, respectively;

[0012] The network is divided into two paths, consisting of a global feature branch f- g and the three local feature branches f -pn The following structures, n = 1, 2, 3, are jointly constructed, where n is the total number of branches. These are used to learn rich and diverse pedestrian representations. In the global branches, a learnable generalized mean (GEM) pooling method is applied to the output features F as a fine-grained instance retrieval method to obtain global features.

[0013] In the local branch, the output feature F is divided into three horizontal strips, and GEM is applied to obtain the locally refined features. The final global and local instance feature sets are obtained, expressed as follows, where t is the training period:

[0014]

[0015] Preferably, step S2 specifically includes:

[0016] The Re-ID multi-view feature map is segmented, and a localized multi-branch network is constructed in the multi-view appearance feature extraction module to obtain diverse feature representations.

[0017] For the branch structure of multi-view feature maps, a multi-channel perceptual attention network is embedded to aggregate appearance feature information at different scales, so as to enrich the high-level semantic information contained in the multi-view pedestrian features.

[0018] Preferably, step S3 specifically includes:

[0019] The instance features of all samples are stored in Memory k (k = 1, 2, 3, 4), and updated during training using the output features of the corresponding branches.

[0020] The fusion hierarchical clustering module performs fusion clustering on the extracted image features, and the multi-view branch features share the same pseudo-label.

[0021] Preferably, updating the output features of the corresponding branch during training includes:

[0022] Update using the following formula;

[0023]

[0024] Where M∈R N×d d is the feature dimension; and These represent the output characteristics of different branches; and These represent the clustering features stored in their respective memories after accumulating different output features; α∈[0,1] is the update rate.

[0025] Preferably, step S4 specifically includes:

[0026] Feature extraction and construction of positive and negative sample pairs;

[0027] The Euclidean distance is used to calculate the distance between feature vectors, measuring their similarity and difference. The closer the cosine distance is to 1, the more similar the two feature vectors are; the closer it is to -1, the less similar the two feature vectors are.

[0028] Define a contrastive loss function to calculate the inner product between the instance feature vector f and the centroid c of its cluster, and measure the similarity between them;

[0029] Calculate the instance feature vector f and the centroid c of the k-th cluster. k The inner product between f is compared with the centroids of clusters that do not belong to it;

[0030] A unified contrastive loss is used to supervise the learning of the network.

[0031] Preferably, step S4 further includes:

[0032] The global features are horizontally segmented, and the contrastive loss between the global branch and the local branch is calculated separately. To balance the loss weights in each branch, the following formula is used as the model loss, where, This represents the loss of the global branch. This represents the loss of the local branch (n=3), where 1 / 3 is the weighting factor;

[0033]

[0034] After assigning a pseudo-label, a unified contrastive loss is constructed using instance features to supervise the network's learning, where the unified contrastive loss function is defined as follows:

[0035]

[0036] Where f represents instance features and represents global features. and local branch features c +c represents the centroid feature center of the cluster to which f belongs. k C represents k The centroid of the cluster to which it belongs, k is the number of clusters, τ is a hyperparameter, and <,> are used to calculate the inner product between two feature vectors to measure their similarity.

[0037] The beneficial effects of this invention are that the method can be widely applied in fields such as intelligent security, video surveillance, and human-computer interaction, providing strong support for pedestrian tracking and behavior analysis.

[0038] This invention effectively reduces the impact of perspective changes and lighting conditions on pedestrian re-identification by fusing multi-view image features and a contrastive learning strategy, significantly improving recognition accuracy. Simultaneously, the unsupervised method avoids dependence on large amounts of labeled data, enabling the model to better adapt to different scenarios and conditions, thus enhancing generalization ability. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a system framework diagram of the contrastive learning-based unsupervised person re-identification method based on multi-view similarity aggregation according to an embodiment of the present invention.

[0041] Figure 2 This is a framework diagram of the multi-view appearance feature extraction module according to an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0043] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0044] like Figure 1 , Figure 2 As shown in the embodiments of this specification, a contrastive learning unsupervised pedestrian re-identification method based on multi-view similarity aggregation is provided, characterized by including: a multi-view appearance feature extraction module, a hierarchical clustering fusion module, and a contrastive learning module;

[0045] Multi-view appearance feature extraction module: In unsupervised pedestrian re-identification tasks, the lack of labeled data during training, coupled with the variability in lighting, shooting angles, and pedestrian poses in real-world environments, makes it impossible to extract sufficient feature representations from unlabeled images, resulting in low model recognition accuracy. To enable the model to have good recognition capabilities in real-world scenes, a multi-view appearance feature extraction module is designed to refine the branches of global and local features from multiple perspectives, thereby capturing multiple sets of diverse feature representations.

[0046] Hierarchical Clustering Fusion Module: The overall network comprises alternating clustering and model training phases. A fusion hierarchical clustering algorithm is designed to integrate pedestrian image features from different viewpoints and cluster them based on feature similarity. This module aims to reduce the impact of viewpoint differences on feature matching and improve the accuracy of clustering results.

[0047] Contrastive Learning Module: This module constructs a multi-level contrastive learning framework. Through contrastive learning between positive sample pairs (different images of the same pedestrian) and negative sample pairs (images of different pedestrians), it further explores the relationships between instances. This module improves the InfoNCE loss function, optimizing the model to make the features of the same pedestrian more similar and the features of different pedestrians more dissimilar, thereby supervising the training and updating of the network.

[0048] Design a multi-view appearance feature extraction module;

[0049] The viewpoint appearance feature extraction module denotes the unlabeled pedestrian dataset as Where xi is the i-th image and N is the number of images. Pedestrian samples, after passing through layer 4 of ResNet50, yield coarse shared global features, denoted as F. θ (x i )∈R C×H×W Where C, H, and W represent the channel size, feature map height, and width, respectively. The network is then split into two paths, with the global feature branch f- g and the three local feature branches f -pn A joint structure (n = 1, 2, 3, where n is the total number of branches) is constructed to learn rich and diverse pedestrian representations. In the global branch, a learnable generalized mean (GEM) pooling method is applied to the output features F as a fine-grained instance retrieval method to obtain global features. In the local branch, the output feature F is divided into three horizontal strips, and GEM is applied to obtain the locally refined features. At this point, the local features incorporate more globally shared semantic information, resulting in stronger expressive power. The final global and local instance feature sets are obtained, represented by formulas (1) and (2), where t is the training period.

[0050]

[0051] Hierarchical clustering fusion module;

[0052] In the first stage, ResNet50 is used to extract global features from all training samples. Then, a clustering method is used to assign pseudo-labels to all training instances globally. Global and local features share the same pseudo-labels. During model training, pedestrian samples are divided and sampled according to a group sampling strategy. A multi-branch network with multi-view local refinement is then constructed to obtain diverse feature representations. These instance features are stored in a corresponding in-memory dictionary so that the training and updates of the network can be supervised using a unified contrastive loss function.

[0053] To preserve the obtained global and local instance features, four identical memory dictionaries Memory k (k = 1, 2, 3, 4) are used to store the instance features of all samples and are updated during training by the output features of the corresponding branches, as shown in Equations (3) and (4).

[0054]

[0055] Where M∈R N×d d is the feature dimension; and These represent the output characteristics of different branches; and These represent the clustering features stored in their respective memories after accumulating different output features; α∈[0,1] is the update rate.

[0056] 4. The unsupervised pedestrian re-identification method based on multi-view similarity aggregation and contrastive learning according to claim 1, characterized in that: a contrastive learning module;

[0057] This method, which utilizes instance features and cluster centroids to construct contrastive losses for supervised network learning, aims to learn discriminative feature representations in an unsupervised manner. The global features are horizontally segmented, and the contrastive losses for the global and local branches are calculated separately. To balance the loss weights in each branch, formula (5) is used as the model loss. This represents the loss of the global branch. This represents the loss of the local branch (n=3), where 1 / 3 is the weighting factor.

[0058]

[0059] After assigning a pseudo-label, a unified contrastive loss is constructed using instance features to supervise the learning of the network, where the unified contrastive loss function is defined as shown in Equation (6).

[0060]

[0061] Where f represents instance features, and in this paper, it represents global features. and local branch features c + c represents the centroid (feature center) of the cluster to which f belongs; k C represents k The centroid of the cluster to which it belongs, k is the number of clusters, τ is a hyperparameter, and <,> are used to calculate the inner product between two feature vectors to measure their similarity.

[0062] 5. The unsupervised pedestrian re-identification method based on multi-view similarity aggregation according to claim 1, characterized by the following steps:

[0063] Step 1: Obtain pedestrian video from surveillance cameras, track pedestrians and crop images to obtain multi-angle images of pedestrians;

[0064] Step 1.1: First, a group sampling strategy is used to divide and sample pedestrians;

[0065] Step 1.2: Read the input image;

[0066] Step 2: Use ResNet50 as the backbone network;

[0067] Step 3: Initialize the network;

[0068] Step 4: Extract global features from all pedestrian training samples;

[0069] Step 4.1: Segment the Re-ID multi-view feature map and construct a multi-branch network for local refinement in the multi-view appearance feature extraction module to obtain diverse feature representations;

[0070] Step 4.2: For the branch structure of the multi-view feature map, embed a multi-channel perceptual attention network to aggregate appearance feature information at different scales to enrich the high-level semantic information contained in the multi-view pedestrian features.

[0071] Step 5: Use the DBSCAN clustering algorithm to assign pseudo-labels to all training instances globally;

[0072] Step 5.1: Store the instance features of all samples using Memory k (k = 1, 2, 3, 4), and update them during training using the output features of the corresponding branches;

[0073] Step 5.2: The fusion hierarchical clustering module performs fusion clustering on the extracted image features, and the multi-view branch features share the same pseudo-label;

[0074] Step 6: Use the multi-level contrastive learning module to obtain the relationships between instances;

[0075] Step 6.1: Feature extraction and construction of positive and negative sample pairs;

[0076] Step 6.2: Use Euclidean distance to calculate the distance between feature vectors to measure the similarity and difference between them; the closer the cosine distance is to 1, the more similar the two feature vectors are; the closer it is to -1, the less similar the two feature vectors are.

[0077] Step 6.3: Define the contrastive loss function, calculate the inner product between the instance feature vector f and the centroid c of its cluster, and measure the similarity between them;

[0078] Step 6.4: Calculate the instance feature vector f and the centroid c of the k-th cluster. k The inner product between f is compared with the centroids of clusters that do not belong to it;

[0079] Step 6.5: Use a unified contrastive loss to supervise the network's learning;

[0080] Step 7: Using the backpropagation algorithm, propagate the gradient of the contrastive loss function to the model parameters, and use the optimizer to update the model parameters.

[0081] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity. Any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A contrastive learning unsupervised person re-identification method based on multi-view similarity aggregation, characterized in that: The following steps are involved: S1. Obtain a video of a pedestrian, track the pedestrian and crop the image to obtain a multi-angle image of the pedestrian; S2, using ResNet50 as the backbone network to extract global features of all pedestrian multi-angle images; S3, use the DBSCAN clustering algorithm to assign pseudo labels to all global training instances; S4, using a multi-level contrastive learning module to obtain the relationship between instances; S5. Calculate the contrast loss, supervise the learning of the network, and obtain the pedestrian re-identification result.

2. The unsupervised person re-identification method based on contrastive learning and multi-view similarity aggregation according to claim 1 is characterized in that: Step S1 specifically includes: The unlabeled pedestrian dataset is denoted as Where xi is the i-th image, N is the number of images; pedestrian samples are processed by layer4 of ResNet50 to obtain a rough shared global feature, denoted as F θ (x i )∈R C×H×W , where C, H, and W represent the size of the channel, the height and width of the feature map respectively; The network is divided into two paths, consisting of the global feature branch f -g And the three local feature branches f -pn , n = 1, 2, 3, jointly constructed, where n is the total number of branches, for learning rich and diverse pedestrian representations. In the global branch, the global feature is obtained by applying a learnable generalized mean (GEM) pooling to the output feature F as a fine-grained instance retrieval. In the local branch, the output feature F is divided into three horizontal strips, and GEM is also applied to obtain the local refined features Finally, the global and local instance feature sets are obtained, which are expressed as follows, where t is the training cycle:

3. The unsupervised person re-identification method based on contrastive learning and multi-view similarity aggregation according to claim 1 is characterized in that: Step S2 specifically includes: The Re-ID multi-view feature map is segmented, and a local refined multi-branch network in the multi-view appearance feature extraction module is constructed to obtain diversified feature representations; Aiming at the branch structure of multi-view feature maps, a multi-channel perceptual attention network is embedded to aggregate appearance feature information of different scales to enrich the high-level semantic information contained in the multi-view pedestrian features.

4. The unsupervised person re-identification method based on contrastive learning and multi-view similarity aggregation according to claim 1, characterized in that: Step S3 specifically includes: Memory k (k = 1, 2, 3, 4) is used to store instance features of all samples and update them through the output features of the corresponding branches during training; The fused hierarchical clustering module performs fusion clustering on the extracted image features, and multi-view branch features share the same pseudo-label.

5. The unsupervised person re-identification method based on contrastive learning and multi-view similarity aggregation according to claim 4 is characterized in that: The updating of the output features of the corresponding branches during the training process includes: Update by the following formula; Where M∈R N×d , d is the characteristic dimension; and Represent the output features of different branches respectively; and They represent the clustering features stored in their respective memories after accumulating different output features; α∈[0,1] is the update rate.

6. The unsupervised person re-identification method based on contrastive learning and multi-view similarity aggregation according to claim 1, characterized in that: Step S4 specifically includes: Feature extraction and construction of positive and negative sample pairs; Use Euclidean distance to calculate the distance between feature vectors and measure the similarity and difference between feature vectors. The closer the cosine distance is to 1, the more similar the two feature vectors are; the closer it is to -1, the less similar the two feature vectors are. Define the contrast loss function, calculate the inner product between the instance feature vector f and the centroid c of the cluster to which it belongs, and measure the similarity between them; Calculate the instance feature vector f and the kth cluster centroid c k The inner product between them compares the similarity between f and the centroid of the cluster that does not belong to it; A unified contrastive loss is used to supervise the learning of the network.

7. The unsupervised person re-identification method based on contrastive learning and multi-view similarity aggregation according to claim 6 is characterized in that: Step S4 further comprises: The global feature level is split and the contrast loss of the global branch and the local branch is calculated separately. In order to balance the loss weights in each branch, the following formula is used as the model loss, where: represents the loss of the global branch, represents the loss of the local branch part (n=3), 1 / 3 is the weight factor; After assigning a pseudo label, the instance features are used to construct a unified contrast loss to supervise the learning of the network, where the unified contrast loss function is defined as follows: Among them, f represents instance features and represents global features and local branch features c + represents the centroid feature center of the cluster to which f belongs, c k Represents C k The centroid of the cluster to which it belongs, k is the number of clusters, τ is a hyperparameter, and <,> is used to calculate the inner product between two feature vectors to measure their similarity.

Images