A Pedestrian Re-identification Method Based on Joint Distance Matrix and Hybrid Contrastive Loss

Abstract

The present invention relates to the field of image processing technology, and in particular to a pedestrian re-identification method based on a joint distance matrix and a hybrid contrast loss, comprising obtaining a pedestrian image and performing preprocessing; using features in an instance-level memory dictionary and combining it with a camera offset distance matrix to calculate a joint distance matrix; then using the DBSCAN algorithm for clustering to generate pseudo labels; selecting a certain number of query samples from the training samples and inputting them into a backbone network to extract features of the query samples, which are used to calculate a hybrid contrast loss with the features in the clustering memory dictionary; extracting features of a global average pooling layer and using Euclidean distance for retrieval. The present invention guides camera perception clustering by designing a joint distance matrix, thereby alleviating the impact of camera differences and making it easier to cluster images of the same identity under different cameras; at the same time, using a hybrid contrast loss strategy, it is possible to effectively utilize clustering information between samples and improve model performance.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a pedestrian re-identification method based on a joint distance matrix and a hybrid contrast loss. Background Art

[0002] In recent years, self-supervised contrastive learning has garnered increasing attention due to its effectiveness in unsupervised representation learning. The performance of unsupervised methods also relies on learned feature representations. Recently, advanced person re-identification methods have achieved excellent performance by using a memory dictionary to store all instance features, leveraging pseudo-labels generated by clustering, and combining this with contrastive learning. Contrastive learning can be categorized into two types based on how it utilizes supervised information: instance-level contrastive learning and cluster-level contrastive learning.

[0003] Instance-level contrastive learning relies on enhanced information about samples. It treats each sample as a separate class and learns features by matching images augmented using different data augmentation methods. However, in person re-identification datasets, each class contains more than one positive sample, resulting in inconsistent updates for each class. Furthermore, using instance-level contrastive loss exploits the self-supervisory information of each sample without fully considering the structural and correlation information (such as clustering) between samples. While cluster-level contrastive learning captures structural and correlation information, it introduces excessive additional structural information to the negative sample data, which is useless in practical applications.

[0004] Camera variability is a significant factor in person re-identification (RID) tasks. In RID datasets, images of people captured by different cameras have varying poses, lighting, and resolutions. In other words, even images of people belonging to the same identity can differ significantly due to camera variations. This is particularly true in unsupervised RID tasks, where camera variability can impact the accuracy of pseudo-labeling. Previous work has considered the impact of camera variability. Despite various attempts, mitigating the impact of pseudo-labeling in unsupervised scenarios remains an open question. Summary of the Invention

[0005] To address the shortcomings of existing algorithms, this paper designs a joint distance matrix to guide camera-aware clustering, alleviates the impact of camera differences, and makes it easier to cluster images of the same identity under different cameras; at the same time, the use of a mixed contrast loss strategy can effectively utilize the clustering information between samples and improve the performance of the model.

[0006] The technical solution adopted by the present invention is: a pedestrian re-identification method based on a joint distance matrix and a hybrid contrast loss comprises the following steps:

[0007] Step 1: Obtain a pedestrian image dataset and preprocess the images;

[0008] Furthermore, preprocessing includes: performing random data augmentation, unifying the image resolution, feeding the unlabeled pedestrian images into the feature extraction model, determining the initial features of the pedestrian images, and storing the extracted features in the instance memory dictionary;

[0009] Furthermore, the feature extraction model uses the deep residual network Resnet50 model to extract the features of pedestrian images.

[0010] Step 2: Use the features in the instance-level memory dictionary and the camera offset distance matrix to calculate the joint distance matrix; then use the DBSCAN algorithm for clustering and generate pseudo labels;

[0011] Further, it specifically includes:

[0012] Step 21: The camera ID corresponding to each training sample is expressed as Where N represents the number of pedestrian samples in the training data set, cam i Indicates the camera label corresponding to the current i-th training sample, and sets a camera domain offset matrix M cam ∈N cam ×N cam , N cam Indicates the number of cameras;

[0013] Step 22, through the original distance matrix M i,j and the offset matrix M cam Calculate a joint distance matrix

[0014] Furthermore, the original distance matrix M i,j The calculation formula is:

[0015]

[0016] Among them, f θ (x i ) is the sample feature extracted by the backbone network;

[0017] Step 23: Calculate the difference between camera u and camera v in the domain offset matrix;

[0018] Furthermore, the difference is calculated as:

[0019]

[0020] Among them, |f θ (x i ) u| represents the number of instance samples captured by camera u, |f θ (x j ) v | represents the number of instance samples captured by camera v;

[0021] Step 24: Subtract the domain offset matrix from the original distance matrix as a penalty term to obtain a joint distance matrix;

[0022] Furthermore, the joint distance matrix The formula is:

[0023]

[0024] Among them, α is a parameter that controls the effect of camera difference, M i,j is the original distance matrix, is the difference between camera u and camera v in the domain offset matrix;

[0025] Step 3: Select a certain number of query samples from the training samples and input them into the backbone network to extract the features of the query samples, which are used to calculate the hybrid contrast loss with the features in the cluster memory dictionary;

[0026] Further, it specifically includes:

[0027] Step 31, calculating the contrast loss using the similarity between the input image and the cluster centroid features;

[0028] Furthermore, the cluster centroid is expressed as:

[0029]

[0030] Among them, C y represents the centroid of the y-th cluster, |F y | represents the number of all features in the y-th cluster set, f i represents the characteristics of a sample, and

[0031] Step 32: retain the clustering information of the positive samples, cluster similar samples together, and treat all remaining samples except the positive samples as negative samples;

[0032] Furthermore, the calculation formula of the mixed contrast loss is:

[0033]

[0034] in, Represents the centroid of the y-th cluster, which is used to represent the positive sample. Represents the set of samples that do not currently belong to the current cluster.

[0035] Step 4: Extract the features of the global average pooling layer and use Euclidean distance for retrieval.

[0036] Beneficial effects of the present invention:

[0037] 1. The method of the present invention is compared with the currently advanced unsupervised pedestrian re-identification algorithms JNTL, JGCL, SPCL, RSS and CACL. Through the horizontal comparison of the mean average precision (mAP) and the probability of correct matching in the top k lists (Rank-k), it can be seen that the comprehensive performance of the present invention is significantly better than those of other algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the pedestrian re-identification method based on joint distance matrix and hybrid contrast loss of the present invention;

[0039] Figure 2 Comparison between the proposed method and five other commonly used methods on the Market-1501 dataset;

[0040] Figure 3 Comparison between the proposed method and five other commonly used methods on the DukeMTMC dataset;

[0041] Figure 4 This figure compares the proposed method with five other commonly used methods on the MSMT17 dataset. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0043] Since the styles of images taken by different cameras are different, the present invention proposes a joint distance matrix to alleviate the impact of camera differences. Specifically, for each intra-camera and inter-camera situation, the average value of the cosine similarity between the target instance samples is calculated, which implicitly reflects the distance difference between samples caused by camera changes. Before starting clustering each time, the value of the camera domain offset matrix is subtracted from the similarity of the instance sample pair according to the camera label of the instance sample pair, and then a softmax operation is performed on the distance vector of each instance sample. In this way, the differences between cameras can be narrowed, and images of the same identity taken by different cameras will be more likely to be clustered into the same category. In order to make up for the shortcomings of instance-level and cluster-level contrast losses and to be able to efficiently utilize the information of samples, the present invention uses a mixed contrast loss to train the model. First, the clustering information of the positive samples is retained, similar samples are clustered together, and all remaining samples except the positive samples are treated as negative samples.

[0044] like Figure 1 As shown in FIG, a pedestrian re-identification method based on a joint distance matrix and a hybrid contrast loss includes:

[0045] Step 1: Obtain the MarKet-1501, DukeMTMC-reID, and MSMT17 image datasets for pedestrian re-identification, perform random data augmentation on the datasets, where random data augmentation includes random flipping, random cropping, and random erasing, unify the image resolution to 256×128, and input the unlabeled pedestrian images into the feature extraction model. This embodiment uses the residual network Resnet50 to extract the features of the pedestrian images, determine the initial features of the pedestrian images, and store the extracted features in the instance memory dictionary.

[0046] In this step, person re-identification, also known as pedestrian re-ID, uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence. For example, given a monitored pedestrian image with an identity tag from camera A, the system can retrieve images of the same pedestrian from a cross-device device, such as camera B. Person re-ID can overcome the visual limitations of fixed cameras and can be combined with pedestrian detection and tracking technologies for widespread application in intelligent video surveillance, smart security, and other fields. For example, if camera A captures an image of a pedestrian at an intersection and the pedestrian enters an alley at the corner of the intersection, camera A, being fixed, cannot capture the pedestrian once they enter the alley. Therefore, camera B in the alley is needed to search for the pedestrian image.

[0047] Therefore, in order to search for pedestrian images taken by camera A from the pedestrian images taken by camera B, the traditional method uses the images taken by camera A as a training set and trains the pedestrian re-identification model based on the deep learning method. However, the traditional method is extremely dependent on the training data set, that is, it can only recognize the image features taken by camera A in the training set (such as the features of the image taken under the perspective or lighting of camera A). Once it is switched to a new application scenario (such as the image taken by camera B across different perspectives or lighting), since the traditional method is trained based on the images taken by camera A, it will judge that the image features of the same pedestrian are different (the features of the images of the same pedestrian taken by camera A and camera B are different), and thus it is believed that the pedestrian image taken by camera A does not exist in the image taken by camera B, resulting in misjudgment.

[0048] In order to solve the above-mentioned technical problem that it is impossible to accurately identify whether there is a pedestrian image taken by camera A in the pedestrian image taken by camera B due to cross-viewpoint differences and / or lighting differences in pedestrian images, this embodiment uses a joint distance matrix to guide camera perception clustering to alleviate the impact of camera differences and make images of the same identity under different cameras easier to cluster; at the same time, to address the problem that the relationship between different instances in instance-level contrastive learning is not fully utilized, a mixed contrast loss is used to enhance the robustness of the model.

[0049] This embodiment uses a proximity algorithm to classify image features in the feature extraction module, so that the image features of pedestrians with the same identity are closer, and the image features of pedestrians with different identities are farther apart, thereby improving the similarity between the image features to be re-identified and the image features of pedestrians with the same identity, making the recognition results more accurate and highly robust.

[0050] Step 2: Use the features in the instance-level memory dictionary and the camera offset distance matrix to calculate the joint distance matrix, then use the DBSCAN algorithm for clustering to generate pseudo labels;

[0051] Before starting training, the joint distance matrix is combined and clustered using the DBSCAN algorithm to generate pseudo labels. For the DBSACN algorithm, the maximum distance between two samples is set to 0.5.

[0052] The camera ID corresponding to each training sample is represented as Where N represents the number of pedestrian samples in the training data set, cam i Indicates the camera label corresponding to the current i-th training sample. In order to reduce the impact of camera differences, a camera domain offset matrix M is set cam ∈N cam ×N cam To reflect the image differences between different camera domains or within the same domain, where N cam Represents the number of cameras. Before calculating the distance matrix for clustering, try to get the original distance matrix M i,j ∈N×N to reduce the camera difference; in each iteration, the domain offset matrix M cam It is derived from the average similarity of instance sample pairs belonging to the corresponding camera domain; then, through the original distance matrix M and the offset matrix M cam Calculate a joint distance matrix This matrix can better reflect the differences between identities; the original distance matrix M i,j is the sample feature f extracted from the backbone network θ (x i ) is calculated based on the similarity between the original distance matrix M i,jThe calculation formula is:

[0053]

[0054] Although the number of pedestrian identities is much larger than the number of camera domains, the average of the similarities belonging to each camera pair implicitly represents the differences caused by the corresponding inter-domain or intra-domain changes; the difference between camera u and camera v in the domain offset matrix is calculated as follows:

[0055]

[0056] Among them, |f θ (x i ) u | represents the number of instance samples taken by camera u; Finally, the value of the domain offset matrix is subtracted from the original distance matrix as a penalty term to obtain the joint distance matrix

[0057]

[0058] Among them, α is a parameter that controls the effect of camera difference.

[0059] Step 3: During the training phase, a certain number of query samples are selected from the training samples and input into the backbone network to extract the features of the query samples, which are then used to calculate the hybrid contrast loss with the features in the cluster memory dictionary.

[0060] In previous cluster-level contrastive learning, clustering information (i.e., pseudo labels) is generated using a clustering algorithm, and the contrast loss is calculated using the similarity between the input image and the cluster centroid (i.e., the average vector of each cluster). The centroid of each cluster can be expressed as:

[0061]

[0062] Among them, C y represents the centroid of the y-th cluster, |F y | represents the number of all features in the y-th cluster set, f i represents the characteristics of a sample, and

[0063] To compensate for the shortcomings of instance-level and cluster-level contrast losses and to efficiently utilize sample information, this paper uses a hybrid contrast loss to train the model. First, the clustering information of positive samples is retained, similar samples are clustered together, and all other samples except positive samples are treated as negative samples. The specific calculation method is as follows:

[0064]

[0065] in, Represents the centroid of the y-th cluster, which is used to represent the positive sample. Represents the set of samples that do not currently belong to the current cluster.

[0066] During training, the initial learning rate was 0.00035, which was then reduced to 1 / 10 of the previous rate every 20 training epochs. The Adam optimizer was used to optimize the model, with weight decay set to 0.0005, for a total of 50 epochs. In addition, when training with the hybrid contrast loss, the temperature coefficient was set to 0.05 and the momentum parameter was set to 0.2.

[0067] Step 4: During testing, extract the features of the global average pooling layer and use Euclidean distance for retrieval.

[0068] Calculate the similarity between the pedestrian pictures in the test set (query) and the pedestrian pictures in the gallery (gallery), and then sort them according to the similarity to obtain the model results.

[0069] like Figure 2-4 As shown in the figure, the method of the present invention is compared with the unsupervised pedestrian re-identification algorithms JNTL, JGCL, SPCL, RSS and CACL on three datasets. Through the horizontal comparison of the average precision (mAP) and the probability of correct matching in the top k lists (Rank-k), it can be seen that the comprehensive performance of the present invention is significantly better than that of other algorithms.

[0070] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A person re-identification method based on joint distance matrix and hybrid contrast loss, characterized in that: The following steps are involved: Step 1: Obtain a pedestrian image dataset and preprocess the images; Step 2: Use the features in the instance-level memory dictionary and the camera offset distance matrix to calculate the joint distance matrix; then use the DBSCAN algorithm for clustering and generate pseudo labels; Step 3: Select a certain number of query samples from the training samples and input them into the backbone network to extract the features of the query samples, which are used to calculate the hybrid contrast loss with the features in the cluster memory dictionary; Step 4: Extract the features of the global average pooling layer and use Euclidean distance for retrieval.

2. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 1 is characterized in that Preprocessing includes: Perform random data augmentation on the image, unify the image resolution, input the unlabeled pedestrian image into the feature extraction model, determine the initial features of the pedestrian image, and store the extracted features in the instance memory dictionary.

3. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 2 is characterized in that The feature extraction model uses the deep residual network Resnet50 model to extract the features of pedestrian images.

4. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 1, characterized in that Step 2 specifically includes: Step 21: The camera ID corresponding to each training sample is expressed as Set a camera domain offset matrix M cam ∈N cam ×N cam , N cam Indicates the number of cameras; Step 22, through the original distance matrix M i,j and the offset matrix M cam Calculate a joint distance matrix Step 23: Calculate the difference between camera u and camera v in the domain offset matrix; Step 24: Subtract the value of the domain offset matrix from the original distance matrix as a penalty term to obtain a joint distance matrix.

5. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 4 is characterized in that Original distance matrix M i,j The calculation formula is: Among them, f θ (x i ) is the sample feature extracted by the backbone network.

6. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 4, characterized in that The difference is calculated as: Among them, |f θ (x i ) u | represents the number of instance samples captured by camera u, |f θ (x j ) v | represents the number of instance samples captured by camera v.

7. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 4 is characterized in that Joint distance matrix The formula is: Among them, α is a parameter that controls the effect of camera difference, M i,j is the original distance matrix, is the difference between camera u and camera v in the domain offset matrix.

8. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 4, characterized in that Step three specifically includes: Step 31, calculating the contrast loss using the similarity between the input image and the cluster centroid features; Step 32: retain the clustering information of the positive samples, cluster similar samples together, and treat all remaining samples except the positive samples as negative samples.

9. The pedestrian re-identification method based on joint distance matrix and hybrid contrast loss according to claim 8, characterized in that The cluster centroid is expressed as: Among them, C y represents the centroid of the y-th cluster, |F y | represents the number of all features in the y-th cluster set, f i represents the characteristics of a sample, and

Images