An unsupervised person re-identification method based on feature consistency

Abstract

The present invention relates to an unsupervised pedestrian re-identification method based on feature consistency. The method comprises the following steps: 1: designing a robust feature extraction model based on random feature masks, and then calculating a time average model based on the feature extraction model; 2: feeding the pedestrian image to the feature extraction model and the time average model twice respectively, obtaining four features, and making the four features close to each other through consistency loss; 3: clustering the features extracted by the feature extraction model, and then adjusting the clustering results according to independence and compactness, and finally fine-tuning the model based on the clustering results. The method based on feature consistency designed by the present invention aims to make the feature extraction model pay more attention to the high-level semantic information of the image by randomly masking part of the information of the image, thereby improving the robustness of the feature extraction model.

Description

Technical field:

[0001] The present invention relates to the field of pedestrian re-identification, and in particular to an unsupervised pedestrian re-identification method based on feature consistency. Background technology:

[0002] In recent years, convolutional neural networks have achieved unprecedented success in computer vision tasks. This success is often based on existing large-scale labeled datasets, but large-scale data annotation is costly and laborious, especially for complex data (such as image retrieval). Compared with methods that rely on a large amount of supervision, self-supervised learning is a way to easily obtain supervisory signals from the data itself and use it to promote its own learning, which does not require a large amount of manual annotation supervision.

[0003] The requirement of person re-identification is to match the person of interest with images of the same identity from non-overlapping cameras. Existing unsupervised person re-identification methods first extract embedded features of unlabeled datasets from the network model, and then apply unsupervised clustering to divide the images into different clusters for training the model. These methods simply use clustering algorithms to generate pseudo-labels as guidance, so they do not fully exploit the semantic information present in the data itself, which often limits the representation ability of the learning model. In addition, current clustering-based methods often focus on capturing local information and low-level features (such as color, contrast, texture, etc.), which have low robustness.

[0004] In order to solve the above problems, the present invention learns feature consistency so that the clustering network can divide images into semantic clusters instead of focusing on low-level features. Similar to unsupervised representation learning, the purpose of feature consistency is to learn feature representations that support visual and temporal consistency reasoning. The present invention focuses on building consistency in static character images so as to perform unsupervised representation learning in the framework of person re-identification. In order to construct consistent information from a single image and convert it into a learning signal for optimizing the network, the present invention feeds the same image twice to the current model and the average model respectively to generate four encoded views of the image (i.e., feature representations). Finally, the consistency between the above four encoded features is maximized by the consistency loss in the latent space to ensure consistency between views. Summary of the invention:

[0005] The purpose of the present invention is to overcome the shortcomings of the existing methods and to solve the problem of low robustness of feature extraction models in unsupervised pedestrian re-identification by using an unsupervised pedestrian re-identification method, particularly an unsupervised pedestrian re-identification method based on feature consistency.

[0006] An unsupervised person re-identification method based on feature consistency, characterized by comprising the following steps:

[0007] Step 1: Design a robust feature extraction model based on random feature masks, and then calculate the time average model based on the feature extraction model;

[0008] Step 2: Feed the pedestrian image to the feature extraction model and the time average model twice respectively to obtain four features, and make the four features close to each other through consistency loss;

[0009] Step 3: Cluster the features extracted by the feature extraction model, then adjust the clustering results based on independence and compactness, and finally fine-tune the model based on the clustering results.

[0010] The implementation of step 1 includes:

[0011] Step 1.1: Add a random mask layer before the ResNet-50-based feature extraction model to preprocess the pedestrian image input to the model: use a random feature mask to cover up part of the information of the pedestrian image;

[0012] Step 1.2: First, pre-train the model on ImageNet and calculate the average model based on the existing models. The parameters of the average model at t iterations are calculated as:

[0013] Where is the parameter of the model at the tth iteration and is the time momentum.

[0014] The implementation of step 2 includes:

[0015] Step 2.1: Input the same image into the model twice to get two image features:

[0016] Where is the image input to the model each time, is the image feature, and F is the parameterized feature extraction model. Since the feature extraction model contains a random mask layer, the same image is masked in different areas in the two inputs, so the obtained feature vectors are often different;

[0017] Step 2.2: Input the same image x into the average model twice to obtain two image features:

[0018] Where is the image feature and is a parameterized feature extraction model. Since the average model also contains a random mask layer, the same image is masked in different areas in the two inputs, so the feature vectors obtained are often different;

[0019] Step 2.3: Use consistency loss to reduce the distance between the above four features. For example, the consistency loss between and is defined as:

[0020] Where is the consistency loss between and , and the present invention uses Euclidean distance as the distance metric. Then the present invention obtains a total of consistency losses.

[0021] The implementation of step 3 includes:

[0022] Step 3.1: Use the feature memory to store the features extracted by the feature extraction model, and then perform bottom-up hierarchical clustering on all features;

[0023] Step 3.2: Determine whether the clustering results meet both independence and compactness, that is, whether the clusters are independent of each other and whether the samples within the clusters are compact enough. If they meet the requirements, keep the results, otherwise split them;

[0024] Step 3.3: Based on the clustering results, fine-tune the model using the hard triplet loss:

[0025] Where is a function used to measure the distance between two features, is a pair of hard positive samples,

[0026] For the hard negative sample pairs, based on KP sampling (first select K identities, then select P samples in each identity), maximize the distance of the most difficult positive sample pair and minimize the distance of the most difficult negative sample pair. m is the minimum distance to be guaranteed between the positive sample pair and the negative sample pair.

[0027] Beneficial effects of the present invention: Taking into account that the current feature extraction model has low robustness and is easily affected by local information and low-level features of the image, the present invention implements random masking on the image, by bringing image features with different masks close to each other and making the features of the model and the average model mutually compare, in an attempt to enable the model to predict the masked area through the unmasked area, thereby making the model have higher robustness to local information and underlying features. Description of the drawings:

[0028] Figure 1 It is a flowchart of an unsupervised person re-identification method based on feature consistency.

[0029] Figure 2 It is an overall framework diagram of an unsupervised person re-identification method based on feature consistency.

[0030] Figure 3 is a schematic diagram of a random mask. Specific implementation method:

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Figure 1It is a specific flow chart of the implementation of the present invention, Figure 2 It is the overall framework diagram of the present invention, such as Figure 1 and Figure 2 As shown, the method includes:

[0033] Step 1: Design a robust feature extraction model based on random feature masks, and then calculate the time average model based on the feature extraction model;

[0034] Step 2: Feed the pedestrian image to the feature extraction model and the time average model twice respectively to obtain four features, and make the four features close to each other through consistency loss;

[0035] Step 3: Cluster the features extracted by the feature extraction model, then adjust the clustering results based on independence and compactness, and finally fine-tune the model based on the clustering results.

[0036] The implementation of step 1 includes:

[0037] Step 1.1: Add a random mask layer before the ResNet-50 based feature extraction model to preprocess the pedestrian image input to the model: Figure 3 As shown, a random feature mask is used to cover up part of the information of the pedestrian image;

[0038] Step 1.2: First, pre-train the model on ImageNet and calculate the average model based on the existing models. The parameters of the average model at t iterations are calculated as:

[0039] Where is the parameter of the model at the tth iteration and is the time momentum.

[0040] The implementation of step 2 includes:

[0041] Step 2.1: Input the same image into the model twice to get two image features:

[0042] Where is the image input to the model each time, is the image feature, and F is the parameterized feature extraction model. Since the feature extraction model contains a random mask layer, the same image is masked in different areas in the two inputs, so the obtained feature vectors are often different;

[0043] Step 2.2: Input the same image into the average model twice to obtain two image features:

[0044] Where is the image feature and is a parameterized feature extraction model. Since the average model also contains a random mask layer, the same image is masked in different areas in the two inputs, so the feature vectors obtained are often different;

[0045] Step 2.3: If Figure 2As shown, the consistency loss is used to reduce the distance between the above four features. For example, the consistency loss between and is defined as:

[0046] Where is the consistency loss between and , and the present invention uses Euclidean distance as the distance metric. Then the present invention obtains a total of consistency losses.

[0047] The implementation of step 3 includes:

[0048] Step 3.1: Use the feature memory to store the features extracted by the feature extraction model, and then perform bottom-up hierarchical clustering on all features;

[0049] Step 3.2: Determine whether the clustering results meet both independence and compactness, that is, whether the clusters are independent of each other and whether the samples within the clusters are compact enough. If they meet the requirements, keep the results, otherwise split them;

[0050] Step 3.3: Based on the clustering results, fine-tune the model using the hard triplet loss:

[0051] Where is a function used to measure the distance between two features. Based on KP sampling (first select K identities, then select P samples from each identity), minimize the distance of the most difficult positive sample pair and maximize the distance of the most difficult negative sample pair.

[0052] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0053] The above description in combination with the accompanying drawings is only a specific implementation method and process of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art should understand that this is only an example, and various changes and substitutions can be made to this implementation method without departing from the essence of the present invention. The scope of the present invention is limited only by the attached claims.

[0054] The embodiments of the present invention described with reference to the accompanying drawings are exemplary and are only used to explain the present invention. They should not be understood as limiting the present invention. The specific scope of the embodiments of the present invention is not limited thereto. On the contrary, all embodiments of the present invention include all changes and modifications that fall within the spirit and connotation of the appended claims.

Claims

1. An unsupervised person re-identification method based on feature consistency, characterized by: The steps include: Step 1: Design a robust feature extraction model based on random feature masks, and then calculate the time average model based on the feature extraction model; Step 2: Feed the pedestrian image to the feature extraction model and the time average model twice respectively to obtain four features, and make the four features close to each other through consistency loss; Step 3: Cluster the features extracted by the feature extraction model, then adjust the clustering results based on independence and compactness, and finally fine-tune the model based on the clustering results; Step 1 specifically includes the following steps: Step 1.1: Add a random mask layer before the ResNet-50-based feature extraction model to preprocess the pedestrian image input to the model: use a random feature mask to cover up part of the information of the pedestrian image; Step 1.2: First, pre-train the model on ImageNet and calculate the average model based on the existing models. t The parameters for the iteration is calculated as: ； where θ t For the model t The parameters for the iteration, is the time momentum; Step 2 specifically includes the following steps: Step 2.1: Input the same image x into the feature extraction model twice to obtain two image features: ； in For each image input to the model, f i is the image feature, F It is a parameterized feature extraction model. Since the feature extraction model contains a random mask layer, the same image is masked in different areas in the two inputs, so the obtained feature vectors are often different. Step 2.2: Input the same image x into the time average model twice to obtain two image features: ； in is the image feature, F m It is a parameterized feature extraction model. Since the average model also contains a random mask layer, the same image is masked in different areas in the two inputs, so the obtained feature vectors are often different. Step 2.3: Use consistency loss to reduce the distance between the above four features. The consistency loss between f1 and f2 is defined as: ； Where L1 is the consistency loss of f1 and f2, and Euclidean distance is used as the distance metric. Then we get A consistency loss; Step 3 specifically includes the following steps: Step 3.1: Use the feature memory to store the features extracted by the feature extraction model, and then perform bottom-up hierarchical clustering on all features; Step 3.2: Determine whether the clustering results meet both independence and compactness, that is, whether the clusters are independent of each other and whether the samples within the clusters are compact. If they meet the requirements, keep the results, otherwise split them; Step 3.3: Based on the clustering results, use the difficult triplet loss Fine-tune the model: ； in It is a function used to measure the distance between two features. Based on KP sampling, K identities are first selected, and then P samples are selected from each identity to maximize the distance of the most difficult positive sample pair and the distance of the most difficult negative sample pair.

Images