Unsupervised object re-identification method based on non-contrast nearest neighbor identity guidance
By employing an unsupervised target re-identification method guided by non-contrast nearest neighbor identity, and utilizing DBSCAN clustering and conjoint network optimization models, the problems of clustering noise and class conflict are solved, thereby improving the accuracy and robustness of the model.
Patent Information
- Application Number
- CN202211371308.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-03
- Publication Date
- 2026-02-10
- Estimated Expiration
- 2042-11-03
AI Technical Summary
In existing unsupervised target re-identification methods, noisy instances and false negative samples caused by clustering methods lead to class conflicts, affecting the model optimization effect.
An unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance is adopted. Features are extracted through conjoined networks, pseudo-labels are generated and centroids are calculated using DBSCAN clustering, and the model is optimized by combining InfoNCE loss and nearest neighbor consistency loss. The conjoined networks are trained using random class samplers and exponential moving average.
It effectively alleviates class conflict issues, improves the accuracy and robustness of the model, reduces the negative impact of noisy instances, and trains high-quality representations.
Smart Images
Figure CN115601579B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pattern recognition technology, and in particular to an unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance. Background Technology
[0002] Object re-identification aims to identify objects of interest from images or videos taken by different cameras using a model. Typically, object re-identification is divided into pedestrian re-identification and vehicle re-identification. Object re-identification is a retrieval task with significant and essential real-world applications, including intelligent transportation, criminal tracking, and finding missing persons. Due to the high cost of data and labeling for object re-identification datasets, unsupervised learning is used. Unsupervised object re-identification does not require labels and can scale to large unlabeled datasets. Pure unsupervised object re-identification utilizes only unlabeled data to train the re-identification model, typically employing clustering to generate pseudo-labels as real identity labels.
[0003] Currently, unsupervised object re-identification methods with self-scheduled contrastive learning generate in-cluster points and outliers for unlabeled data using the DBSCAN clustering algorithm, and generate pseudo-labels based on the in-cluster points. Because contrastive learning is used during training, and a memory is established as the contrast target, the model continuously updates cluster outliers with in-cluster points, thus gradually training a high-performance re-identification model. However, due to the performance limitations of clustering methods, noisy instances inevitably exist within each cluster. Self-scheduled contrastive learning typically requires a large number of negative samples from each class, where false negative samples can cause class conflict issues, leading to incorrect model optimization. Summary of the Invention
[0004] This invention provides an unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance to effectively alleviate class conflict and noise problems.
[0005] To achieve the above objectives, this invention provides an unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance, comprising an epoch process and an iterative process, wherein each epoch process includes multiple iterative processes; the epoch process includes:
[0006] The unlabeled training set X = {x} of the model is tested by setting a query set Q = {p1, p2, ...} and a graph set G = {g1, g2, ...}. i |x1,x2,...,x n The performance of}, where n represents the number of unlabeled training sets X; x ip1 and p2 represent each element in the training set; p1 and p2 represent elements in the query set; g1 and g2 represent elements in the image set.
[0007] Features of the training set X are extracted using an online network of conjoined networks, resulting in the representation matrix Z = {z}. i |z1,z2,…,z n}∈R n×d Where d represents the dimension of the representation, n represents the number of training sets; z i With x in the training set i Correspondingly, this represents the representation of the i-th image in the training set;
[0008] Pseudo-labels Y = {y} are generated using the DBSCAN clustering algorithm. i |y1,y2,…,y n}∈R n Pseudo-labels only use in-cluster points and discard outliers, where y i Let represent the pseudo-label corresponding to the i-th training image, and n represent the number of training images;
[0009] The centroid is calculated from the cluster interior points, resulting in centroid C = {c k |c1,c2,…,c k}∈R K×d The centroids are stored in a dynamically updated memory database, where K represents the number of pseudo-label classes; c k Let d represent the representation vector of the k-th cluster, where d represents the dimension of the representation;
[0010] The pseudo-labels are reassigned to the training set X, overwriting the original labels;
[0011] In the aforementioned unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance, optionally, the representation vector c of the k-th cluster... k for;
[0012]
[0013] Where, N k This represents the k-th cluster subset containing all representation vectors; |·| represents the counting operator;
[0014] In the above-described unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance, the iterative process may optionally include:
[0015] The training set X is sampled using a random class sampler, and the resulting dataset is divided into multiple mini-batches.
[0016] Two independent data augmentations, t and t′, are sampled from different training datasets using a conjoint network. The augmented mini-batch samples are: and
[0017] augmented mini-batch samples and By inputting the online network and the target network of the conjoined network respectively, the normalized representation h is obtained. i ∈R d and h j ∈R d ,Right now
[0018] The InfoNCE loss and nearest neighbor consistency loss are combined as the final loss to optimize the model, where the final loss is:
[0019] L = L q +λL ncc
[0020] Where λ represents an adjustable coefficient;
[0021] The InfoNCE loss is as follows:
[0022]
[0023] Where τ represents the temperature hyperparameter; c + h i The centroid representation corresponding to each cluster category; K represents the sum of the cluster categories;
[0024] The nearest neighbor consistency loss is:
[0025]
[0026] Where, q θ The predictor in an online network; SCNN represents the nearest neighbor sampler for the same class; h i H represents the normalized representation of the online network output, |||2 represents the L2 norm, and H j ={h j |h1,h2,…,h N}∈R N×d h represents the normalized representation matrix output by the target network. j This represents the normalized representation of the target network output, where N represents the batch size and d represents the dimension of the representation.
[0027] In the above-mentioned unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance, the SCNN sampler can optionally be:
[0028]
[0029] in, This indicates that in a small batch with H j h j Representations of the same category, h j Let represent the normalized representation of the target network, p represent the number of identical classes in the mini-batch, || ||2 represents the L2 norm, and arg min represents keeping only the minimum value.
[0030] This invention provides an unsupervised target re-identification method based on non-contrastive nearest neighbor identity guidance. This method provides the model with highly accurate priors. Specifically, it relies on a random class sampler to provide the network with regression targets of nearest neighbors with the same identity within a mini-batch. It encodes increasingly more information during iteration using a conjoined network with exponential moving averages to train high-quality representations, mitigating the negative impact of noisy instances and correcting class conflicts during training.
[0031] The structure of the present invention, as well as its other inventive objects and beneficial effects, will become more apparent from the description of preferred embodiments taken in conjunction with the accompanying drawings. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 A schematic diagram of the epoch process of the unsupervised target re-identification method based on non-comparative nearest neighbor identity guidance provided in an embodiment of the present invention;
[0034] Figure 2 This is a schematic diagram illustrating the iterative process of the unsupervised target re-identification method based on non-comparative nearest neighbor identity guidance provided in an embodiment of the present invention. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be described in more detail below with reference to the accompanying drawings of the preferred embodiments. The described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0036] like Figure 1 As shown, this invention provides an unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance, including an epoch process and an iterative process, wherein each epoch process contains multiple iterative processes; wherein, the epoch process includes:
[0037] Step 110: Test the model's unlabeled training set X = {x} by setting the query set Q = {p1, p2, ...} and the graph set G = {g1, g2, ...}. i |x1,x2,...,x n The performance of}, where n represents the number of unlabeled training sets X; x i p1 and p2 represent each element in the training set; p1 and p2 represent elements in the query set; g1 and g2 represent elements in the image set.
[0038] It should be noted that the goal of unsupervised object re-identification is to train a generalized and robust model. Therefore, a query set Q = {p1, p2, ...} and a graph set G = {g1, g2, ...} are set to test the model on an unlabeled training set X = {x}. i |x1,x2,...,x n The performance of a graph set is considered, where each element in the query set is a probe. Given a probe, the goal is to find an image in the graph set that corresponds to the same identity; that is, the Euclidean distance between the high-dimensional representations of the same identity in the graph set and the probe should be as close as possible.
[0039] Step 120: Extract features from the training set X using the online network of the conjoined network to obtain the representation matrix Z = {z} i |z1,z2,…,z n}∈R n×d Where d represents the dimension of the representation, n represents the number of training sets; z i With x in the training set i Correspondingly, this represents the representation of the i-th image in the training set;
[0040] It should be noted that the two branches of the Siamese network are called the online network ResNet50 and the target network ResNet50-ibn, respectively. The models of the online network and the target network are defined as f. θ and f ξHere, θ and ξ represent the model weights of the online network and the target network, respectively. Since the target network provides the regression objective for the online network, only the weights θ of the online network are backpropagated. The weights ξ of the target network are updated in each iteration by the exponential moving average of the online network θ. The exponential moving average allows the target network to learn information from different iterations over a period, providing a rich regression objective for the online network. The update process is as follows:
[0041] ξ←m t ξ+(1-m t )θ
[0042] Where, m t This represents the target update factor.
[0043] The target network stops gradient averaging and relies on the online network to update network weights via exponential moving averages. We add a predictor to the online network to predict the target network's output and a SINN sampler to sample the target network's nearest neighbor representation to replace the original output.
[0044] Step 130: Generate pseudo-labels using the DBSCAN clustering algorithm to obtain pseudo-labels Y = {y i |y1,y2,…,y n}∈R n Pseudo-labels only use in-cluster points and discard outliers, where y i Let represent the pseudo-label corresponding to the i-th training image, and n represent the number of training images.
[0045] It should be noted that the DBSCAN clustering algorithm generates in-cluster points and outliers. Only in-cluster points are taken and outliers are discarded because outliers may be noise points in the current training period.
[0046] Step 140: Calculate the centroid based on the cluster interior points, obtaining the centroid C = {c k |c1,c2,…,c k}∈R K×d The centroids are stored in a dynamically updated memory database, where K represents the number of pseudo-label classes; c k Let d represent the representation vector of the k-th cluster, where d represents the dimension of the representation;
[0047] The representation vector c of the k-th cluster k for;
[0048]
[0049] Where, N k This represents the k-th cluster subset containing all representation vectors; |·| represents the counting operator;
[0050] It should be noted that, based on the intra-cluster points, the centroids C = {c...} of the same pseudo-label features are... k |c1,c2,…,c k}∈R K ×d Stored in the memory bank.
[0051] Step 150: Reassign the pseudo-labels to the training set X, overwriting the original labels;
[0052] It's important to note that during training, due to limitations in device processing capacity, it's impossible to train all images in the training set simultaneously. Therefore, it's necessary to train the images in batches, i.e., splitting them into groups for training. Each training group of images can be considered an iteration. We set a limit on the number of iterations, such as 200 or 400, meaning training 200 or 400 groups of images constitutes one epoch.
[0053] like Figure 2 As shown, the iterative process includes:
[0054] Step 210: Sample the training set X using a random class sampler to divide the resulting dataset into multiple mini-batches;
[0055] It should be noted that a random class sampler is used to sample the training set X, and the dataset is divided into multiple mini-batches. Specifically, each mini-batch consists of I classes, and each class includes P instances, for a total of I×P=N instances.
[0056] Step 220: Sample two independent data augmentations, t and t′, from different training datasets using a conjoined network. The augmented mini-batch samples are... and
[0057] Step 230: Augmenting the data in mini-batch samples and By inputting the online network and the target network of the conjoined network respectively, the normalized representation h is obtained. i ∈R d and h j ∈R d ,Right now
[0058] It should be noted that H is defined i ={h i |h1,h2,…,h N}∈R N×d Let H be the normalized representation matrix output by the online network; define H j ={h j |h1,h2,…,hN}∈R N×d This is the normalized representation matrix output by the target network.
[0059] Step 240: Combine the InfoNCE loss and the nearest neighbor consistency loss as the final loss to optimize the model, where the final loss is:
[0060] L = L q +λL ncc
[0061] Here, λ represents an adjustable coefficient, which is set to 0.5.
[0062] The InfoNCE loss is as follows:
[0063]
[0064] Where τ represents the temperature hyperparameter, the smaller the value of τ, the more focused it is on separating this sample from the most similar negative sample; c + h i The centroid representation corresponding to each cluster category; K represents the sum of the cluster categories, which changes continuously in each epoch;
[0065] It should be noted that InfoNCE loss is widely used in unsupervised representation learning. Given a representation obtained through an online network, InfoNCE loss can achieve high similarity between this representation and centroids of the same class through dot product, but lower similarity with centroids of different classes. Here, the centroids are stored in a memory bank, which is dynamic because the centroid representation changes during training, and this loss effectively solves the problem of memory bank update consistency. The update formula is:
[0066] c k ←mc k +(1-m)h i
[0067] Where m∈[0,1] represents the momentum update factor;
[0068] The nearest neighbor consistency loss is:
[0069]
[0070] Where, q θ The predictor in an online network; SCNN represents the nearest neighbor sampler for the same class; h i H represents the normalized representation of the online network output, |||2 represents the L2 norm, and H j ={h j |h1,h2,…,h N}∈R N×dh represents the normalized representation matrix output by the target network. j This represents the normalized representation of the target network output, where N represents the batch size and d represents the dimension of the representation.
[0071] It should be noted that noisy instances inevitably exist within each cluster, and false negatives can cause class conflict issues. Therefore, a target network is introduced to improve the latent representation and optimize the compactness within the cluster, with the aim of finding a correct regression instance for the online network. Non-contrastive methods typically require two independent data augmentations to increase the difference between representations, allowing the neural network to learn more knowledge; however, strong data augmentation is not suitable for self-paced unsupervised few-shot learning. Since the random class sampler samples P instances in each class, the nearest neighbors of the same class (excluding itself) of each instance in the mini-batch are used as positive pairs.
[0072] The SCNN sampler is:
[0073]
[0074] in, This indicates that in a small batch with H j h j Representations of the same category, h j Let h represent the normalized representation of the target network, p represent the number of identical classes in the mini-batch, ||2|| represent the L2 norm, and argmin represent keeping only the minimum value; h can be obtained through the SCNN sampler. j The nearest neighbor representation of the same category, and instead of h j It is involved in the loss. Since this loss does not require negative samples, there is no class conflict problem.
[0075] This invention provides an unsupervised target re-identification method based on non-contrastive nearest neighbor identity guidance. This method provides the model with highly accurate priors. Specifically, it relies on a random class sampler to provide the network with regression targets of nearest neighbors with the same identity within a mini-batch. It encodes increasingly more information during iteration using a conjoined network with exponential moving averages to train high-quality representations, mitigating the negative impact of noisy instances and correcting class conflicts during training.
[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance, characterized in that, This includes an epoch process and an iteration process, with each epoch process comprising multiple iteration processes; the epoch process includes: The unlabeled training set X = {x} of the model is used to test the query set Q = {p1, p2, ...} and the graph set G = {g1, g2, ...}. i |x1,x2,...,x n The performance of}, where n represents the number of unlabeled training sets X; X i p1 and p2 represent each element in the training set; p1 and p2 represent elements in the query set; g1 and g2 represent elements in the image set. Features of the training set X are extracted using an online network of conjoined networks, resulting in the representation matrix Z = {z}. i |z1,z2,…,z n }∈R n ×d Where d represents the dimension of the representation; n represents the number of training sets; z i With x in the training set i Correspondingly, this represents the representation of the i-th image in the training set; Pseudo-labels Y = {y} are generated using the DBSCAN clustering algorithm. i |y1,y2,…,y n }∈R n Pseudo-labels only use in-cluster points and discard outliers, where y i Let represent the pseudo-label corresponding to the i-th training image, and n represent the number of training images; The centroid is calculated from the cluster interior points, resulting in centroid C = {c k |c1,c2,…,c k }∈R K×d The centroids are stored in a dynamically updated memory database, where K represents the number of pseudo-label classes; c k Let d represent the representation vector of the k-th cluster, where d represents the dimension of the representation; The pseudo-labels are reassigned to the training set X, overwriting the original labels; The representation vector c of the k-th cluster k for; Where, N k This represents the k-th cluster subset containing all representation vectors; |·| represents the counting operator; The iterative process includes: The training set X is sampled using a random class sampler, and the resulting dataset is divided into multiple mini-batches. Two independent data augmentations, t and t′, are sampled from different training datasets using a conjoint network. The augmented mini-batch samples are: and augmented mini-batch samples and By inputting the online network and the target network of the conjoined network respectively, the normalized representation h is obtained. i ∈R d and h j ∈R d ,Right now The InfoNCE loss and nearest neighbor consistency loss are combined as the final loss to optimize the model, where the final loss is: L=L q +λL ncc Where λ represents an adjustable coefficient; The InfoNCE loss is as follows: Where τ represents the temperature hyperparameter; c + h i The centroid representation corresponding to each cluster category; K represents the sum of the cluster categories; The nearest neighbor consistency loss is: Where, q θ The predictor in an online network; SCNN represents the nearest neighbor sampler for the same class; h i H represents the normalized representation of the online network output, |||2 represents the L2 norm, and H j ={h j |h1,h2,…,h N }∈R N×d h represents the normalized representation matrix output by the target network. j This represents the normalized representation of the target network output, where N represents the batch size and d represents the dimension of the representation.
2. The unsupervised target re-identification method based on non-contrast nearest neighbor identity guidance according to claim 1, characterized in that, The SCNN sampler is: in, This indicates that in a small batch with H j h j Representations of the same category, h j denoted by , p represents the number of identical classes in the mini-batch, ||2 represents the L2 norm, and argmin represents keeping only the minimum value.