An unsupervised pedestrian re-identification method based on contrastive learning
By designing a two-stage framework and making reasonable use of cluster outlier information, the problem of poor training effect in unsupervised person re-identification is solved, and more efficient feature clustering and recognition performance are achieved.
Patent Information
- Application Number
- CN202310381255.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-11
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2043-04-11
AI Technical Summary
Existing unsupervised pedestrian re-identification methods fail to make reasonable use of the information carried by outliers in clusters, resulting in poor training performance. In particular, the use of difficult samples from small batches to update the memory dictionary in the early stages of training is unreasonable and hinders training progress.
A two-stage framework is adopted. First, clustering is performed by setting a large EPS parameter. The feature vectors obtained by the DBSCAN clustering method are used to form the feature vectors. A two-stage framework is designed. In the first stage, the mean of the mini-batch samples is used to update the in-memory dictionary and outlier loss is added. In the second stage, the hard samples in the mini-batch are used to update the in-memory dictionary. The model is trained iteratively until convergence.
It significantly improved the model's performance, solved the problem of unreasonable handling of outliers in clustering, improved the model's recognition accuracy and training efficiency, and achieved higher quality feature clustering and recognition results.
Smart Images
Figure CN116524534B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer application, and in particular to an unsupervised pedestrian re-identification method based on contrastive learning. BACKGROUND
[0002] In daily traffic scenes, monitoring cameras can be seen everywhere. Specific analysis of a large amount of video data taken by monitoring cameras can be a very important auxiliary means for criminal investigation and intelligent monitoring. However, if the specified target is to be found by human eyes from the massive monitoring data, it will inevitably cost a lot of manpower and material resources. With the tide of artificial intelligence sweeping the world, people inevitably have the idea of using advanced computer vision technology to intelligently analyze and identify monitoring images and videos, and pedestrian re-identification is a very important part of it. Given a pedestrian picture to be searched, an effective pedestrian re-identification algorithm model needs to find a picture with the same pedestrian in the picture library with a high accuracy. According to the search result, other time and place where the pedestrian appears can be found, which facilitates subsequent tracking and positioning. An efficient pedestrian re-identification algorithm greatly reduces the burden of staff and improves the efficiency of criminal investigation and security work. As can be seen, the pedestrian re-identification task has important research value and practical application prospect.
[0003] Current supervised pedestrian re-identification has made extraordinary achievements on many public data sets. However, public data sets in the academic field also require manual labeling, but the amount of data in the actual traffic scene is obviously much larger than the current largest data set, and manual labeling one by one is obviously unrealistic, so the unsupervised pedestrian re-identification task has entered the field of view. The unsupervised pedestrian re-identification task directly trains the model based on the unlabeled data set, which has stronger scalability in the real world. Existing unsupervised pedestrian re-identification methods can be roughly divided into two categories, namely pure unsupervised pedestrian re-identification (USL) and unsupervised domain adaptive pedestrian re-identification (UDA). The USL method only uses unlabeled data, while the UDA method needs an additional labeled source data set to pre-train the model, and then further fine-tune the model on the completely unlabeled target data set. Generally speaking, the performance of the UDA method is superior to that of the USL method due to the use of an external source domain. At this time, another problem arises, the performance of the UDA method is severely restricted by the similarity between the source domain and the target domain, and it may be difficult to find a similar source domain in the real scene, so the use of the UDA method is still limited. SUMMARY
[0004] Embodiments of the present application provide an unsupervised pedestrian re-identification method based on contrastive learning to solve the problems in the prior art.
[0005] To achieve the above object, the present application adopts the following technical solutions.
[0006] An unsupervised pedestrian re-identification method based on contrastive learning, comprising:
[0007] S1 Based on the unlabeled pedestrian re-identification dataset X, using a backbone encoder Encode the dataset X Obtain a feature vector set D;
[0008] S2 The feature vectors in the feature vector set D are clustered into N classes in a high-dimensional space by the DBSCAN clustering method, each feature vector is assigned a pseudo label, each cluster is represented by a feature vector and maintained in the memory dictionary M, and a first cluster is obtained.
[0009] S3 Through the first cluster, and using the cluster-level contrastive loss to iteratively train the network model; in each iteration round, a small batch of feature vectors are randomly extracted from the clusters used in the last round , and add outlier loss auxiliary training in the previous multiple training periods, and update the memory dictionary;
[0010] S4 Repeat steps S1 to S3 iteratively until the network model converges.
[0011] The converged network model is used for pedestrian re-identification.
[0012] Preferably, the network model deletes all sub-model layers after the 4th layer, and has global pooling, batch normalization layer and L2 regularization layer.
[0013] Preferably, in step S2:
[0014] Before the start of each training period, calculate the Jaccard distance between the feature vectors in the feature vector set D;
[0015] In the DBSCAN clustering method, the minimum number of points is set to 4.
[0016] Preferably, step S3 comprises:
[0017] The previous plurality of training cycles are the first 20 training cycles, and in the first 20 training cycles, the EPS parameter is set to 0.65 for the Market1501 data and is set to 0.7 for the DukeMTMC-reID data set; after the first 20 training cycles, the EPS parameter is set to 0.7 for the DukeMTMC-reID data set, and the EPS parameter is set to 0.45 for the Market1501 data set and is set to 0.55 for the DukeMTMC-reID data set;
[0018] Before each round of training, the memory dictionary is initialized by formula
[0019] (1)
[0020] The memory dictionary is initialized; in the formula, represents the number of samples contained in the i-th class at the current stage;
[0021] In the first 20 training cycles, the memory dictionary is updated by formula
[0022] (2) and
[0023] (3)
[0024] The difficult sample of the memory dictionary is updated; in the formula, represents the average value of the i-th class in the extracted small batch of samples, represents the K sample features extracted for the i-th class; the cluster center is updated in a momentum manner, is the update weight in the momentum method;
[0025] After the first 20 training cycles, the memory dictionary is updated by formula
[0026] and
[0027] (5)
[0028] The difficult sample of the memory dictionary is updated; in the formula, represents the feature vector with the smallest cosine similarity to the cluster center of the i-th class in the i-th class of the small batch of samples;
[0029] The memory dictionary is updated by formula
[0030] (6)
[0031] The cluster level loss function of the first 20 training cycles is calculated; in the formula, q represents any query vector, P represents the number of categories extracted in the current iteration batch, represents the cluster center of the extracted P categories, represents the cluster center of the category to which q belongs;
[0032] In the first 20 training cycles, by formula
[0033] (7) and
[0034] (8)
[0035] Processing outlier loss; in the formula, is the proportion of clusters that are considered unreliable with respect to the current outlier, represents the number of outliers extracted in this round of iteration batch, is the balance factor between the two loss functions.
[0036] As can be seen from the technical solutions provided by the above embodiments of the present application, the present application provides an unsupervised pedestrian re-identification method based on contrast learning, which comprises: using a ResNet-50 network pre-trained on ImageNet to initialize a backbone encoder ; using the initialized encoder to extract feature vectors from the original data set; using a DBSCAN clustering algorithm to cluster the feature vectors, and initializing a memory dictionary according to the vectors in each clustering cluster; iteratively extracting small batches of feature vectors combined with the current memory dictionary to train the encoder using cluster contrast loss and outlier loss ; updating the feature vectors corresponding to the corresponding clusters in the memory dictionary; iteratively performing DBSCAN clustering to train the model encoder The steps between the iterations are performed until the model converges. The present application is based on a contrast learning method, which improves the encoding ability of the network model through contrast learning iteration, so that the input pictures that are actually similar have a close distance in the high-dimensional space, and vice versa, the pictures that are different have a larger distance in the high-dimensional space. The encoder with such ability can quickly converge and achieve better performance when performing downstream tasks, because the network has learned how to distinguish different data in the process of continuous contrast learning; the out-of-cluster values generated by DBSCAN clustering are pushed away from the clusters that they do not belong to in each iteration process through a well-designed out-of-cluster loss function, so that the out-of-cluster values of the last clustering can be reasonably classified in the next iteration process; by reasonably designing the updating method of the memory dictionary, the mean of the small batch of samples is used in the initial iteration round, and the hard sample of the small batch of samples is used to update the memory dictionary in the subsequent iteration round, so that the best feature vector representing the current clustering is always maintained in the memory dictionary during the whole training process. Through the above improvements, the present application has a large performance improvement compared with the current best unsupervised pedestrian re-identification method, and reasonably solves the unreasonable problem of processing clustering out-of-cluster values in the previous unsupervised pedestrian re-identification method.
[0037] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following description and drawings. BRIEF DESCRIPTION OF DRAWINGS
[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.
[0039] Figure 1 A processing flowchart of an unsupervised pedestrian re-identification method based on contrast learning provided by the present application is shown in the figure.
[0040] Figure 2 A detailed training workflow diagram of the model of the unsupervised pedestrian re-identification method based on contrast learning provided by the present application is shown in the figure. DETAILED DESCRIPTION
[0041] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be interpreted as a limitation of the present application.
[0042] It is to be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It is further understood that the terms "comprise" (or comprise), "comprises" (or comprises) and "comprising" (or comprising) when used in this specification, specify the presence of stated features, integers, steps, operations, elements, or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof. It is further understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can be present. In addition, the use of "connection" or "coupling" herein also includes wireless connection or coupling. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.
[0043] It is to be understood that the terms so used are intended to encompass like and related terms. Furthermore, this description uses terminology commonly employed for the purposes of description but not necessarily limited to these terms. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
[0044] For the purpose of clarity, the following further describes the embodiments of the present application with reference to the accompanying drawings, in which several embodiments are shown by way of example, and the embodiments are not intended to limit the embodiments of the present application.
[0045] The present application provides a new method for reasonably utilizing outlier samples, which is used to solve the following problems existing in the prior art:
[0046] The current most advanced purely unsupervised pedestrian re-identification method first uses the DBSCAN clustering algorithm to obtain pseudo-labels, and then uses a novel self-paced contrastive learning framework to continuously optimize the label quality. Subsequently, many methods have been further improved based on this method, such as completely discarding the clustering outliers, using cluster contrastive loss to train the model, combining the memory dictionary updating method based on difficult samples, and achieving more excellent performance. In the training process, the outliers generated by clustering are also training samples, and also carry valuable information. However, previous methods have not reasonably utilized the clustering outliers. They either regard all outliers as negative samples, or directly discard all outliers. And the applicant found that it is unreasonable to use difficult samples in small batches throughout the training to update the memory dictionary, especially in the early stage of training, which may even hinder the training.
[0047] The prior art method is unreasonable for clustering outliers. They either directly discard these outliers, which is equivalent to directly discarding a part of the training samples, which obviously also carries information, or consider all outliers as negative samples in the contrast learning process. These outliers are pushed away from each cluster in high-dimensional space, but these outlier samples may belong to a previous cluster in the next iteration process. This approach obviously hinders subsequent training.
[0048] The current most advanced method uses difficult samples in the iteratively extracted small batch samples to update the memory dictionary. They believe that the difficult samples in the small batch samples extracted from each cluster are the middle samples of the entire cluster. However, at the beginning of training, each cluster is very small, and the extracted small batch samples are almost equivalent to all samples of the entire cluster. Therefore, it is unreasonable to use difficult samples to update the memory dictionary at the beginning of training.
[0049] In view of the above problems existing in the prior art, the application innovatively proposes a two-stage framework of first clustering and merging, and then segmenting and more finely merging.
[0050] The first stage is called the aggregation stage. In order to achieve the best training effect, a larger distance threshold parameter (EPS parameter) is set for DBSCAN clustering in this stage. At this time, the obtained clusters are relatively large in high-dimensional space, and each cluster may contain multiple similar but different labeled samples. At this time, the training accuracy is obviously not high. In this stage, the average value of the iteratively extracted small batch samples is used to update the memory dictionary, and an outlier loss is added to the original clustering level contrast loss, which improves the possibility that the outliers are correctly processed by the network model in the subsequent stage, reduces the number of outliers as much as possible, and improves the quality of the clusters, and prepares for the training in the subsequent stage.
[0051] The second stage is called the segmentation and aggregation stage. The training in the first stage brings larger clusters, and each cluster may contain multiple different but similar samples. At this time, a smaller distance threshold parameter is set for DBSCAN clustering to finely segment the current clusters. The segmented high-dimensional sample space will present the following situation: the segmented clusters are still of considerable size, and the outliers are no longer chaotic because they are also generated by the segmentation of the clusters in the previous stage and still satisfy the distance threshold in the previous stage. Since the size of the segmented clusters is still considerable, the difficult samples in the small batch samples are the middle samples of the clusters, which are the appropriate feature representation of the clusters, which can ensure that the memory dictionary is effectively updated.
[0052] Reference Figure 1The application provides a contrast learning-based unsupervised pedestrian re-identification method, comprising the following steps:
[0053] S1. Based on a label-free pedestrian re-identification dataset X, a backbone encoder is used The dataset X is encoded A feature vector set D is obtained.
[0054] S2. The feature vectors in the feature vector set D are clustered into N classes in a high-dimensional space by a DBSCAN clustering method, each feature vector is assigned a pseudo label, each cluster is represented by a feature vector and maintained in a memory dictionary M, and a first cluster (i.e., a large cluster) is obtained.
[0055] S3. The first cluster is used to iteratively train a network model by using a cluster-level contrast loss, and in each iteration round, a small batch of feature vectors is randomly extracted from the cluster used in the last round The large cluster is segmented, and an outlier loss is added for auxiliary training in the previous multiple training periods, and the memory dictionary is updated.
[0056] S4. Steps S1 to S3 are iteratively repeated until the network model converges.
[0057] The trained network model until convergence is used for pedestrian re-identification.
[0058] The method provided by the application is suitable for the characteristics of large clusters based on the memory dictionary updating method of difficult samples, by setting the eps parameter value in DBSCAN clustering to a larger value, and using a mean-based memory dictionary updating method combined with an outlier loss, the number of outliers in the memory space is greatly reduced, and large clusters are obtained. Then, set the eps parameter value to a smaller value, indicating that the model requires a more fine-grained density requirement for the current cluster. At this time, the large cluster in the last round is divided into several small clusters and some outliers. Note that due to the large cluster in the training process in the last round, the clusters after being divided still have considerable size, and the efficiency of the memory dictionary updating method based on difficult samples is higher. Then, through training, these new outliers and clusters are aggregated again until the model converges, and higher-quality clusters are obtained.
[0059] In the preferred embodiments provided by the application, the specific execution process of each step is as follows.
[0060] Step S1: Given a label-free pedestrian re-identification dataset X, a backbone encoder is used The dataset X is encoded A feature vector set D is obtained.
[0061] Generally, in order to improve the robustness of the model, data augmentation will be performed on the original data set. The data augmentation method adopted by the present application for the input picture includes: random image horizontal flip, 10 pixel edge padding, random cropping, and random erasing. In order to improve the initial performance of the model, some changes are also made to the basic network model. All sub-model layers after the 4th layer are deleted, global pooling (GAP) is added, and then batch normalization and L2 regularization layers are added, finally a 2048-dimensional feature vector is obtained. The data set used by the present application is the public Market1501 data set and the DukeMTMC-reID data set, and in order to facilitate training, the size of the input image is adjusted to 256x128.
[0062] Step S2: cluster the feature vectors in high-dimensional space into N classes by DBSCAN clustering, give each feature vector a pseudo label, and maintain each cluster in a memory dictionary M.
[0063] Before the start of each training period, the present application needs to first calculate the Jaccard distance between the feature vectors in the set D, and then use the DBSCAN clustering algorithm to cluster the feature vectors. The DBSCAN clustering algorithm has two important parameters, the minimum number of points (MinPts) of the core object within the domain radius and the domain radius (EPS), wherein MinPts is set to 4, and in the first 20 training periods, the EPS parameter is set to 0.65 for the Market1501 data set and 0.7 for the DukeMTMC-reID data set; after 20 training periods, the EPS parameter is set to 0.45 for the Market1501 data set and 0.55 for the DukeMTMC-reID data set. The size of the memory dictionary M is N x 2048, which stores a cluster center for each cluster. The present application initializes the memory dictionary as the average value of all feature vectors in the corresponding cluster.
[0064] Step S3: set B iteration rounds, randomly extract of the small batch of feature vectors in each iteration round, train the network using cluster-level contrastive loss, add outlier loss auxiliary training in the first 20 training periods, and update the memory dictionary at the same time.
[0065] Through comparison by experiments, the value of iteration round B is set to 200 in the application, P = 16 categories are extracted in each iteration process, K = 16 sample vectors are extracted for each category for training, and 256 outliers are extracted. As mentioned earlier, in the early training round, the samples in each cluster are actually very few, and when the number of samples in a cluster is less than 16, all the samples in the entire cluster are directly selected. For example, for the experiment of the Market1501 dataset with 12936 training samples, after the first training round, about 400 clusters and 4000 outliers are obtained, and obviously the number of samples in many clusters is less than 16. The application considers that it is unreasonable to update the memory dictionary with difficult samples in the small batch at the initial state, because the difficult samples in the small batch at this time are the edge samples in the cluster. Therefore, in the early training stage, the mean value of the small batch is selected to update the memory dictionary, and at this time, the outliers in the training process do not participate in the cluster level contrast learning, and still maintain the original state in the high-dimensional space, which is chaotic, so the outlier loss needs to be added to normalize the outliers for better classification; when the training result tends to be stable (about 20-25 training rounds, experiments prove that any value between 20-25 has no effect on the experimental result, so the application takes 20 training rounds), at this time, the number of outliers is less than 100, and the EPS parameter in the DBSCAN clustering algorithm is adjusted to segment the clusters, the previous training stage of the application brings two advantages to this training stage: first, the segmented clusters still maintain an observable size, from the original 400 or so clusters to 700 clusters, so it is suitable to update the memory dictionary with difficult samples in the small batch at this time, because the difficult samples in the small batch at this time are equivalent to non-edge samples in the high-dimensional space; second, the number of outliers at this time is greatly reduced from the beginning of the training, only about 1000. Most of these outliers are segmented because they do not meet the new smaller EPS distance threshold, which exactly shows that these segmented outliers are difficult to classify and need to be correctly classified through further fine training; and at this time, these outliers are no longer chaotic, because they are segmented from the same cluster in the previous stage, so the application no longer uses the outlier loss to normalize the outliers in this stage.
[0066] Based on the above bottom principle analysis, the specific implementation of the application will be further given.
[0067] First, introduce the updating method of the memory dictionary. The memory dictionary is reinitialized to the average value of all feature vectors in each cluster in each training round, which is equivalent to the cluster center of each cluster, which is denoted as {C1, C2... CN} is represented, and the memory dictionary is initialized as
[0068] (1)
[0069] wherein represents the number of samples contained in the i-th class in the current stage, represents a sample feature vector of the i-th cluster. In the first stage, i.e., the first 20 training rounds, the memory dictionary is updated as follows:
[0070] (2) and
[0071] (3)
[0072] wherein represents K sample features extracted for the i-th class; the cluster center is updated in a momentum manner, is the update weight in the momentum method.
[0073] In the second training stage, the memory dictionary is updated by using the difficult samples in the cluster as follows:
[0074] and
[0075] (5)
[0076] wherein represents the average value of the i-th class in the extracted mini-batch sample, represents a feature vector with the smallest cosine similarity to the cluster center of the i-th class in the i-th class of the mini-batch sample.
[0077] Secondly, the loss function of the model designed in the application is introduced. In the first 20 training rounds, outliers need to be processed, so the outlier loss function and the cluster level loss function are used at the same time; in the second stage, outliers do not need to be processed, so only the outlier loss function can be used. Given an instance feature with query , the InfoNCE loss function is used as the cluster level loss function
[0078] (6)
[0079] wherein q represents any to-be-queried vector, P is the number of classes in the current extracted mini-batch sample, represents the cluster center of the cluster to which the to-be-queried vector q belongs, is a temperature hyperparameter, represents the cluster centers of the extracted P categories. Another significant improvement of the present invention is to add the processing of outliers to the previous invention. In order to better handle outliers, the outlier loss is designed as follows:
[0080] (7) and
[0081] (8)
[0082] wherein, is the proportion of clusters considered unreliable with respect to the current outlier, represents the number of outliers extracted in this round of iteration batch, is a balance factor between the two loss functions.
[0083] The last step is to take the Market1501 dataset as an example to introduce the understanding of the above three hyperparameters, and supplement some experimental details not mentioned above:
[0084] τ is the temperature parameter, the larger the value, the closer to 1, indicating that all samples are treated equally; when the value is smaller, the closer to 0, indicating more attention to difficult samples. The present invention sets τ = 0.1 to obtain the best experimental results.
[0085] is the proportion of clusters considered unreliable with respect to the current outlier, when its value is close to 1, it is equivalent to directly considering that this outlier belongs to the cluster with the highest similarity, which is obviously unreasonable, because with training each cluster is constantly changing, so the most similar cluster is not necessarily the correct cluster, and the similarity between the cluster and the outlier is determined by the cosine similarity between the cluster center in the memory dictionary and the outlier vector, but the memory dictionary is also constantly updated, so the similarity is also constantly changing. Therefore, it is impossible to determine which cluster the current outlier belongs to, but by setting a proportion threshold, the clusters that the outlier certainly does not belong to can be found, and by pushing their distance away in high-dimensional space, it can help the outlier to be better classified in the subsequent training process. In the present invention, it is set to 0.9.
[0086] represents the weight of the outlier loss in the total loss, during the entire training process, the outlier loss is obviously in a secondary position, therefore it should not occupy too high a weight, the present invention sets it to 0.1 by default.
[0087] During training, the present invention uses ADAM as the optimizer, the weight decay coefficient is set to 0.0005; the initial learning rate is set to 0.00035; the total number of training cycles is 50.
[0088] Step S4: continuously iterate steps S1-S3 until the model converges, which is generally after 50 training rounds.
[0089] More specifically, the training and optimization workflow of the unsupervised pedestrian re-identification based on contrastive learning provided by the embodiment of the application is as shown in the figure Figure 2 Some preliminary work is omitted here, and only the core part of the algorithm is retained:
[0090] (1) First, determine whether the current training round is greater than 20, that is, determine which stage of training the current is in;
[0091] (2) If the current is in the first stage of training, train the network using the cluster level contrastive loss combined with the outlier loss, and update the memory dictionary using the mean of the small batch of samples, and iterate the training for B rounds;
[0092] (3) If the current is in the second stage of training, train the network using only the cluster level contrastive loss, and update the memory dictionary using the hard sample in the small batch of samples, and iterate the training for B rounds;
[0093] (4) Check whether the current model converges, if not, iterate the training, generally when the training round is 50, the model has already converged.
[0094] In summary, the present application provides an unsupervised pedestrian re-identification method based on contrastive learning, a two-stage framework is designed, in the first stage, the mean of the small batch of samples is used to update the memory dictionary, and the outlier loss is added on the basis of the original cluster level contrastive loss, which improves the possibility of correct processing of outliers by the network model in the subsequent stage; in the second stage, the hard sample is used to update the memory dictionary when the samples in each cluster are sufficient. The two-stage training framework can solve the above two problems existing in the previous method at the same time, and the performance of the model is greatly improved. The method provided by the present application has the following advantages:
[0095] For the first time, the problem of how to reasonably use the cluster outliers in the pure unsupervised pedestrian re-identification task is considered. Outliers also carry a lot of information as training samples, and should not be simply discarded or treated as negative samples of all clusters. The designed outlier loss fully and reasonably utilizes the information carried by the outliers, and greatly improves the performance of the algorithm;
[0096] An innovative two-stage framework of first aggregation, then segmentation and re-aggregation is designed. Through experiments, the applicant finds that it is not the best solution to update the memory dictionary with one strategy throughout the training process, because the cluster size is constantly changing during the iterative training process, and the difficult samples in the small batch of samples have considerable size in the cluster generated by clustering, and are very suitable for updating the memory dictionary, but when the cluster is very small at the beginning of training, it is not applicable. The two-stage framework of the application provides a better updating strategy for the memory dictionary, and the performance of the algorithm is greatly improved.
[0097] Those skilled in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or flows in the drawings are not necessarily required to implement the application.
[0098] From the above description of the embodiments, those skilled in the art can clearly understand that the application can be implemented by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.
[0099] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts can be referred to the part of the method embodiments. The above-described device and system embodiments are only schematic, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to the actual needs. Those skilled in the art can understand and implement without creative labor.
[0100] The above is only the preferred specific implementation of the application, but the protection scope of the application is not limited to this. Any changes or replacements within the technical range disclosed by the application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.
Claims
1. A contrastive learning based unsupervised pedestrian re-identification method, characterized in that, Comprise: S1 encoding the dataset X using a backbone encoder encoding the dataset X obtaining a set of feature vectors D; S2, the feature vectors in the feature vector set D are clustered into N classes in high-dimensional space by the DBSCAN clustering method, each feature vector is assigned a pseudo label, each cluster is represented by a feature vector and maintained in the memory dictionary M, and a first cluster is obtained; S3 passes through the first cluster, and iteratively trains the network model by using the cluster-level contrast loss; in each iteration round, a small batch of feature vectors are randomly extracted from the cluster used in the last round and an outlier loss auxiliary training is added in the previous multiple training periods, while updating the memory dictionary; specifically comprising: Step S3 comprises: The previous multiple training periods are the previous 20 training periods, in the previous 20 training periods, the EPS parameter is set to 0.65 for the Market1501 data, and the EPS parameter is set to 0.7 for the DukeMTMC-reID data set; After the previous 20 training periods, the EPS parameter is set to 0.7 for the DukeMTMC-reID data set, and the EPS parameter is set to 0.45 for the Market1501 data set, and the EPS parameter is set to 0.55 for the DukeMTMC-reID data set; Before each round of training, the formula （1） Initialize the memory dictionary; where, represents the number of samples contained in the i-th class in the current stage; represents a sample feature vector of the i-th cluster; represents the cluster center of the i-th class; In the previous 20 training periods, the formula (2) and （3） updating the memory dictionary with hard samples; wherein, denotes the average value of the i-th class in the extracted mini-batch, denotes the K sample features extracted for the i-th class; the cluster center is updated in a momentum manner, is the updated weight value in the momentum method; After the previous 20 training periods, the formula and （5） Updating the memory dictionary with difficult samples; wherein, represents the cosine similarity of the feature vector with the cluster center of the i-th class in the i-th class of small batch samples of the i-th class The formula （6） calculating the loss function of the cluster level of the first 20 training periods; wherein q represents any to-be-queried vector, P represents the number of categories extracted in the current iteration batch, denotes the cluster center of the extracted P categories, denotes the cluster center of the category to which q belongs; In the previous 20 training periods, the formula (7) and （8） handling outliers loss; where, is the proportion of clusters that are considered unreliable with respect to the current outlier, is the number of outliers drawn in the current iteration batch, is a balancing factor between the two loss functions; S4, steps S1 to S3 are iteratively repeated until the network model converges; The converged network model is used for pedestrian re-identification.
2. The method of claim 1, wherein, The network model deletes all sub-model layers after the fourth layer and has global pooling, batch normalization layer and L2 regularization layer.
3. The method of claim 1, wherein, In step S2: Before the start of each training period, the Jaccard distance between the feature vectors in the feature vector set D is calculated; In the DBSCAN clustering method, the minimum number of points is set to 4.