Unsupervised pedestrian re-identification method, device, equipment and readable storage medium
Through deep learning models and clustering algorithms, pedestrian re-identification methods are optimized, and hybrid contrast learning strategies are used to focus on difficult sample features, which solves the problem of difficult pedestrian characteristics and large gaps in the class, and improves the accuracy of pedestrian re-identification.
Patent Information
- Application Number
- CN202210533483.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-13
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-05-13
AI Technical Summary
The accuracy of pedestrian re-identification results in the prior art is low, mainly because images that are difficult to identify pedestrian characteristics have not been fully tapped and utilized, and the gap in class caused by the difference in camera angle and light intensity is large, which affects the recognition effect. .
The features of pedestrian images are obtained through deep learning models, the clustering algorithm allocates false labels, constructs category feature memory and feature memory, randomly preprocesses image samples, optimizes the deep learning model to mix and contrast learning strategies, pays attention to difficult sample features, and reduces the in-class Gap, expand the gap between categories.
The accuracy of pedestrian re-identification results is improved. By fully learning pedestrian characteristics, the model's attention to difficult samples is enhanced, the gap within the class is reduced, and the accuracy of matching results is improved.
Smart Images

Figure CN114821661B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to an unsupervised pedestrian re-identification method, device, equipment and readable storage medium. Background Art
[0002] Pedestrian re-identification is the process of searching for the same pedestrian in different surveillance cameras. However, due to the diversity of camera shooting angles, lighting intensity, and human body postures, the images of the same pedestrian taken by the cameras have large differences, making pedestrian re-identification through pedestrian images more difficult.
[0003] At present, in the process of pedestrian re-identification through pedestrian images, the importance of each image is considered to be equal. The difficulty of pedestrian feature recognition on the pedestrian image is not distinguished, and the same attention is paid to all pedestrian images, so that pedestrian images with easy-to-recognize pedestrian features are fully exploited, while pedestrian images with difficult-to-recognize pedestrian features are not fully exploited, resulting in a low accuracy rate of recognition results for pedestrian re-identification through pedestrian images. Summary of the invention
[0004] The main purpose of the present invention is to provide an unsupervised pedestrian re-identification method, device, equipment and readable storage medium, aiming to solve the technical problem of low recognition accuracy of pedestrian re-identification results in the prior art.
[0005] In a first aspect, the present invention provides an unsupervised pedestrian re-identification method, the unsupervised pedestrian re-identification method comprising:
[0006] Obtain the first pedestrian feature corresponding to the pedestrian image sample through the deep learning model;
[0007] Clustering the pedestrian image samples by using a clustering algorithm and according to the first pedestrian features corresponding to the pedestrian image samples, and assigning a false label to each pedestrian image sample of the same category based on the clustering results, wherein the false labels of pedestrian image samples of the same category are the same;
[0008] A category feature memory is constructed based on the clustering result and the first pedestrian feature, and a feature memory is constructed based on the total number of pedestrian image samples and the first pedestrian feature;
[0009] Obtain K types of pedestrian image samples according to the false labels, and obtain P pedestrian image samples from each type of pedestrian image samples, where K and P are positive integers;
[0010] Randomly preprocess K*P pedestrian image samples;
[0011] Input the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model;
[0012] Optimizing the deep learning model based on the category feature memory, the feature memory, and the second pedestrian feature;
[0013] The optimized deep learning model is used as the deep learning model, and the step of obtaining the first pedestrian feature corresponding to the pedestrian image sample through the deep learning model is returned.
[0014] When the number of cycles reaches the preset number, the pedestrian features corresponding to the target pedestrian image are obtained using the final deep learning model, and the pedestrian features corresponding to the target pedestrian image are matched with the pedestrian features in the database to obtain the pedestrian recognition result.
[0015] Optionally, the steps of constructing a category feature memory based on the clustering result and the first pedestrian feature, and constructing a feature memory based on the total number of pedestrian image samples and the first pedestrian feature include:
[0016] Determine the size of the category feature memory to be constructed according to the product of the total number of categories obtained by clustering and the dimension of the first pedestrian feature, and construct the category feature memory based on the size of the category feature memory to be constructed;
[0017] The size of the feature memory to be constructed is determined according to the product of the total number of pedestrian image samples and the dimension of the first pedestrian feature, and the feature memory is constructed based on the size of the feature memory to be constructed.
[0018] Optionally, the step of optimizing the deep learning model based on the category feature memory, the feature memory and the second pedestrian feature includes:
[0019] Based on the category feature memory and the second pedestrian feature, the category level contrast learning is designed, and the dynamic contrast loss function corresponding to the second pedestrian feature is constructed. The dynamic contrast loss function corresponding to the second pedestrian feature is expressed as:
[0020]
[0021] Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of categories obtained by clustering, τ represents the hyperparameter, k t represents the category feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, k a represents the ath category feature stored in the category feature memory;
[0022] Based on the feature memory and the second pedestrian feature, image-level contrastive learning is designed to construct a sample-constrained loss function corresponding to the second pedestrian feature. The sample-constrained loss function corresponding to the second pedestrian feature is expressed as:
[0023]
[0024] Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Indicates the hard positive sample features corresponding to the second pedestrian features;
[0025] The deep learning model is optimized based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, where the optimized deep learning model is expressed as:
[0026] L HCM =L cla +β*L img
[0027] Here, β represents a hyperparameter.
[0028] Optionally, after the step of inputting the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model, the method further includes:
[0029] The category feature memory and the feature memory are updated according to the second pedestrian features corresponding to the K*P pedestrian image samples.
[0030] Optionally, the preprocessing includes horizontal flipping, cropping, erasing and normalization.
[0031] In a second aspect, the present invention further provides an unsupervised pedestrian re-identification device, the unsupervised pedestrian re-identification device comprising:
[0032] A first acquisition module, used to obtain a first pedestrian feature corresponding to a pedestrian image sample through a deep learning model;
[0033] an assignment module, configured to cluster the pedestrian image samples by using a clustering algorithm and according to the first pedestrian features corresponding to the pedestrian image samples, and assign a false label to each pedestrian image sample of the same category based on the clustering results, wherein the false labels of the pedestrian image samples of the same category are the same;
[0034] A construction module is used to construct a category feature memory based on the clustering result and the first pedestrian feature, and to construct a feature memory based on the total number of pedestrian image samples and the first pedestrian feature;
[0035] A second acquisition module is used to acquire K types of pedestrian image samples according to the false labels, and acquire P pedestrian image samples from each type of pedestrian image samples, where K and P are positive integers;
[0036] A processing module, used for randomly preprocessing K*P pedestrian image samples;
[0037] An input module, used to input the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model;
[0038] An optimization module, for optimizing a deep learning model based on a category feature memory, a feature memory, and a second pedestrian feature;
[0039] The loop module is used to use the optimized deep learning model as the deep learning model and notify the first acquisition module to perform its corresponding action.
[0040] The matching module is used to obtain the pedestrian features corresponding to the target pedestrian image with the final deep learning model when the number of cycles reaches a preset number, and match the pedestrian features corresponding to the target pedestrian image with the pedestrian features in the database to obtain the pedestrian recognition result.
[0041] Optionally, the building block is used to
[0042] Determine the size of the category feature memory to be constructed according to the product of the total number of categories obtained by clustering and the dimension of the first pedestrian feature, and construct the category feature memory based on the size of the category feature memory to be constructed;
[0043] The size of the feature memory to be constructed is determined according to the product of the total number of pedestrian image samples and the dimension of the first pedestrian feature, and the feature memory is constructed based on the size of the feature memory to be constructed.
[0044] Optional, optimization module for:
[0045] Based on the category feature memory and the second pedestrian feature, the category level contrast learning is designed, and the dynamic contrast loss function corresponding to the second pedestrian feature is constructed. The dynamic contrast loss function corresponding to the second pedestrian feature is expressed as:
[0046]
[0047] Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of categories obtained by clustering, τ represents the hyperparameter, k t Represents the category feature corresponding to the pedestrian feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, ka represents the a-th category feature stored in the category feature memory;
[0048] Based on the feature memory and the second pedestrian feature, image-level contrastive learning is designed to construct a sample-constrained loss function corresponding to the second pedestrian feature. The sample-constrained loss function corresponding to the second pedestrian feature is expressed as:
[0049]
[0050] Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Indicates the hard positive sample features corresponding to the second pedestrian features;
[0051] The deep learning model is optimized based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, where the optimized deep learning model is expressed as:
[0052] L HCM =L cla +β*L img
[0053] Here, β represents a hyperparameter.
[0054] In a third aspect, the present invention also provides an unsupervised pedestrian re-identification device, which includes a processor, a memory, and an unsupervised pedestrian re-identification program stored in the memory and executable by the processor, wherein when the unsupervised pedestrian re-identification program is executed by the processor, the steps of the unsupervised pedestrian re-identification method as described above are implemented.
[0055] In a fourth aspect, the present invention further provides a readable storage medium, on which an unsupervised pedestrian re-identification program is stored, wherein when the unsupervised pedestrian re-identification program is executed by a processor, the steps of the unsupervised pedestrian re-identification method as described above are implemented.
[0056] In the present invention, a first pedestrian feature corresponding to a pedestrian image sample is obtained through a deep learning model; the pedestrian image samples are clustered through a clustering algorithm and according to the first pedestrian feature corresponding to the pedestrian image samples, and a false label is assigned to each pedestrian image sample of the same category based on the clustering result, wherein the false labels of pedestrian image samples of the same category are the same; a category feature memory is constructed based on the clustering result and the first pedestrian feature, and a feature memory is constructed based on the total number of pedestrian image samples and the first pedestrian feature; K categories of pedestrian image samples are obtained according to the false labels, and P pedestrian image samples are obtained from each category of pedestrian image samples, wherein K and P are positive integers; K*P pedestrian image samples are randomly selected. The method comprises the following steps: preprocessing is performed; the preprocessed K*P pedestrian image samples are input into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model; the deep learning model is optimized based on the category feature memory, the feature memory and the second pedestrian features; the optimized deep learning model is used as the deep learning model, and the step of obtaining the first pedestrian features corresponding to the pedestrian image samples through the deep learning model is returned to be executed; when the number of cycles reaches the preset number, the pedestrian features corresponding to the target pedestrian image are obtained by the finally obtained deep learning model, the pedestrian features corresponding to the target pedestrian image are matched with the pedestrian features in the database, and the pedestrian recognition results are obtained. Through the present invention, after constructing a category feature memory and a feature memory and obtaining the second pedestrian features corresponding to K*P pedestrian image samples, the deep learning model is optimized. In the process of optimizing the deep learning model, the problem that pedestrian images with difficult pedestrian features are not fully mined and utilized is solved through a hybrid contrast learning strategy, that is, the difficult sample features are explored from different angles, so that the deep learning model can pay attention to more difficult sample information, and at the same time, the problem of large intra-class gaps caused by the lens is solved, and the pedestrian features are fully learned to reduce the intra-class gaps of pedestrian samples and increase the inter-class gaps of pedestrian samples, so that the matching results of the pedestrian features corresponding to the target pedestrian image based on the final deep learning model are more accurate, that is, the accuracy of the recognition results of pedestrian re-identification is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 Schematic diagram of the hardware structure of the unsupervised pedestrian re-identification device involved in the embodiment of the present invention;
[0058] Figure 2 A schematic diagram of a flow chart of an embodiment of an unsupervised pedestrian re-identification method of the present invention;
[0059] Figure 3 for Figure 2 Step S30 is a schematic diagram of a detailed process flow;
[0060] Figure 4 for Figure 2 Step S70 is a detailed flow chart;
[0061] Figure 5 Schematic diagram of functional modules of an unsupervised person re-identification device according to an embodiment of the present invention.
[0062] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0063] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0064] In a first aspect, an embodiment of the present invention provides an unsupervised person re-identification device, which may be a device with data processing function, such as a personal computer (PC), a laptop computer, or a server.
[0065] Reference Figure 1 , Figure 1 The hardware structure diagram of the unsupervised pedestrian re-identification device involved in the embodiment of the present invention. In the embodiment of the present invention, the unsupervised pedestrian re-identification device may include a processor 1001 (such as a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a wireless fidelity WIreless-FIdelity, WI-FI interface); the memory 1005 may be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as a disk storage, and the memory 1005 may optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0066] Continue to refer to Figure 1 , Figure 1The memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an unsupervised person re-identification program. The processor 1001 may call the unsupervised person re-identification program stored in the memory 1005 and execute the unsupervised person re-identification method provided in the embodiment of the present invention.
[0067] In a second aspect, an embodiment of the present invention provides an unsupervised pedestrian re-identification method.
[0068] In one embodiment, referring to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of the unsupervised pedestrian re-identification method of the present invention. Figure 2 As shown in Figure 2, unsupervised person re-identification methods include:
[0069] Step S10, obtaining a first pedestrian feature corresponding to the pedestrian image sample through a deep learning model;
[0070] In this embodiment, before optimizing the deep learning model, the first pedestrian feature corresponding to the pedestrian image sample stored in the Market1501 database is obtained through the deep learning model.
[0071] Step S20, clustering the pedestrian image samples by a clustering algorithm according to the first pedestrian features corresponding to the pedestrian image samples, and assigning a false label to each pedestrian image sample of the same category based on the clustering results, wherein the false labels of pedestrian image samples of the same category are the same;
[0072] In this embodiment, the pedestrian image samples are clustered by the DBSCAN clustering algorithm according to the first pedestrian features corresponding to the pedestrian image samples, and then a false label is assigned to each pedestrian image sample of the same category based on the clustering results, wherein the false labels assigned to the pedestrian image samples of the same category are the same, and before each optimization of the ResNet-50 deep learning model, all pedestrian image samples are input into the ResNet-50 deep learning model to obtain the convolution activation map corresponding to the image, and an average pooling operation is performed on the convolution activation map to obtain a 2048-dimensional feature, and the 2048-dimensional feature is used as the first pedestrian feature corresponding to the pedestrian image sample.
[0073] Specifically, in this embodiment, a ResNet-50 deep learning model is used. After the parameters of the ResNet-50 deep learning model are initialized, a pedestrian image sample is input into the ResNet-50 deep learning model to obtain the first pedestrian feature corresponding to the pedestrian image sample output by the ResNet-50 deep learning model. The DBSCAN clustering algorithm is used to cluster the pedestrian image samples according to the first pedestrian feature corresponding to the pedestrian image sample, that is, the pedestrian image samples are divided into multiple categories, and then a false label is assigned to each pedestrian image sample of the same category, wherein pedestrian images of the same category are clustered. The false labels assigned to the samples are the same. The DBSCAN clustering algorithm assigns a label to all features based on the density attribute of the features. Because this label is generated by the clustering algorithm and is not real, it is called a false label. The false labels generated range from -1, 0, 1, 2, ..., T, where T is the maximum number of false labels generated. There are a total of (T+1) labels, and the total number generated each time is uncertain. The features assigned to -1 represent edge features and are discarded. The remaining normal features are assigned any number between 0 and T. This number is considered to be the label of the pedestrian image sample, that is, the category.
[0074] Step S30, constructing a category feature memory based on the clustering result and the first pedestrian feature, and constructing a feature memory based on the total number of pedestrian image samples and the first pedestrian feature;
[0075] In this embodiment, a category feature memory is constructed based on the clustering results and the first pedestrian feature, and a feature memory is constructed based on the total number of pedestrian image samples and the first pedestrian feature, wherein the category feature memory stores the category features of the pedestrian image samples, and the feature memory stores the first pedestrian feature corresponding to each pedestrian image sample.
[0076] Further, in one embodiment, referring to Figure 3 , Figure 3 for Figure 2 The detailed flow chart of step S30 is as follows: Figure 3 As shown, step S30 includes:
[0077] Step S301, determining the size of the category feature memory to be constructed according to the product of the total number of categories obtained by clustering and the dimension of the first pedestrian feature, and constructing the category feature memory based on the size of the category feature memory to be constructed;
[0078] Step S302: determine the size of the feature memory to be constructed according to the product of the total number of pedestrian image samples and the dimension of the first pedestrian feature, and construct the feature memory based on the size of the feature memory to be constructed.
[0079] In this embodiment, the dimension of the first pedestrian feature is 2048. If the total number of categories clustered in the Market1501 database is 800, the size of the category feature memory to be constructed is 800*2048. Based on the size of the category feature memory to be constructed, a category feature memory of size 800*2048 is constructed, and then the category feature memory is initialized. Since the first pedestrian feature corresponding to the pedestrian image sample stored in the Market1501 database obtained after optimization by the deep learning model may be different from the first pedestrian feature corresponding to the pedestrian image sample stored in the Market1501 database before optimization by the deep learning model, the clustering results obtained by clustering the pedestrian image samples by the clustering algorithm and according to the first pedestrian feature corresponding to the pedestrian image samples before and after optimization of the deep learning model, that is, the total number of categories, are also different. Therefore, the deep learning model is optimized once, and the category feature memory is initialized once.
[0080] If the total number of pedestrian image samples in the Market1501 database is 12936, the size of the feature memory to be constructed is 12936*2048. A feature memory of size 12936*2048 is constructed based on the size of the feature memory to be constructed, and then the feature memory is initialized based on the first pedestrian feature. Because the Market1501 database contains a total of 12936 images, the constructed memory size is 12936*2048, and the position of each image is fixed. The first pedestrian feature is stored in the corresponding position as initialization, and the first pedestrian feature is updated in the subsequent optimization process.
[0081] Step S40, obtaining K types of pedestrian image samples according to the false labels, and obtaining P pedestrian image samples from each type of pedestrian image samples, where K and P are positive integers;
[0082] In this embodiment, a false label is assigned to each pedestrian image sample using the DBSCAN clustering algorithm. The value of the false label starts from 0, and the maximum label is assigned to T. The maximum label each time is uncertain, but the number of false labels generated is far less than the total number of images. For example, there are 12,936 images in the Market1501 database, and generally about 800 false labels are generated. The number of images assigned with the same label (same number, same category) may be dozens. K-class pedestrian image samples are pedestrian image samples with K-class labels, that is, different numbers assigned indicate different categories. After obtaining K-class pedestrian image samples, P pedestrian image samples are obtained from each class of pedestrian image samples.
[0083] Specifically, taking K as 8 and P as 16 as an example, pedestrian image samples with 8 categories of labels are selected, and 16 pedestrian image samples are obtained from each category of pedestrian image samples to obtain 8*16 pedestrian image samples.
[0084] Step S50, randomly preprocessing K*P pedestrian image samples;
[0085] In this embodiment, K*P pedestrian image samples are randomly preprocessed, that is, 8*16 pedestrian image samples are randomly preprocessed. The diversity of pedestrian image samples is increased by preprocessing, because the more pedestrian image samples there are, the better the effect of the general model is. The samples in the database are limited. In order to increase the diversity, data enhancement is generally performed, that is, K*P pedestrian image samples are randomly preprocessed.
[0086] Furthermore, in one embodiment, the preprocessing includes horizontal flipping, cropping, erasing and normalization.
[0087] In this embodiment, after K*P pedestrian image samples are horizontally flipped and cropped, they are erased, and finally, the images are normalized. It is easy to understand that because the K*P pedestrian image samples are randomly pre-processed, the probabilities of the K*P pedestrian image samples being horizontally flipped, cropped, erased, and normalized are different. It is possible that only 5 of the K*P pedestrian image samples are horizontally flipped, or it is possible that 15 of the K*P pedestrian image samples are horizontally flipped, and the same applies to cropping, erasing, and normalization. Among them, the normalization process does not change the information storage of the image itself, but only converts the value range from 0 to 255 to between 0 and 1.
[0088] Step S60, inputting the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model;
[0089] In this embodiment, the preprocessed K*P pedestrian image samples are input into the deep learning model to obtain the second pedestrian feature corresponding to the K*P pedestrian image samples output by the deep learning model, wherein the second pedestrian feature has the same dimension as the first pedestrian feature but different feature results, and the second pedestrian feature is stored in the feature memory.
[0090] Step S70, optimizing the deep learning model based on the category feature memory, the feature memory and the second pedestrian feature;
[0091] In this embodiment, the deep learning model can be optimized based on the category feature memory, the feature memory and the second pedestrian feature to obtain an optimized deep learning model.
[0092] Further, in one embodiment, referring to Figure 4 , Figure 4 for Figure 2 The detailed flow chart of step S70 is shown in FIG. Figure 4 As shown, step S70 includes:
[0093] Step S701, based on the category feature memory and the second pedestrian feature, design category-level contrast learning, and construct a dynamic contrast loss function corresponding to the second pedestrian feature, wherein the dynamic contrast loss function corresponding to the second pedestrian feature is expressed as:
[0094]
[0095] Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of categories obtained by clustering, τ represents the hyperparameter, k t represents the category feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, k a represents the ath category feature stored in the category feature memory;
[0096] Step S702, based on the feature memory and the second pedestrian feature, design image-level contrast learning, and construct a sample constraint loss function corresponding to the second pedestrian feature, wherein the sample constraint loss function corresponding to the second pedestrian feature is expressed as:
[0097]
[0098] Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Indicates the hard positive sample features corresponding to the second pedestrian features;
[0099] Step S703, optimizing the deep learning model based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, wherein the optimized deep learning model is expressed as:
[0100] L HCM =L cla +β*L img
[0101] Here, β represents a hyperparameter.
[0102] In this embodiment, based on the category feature memory and the second pedestrian feature, a category-level contrast learning is designed to construct a dynamic contrast loss function corresponding to the pedestrian features corresponding to K*P pedestrian image samples, wherein the dynamic contrast loss function L corresponding to the pedestrian features corresponding to K*P pedestrian image samples is cla It is expressed as:
[0103]
[0104] Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of clustered categories, each clustering is obtained by the BSCAN clustering algorithm, τ represents the hyperparameter, k t represents the category feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, k a represents the ath category feature stored in the category feature memory, where N a represents the total number of pedestrian image samples in the ath category, M a represents the second pedestrian feature set corresponding to all pedestrian image samples of the ath category, f i a represents the second pedestrian feature corresponding to the i-th pedestrian image sample of the a-th category; k t Stored in the category feature memory, the dimension is also 2048 dimensions; τ can be set to 0.05, λ can be set to 0.6; the hybrid contrastive learning method includes two types of contrasts, one is category-level contrastive learning, and the other is image-level contrastive learning. In category-level contrastive learning, a category memory C is constructed, where C[a] represents the ath category. By constraining the feature distribution of the second pedestrian feature and the category feature corresponding to each pedestrian image sample, that is, reducing the intra-class gap, all images with the same label generated by clustering have similar distributions. For category-level contrastive learning, a dynamic contrast loss function is proposed, where the dynamic factor represents the difficulty of re-identification of pedestrian image samples. The higher the dynamic factor, the more difficult it is to re-identify the pedestrian image sample. The lower the dynamic factor, the easier it is to re-identify the pedestrian image sample. A large dynamic factor is generated for difficult samples, which generates greater attention, thereby learning better features and reducing the intra-class gap.
[0105] Based on the feature memory and the second pedestrian feature, image-level contrastive learning is designed to construct a sample constraint loss function corresponding to the extracted pedestrian feature. The sample constraint loss function L corresponding to the extracted pedestrian feature is img It is expressed as:
[0106]
[0107] Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Represents the hard positive sample features corresponding to the second pedestrian features.
[0108] The method for selecting the hard negative sample corresponding to the second pedestrian feature corresponding to the jth pedestrian image sample includes: calculating the cosine similarity between the second pedestrian feature and all the first pedestrian features stored in the feature memory M, and sorting all the similarities from high to low, and selecting N pedestrian image samples corresponding to the high similarity starting from the 100th high similarity. When optimizing in the Market1501 database, N can be set to 9900.
[0109] The high confidence sample feature f corresponding to the second pedestrian feature c It is expressed as:
[0110] f c = arg max sim(f j ,f c )
[0111] f c ∈Q j
[0112] Among them, Q j represents a feature set, each feature in the set is selected in the feature memory M, where each feature is related to f j With the same false label, the pedestrian image sample corresponding to each feature is the same as f j The corresponding pedestrian image samples come from different shots, and sim(·,·) represents the calculation of cosine similarity.
[0113] The high confidence sample feature f corresponding to the second pedestrian feature h It is expressed as:
[0114] f h = arg max D(f j ,f h )
[0115] f h ∈R j
[0116] Among them, R j Indicates that f j For a set of features with the same false label, D(·,·) represents the calculation of the Euclidean distance.
[0117] For contrastive learning at the image level, a sample constraint loss function is proposed. For positive sample pairs, the higher the similarity, the easier it is to re-identify the pedestrian image sample, and the lower the similarity, the more difficult it is to re-identify the pedestrian image sample, that is, the difficult sample; for negative sample pairs, the higher the similarity, the more difficult it is to re-identify the pedestrian image sample, that is, the difficult sample, and the lower the similarity, the easier it is to re-identify the pedestrian image sample. In contrastive learning at the image level, since the similarity of positive sample pairs that are difficult to re-identify pedestrian image samples is low, the intra-class gap is large. By solving the problem of the large intra-class gap of difficult positive samples, the difficult positive sample pairs have a high similarity, thereby reducing the intra-class gap. Since the similarity of negative sample pairs that are difficult to re-identify pedestrian image samples is high, the inter-class gap is large. By solving the problem of the large inter-class gap of difficult negative samples, the difficult negative sample pairs have a low similarity, thereby increasing the inter-class gap. The pedestrian re-identification method of this scheme effectively solves the problem that pedestrian images with difficult human features are not fully mined and utilized, and greatly improves the correct search rate of pedestrian re-identification.
[0118] The deep learning model is optimized based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, where the optimized deep learning model is expressed as:
[0119] L HCM =L cla +β*L img
[0120] Where β is a hyperparameter. When performing person re-identification in the Market1501 database, β can be set to 0.8.
[0121] Step S80, using the optimized deep learning model as the deep learning model, and returning to execute the step of obtaining the first pedestrian feature corresponding to the pedestrian image sample through the deep learning model;
[0122] In this embodiment, after obtaining the optimized deep learning model, the optimized deep learning model is used as the deep learning model, and the process returns to step S10, that is, returns to the step of obtaining the first pedestrian feature corresponding to the pedestrian image sample through the deep learning model.
[0123] Step S90, when the number of loops reaches the preset number, the pedestrian features corresponding to the target pedestrian image are obtained by using the final deep learning model, and the pedestrian features corresponding to the target pedestrian image are matched with the pedestrian features in the database to obtain the pedestrian recognition result.
[0124] In this embodiment, when the number of loops reaches a preset number, the target pedestrian image is input into the final deep learning model to obtain the pedestrian features corresponding to the target pedestrian image output by the final deep learning model, and the pedestrian features corresponding to the target pedestrian image are matched with the pedestrian features in the database to obtain the pedestrian recognition results.
[0125] Furthermore, the performance of the hybrid contrastive learning model can be evaluated based on the search accuracy. For example, using a large-scale online person re-identification database as the test object, when tested on the Market 1501 database, the correct search rate of pedestrian re-identification in this solution can reach 91.8%, and the performance of the hybrid contrastive learning model is good.
[0126] In this embodiment, a first pedestrian feature corresponding to a pedestrian image sample is obtained through a deep learning model; the pedestrian image samples are clustered through a clustering algorithm and according to the first pedestrian feature corresponding to the pedestrian image samples, and a false label is assigned to each pedestrian image sample of the same category based on the clustering result, wherein the false labels of pedestrian image samples of the same category are the same; a category feature memory is constructed based on the clustering result and the first pedestrian feature, and a feature memory is constructed based on the total number of pedestrian image samples and the first pedestrian feature; K categories of pedestrian image samples are obtained according to the false labels, and P pedestrian image samples are obtained from each category of pedestrian image samples, wherein K and P are positive integers; K*P pedestrian images are randomly selected. The samples are preprocessed; the preprocessed K*P pedestrian image samples are input into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model; the deep learning model is optimized based on the category feature memory, the feature memory and the second pedestrian features; the optimized deep learning model is used as the deep learning model, and the step of obtaining the first pedestrian features corresponding to the pedestrian image samples through the deep learning model is returned to execute; when the number of cycles reaches the preset number, the pedestrian features corresponding to the target pedestrian image are obtained by the final deep learning model, and the pedestrian features corresponding to the target pedestrian image are matched with the pedestrian features in the database to obtain the pedestrian recognition result. Through this embodiment, after constructing a category feature memory and a feature memory and obtaining the second pedestrian features corresponding to K*P pedestrian image samples, the deep learning model is optimized. In the process of optimizing the deep learning model, the problem that pedestrian images with difficult pedestrian features are not fully mined and utilized is solved through a hybrid contrast learning strategy, that is, exploring difficult sample features from different angles, enabling the deep learning model to pay attention to more difficult sample information, and at the same time solving the problem of large intra-class gaps caused by the lens, and fully learning pedestrian features to reduce the intra-class gaps of pedestrian samples and increase the inter-class gaps of pedestrian samples, so that the matching results of pedestrian features corresponding to the target pedestrian image based on the final deep learning model are more accurate, that is, the accuracy of the recognition results of pedestrian re-identification is improved.
[0127] Furthermore, in one embodiment, after step S50, the following steps are included:
[0128] The category feature memory and the feature memory are updated according to the second pedestrian features corresponding to the K*P pedestrian image samples.
[0129] In this embodiment, the method of updating the category feature memory C is expressed as:
[0130] C′[a]←ψC[a]+(1-ψ)f i a
[0131] Where C′[a] represents the a-th category feature stored in the category feature memory after update, ψ represents the update rate, C[a] represents the a-th category feature stored in the category feature memory before update, and f i a The second pedestrian feature corresponding to the i-th pedestrian image sample of the a-th category is represented by linear calculation and normalization of the category feature before updating in the category feature memory and the second pedestrian feature before updating, wherein C[a] is normalized by the L2 norm C[a]←||C[a]||2.
[0132] The method of updating the feature memory M is expressed as:
[0133] M′[j]←ψM[j]+(1-ψ)f j
[0134] Wherein, M′[j] represents the second pedestrian feature corresponding to the j-th pedestrian image sample stored in the feature memory after update, M[j] represents the first pedestrian feature corresponding to the j-th pedestrian image sample stored in the feature memory before update, and f j represents the second pedestrian feature corresponding to the j-th pedestrian image sample, ψ represents the update rate, which can be set to 0.1.
[0135] In a third aspect, an embodiment of the present invention further provides an unsupervised pedestrian re-identification device.
[0136] In one embodiment, referring to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the functional modules of an embodiment of the unsupervised pedestrian re-identification device of the present invention. Figure 5 As shown, the unsupervised pedestrian re-identification device includes:
[0137] A first acquisition module 10, used to obtain a first pedestrian feature corresponding to a pedestrian image sample through a deep learning model;
[0138] an assignment module 20, configured to cluster the pedestrian image samples by using a clustering algorithm and according to the first pedestrian features corresponding to the pedestrian image samples, and assign a false label to each pedestrian image sample of the same category based on the clustering results, wherein the false labels of the pedestrian image samples of the same category are the same;
[0139] A construction module 30 is used to construct a category feature memory based on the clustering result and the first pedestrian feature, and to construct a feature memory based on the total number of pedestrian image samples and the first pedestrian feature;
[0140] A second acquisition module 40 is used to acquire K types of pedestrian image samples according to the false labels, and acquire P pedestrian image samples from each type of pedestrian image samples, where K and P are positive integers;
[0141] A processing module 50 is used to randomly pre-process K*P pedestrian image samples;
[0142] An input module 60 is used to input the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model;
[0143] An optimization module 70, configured to optimize the deep learning model based on the category feature memory, the feature memory, and the second pedestrian feature;
[0144] A loop module 80 is used to use the optimized deep learning model as the deep learning model and notify the first acquisition module to perform its corresponding action;
[0145] The matching module 90 is used to obtain the pedestrian features corresponding to the target pedestrian image with the final deep learning model when the number of cycles reaches a preset number, and match the pedestrian features corresponding to the target pedestrian image with the pedestrian features in the database to obtain the pedestrian recognition result.
[0146] Furthermore, in one embodiment, the construction module 20 is used to:
[0147] Determine the size of the category feature memory to be constructed according to the product of the total number of categories obtained by clustering and the dimension of the first pedestrian feature, and construct the category feature memory based on the size of the category feature memory to be constructed;
[0148] The size of the feature memory to be constructed is determined according to the product of the total number of pedestrian image samples and the dimension of the first pedestrian feature, and the feature memory is constructed based on the size of the feature memory to be constructed.
[0149] Furthermore, in one embodiment, the optimization module 60 is used to:
[0150] Based on the category feature memory and the second pedestrian feature, the category level contrast learning is designed, and the dynamic contrast loss function corresponding to the second pedestrian feature is constructed. The dynamic contrast loss function corresponding to the second pedestrian feature is expressed as:
[0151]
[0152] Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of categories obtained by clustering, τ represents the hyperparameter, k t represents the category feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, k a represents the ath category feature stored in the category feature memory;
[0153] Based on the feature memory and the second pedestrian feature, image-level contrastive learning is designed to construct a sample-constrained loss function corresponding to the second pedestrian feature. The sample-constrained loss function corresponding to the second pedestrian feature is expressed as:
[0154]
[0155] Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Indicates the hard positive sample features corresponding to the second pedestrian features;
[0156] The deep learning model is optimized based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, where the optimized deep learning model is expressed as:
[0157] L HCM =L cla +β*L img
[0158] Here, β represents a hyperparameter.
[0159] Furthermore, in one embodiment, the unsupervised person re-identification apparatus further includes an updating module, which is used to:
[0160] The category feature memory and the feature memory are updated according to the second pedestrian features corresponding to the K*P pedestrian image samples.
[0161] Furthermore, in one embodiment, the preprocessing includes horizontal flipping, cropping, erasing and normalization.
[0162] Among them, the functional implementation of each module in the above-mentioned unsupervised pedestrian re-identification device corresponds to the various steps in the above-mentioned unsupervised pedestrian re-identification method embodiment, and its functions and implementation processes are no longer repeated here one by one.
[0163] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium.
[0164] The readable storage medium of the present invention stores an unsupervised pedestrian re-identification program, wherein when the unsupervised pedestrian re-identification program is executed by a processor, the steps of the unsupervised pedestrian re-identification method described above are implemented.
[0165] Among them, the method implemented when the unsupervised pedestrian re-identification program is executed can refer to the various embodiments of the unsupervised pedestrian re-identification method of the present invention, and will not be repeated here.
[0166] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0167] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.
[0169] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. An unsupervised person re-identification method, characterized in that: The unsupervised person re-identification method comprises: Obtain the first pedestrian feature corresponding to the pedestrian image sample through the deep learning model; Clustering the pedestrian image samples by using a clustering algorithm and according to the first pedestrian features corresponding to the pedestrian image samples, and assigning a false label to each pedestrian image sample of the same category based on the clustering results, wherein the false labels of pedestrian image samples of the same category are the same; A category feature memory is constructed based on the clustering result and the first pedestrian feature, and a feature memory is constructed based on the total number of pedestrian image samples and the first pedestrian feature; Obtain K types of pedestrian image samples according to the false labels, and obtain P pedestrian image samples from each type of pedestrian image samples, where K and P are positive integers; Randomly preprocess K*P pedestrian image samples; Input the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model; Optimizing the deep learning model based on the category feature memory, the feature memory, and the second pedestrian feature; The optimized deep learning model is used as the deep learning model, and the step of obtaining the first pedestrian feature corresponding to the pedestrian image sample through the deep learning model is returned. When the number of cycles reaches the preset number, the pedestrian features corresponding to the target pedestrian image are obtained by using the final deep learning model, and the pedestrian features corresponding to the target pedestrian image are matched with the pedestrian features in the database to obtain the pedestrian recognition result; The step of optimizing the deep learning model based on the category feature memory, the feature memory and the second pedestrian feature includes: Based on the category feature memory and the second pedestrian feature, the category level contrast learning is designed, and the dynamic contrast loss function corresponding to the second pedestrian feature is constructed. The dynamic contrast loss function corresponding to the second pedestrian feature is expressed as: Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of categories obtained by clustering, τ represents the hyperparameter, k t represents the category feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, k a represents the ath category feature stored in the category feature memory; Based on the feature memory and the second pedestrian feature, image-level contrastive learning is designed to construct a sample-constrained loss function corresponding to the second pedestrian feature. The sample-constrained loss function corresponding to the second pedestrian feature is expressed as: Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Indicates the hard positive sample features corresponding to the second pedestrian features; The deep learning model is optimized based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, where the optimized deep learning model is expressed as: L HCM =L cla +β*L img Here, β represents a hyperparameter.
2. The unsupervised person re-identification method according to claim 1, characterized in that: The steps of constructing a category feature memory based on the clustering result and the first pedestrian feature, and constructing a feature memory based on the total number of pedestrian image samples and the first pedestrian feature include: Determine the size of the category feature memory to be constructed according to the product of the total number of categories obtained by clustering and the dimension of the first pedestrian feature, and construct the category feature memory based on the size of the category feature memory to be constructed; The size of the feature memory to be constructed is determined according to the product of the total number of pedestrian image samples and the dimension of the first pedestrian feature, and the feature memory is constructed based on the size of the feature memory to be constructed.
3. The unsupervised person re-identification method according to claim 1, characterized in that: After the step of inputting the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model, the method includes: The category feature memory and the feature memory are updated according to the second pedestrian features corresponding to the K*P pedestrian image samples.
4. The unsupervised person re-identification method according to claim 1, characterized in that: The preprocessing includes horizontal flipping, cropping, erasing and normalization.
5. An unsupervised pedestrian re-identification device, characterized in that: The unsupervised pedestrian re-identification device comprises: A first acquisition module, used to obtain a first pedestrian feature corresponding to a pedestrian image sample through a deep learning model; an assignment module, configured to cluster the pedestrian image samples by using a clustering algorithm and according to the first pedestrian features corresponding to the pedestrian image samples, and assign a false label to each pedestrian image sample of the same category based on the clustering results, wherein the false labels of the pedestrian image samples of the same category are the same; A construction module is used to construct a category feature memory based on the clustering result and the first pedestrian feature, and to construct a feature memory based on the total number of pedestrian image samples and the first pedestrian feature; A second acquisition module is used to acquire K types of pedestrian image samples according to the false labels, and acquire P pedestrian image samples from each type of pedestrian image samples, where K and P are positive integers; A processing module, used for randomly preprocessing K*P pedestrian image samples; An input module, used to input the preprocessed K*P pedestrian image samples into the deep learning model to obtain the second pedestrian features corresponding to the K*P pedestrian image samples output by the deep learning model; An optimization module, for optimizing a deep learning model based on a category feature memory, a feature memory, and a second pedestrian feature; A loop module, used to use the optimized deep learning model as the deep learning model and notify the first acquisition module to perform its corresponding action; A matching module is used to obtain pedestrian features corresponding to the target pedestrian image with the final deep learning model when the number of cycles reaches a preset number, and match the pedestrian features corresponding to the target pedestrian image with the pedestrian features in the database to obtain pedestrian recognition results; Optimization modules for: Based on the category feature memory and the second pedestrian feature, the category level contrast learning is designed, and the dynamic contrast loss function corresponding to the second pedestrian feature is constructed. The dynamic contrast loss function corresponding to the second pedestrian feature is expressed as: Among them, f j represents the second pedestrian feature corresponding to the jth pedestrian image sample, A represents the total number of categories obtained by clustering, τ represents the hyperparameter, k t Represents the category feature corresponding to the pedestrian feature corresponding to the jth pedestrian image sample, represents the dynamic factor, λ represents the attention parameter, k a represents the a-th category feature stored in the category feature memory; Based on the feature memory and the second pedestrian feature, image-level contrastive learning is designed to construct a sample-constrained loss function corresponding to the second pedestrian feature. The sample-constrained loss function corresponding to the second pedestrian feature is expressed as: Where S is a set of N hard negative samples, each negative sample is selected in the feature memory, f n represents the hard negative sample feature corresponding to the second pedestrian feature, f c represents the high confidence sample feature corresponding to the second pedestrian feature, f h Indicates the hard positive sample features corresponding to the second pedestrian features; The deep learning model is optimized based on the dynamic contrast loss function and the sample constraint loss function to obtain an optimized deep learning model, where the optimized deep learning model is expressed as: L HCM =L cla +β*L img Here, β represents a hyperparameter.
6. The unsupervised person re-identification device according to claim 5, characterized in that: The building blocks are used Determine the size of the category feature memory to be constructed according to the product of the total number of categories obtained by clustering and the dimension of the first pedestrian feature, and construct the category feature memory based on the size of the category feature memory to be constructed; The size of the feature memory to be constructed is determined according to the product of the total number of pedestrian image samples and the dimension of the first pedestrian feature, and the feature memory is constructed based on the size of the feature memory to be constructed.
7. An unsupervised person re-identification device, characterized in that: The unsupervised pedestrian re-identification device includes a processor, a memory, and an unsupervised pedestrian re-identification program stored in the memory and executable by the processor, wherein when the unsupervised pedestrian re-identification program is executed by the processor, the steps of the unsupervised pedestrian re-identification method as described in any one of claims 1 to 4 are implemented.
8. A readable storage medium, characterized in that: The readable storage medium stores an unsupervised pedestrian re-identification program, wherein when the unsupervised pedestrian re-identification program is executed by a processor, the steps of the unsupervised pedestrian re-identification method according to any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Pedestrian re-identification method and device based on unsupervised learning and medium
CN110263697A
Unsupervised pedestrian re-identification method and system based on deep clustering and sample learning
CN111401281A
Unsupervised pedestrian re-identification method based on pixel and feature transfer fusion
CN112347995A
Unsupervised domain adaptation target re-identification method
WO2022001489A1
