A collaborative multi-feature clustering unsupervised pedestrian re-identification method and system

Abstract

The application provides a collaborative multi-feature clustering unsupervised pedestrian re-identification method and system, which comprises the following steps: data preprocessing is performed on source domain pedestrian images in a training set; the preprocessed source domain pedestrian images are sent into an initial pedestrian re-identification model for pre-training; the source domain pedestrian images are input into a multi-domain translation network, are migrated to a target domain data camera style, and a target domain labeled data set is constructed according to original data labels, and the initial pedestrian re-identification model is retrained; two double-branch networks with the same structure are constructed for unsupervised clustering and supervised training; pedestrian images in a test set are sent into the double-branch network to extract double-branch joint features, the Euclidean distances between the double-branch joint features and corresponding joint features of candidate set images are calculated, and a ranking result is output. The application can be effectively applied to a data unlabeled scene, is more robust to inter-domain image style differences existing in data, and is suitable for complex situations in real scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and in particular to a collaborative multi-feature clustering unsupervised pedestrian re-identification method and system based on camera style domain adaptation. BACKGROUND

[0002] In real-world scenarios, due to the high cost of labeling pedestrian datasets, data often lacks labels, but there is usually a labeled source domain dataset. However, since the two datasets are collected in different environments, there is a significant difference in image style between the two datasets. Therefore, directly migrating the model trained on the source domain data to the target domain data for testing will result in a significant performance drop.

[0003] To solve this problem, existing methods mostly transfer source domain label knowledge or need to assume strong prior knowledge, or assume that manual features are already sufficiently discriminative on target domain data. These methods mostly use unsupervised domain adaptation strategies to fully utilize the semantic information present in the source domain data. Due to the camera style difference across domains, the above methods have limited improvement on cross-domain pedestrian re-identification datasets. SUMMARY

[0004] To solve the problem of cross-domain unsupervised pedestrian re-identification often encountered in real-world scenarios, that is, to alleviate the difference in camera style between source domain images and target domain images and improve the robustness of the model to domain-independent information, the present application provides a collaborative multi-feature clustering unsupervised pedestrian re-identification method and system based on camera style domain adaptation.

[0005] The object of the present application can be achieved by the following technical solutions:

[0006] A collaborative multi-feature clustering unsupervised pedestrian re-identification method based on camera style domain adaptation, comprising the following steps:

[0007] For a pedestrian re-identification dataset, data preprocessing is performed on source domain pedestrian images in the training set;

[0008] The preprocessed source domain pedestrian images are input into an initial pedestrian re-identification model for pre-training;

[0009] The source domain pedestrian images are input into a multi-domain translation network, migrated to the camera style of the target domain data, and a labeled target domain dataset is constructed according to the original data labels. The initial pedestrian re-identification model is retrained according to the labeled dataset to obtain a fine-tuned pedestrian re-identification model;

[0010] Based on the fine-tuned pedestrian re-identification model, two double-branch networks with the same structure are constructed, two double-branch networks respectively use the features generated by the corresponding branches of each other for unsupervised clustering, and the labels obtained by the unsupervised clustering are used for supervised training of the double-branch networks;

[0011] The pedestrian images in the test set are input into the double-branch network to extract double-branch joint features, the Euclidean distance between the double-branch joint features and the corresponding joint features of the candidate set images is calculated, and the ranking result is output.

[0012] Further, the preprocessing includes: reading the pedestrian images in the training set into the memory, first performing normalization processing, that is, converting the pixel values to a mean value of 0 and a variance of 1; then using random erasing and boundary expansion operations for data enhancement; finally, the enhanced data is converted into a data format supported by the Pytorch deep learning framework.

[0013] Further, the pre-training of the pre-processed source domain pedestrian image into the initial pedestrian re-identification model includes:

[0014] The ResNet-50 model pre-trained on ImageNet is used as the initial basic backbone model of the pedestrian re-identification network, and cross-entropy and triple loss are used for supervised training; given a single batch of labeled data, the initial basic backbone model is trained using cross-entropy loss and triple loss of hard negative mining, wherein the cross-entropy loss uses the output of the last fully connected layer as the input, and a label smoothing strategy is used.

[0015] Further, the multi-domain translation network includes a pedestrian image generator network, an image authenticity discriminator network, and an image domain classification discriminator network; the source domain pedestrian image is input into the multi-domain translation network, migrated to the target domain camera style, and the target domain labeled data set is constructed according to the original data label, and the initial pedestrian re-identification model is retrained according to the labeled data set, including:

[0016] First, the source domain pedestrian image is input into the pedestrian image generator network to generate a migrated image, and then the migrated image and the original image are simultaneously input into the image authenticity discriminator network and the image domain classification discriminator network for adversarial training, so that the pedestrian image generator network generates a pedestrian image of the target domain style; then, the pre-trained initial pedestrian re-identification model is fine-tuned based on the target domain style pedestrian image migrated by the generator.

[0017] Further, the adversarial training and fine-tuning include the following loss functions:

[0018] Adversarial generative learning loss, domain classification loss, identity mapping loss, pedestrian identity preservation loss, and pedestrian image reconstruction loss.

[0019] Further, the supervised training includes global feature branch training and local feature branch training.

[0020] The global feature branch training includes: given an unlabeled image, first inputting the unlabeled image into a pre-trained pedestrian recognition model for feature extraction to obtain a feature map; then, performing global mean pooling operation on the obtained feature map to obtain a feature vector; for each image in the target domain dataset, performing feature extraction and combining the obtained feature vectors into a set; based on the feature vector set, using an unsupervised clustering method to divide the target domain dataset into different groups; according to the clustering result, each sample is assigned a pseudo label to obtain a labeled training dataset; then, based on the labeled training dataset, the pedestrian recognition model is fine-tuned in a supervised manner.

[0021] The local feature branch training includes: dividing the feature map of the pedestrian image into two regions in the horizontal dimension, corresponding to the upper body and the lower body of the pedestrian respectively, and the two divided feature maps are used for pseudo label generation of the pedestrian image respectively.

[0022] Further, the global feature branch training further includes:

[0023] An activation map is generated from the feature map of the unlabeled image, and the point with the maximum activation value in each sample activation map is recorded, which represents the area that the current model pays most attention to;

[0024] A mask region of random size and random aspect ratio is generated around the point with the maximum activation value to occlude the image, and then the occluded image is input into the decoder network to generate a reconstruction result.

[0025] In order to achieve the above purpose, the application further provides a collaborative multi-feature clustering unsupervised pedestrian re-identification system, which comprises:

[0026] The preprocessing module performs data preprocessing on the source domain pedestrian images in the training set for the pedestrian re-identification dataset.

[0027] The pre-training module sends the pre-processed source domain pedestrian images into an initial pedestrian re-identification model for pre-training.

[0028] The transfer fine-tuning module inputs the source domain pedestrian images into a multi-domain translation network, transfers to the target domain data camera style, and constructs a target domain labeled dataset according to the original data label, and re-trains the initial pedestrian re-identification model according to the labeled dataset to obtain a fine-tuned pedestrian re-identification model.

[0029] The supervision training module constructs two double-branch networks with the same structure based on the fine-tuned pedestrian re-identification model, and the two double-branch networks respectively use the features generated by the corresponding branches of each other to perform unsupervised clustering, and the double-branch networks are supervised trained according to the labels obtained by the unsupervised clustering.

[0030] The calculation ranking module inputs the pedestrian images in the test set into the double-branch network to extract double-branch joint features, calculates the Euclidean distance between the double-branch joint features and the corresponding joint features of the candidate set images, and outputs the ranking result.

[0031] Compared with the prior art, the present application has the following advantages:

[0032] Firstly, the present application can effectively reduce the distribution difference between the source domain data and the target domain data, thereby fully mining the pedestrian identity related discriminative information existing in the target domain data and improving the cross-domain performance of the model.

[0033] Secondly, the present application proposes a plurality of mapping consistency preserving losses, which can keep the pedestrian identity information unchanged before and after migration, and fully utilize the style mapping relationship between the source domain and the target domain data sets, so that the generated images are as close as possible to the real images.

[0034] Thirdly, the present application designs an effective collaborative multi-feature clustering framework, which uses the pseudo labels obtained by using each other's features to train in parallel to alleviate the negative effects of consistency bias and improve the robustness of the model to the noise labels generated by clustering.

[0035] Fourthly, the present application provides an unsupervised pedestrian re-identification system which is robust to image camera style changes and has more application value in real-world scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0036] In the drawings, like reference numerals designate like elements throughout the several views, unless otherwise specified. The drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the application. It should be understood that the drawings are merely depictions of some embodiments of the application and should not be construed as limiting the scope of the application.

[0037] Figure 1 Flow chart of the collaborative multi-feature clustering unsupervised pedestrian re-identification method of the present application.

[0038] Figure 2 Pedestrian image diagram after camera style migration generated by the present application.

[0039] Figure 3 Working principle diagram of the double-branch network constructed by the present application.

[0040] Figure 4A constitution diagram of a cooperative multi-feature clustering unsupervised pedestrian re-identification system according to an embodiment of the present application is shown.

[0041] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0042] Figure 6 A schematic diagram of a storage medium provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the related application, but not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0044] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0045] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the embodiments, specifically as Figure 1 shown in the method flowchart, the present application will be further described in detail. It should be understood that the specific embodiments described herein are merely intended to explain the present application, but not to limit the present application.

[0046] The present application provides a cooperative multi-feature clustering unsupervised pedestrian re-identification method based on camera style domain adaptation, so that the model can fully utilize the labeled data of the source domain to mine the discriminative information of the unlabeled images of the target domain, alleviate the consistency bias problem caused by using its own features to generate pseudo-labels, and let the model mine the multi-granularity information of the identity correlation existing in the image. As Figure 1 shown, the present application includes the following steps:

[0047] Step 1: Data preprocessing. Read the training set in the pedestrian re-identification dataset into the memory and perform preprocessing. The implementation is as follows:

[0048] Read the pedestrian images in the training set into the memory using Python programming language, first perform normalization, i.e. convert the pixel values to mean 0 and variance 1. Then perform data augmentation using random erasing, boundary expansion, etc. Finally, convert to a data format supported by the Pytorch deep learning framework.

[0049] Step 2: Build an initial pedestrian re-identification model, divide the labeled source domain pedestrian image dataset into multiple batches and input them into the network to obtain the prediction results and calculate the classification loss.

[0050] The implementation is as follows:

[0051] In order to learn the discriminative information existing in the labeled source domain data, first, supervised training is performed on the source domain data, specifically, using a ResNet-50 model pre-trained on ImageNet as the initial basic backbone model of the pedestrian re-identification network, and using cross-entropy and triplet loss for supervised training. Given a single batch of labeled data, the initial basic backbone model is trained using cross-entropy loss and triplet loss for hard negative mining, wherein the cross-entropy loss uses the output of the last fully connected layer as input, and at the same time, a label smoothing strategy is used to avoid overfitting to the training data. Therefore, the final form of the cross-entropy loss can be expressed as follows:

[0052]

[0053]

[0054] where n s is the number of images in the batch, p i (k)∈[0,1] is the predicted probability value of the model for class k, q i (k) is the label distribution, y is the real class label, ξ is a small perturbation term, and M S represents the number of classes of the source domain dataset. The triplet loss is used to further consolidate the intra-class closeness of the features and strengthen the inter-class difference, and its form is as follows:

[0055]

[0056] where x a represents the anchor image, x p and x n represent the farthest positive example and the nearest negative example in the batch, respectively. m is a boundary hyperparameter with a fixed value, and D(·) is the Euclidean distance calculation function between two features. Therefore, the overall loss function of the model can be summarized as follows:

[0057] L baseline =λ t L tti +L cross

[0058] where λ t is set to 1 so that the triplet loss can fully optimize the discriminative nature of the features.

[0059] Step three: constructing a multi-domain image translation network: including a pedestrian image generator network, an image authenticity discriminator network, and an image domain classification discriminator network. The pedestrian image generator network is based on Stylegan, and the image authenticity discriminator network and the image domain classification discriminator network use ResNet50 as the network structure. Specifically, first, the real source domain image is input into the generator network to generate the migrated image, and then the migrated image and the original image are input into the source domain pre-trained image authenticity discriminator network and the image domain classification discriminator network for adversarial training, so that the generator can generate pedestrian images with the style of the target domain; then, the model obtained in step two is fine-tuned based on the target domain style data migrated by the generator.

[0060] The specific implementation is as follows:

[0061] In order to sufficiently reduce the distribution difference between the source domain data and the target domain data, each camera is regarded as a separate data domain for image-image migration. Given a source domain data set and a target domain data set, and both data sets provide camera labels. The goal is to train a generator G that can accurately learn the mapping relationship between domains. In this way, given a labeled image and its camera label, the generator G can be converted into an image with the camera style of the target domain while keeping the pedestrian identity information unchanged. Specifically, the training process includes five kinds of loss functions:

[0062] ① Adversarial generative learning loss: this loss is used to make the generated image as close to the real image as possible. Specifically, the generator G aims to minimize this loss to make the generated image as realistic as possible, while the discriminator aims to maximize this loss to make it more discriminative. The form of this loss is as follows:

[0063] L adv =E x [logD src (x)]+E x，c [log(1-D src (G(x,c))]]

[0064] Where E, D src represent the cumulative average function, the source domain classification discriminator, and the subscript represents the variables required for calculation. C represents the target domain label of the real image x.

[0065] ② Domain classification loss: in order to distinguish the domain labels of real / generated images, an auxiliary classifier is introduced at the end of the discriminator and uses cross-entropy loss to optimize the generator and discriminator so that the generator can produce images that fully integrate the style of the target domain image data. In particular, the domain classification loss of the real image is used to optimize the discriminator, while the domain classification loss of the generated image is used to optimize the generator, which is as follows:

[0066]

[0067]

[0068] where c’ represents the original domain label of the real image x, c is the target domain label, D cls is the target domain classification discriminator.

[0069] ③ Identity mapping loss: The identity mapping loss regularizes the generator to accurately learn the identity mapping matrix of the target domain samples. At the same time, this loss can also maintain the color composition between the input and output images in the mapping process to some extent. Specifically, the identity mapping loss is as follows:

[0070] L idt = E x，c [||G(x, y) - x||1]

[0071] ④ Pedestrian identity preservation loss: In order to fully utilize the transferred image to perform supervised training of the pedestrian re-identification model on the target domain data while changing the style of the image, an identity preservation loss is introduced, which optimizes the image foreground change before and after pedestrian transfer. In order to consolidate the robustness of the generator, a consistency regularization loss is introduced, which constrains the output of the original image and the reconstructed image to be consistent, and the loss form is as follows:

[0072]

[0073] M(x) represents the foreground mask matrix of the input x.

[0074] ⑤ Pedestrian image reconstruction loss: In order to maintain the content of the input image while changing only the domain-related style information in the image, a reconstruction loss is used to construct the forward loop consistency. Specifically, the loss is as follows:

[0075] L rec = E x，c，c′ [||x - G(x, c), c’)||1] In summary, the overall objective function for optimizing the generator G and the discriminator D in the multi-domain image style transfer stage can be represented as follows:

[0076]

[0077]

[0078] where λ cls , λ rec , λ idt , λ pidThe superparameters of the control domain adaptive classification loss, the reconstruction loss, the identity mapping loss, and the pedestrian identity preservation loss, respectively.

[0079] Step four: based on the fine-tuned model, two structurally identical double-branch networks are constructed, as shown in Figure 3 The two networks respectively use the features generated by the corresponding branches of each other for unsupervised clustering, and the labels obtained from the clustering are used to supervise the training of the double-branch to alleviate the influence of model bias on the quality of pseudo labels caused by a single model. During testing, the distance between images is calculated to perform pedestrian feature matching.

[0080] The specific implementation is as follows:

[0081] (1) Global feature branch training: given an unlabeled image First, the sample is input into the pre-trained model for feature extraction. The feature map of the image is represented as The result is taken from the output of the fifth layer of the ResNet-50 model. Subsequently, a global average pooling operation is used to process the obtained feature map to obtain a feature vector For each image in the target domain dataset, feature extraction is performed and the obtained feature vectors are combined into a set, which is represented as Based on the feature vector set, an unsupervised clustering method is used to divide the target domain dataset into different groups. Specifically, the DBSCAN method is used to perform the operation. According to the clustering result, each sample is assigned a pseudo label In this way, a new labeled training dataset can be composed of target domain images and pseudo labels generated by corresponding clustering, so the dataset can be represented in the following form: Subsequently, the model is fine-tuned in a supervised manner based on the dataset. Specifically, batch-hard triplet loss and cross-entropy loss are used in the fine-tuning process. The average feature of each cluster is used as the initialization parameter. Therefore, the parameter W c of the classifier is initialized as follows: where C is the number of clusters in each iteration process. Finally, a new reconstruction branch based on adversarial erasing is proposed to improve the discriminability of the model based on the feature map input by the global branch. Specifically, the feature map of the image is first used to generate an activation map, and then the point with the maximum activation value in the activation map of each sample is recorded, which is considered to represent the area that the current model pays most attention to.

[0082] In addition, the global max pooling and the global average pooling are non-parametric processes. Specifically, the global average pooling calculates the average of all pixels in the pooling region and replaces the value of the center point of the current window with the average. The global max pooling only considers the maximum value of the pixels in the current window. Since the global max pooling can effectively maintain the significant regions in the image, the framework selects the global max pooling for high-quality feature extraction.

[0083] (2) To alleviate the consistency bias of the model, the model focuses on more detailed discriminative regions in the image. A mask region with a random size and a random aspect ratio is generated around the point with the maximum activation value to obscure the image. Then, the obscured image is input into another decoder network to generate a reconstruction result. The decoder network aims to reconstruct the original image that has not been erased according to the feature map generated by the adversarial erasing branch, thereby training in a self-supervised manner. Specifically, the reconstruction result is the reconstruction of the original unobscured image. The reconstruction loss used by the network is the same as L rec In this way, the consistency bias of the network and the excessive reliance on noisy labels can be alleviated, so that the network can make a comprehensive judgment on the pedestrian identity from multiple regions in the image.

[0084] (3) Local feature branch training: Before training in the target domain, the model first performs supervised training on the source domain dataset and the image dataset obtained by migration in the previous section to learn high-quality discriminative features of pedestrian images. However, the global feature model obtained in this way still performs poorly on the target domain data. In the pedestrian re-identification task, the fusion of local feature learning usually achieves superior results because the feature contains more fine-grained identity information. The feature map of the pedestrian image is divided into two regions in the horizontal dimension, corresponding to the upper body and the lower body of the pedestrian. The two divided feature maps are used for pseudo-label generation of the pedestrian image.

[0085] (4) The Rank-1 / 5 / 10 and the average mean precision (mAP) are used to evaluate the performance of the present application. The cross-domain experiments are performed on two pedestrian re-identification datasets, and the comparison results with the most advanced method are shown in the following table:

[0086]

[0087]

[0088] The specific implementation steps of the present application are as follows:

[0089] Figure 1 The implementation flowchart of the present application is shown in FIG. 1, and the specific implementation is as follows:

[0090] 1. For pedestrian re-identification dataset, data preprocessing is performed on pedestrian images in the training set.

[0091] 2. The preprocessed source domain pedestrian images are input into the network for pre-training.

[0092] 3. The source domain pedestrian images are input into the multi-domain translation network, migrated to the target domain data camera style, and a target domain labeled dataset is constructed according to the original data label. The model in 2 is retrained according to the labeled data.

[0093] 4. Based on the fine-tuned model, two double-branch networks are constructed, and the features generated by the corresponding branches of the two networks are used for unsupervised clustering, and the labels obtained by the clustering are used for supervised training of the double-branch.

[0094] 5. The pedestrian images in the test set are input into the double-branch network to extract the double-branch joint features, calculate the Euclidean distance between the double-branch joint features and the corresponding joint features of the candidate set images, and output the ranking result. This part is trained based on the feature map of the global feature branch and belongs to the data augmentation strategy of the entire network.

[0095] Figure 2 For the pedestrian visualization based on domain camera adaptive migration of the present application, the various mapping consistency preservation losses of the present application can keep the pedestrian identity information unchanged before and after migration, and fully utilize the style mapping relationship between the source domain and the target domain dataset, so that the generated image is as close as possible to the real image.

[0096] The application embodiment provides a collaborative multi-feature clustering unsupervised pedestrian re-identification system, which is used to execute the collaborative multi-feature clustering unsupervised pedestrian re-identification method described in the above embodiment, as shown in Figure 4 The system comprises:

[0097] The preprocessing module 401 performs data preprocessing on the source domain pedestrian images in the training set for the pedestrian re-identification dataset.

[0098] The pre-training module 402 inputs the preprocessed source domain pedestrian images into the initial pedestrian re-identification model for pre-training.

[0099] The migration fine-tuning module 403 inputs the source domain pedestrian images into the multi-domain translation network, migrates to the target domain data camera style, and constructs a target domain labeled dataset according to the original data label. The initial pedestrian re-identification model is retrained according to the labeled dataset, and a fine-tuned pedestrian re-identification model is obtained.

[0100] The supervision training module 404 constructs two double-branch networks with the same structure based on the fine-tuned pedestrian re-identification model, and the two double-branch networks perform unsupervised clustering using the features generated by the corresponding branches of each other, and the double-branch networks are supervised trained according to the labels obtained by the unsupervised clustering.

[0101] The calculation ranking module 405 inputs the pedestrian images in the test set into the double-branch network to extract double-branch joint features, calculates the Euclidean distance between the double-branch joint features and the corresponding joint features of the candidate set images, and outputs the ranking result.

[0102] The above-mentioned embodiments of the application provide a cooperative multi-feature clustering unsupervised pedestrian re-identification system, and the method for cooperative multi-feature clustering unsupervised pedestrian re-identification provided by the embodiments of the application have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0103] The embodiments of the application further provide an electronic device corresponding to the method for cooperative multi-feature clustering unsupervised pedestrian re-identification provided by the above-mentioned embodiments, to execute the above-mentioned method for cooperative multi-feature clustering unsupervised pedestrian re-identification. The embodiments of the application are not limited.

[0104] Please refer to Figure 5 , which shows a schematic diagram of an electronic device provided by some embodiments of the application. As Figure 5 shown, the electronic device 20 includes a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program executable on the processor 200, and the processor 200 executes the computer program to perform the method for cooperative multi-feature clustering unsupervised pedestrian re-identification provided by any one of the preceding embodiments of the application.

[0105] Among them, the memory 201 can contain a high-speed random access memory (RAM: Random Access Memory), and can also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 203 (which can be wired or wireless), and the Internet, wide area network, local network, metropolitan area network, etc. can be used.

[0106] The bus 202 can be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is configured to store a program, and the processor 200 executes the program after receiving an execution instruction. The cooperative multi-feature clustering unsupervised pedestrian re-identification method disclosed in any of the embodiments of the present application can be applied to the processor 200 or implemented by the processor 200.

[0107] The processor 200 can be an integrated circuit chip with a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 200 or an instruction in the form of software. The processor 200 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and combines the hardware to complete the steps of the above method.

[0108] The electronic device provided by the embodiments of the present application and the cooperative multi-feature clustering unsupervised pedestrian re-identification method provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by them.

[0109] The embodiments of the present application also provide a computer readable storage medium corresponding to the cooperative multi-feature clustering unsupervised pedestrian re-identification method provided by the preceding embodiments. Please refer to Figure 6 The computer readable storage medium shown in the figure is an optical disc 30, and a computer program (i.e. program product) is stored on the optical disc 30. When the computer program is run by a processor, the cooperative multi-feature clustering unsupervised pedestrian re-identification method provided by any of the preceding embodiments is executed.

[0110] It should be noted that examples of the computer-readable storage medium can also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical, magnetic storage mediums, and the like, without limitation.

[0111] The computer-readable storage medium provided in the foregoing embodiments of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the unsupervised pedestrian re-identification method with collaborative multi-feature clustering provided by the embodiments of the present application.

[0112] It should be noted that:

[0113] The methods and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general purpose systems can be used with these methods based on the teachings herein. General structure for such systems for construction as will be apparent to those of skill in the art in light of the above descriptions is within the scope of the present application. Furthermore, the present application is not intended to be limited to any particular programming language. It will be appreciated that there are many programming languages that can be used to implement the teachings of the present application as described herein, and any such programming language is intended to be encompassed by the present application.

[0114] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to not obscure the understanding of this description.

[0115] Similarly, it is to be understood that the above description is only illustrative of the application and certain examples thereof, and is not to be taken as limiting the scope of the application, as the same can be modified and practiced in several specific ways and that the intent is to be limited strictly only by the following claims. Numerous specific aspects of the application are set forth in the following claims.

[0116] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and furthermore can be split into multiple sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or process or device of any combination of the features disclosed in the specification (including the accompanying claims, abstract and drawings) can be taken, except that at least some of such features and / or processes or units are mutually exclusive, unless explicitly stated otherwise. Each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features providing the same, equivalent, or similar functions unless stated explicitly otherwise.

[0117] Furthermore, those skilled in the art will appreciate that different embodiments of the application have different features and that the features of one embodiment can not be included in another embodiment. For example, in the following claims, any of the claims can be used to enable a single claim, multiple claims or single and multiple claims in combination. The following claims are therefore merely exemplary and are not intended to limit the application or scope of the application in any way.

[0118] The various component embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. Those skilled in the art will appreciate that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the system for creating a virtual machine according to the embodiments of the present application. The present application can also be implemented as a program of apparatus or system (for example, a computer program and a computer program product) for performing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0119] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0120] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A collaborative multi-feature clustering unsupervised pedestrian re-identification method, characterized in that, Includes the following steps: For the pedestrian re-identification dataset, data preprocessing is performed on the source domain pedestrian images in the training set; The preprocessed source domain pedestrian images are fed into the initial pedestrian re-identification model for pre-training. The source domain pedestrian image is input into the multi-domain translation network, transferred to the target domain data camera style, and a target domain labeled dataset is constructed based on the original data labels. The initial pedestrian re-identification model is retrained based on the labeled dataset to obtain the fine-tuned pedestrian re-identification model. Based on the fine-tuned pedestrian re-identification model, two identical dual-branch networks are constructed. The two dual-branch networks use the features generated by their corresponding branches to perform unsupervised clustering, and supervised training is performed on the two dual-branch networks based on the labels obtained from the unsupervised clustering. The pedestrian images in the test set are fed into two dual-branch networks to extract the dual-branch joint features. The Euclidean distance between the dual-branch joint features and the corresponding joint features of the candidate set images is calculated, and the ranking result is output.

2. The unsupervised pedestrian re-identification method based on collaborative multi-feature clustering according to claim 1, characterized in that, The preprocessing includes: reading pedestrian images from the training set into memory, first performing normalization processing, that is, converting their pixel values ​​to a mean of 0 and a variance of 1; then performing data augmentation using random erasure and boundary expansion operations; and finally converting the augmented data into a data format supported by the PyTorch deep learning framework.

3. The unsupervised pedestrian re-identification method based on collaborative multi-feature clustering according to claim 1, characterized in that, The step of feeding the preprocessed source domain pedestrian image into the initial pedestrian re-identification model for pre-training includes: The ResNet-50 model pre-trained on ImageNet was used as the initial basic backbone model of the pedestrian re-identification network, and supervised training was performed using cross-entropy and triplet loss. Given a single batch of labeled data, the initial basic backbone model was trained using cross-entropy loss and triplet loss for hard negative example mining, where the cross-entropy loss used the output of the last fully connected layer as input, and a label smoothing strategy was used.

4. The unsupervised pedestrian re-identification method based on collaborative multi-feature clustering according to claim 1, characterized in that, The multi-domain translation network includes a pedestrian image generator network, an image authenticity discriminator network, and an image domain classification discriminator network. The process of inputting source domain pedestrian images into the multi-domain translation network, transferring them to the target domain data camera style, constructing a labeled dataset for the target domain based on the original data labels, and retraining the initial pedestrian re-identification model based on the labeled dataset includes: First, the source domain pedestrian image is input into the pedestrian image generator network to generate a transferred image. Then, the transferred image and the original image are simultaneously input into the image authenticity discriminator network and the image domain classification discriminator network for adversarial training, so that the pedestrian image generator network generates pedestrian images in the target domain style. Subsequently, the pre-trained initial pedestrian re-identification model is fine-tuned based on the target domain style pedestrian images obtained by the generator transfer.

5. The unsupervised pedestrian re-identification method based on collaborative multi-feature clustering according to claim 4, characterized in that, The adversarial training and fine-tuning training include the following loss functions: Adversarial generative learning loss, domain classification loss, identity mapping loss, pedestrian identity preservation loss, and pedestrian image reconstruction loss.

6. The unsupervised pedestrian re-identification method based on collaborative multi-feature clustering according to claim 1, characterized in that, The supervised training includes global feature branch training and local feature branch training: The global feature branch training includes: given an unlabeled image, firstly, inputting the unlabeled image into a pre-trained pedestrian recognition model for feature extraction to obtain a feature map; processing the obtained feature map using global mean pooling to obtain feature vectors; for each image in the target domain dataset, performing feature extraction and combining the obtained feature vectors into a feature vector set; based on this feature vector set, using an unsupervised clustering method to divide the target domain dataset into different groups; according to the clustering results, each sample is assigned a pseudo-label to obtain a labeled training dataset; and fine-tuning the pedestrian recognition model in a supervised manner based on this labeled training dataset. The local feature branch training includes: dividing the feature map of the pedestrian image into two regions in the horizontal dimension, corresponding to the upper and lower body of the pedestrian respectively, and the two feature maps after division are used for pseudo-label generation of the pedestrian image.

7. The unsupervised pedestrian re-identification method based on collaborative multi-feature clustering according to claim 6, characterized in that, The global feature branch training further includes: Activation maps are generated from the feature maps of unlabeled images, and the point with the largest activation value in the activation map of each sample is recorded. The point with the largest activation value represents the region that the current model is most interested in. The unlabeled image is occluded by generating a mask region of random size and aspect ratio centered on the point with the largest activation value. The occluded image is then input into the decoder network to generate the reconstruction result.

8. A collaborative multi-feature clustering unsupervised pedestrian re-identification system, characterized in that, include: The preprocessing module performs data preprocessing on the source domain pedestrian images in the training set for the pedestrian re-identification dataset. The pre-training module feeds the pre-processed source domain pedestrian images into the initial pedestrian re-identification model for pre-training. The transfer fine-tuning module inputs the source domain pedestrian image into the multi-domain translation network, transfers it to the target domain data camera style, constructs a target domain labeled dataset based on the original data labels, and retrains the initial pedestrian re-identification model based on the labeled dataset to obtain the fine-tuned pedestrian re-identification model. The supervised training module constructs two identical dual-branch networks based on the fine-tuned pedestrian re-identification model. The two dual-branch networks use the features generated by their corresponding branches to perform unsupervised clustering, and supervise the training of the two dual-branch networks based on the labels obtained from the unsupervised clustering. The ranking module feeds pedestrian images from the test set into two dual-branch networks to extract dual-branch joint features, calculates the Euclidean distance between the dual-branch joint features and the corresponding joint features of the candidate set images, and outputs the ranking results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.

Images