Unsupervised pedestrian re-identification method based on complementary channel and global information

Abstract

This invention discloses an unsupervised pedestrian re-identification method based on complementary channel information and global information, comprising: acquiring an unlabeled image; inputting the image into a deep convolutional neural network to extract global features; dividing the global features into multiple local features according to the feature channel direction, and calculating the feature proximity parameter between each global feature and its local features; bidirectionally optimizing the label classification result based on the feature proximity parameter, local features, and global features to obtain noise-resistant global pseudo-labels and smoothed local feature labels; using the noise-resistant global pseudo-labels and smoothed local feature labels as image labels, and training the deep convolutional neural network using the image and its image labels to obtain a trained deep convolutional neural network; inputting the image to be identified into the trained deep convolutional neural network, and obtaining the pedestrian re-identification result based on the output of the deep convolutional neural network.

Description

Technical Field

[0001] This invention relates to an unsupervised pedestrian re-identification method based on complementary channel information and global information, belonging to the field of digital image processing technology. Background Technology

[0002] Pedestrian re-identification aims to match pedestrians with the same label name in image resources captured by different cameras, enabling the retrieval of specific individuals under varying background settings. Thanks to the rapid development of deep convolutional neural networks, recent research has yielded encouraging results. However, all this success relies on having a large, even massive, amount of labeled image resources, allowing us to acquire rich prior knowledge using given label information, camera data, and timestamps. Therefore, we need to consider how to find critical feature representations among unlabeled images in practical applications. This would help reduce the time and manpower consumed by labeling instances.

[0003] To address the lack of labeled resources, previous experiments primarily utilized K-means or Density-Based Spatial Clustering of Applications with Noise as clustering methods. These methods allowed the computer to identify hidden patterns within large amounts of noisy data, dividing it into multiple clusters and assigning a pseudo-label to each cluster. When using K-means for large-scale data, the number of clusters must be predetermined. However, it's difficult to accurately select a value for k within a reliable range at the start of training. Furthermore, coarse clustering methods can easily lead to the loss of data with inconspicuous features, making it difficult to obtain sufficient training data. Therefore, density-based clustering algorithms, such as Density-Based Spatial Clustering of Applications with Noise, have become the preferred choice. These algorithms define a cluster as the largest set of density-connected points, enabling the division of sufficiently high-density regions into clusters and the discovery of clusters of arbitrary shapes in noisy spatial databases. Current unsupervised training typically involves first performing clustering to generate pseudo-labels, and then optimizing the neural network using data carrying these pseudo-labels. Improving the network inevitably focuses on how to further refine the accuracy of pseudo-labels to reduce noise, as this has a significant impact on each iteration of the network.

[0004] To address the challenge of labeling image datasets with lower noise levels, researchers have proposed applying transfer learning to unsupervised person re-identification. Mainstream domain-adaptive unsupervised re-identification methods, such as those employed by Ge Yixiao et al., use labeled data in the source domain for auxiliary training, then use the model obtained from the source domain to extract features from the unlabeled data in the target domain, followed by clustering and generating pseudo-labels for each cluster to train the model. Although the latest domain-adaptive methods have achieved near-supervised accuracy, the difference in feature space distribution between the source and target domain data remains a core problem that continues to challenge researchers.

[0005] Unsupervised methods that do not rely on labeled data generally consist of three stages: preserving the lexicographical order of the original data, generating and refining pseudo-labels, and training the neural network. Lin Yutian et al. iterated through multiple training iterations by adding auxiliary information such as camera labels, gradually grouping similar samples into clusters to generate identical pseudo-labels. Ge Yixiao et al. used student and teacher networks to refine the pseudo-labels. Summary of the Invention

[0006] Unlike existing technologies that only consider global features to predict pseudo-labels, this invention focuses on the complementarity of multi-channel information and global information, improving the performance of the person re-identification network based on bidirectional optimization of label accuracy.

[0007] Purpose of the invention: In order to overcome the shortcomings of the existing technology, the present invention provides an unsupervised pedestrian re-identification method based on the complementarity of channel information and global information. The method uses difference information as a condition for refining labels, selects negative samples as a threshold for judging the accuracy of global labels, and then uses reliable global labels to smooth each local feature label. This method improves the accuracy of training sample labels while reducing manual costs, so as to meet the needs of the deployment of pedestrian re-identification.

[0008] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0009] In a first aspect, the present invention provides an unsupervised pedestrian re-identification method based on complementary channel information and global information, comprising:

[0010] Step 1: Obtain unlabeled images;

[0011] Step 2: Input the image into a deep convolutional neural network to extract global features;

[0012] Step 3: Divide the global feature into multiple local features according to the feature channel direction, and calculate the feature affinity parameter between each global feature and its local features;

[0013] Step 4: Based on the feature proximity parameter, local features, and global features, optimize the label classification results bidirectionally to obtain noise-resistant global pseudo-labels and smoothed local feature labels;

[0014] Step 5: Use the noise-resistant global pseudo-labels and smoothed local feature labels as image labels, and use the images and their image labels to train the deep convolutional neural network to obtain the trained deep convolutional neural network.

[0015] Step 6: Input the image to be identified into the trained deep convolutional neural network, and obtain the pedestrian re-identification result based on the output of the deep convolutional neural network.

[0016] In some embodiments, step 2, inputting the image into a deep convolutional neural network to extract global features, includes:

[0017]

[0018] In the formula: y[i] represents the convolution output at each position i, x[i] represents the input at each position i, w[k] represents the convolution filter of length k, and r represents the sampling step size.

[0019] In some embodiments, step 3 involves dividing the global features into multiple local features according to the feature channel direction, including:

[0020] The global features are divided along the channels, and the features f obtained by the convolutional layer are... i g ∈R C×H×W Where C, H, and W represent the number of channels, height, and width of the feature map, respectively; i represents the lexicographical order of the global features; and g represents the number of channels for the global features. Set to 4, the scale of local features is Where P n This represents a local feature with lexicographical order n.

[0021] In some embodiments, step 3 involves calculating the feature proximity parameter between each global feature and its local features, including:

[0022] The Jaccard similarity coefficient is used to calculate the feature proximity parameter between global features and their local features:

[0023]

[0024] In the formula: C(|·|) is the set of all global and local features in a cluster, n is the number of channel partitions for the global features, and S n (f i g) represents the affinity parameter between the current i-th global feature and the n-th local feature.

[0025] In some embodiments, step 4 involves bidirectionally optimizing the label classification result based on the feature proximity parameter, local features, and global features to obtain noise-resistant global pseudo-labels and smoothed local feature labels, including:

[0026] Step 4.1: For each global feature, select the local feature with the lowest feature proximity parameter as a negative sample to optimize the label of the global feature and obtain a noise-resistant global pseudo label;

[0027] Step 4.2: Based on the obtained noise-resistant global pseudo-labels, smooth all local feature labels corresponding to the global features to obtain smoothed local feature labels.

[0028] Further, in step 4.1: For each global feature, select the local feature with the lowest feature proximity parameter as a negative sample to optimize the global feature label, obtaining a noise-resistant global pseudo-label, including:

[0029]

[0030] In the formula: For the output noise-resistant global pseudo-labels, y i It is the result of a classifier based on global features. S is the predicted vector obtained by the negative sample through the local feature classifier. n (f i g ), S k (f i g ) represent the affinity parameters of the current i-th global feature and the nth and kth local features, respectively. It is the normalized value of the negative sample ranking in the same local features of the source data. It is used to soften the influence of negative samples on the global pseudo label and prevent extreme results when the data difference is too large. u is a uniform vector. As a weighting coefficient, it is used to control the selection tendency of global pseudo-labels and negative samples;

[0031] Further, step 4.2: Based on the obtained noise-resistant global pseudo-labels, smooth all local feature labels corresponding to the global features to obtain smoothed local feature labels, including:

[0032]

[0033] In the formula: These are the smoothed local feature labels, β = S n (f i g ) is the characteristic intimacy parameter.

[0034] In some embodiments, step 5, training the deep convolutional neural network using the image and its image labels, includes:

[0035] The loss function used during training is a combination of local feature loss and global feature loss:

[0036]

[0037] In the formula: Let K be the loss function, and let K and N represent the number of classes in the classifier. The local feature loss value, These are the smoothed local feature labels. It is the predicted vector obtained by the negative sample through the local feature classifier. This represents the global feature loss value. For the output noise-resistant global pseudo-labels, q i g The predicted vector is obtained by the classifier from the global features.

[0038] Secondly, the present invention provides an unsupervised pedestrian re-identification device based on complementary channel information and global information, including a processor and a storage medium;

[0039] The storage medium is used to store instructions;

[0040] The processor is configured to operate according to the instructions to perform the steps of the method according to the first aspect.

[0041] Thirdly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0042] Fourthly, the present invention provides an apparatus comprising,

[0043] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described in the first aspect.

[0044] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0045] This invention discloses an unsupervised person re-identification method based on complementary channel and global information. It obtains global features of the input image through a deep convolutional neural network, divides it into multiple local features according to the feature channel direction, calculates the distance between each global feature and its local features, and optimizes the classifier results of global and local features bidirectionally based on the feature space distance difference. The optimization results are then used as image labels to train the parameters of the deep convolutional neural network. This method improves the accuracy of training sample labels while reducing manual labor costs, thus meeting the needs of practical deployment of person re-identification.

[0046] This invention addresses the characteristic that local features in horizontal image segmentation contain irrelevant background information or key information is occluded. It obtains local features from a channel perspective, avoiding the process of distinguishing noise interference sources and feature alignment. In the process of using a bidirectional optimized labeling algorithm, each global feature captures the local feature farthest from itself based on the distance in the feature space. This local feature can be selected as an optimization factor, and the position of negative samples is dynamically updated in each training round. A dynamic threshold replaces manual setting, and the pseudo-labels continuously learn the most discriminative channel difference information, gradually outputting robust and refined labels, thus improving the reliability of unsupervised re-identification. Attached Figure Description

[0047] Figure 1 This is a flowchart of the training process of the present invention;

[0048] Figure 2 This is a schematic diagram of loss training according to the present invention;

[0049] Figure 3 This is a schematic diagram of the accuracy training of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be used to limit the scope of protection of the present invention.

[0051] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0052] In the description of this invention, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0053] Example 1

[0054] An unsupervised pedestrian re-identification method based on complementary channel information and global information includes:

[0055] Step 1: Obtain unlabeled images;

[0056] Step 2: Input the image into a deep convolutional neural network to extract global features;

[0057] Step 3: Divide the global feature into multiple local features according to the feature channel direction, and calculate the feature affinity parameter between each global feature and its local features;

[0058] Step 4: Based on the feature proximity parameter, local features, and global features, optimize the label classification results bidirectionally to obtain noise-resistant global pseudo-labels and smoothed local feature labels;

[0059] Step 5: Use the noise-resistant global pseudo-labels and smoothed local feature labels as image labels, and use the images and their image labels to train the deep convolutional neural network to obtain the trained deep convolutional neural network.

[0060] Step 6: Input the image to be identified into the trained deep convolutional neural network, and obtain the pedestrian re-identification result based on the output of the deep convolutional neural network.

[0061] In some specific embodiments, such as Figure 1 As shown, the unsupervised pedestrian re-identification method based on complementary channel information and global information provided in this embodiment of the invention includes the following steps:

[0062] Step 1: Obtain unlabeled images;

[0063] Step 2: Input the image into a deep convolutional neural network to extract global features;

[0064] The feature map of the input image is obtained. To test the effectiveness of this method in complex environments, this invention uses the VeRi-776 vehicle dataset in addition to the general pedestrian dataset Market-1501. Complex scene images contain a large number of noise components such as occlusion, pose tilt, and uneven lighting distribution. Under this dataset, the final prediction effect of this invention is still excellent.

[0065] The image is input into a deep convolutional neural network to extract global features. The convolution process is as follows:

[0066]

[0067] In the formula: y[i] represents the convolution output at each position i, x[i] represents the input at each position i, w[k] represents the convolution filter of length k, and r represents the sampling step size, which is set to r=2 in this part.

[0068] Step 3: Divide the global features obtained in Step 2 into multiple local features according to the feature channel direction, and calculate the feature proximity parameter between each global feature and its local features; there is discriminative information between each local feature to guide label optimization, which specifically includes the following steps:

[0069] Step 3.1: The features f obtained from the convolutional layer i g ∈R C×H×W Where C, H, and W represent the number of channels, height, and width of the feature map, respectively; i represents the lexicographical order of the global features; and g represents the number of channels for the global features. Set to 4, the scale of local features is Where P n Represents a local feature with lexicographical order n;

[0070] Step 3.2: Calculate the spatial distance between global and local features to obtain the feature affinity parameter. Use the Jaccard similarity coefficient to compare the similarity and differences between finite sample sets. The process expression is as follows:

[0071]

[0072] In the formula: C(|·|) is the set of all global and local features in a cluster, n is the number of channel partitions for the global features, and S n (f i g ) represents the affinity parameter between the current i-th global feature and the n-th local feature.

[0073] Step 4: Based on the feature proximity parameters obtained in Step 3, local features and global features are used to bidirectionally optimize the label classification results, resulting in noise-resistant global pseudo-labels and smoothed local feature labels. This specifically includes the following steps:

[0074] Step 4.1: For each global feature, select the local feature with the lowest feature proximity parameter as a negative sample to optimize the global feature label, obtaining a noise-resistant global pseudo-label; the process expression is as follows:

[0075]

[0076] In the formula: For the output noise-resistant global pseudo-labels, y i It is the result of a classifier based on global features. S is the predicted vector obtained by the negative sample through the local feature classifier. n (f i g ), S k (f i g ) represent the affinity parameters of the current i-th global feature and the nth and kth local features, respectively. It is the normalized value of the negative sample ranking in the same local features of the source data. It is used to soften the influence of negative samples on the global pseudo label and prevent extreme results when the data difference is too large. u is a uniform vector. As a weighting coefficient, it is used to control the selection tendency of global pseudo-labels and negative samples.

[0077] Step 4.2: Based on the obtained noise-resistant global pseudo-labels, smooth all local feature labels corresponding to the global features to obtain smoothed local feature labels, including:

[0078]

[0079] In the formula: These are the smoothed local feature labels, β = S n (f i g ) is the characteristic intimacy parameter.

[0080] Step 5: Using the noise-resistant global pseudo-labels and smoothed local feature labels as image labels, train the deep convolutional neural network using the images and their image labels to obtain a trained deep convolutional neural network; including:

[0081] The loss function used during training is a combination of local feature loss and global feature loss, and its expression is as follows:

[0082]

[0083] In the formula: Let K be the loss function, and let K and N represent the number of classes in the classifier. The local feature loss value, These are the smoothed local feature labels. It is the predicted vector obtained by the negative sample through the local feature classifier. This represents the global feature loss value. For the output noise-resistant global pseudo-labels, q i g The predicted vector is obtained by the classifier from the global features.

[0084] Step 6: Input the image to be identified into the trained deep convolutional neural network, and obtain the pedestrian re-identification result based on the output of the deep convolutional neural network.

[0085] Figure 2 and Figure 3 The diagrams shown are a loss training diagram and an accuracy statistics diagram for the experiment of this invention. During the training process, the number of channels for global features is 4, and the 4th block with the richest discriminative information is selected as the negative sample. The output value of the loss function continues to decrease, indicating that the deep convolutional neural network is converging. The final accuracy of this invention can reach up to 94.7%. According to actual software calculations, the mean average precision of this invention in a general dataset is 77.2%.

[0086] Example 2

[0087] Secondly, this embodiment provides an unsupervised pedestrian re-identification device based on complementary channel information and global information, including a processor and a storage medium;

[0088] The storage medium is used to store instructions;

[0089] The processor is configured to operate according to the instructions to perform the steps of the method according to Embodiment 1.

[0090] Example 3

[0091] Thirdly, this embodiment provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0092] Example 4

[0093] Fourthly, the present invention provides an apparatus comprising,

[0094] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described in Embodiment 1.

[0095] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0099] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An unsupervised pedestrian re-identification method based on complementary channel information and global information, characterized in that, The method comprises the following steps: Step 1, obtaining an image without a label; Step 2, inputting the image into a deep convolutional neural network to extract global features; Step 3, dividing the global features into multiple local features according to the feature channel direction, and calculating a feature affinity parameter between each global feature and its local feature; Step 4, bidirectional optimization of label classification results according to the feature affinity parameter, local feature and global feature, to obtain an anti-noise global pseudo label and a smoothed local feature label, comprising: Step 4.1: for each global feature, selecting the local feature with the lowest feature affinity parameter as a negative sample to optimize the label of the global feature, and obtaining an anti-noise global pseudo label, denoted as: In the formula: y is the output anti-noise global pseudo label, i is the classifier result of the global feature, S is the prediction vector of the negative sample through the local feature classifier, n (f i g ), S k (f i g respectively represent the affinity parameters of the current i-th global feature and the n-th and k-th local features, is the ranking normalized value of the negative sample in the same local feature of the source data, which is used to soften the influence of the negative sample on the global pseudo label, prevent extreme results when the data gap is too large, and u is a uniform vector; as a weight coefficient to control the selection trend of the global pseudo label and the negative sample; Step 4.2: based on the obtained anti-noise global pseudo label, smoothing all local feature labels corresponding to the global feature to obtain a smoothed local feature label, denoted as: In the formula: is the smoothed local feature label, β = S n (f i g is a feature affinity parameter; Step 5, using the anti-noise global pseudo label and the smoothed local feature label as the image label, training the deep convolutional neural network using the image and its image label to obtain a trained deep convolutional neural network; the loss function used in the training is the superposition of the local feature loss function and the global feature loss function; Step 6, inputting an image to be identified into the trained deep convolutional neural network, and obtaining a pedestrian re-identification result according to the output of the deep convolutional neural network.

2. The unsupervised pedestrian re-identification method based on complementary channel information and global information according to claim 1, wherein, In step 2, the image is input into a deep convolutional neural network to extract global features, comprising: In the formula, y[i] represents the convolution output at each position i, x[i] represents the input at each position i, w[k] represents a convolution filter with a length of k, and r represents a sampling step. 3.The unsupervised pedestrian re-identification method based on complementary channel information and global information according to claim 1, wherein, In step 3, the global features are divided into multiple local features according to the feature channel direction, comprising: The global feature is divided along the channel, and the feature f obtained by the convolution layer i g ∈R C×H×W where C, H, W are the channel number, height and width of the feature map respectively, i is the dictionary order of the global feature, g represents the global feature, and the channel division number is set to 4, and the scale of the local feature is where P n represents the local feature with the dictionary order n. 4.The unsupervised pedestrian re-identification method based on complementary channel information and global information according to claim 1, wherein, In step 3, the feature affinity parameter between each global feature and its local feature is calculated, comprising: The Jaccard similarity coefficient is used to calculate the feature affinity parameter between the global feature and its local feature: wherein C(|·|) is a set of all global features and local features in a cluster, n is a channel division number of the global features, S n (f i g represents an affinity parameter of the current ith global feature and the nth local feature.

5. The unsupervised pedestrian re-identification method based on complementary channel information and global information according to claim 1, wherein, In step 5, the deep convolutional neural network is trained using the image and its image label, comprising: The loss function used in the training is represented as: In the formula: is a loss function, K, N represents the number of categories of the classifier, is a local feature loss value, is a smoothed local feature label, is a prediction vector obtained by a local feature classifier, is a global feature loss value, is an output anti-noise global pseudo label, is a prediction vector obtained by a global feature classifier.

6. An unsupervised pedestrian re-identification device based on complementary channel information and global information, characterized in that, The system comprises a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to perform the steps of the method according to any one of claims 1 to 5.

7. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

8. A computer device, comprising: The system comprises: one or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing any one of the methods according to claims 1 to 5.

Images