Pedestrian re-identification pseudo label refining method based on attention mechanism

Abstract

The invention relates to the technical field of image retrieval, and particularly provides a pedestrian re-identification false label refining method based on an attention mechanism. The method comprises the following steps: clustering an obtained training data set, and distributing pseudo labels; inputting the obtained training data set into a neural network containing a space attention mechanism and a channel attention mechanism, and carrying out pre-training to obtain local features and global features; according to the obtained local features and global features, respectively calculating first k feature index lists which are closest to each other and correspond to the local features and the global features; calculating the Jaccard similarity between the first k mutual nearest neighbors of the global feature and the local feature to obtain a weight; according to the method, the pseudo labels are refined according to the obtained weights, the pseudo labels are dynamically refined according to the obtained weights, it is guaranteed that the model pays more attention to important information, the accuracy of pseudo label distribution is improved, and the stability and accuracy of the model are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of image retrieval technology, and in particular to a method for refining pseudo labels for pedestrian re-identification based on an attention mechanism. Background Art

[0002] With the rapid development of artificial intelligence, person re-identification technology in image retrieval technology has attracted much attention. Through the photos of specific pedestrians, deep learning technology is used to finally find the photos of this person in massive data. Person re-identification mainly includes two types of methods: supervised and unsupervised learning. Supervised learning is trained in labeled pictures, while unsupervised learning is trained in unlabeled data.

[0003] Traditional person re-identification mainly relies on supervised learning methods, which require a large amount of labeled data for training. However, obtaining large-scale labeled datasets is not only time-consuming and laborious, but also in practical applications, the cross-domain problem greatly limits the generalization ability of the model. Therefore, person re-identification technology based on unsupervised learning came into being. Unsupervised learning can effectively train models by exploring the intrinsic structure, feature distribution and potential associations of pedestrian identities in the absence of manual annotations. This technology not only reduces the dependence on labeled data, but also improves the robustness and generalization ability in cross-domain scenarios, greatly promoting the development of the field of person re-identification. However, due to the presence of a large amount of unreliable information such as background and occlusions in the picture, the accuracy of the pseudo-labels assigned by unsupervised learning is not high. Summary of the invention

[0004] In view of this, the present invention provides a method for refining pseudo labels for person re-identification based on an attention mechanism, so as to improve the accuracy of pseudo label assignment and ensure the stability and accuracy of the model.

[0005] In a first aspect, the present invention provides a method for refining pseudo labels for person re-identification based on an attention mechanism, the method comprising: Step 1: Obtain a training dataset for a person re-identification model; Step 2: Perform feature extraction and clustering on the acquired training data set and assign pseudo labels; Step 3: Input the acquired training data set into the neural network with spatial attention mechanism and channel attention mechanism for pre-training to obtain local features. And global features ; Step 4: Based on the local features and global features obtained, the corresponding feature index lists of the first k nearest neighbors are calculated, expressed as , ; Step 5: Calculate the Jaccard similarity between the first k nearest neighbors of the global feature and the local feature, and obtain the weight, which is expressed as ; Step 6: Refine the pseudo-labels according to the obtained weights to obtain refined pseudo-labels.

[0006] Optionally, step 1 includes: In the given form, let represents the unlabeled training dataset, where Represents an image, Represents the number of images; first extract the feature map represented as , where C, H and W are the channel size, height and width of the feature map respectively; given a feature map, the global feature set is obtained by performing global average pooling on the feature map , by dividing the feature map into Evenly divided area And average pooling is performed on each region to obtain the local feature set .

[0007] Optionally, the step 2 includes: Global feature set for the training dataset Perform DBSCAN clustering and assign clusters as pseudo-labels; The pseudo label is recorded as , which is a one-hot encoding with hard assignment of K clusters.

[0008] Optionally, the step 3 includes: In the spatial attention mechanism branch, for the feature map , firstly, the height average pooling and width average pooling are performed respectively, then they are spliced ​​together and two-dimensional convolution operation is performed, then normalization and activation function are performed, and then they are split again. After splitting, two-dimensional convolution operation and activation function operation are performed in sequence, and finally they are expressed as and ; or Each element in represents The importance of the local features of the corresponding columns or rows in identifying pedestrians; then, by using and Reweighting To obtain a new feature map ; In the channel attention mechanism branch, As input to obtain the attention map of the channel dimension, channel average pooling operation, one-dimensional convolution operation, normalization and activation function operation, one-dimensional convolution operation and activation function operation are performed in sequence, and finally the attention map of the channel dimension is obtained. Indicates; similarly, Each element in represents The importance of the semantic information of its corresponding channel in right Re-weighted to get the output feature map , which contains important semantic information of the matching person; after passing through the neural network with spatial attention mechanism and channel attention mechanism, local features and global features are obtained, which are finally expressed as and .

[0009] Optionally, step 4 includes: According to local features and global features, features are sorted based on the similarity between feature vectors, and a feature index list of the first k nearest neighbors is returned; the K nearest neighbor principle is that two objects are one of each other's K nearest neighbors; the similarity between feature vectors is the mutual Euclidean distance, which is calculated as follows: ,in, and Represent the values ​​of the two images in the i-th dimension, and n represents the dimension of the feature vector; finally, the feature index lists of the first k nearest neighbors of the local features and global features are obtained, which are represented as , .

[0010] Optionally, step 5 includes: Calculate the Jaccard similarity between the first k nearest neighbors of the global feature and the local feature. The Jaccard similarity calculation formula is: ,in, and The global features are and local features The index set of the first k samples in the calculated ranking list, and Represents the number of set elements, and finally obtains the weight, expressed as .

[0011] Optionally, step 6 includes: According to the obtained weights, the pseudo labels are refined. Given an image Pseudo-labels , local features The specific formula for label smoothing is: , which is the loss function; where u is a uniform vector, is the weight used to adjust the strength of label balance; in order to dynamically adjust the weight of each part, the Jaccard similarity between the global feature and the first k nearest neighbors of the local feature is used as the weight .

[0012] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for refining pseudo-labels for pedestrian re-identification based on an attention mechanism in the first aspect or any possible implementation of the first aspect.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to perform the method for refining pseudo-labels for pedestrian re-identification based on an attention mechanism in the first aspect or any possible implementation of the first aspect.

[0014] In the technical solution provided by the present invention, the method includes obtaining a training data set for a pedestrian re-identification model; clustering the obtained training data set and assigning pseudo labels; inputting the obtained training data set into a neural network containing a spatial attention mechanism and a channel attention mechanism for pre-training to obtain local features and global features; according to the obtained local features and global features, respectively calculating the feature index lists of the first k nearest neighbors corresponding to them; calculating the Jaccard similarity between the first k nearest neighbors of the global features and the local features to obtain weights; refining the pseudo labels according to the obtained weights to obtain refined pseudo labels. The method dynamically refines the pseudo labels according to the obtained weights, thereby ensuring that the model pays more attention to important information, improving the accuracy of pseudo label assignment, and ensuring the stability and accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of a method for refining pseudo labels for person re-identification based on an attention mechanism provided in an embodiment of the present invention; Figure 2 A schematic diagram of a spatial attention mechanism and a channel attention mechanism provided by an embodiment of the present invention; Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] It should be clear that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0020] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0021] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0022] Figure 1 A flowchart of a method for refining pseudo labels for person re-identification based on an attention mechanism provided in an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes: Step 1: Obtain a training dataset for a person re-identification model; In the embodiment of the present invention, step 1 includes: In the given form, let represents the unlabeled training dataset, where Represents an image, Represents the number of images; first extract the feature map represented as , where C, H and W are the channel size, height and width of the feature map respectively; given a feature map, the global feature set is obtained by performing global average pooling on the feature map , by dividing the feature map into Evenly divided area And average pooling is performed on each region to obtain the local feature set .

[0023] Step 2: Perform feature extraction and clustering on the acquired training data set and assign pseudo labels; In the embodiment of the present invention, step 2 includes: Global feature set for the training dataset Perform DBSCAN clustering and assign clusters as pseudo-labels; The pseudo label is recorded as , which is a one-hot encoding with hard assignment of K clusters.

[0024] In the embodiment of the present invention, DBSCAN clustering: This algorithm defines a cluster as the largest set of density-connected points, can divide areas with sufficiently high density into clusters, and can find clusters of any shape in a noisy spatial database. This algorithm requires the user to input two parameters: the scanning radius and the minimum number of included points. The specific operation is to randomly extract an unprocessed point from the database. If the point is a core point, all objects connected to the density of the point are found to form a cluster. If it is not a core point, then jump out of this loop and look for the next point. Repeat the above steps until all points are processed.

[0025] Step 3: Input the acquired training data set into the neural network with spatial attention mechanism and channel attention mechanism for pre-training to obtain local features. And global features ; In the embodiment of the present invention, Figure 2 As shown, step 3 includes: In the spatial attention mechanism branch, for the feature map , firstly, the height average pooling and width average pooling are performed respectively, then they are spliced ​​together and two-dimensional convolution operation is performed, then normalization and activation function are performed, and then they are split again. After splitting, two-dimensional convolution operation and activation function operation are performed in sequence, and finally they are expressed as and ; or Each element in represents The importance of the local features of the corresponding columns or rows in identifying pedestrians; then, by using and Reweighting To obtain a new feature map ; In the channel attention mechanism branch, As input to obtain the attention map of the channel dimension, channel average pooling operation, one-dimensional convolution operation, normalization and activation function operation, one-dimensional convolution operation and activation function operation are performed in sequence, and finally the attention map of the channel dimension is obtained. Indicates; similarly, Each element in represents The importance of the semantic information of its corresponding channel in right Re-weighted to get the output feature map , which contains important semantic information of the matching person; after passing through the neural network with spatial attention mechanism and channel attention mechanism, local features and global features are obtained, which are finally expressed as and .

[0026] Step 4: Based on the local features and global features obtained, the corresponding feature index lists of the first k nearest neighbors are calculated, expressed as , ; In the embodiment of the present invention, step 4 includes: According to local features and global features, features are sorted based on the similarity between feature vectors, and a feature index list of the first k nearest neighbors is returned; the K nearest neighbor principle is that two objects are one of each other's K nearest neighbors; the similarity between feature vectors is the mutual Euclidean distance, which is calculated as follows: ,in, and Represent the values ​​of the two images in the i-th dimension, and n represents the dimension of the feature vector; finally, the feature index lists of the first k nearest neighbors of the local features and global features are obtained, which are represented as , .

[0027] Step 5: Calculate the Jaccard similarity between the first k nearest neighbors of the global feature and the local feature, and obtain the weight, which is expressed as ; In the embodiment of the present invention, step 5 includes: Calculate the Jaccard similarity between the first k nearest neighbors of the global feature and the local feature. The Jaccard similarity calculation formula is: ,in, and The global features are and local features The index set of the first k samples in the calculated ranking list, and Represents the number of set elements, and finally obtains the weight, expressed as .

[0028] Step 6: Refine the pseudo-labels according to the obtained weights to obtain refined pseudo-labels.

[0029] In the embodiment of the present invention, step 6 includes: According to the obtained weights, the pseudo labels are refined. Given an image Pseudo-labels , local features The specific formula for label smoothing is: , which is the loss function; where u is a uniform vector, is the weight used to adjust the strength of label balance; in order to dynamically adjust the weight of each part, the Jaccard similarity between the global feature and the first k nearest neighbors of the local feature is used as the weight , so as to improve the accuracy of label assignment and obtain better recognition effect.

[0030] Each step of the embodiment of the present invention may be performed by an electronic device, which includes but is not limited to a mobile phone, a tablet computer, a portable PC, a desktop computer, etc.

[0031] In the technical solution provided by the present invention, the method includes obtaining a training data set for a pedestrian re-identification model; clustering the obtained training data set and assigning pseudo labels; inputting the obtained training data set into a neural network containing a spatial attention mechanism and a channel attention mechanism for pre-training to obtain local features and global features; according to the obtained local features and global features, respectively calculating the feature index lists of the first k nearest neighbors corresponding to them; calculating the Jaccard similarity between the first k nearest neighbors of the global features and the local features to obtain weights; refining the pseudo labels according to the obtained weights to obtain refined pseudo labels. The method dynamically refines the pseudo labels according to the obtained weights, thereby ensuring that the model pays more attention to important information, improving the accuracy of pseudo label assignment, and ensuring the stability and accuracy of the model.

[0032] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the electronic device where the computer-readable storage medium is located is controlled to execute an embodiment of the above-mentioned method for refining pseudo-labels for pedestrian re-identification based on the attention mechanism.

[0033] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 3As shown, the electronic device 21 includes: a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, the method for refining pseudo-labels for pedestrian re-identification based on the attention mechanism in the embodiment is implemented. To avoid repetition, they are not described one by one here.

[0034] The electronic device 21 includes, but is not limited to, a processor 211 and a memory 212. Those skilled in the art will appreciate that Figure 3 It is only an example of the electronic device 21 and does not constitute a limitation of the electronic device 21. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0035] The processor 211 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0036] The memory 212 may be an internal storage unit of the electronic device 21, such as a hard disk or memory of the electronic device 21. The memory 212 may also be an external storage device of the electronic device 21, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 21. Further, the memory 212 may also include both an internal storage unit of the electronic device 21 and an external storage device. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 may also be used to temporarily store data that has been output or is to be output.

[0037] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0038] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for refining pseudo labels for person re-identification based on attention mechanism, characterized in that: The method comprises: Step 1: Obtain a training dataset for a person re-identification model; Step 2: Perform feature extraction and clustering on the acquired training data set and assign pseudo labels; Step 3: Input the acquired training data set into the neural network with spatial attention mechanism and channel attention mechanism for pre-training to obtain local features. And global features ; Step 4: Based on the local features and global features obtained, the corresponding feature index lists of the first k nearest neighbors are calculated, expressed as , ; Step 5: Calculate the Jaccard similarity between the first k nearest neighbors of the global feature and the local feature, and obtain the weight, which is expressed as ; Step 6: Refine the pseudo-labels according to the obtained weights to obtain refined pseudo-labels.

2. The method according to claim 1, characterized in that The step 1 comprises: In the given form, let represents the unlabeled training dataset, where Represents an image, Represents the number of images; first extract the feature map represented as , where C, H and W are the channel size, height and width of the feature map respectively; given a feature map, the global feature set is obtained by performing global average pooling on the feature map , by dividing the feature map into Evenly divided area And average pooling is performed on each region to obtain the local feature set .

3. The method according to claim 1, characterized in that The step 2 comprises: Global feature set for the training dataset Perform DBSCAN clustering and assign clusters as pseudo-labels; The pseudo label is recorded as , which is a one-hot encoding with hard assignment of K clusters.

4. The method according to claim 1, characterized in that: The step 3 comprises: In the spatial attention mechanism branch, for the feature map , firstly, the height average pooling and width average pooling are performed respectively, then they are spliced ​​together and two-dimensional convolution operation is performed, then normalization and activation function are performed, and then they are split again. After splitting, two-dimensional convolution operation and activation function operation are performed in sequence, and finally they are expressed as and ; or Each element in represents The importance of the local features of the corresponding columns or rows in the recognition of pedestrians; then, by using and Reweighting To obtain a new feature map ; In the channel attention mechanism branch, As input to obtain the attention map of the channel dimension, channel average pooling operation, one-dimensional convolution operation, normalization and activation function operation, one-dimensional convolution operation and activation function operation are performed in sequence, and finally the attention map of the channel dimension is obtained. Indicates; similarly, Each element in represents The importance of the semantic information of its corresponding channel in right Re-weighted to get the output feature map , which contains important semantic information of the matching person; after passing through the neural network with spatial attention mechanism and channel attention mechanism, local features and global features are obtained, which are finally expressed as and .

5. The method according to claim 1, characterized in that The step 4 comprises: According to local features and global features, features are sorted based on the similarity between feature vectors, and a feature index list of the first k nearest neighbors is returned; the K nearest neighbor principle is that two objects are one of each other's K nearest neighbors; the similarity between feature vectors is the mutual Euclidean distance, which is calculated as follows: ,in, and Represent the values ​​of the two images in the i-th dimension, and n represents the dimension of the feature vector; finally, the feature index lists of the first k nearest neighbors of the local features and global features are obtained, which are represented as , .

6. The method according to claim 1, characterized in that The step 5 comprises: Calculate the Jaccard similarity between the first k nearest neighbors of the global feature and the local feature. The Jaccard similarity calculation formula is: ,in, and The global features are and local features The index set of the first k samples in the calculated ranking list, and Represents the number of set elements, and finally obtains the weight, expressed as .

7. The method according to claim 1, characterized in that The step 6 comprises: According to the obtained weights, the pseudo labels are refined. Given an image Pseudo-labels , local features The specific formula for label smoothing is: , which is the loss function; where u is a uniform vector, is the weight used to adjust the strength of label balance; in order to dynamically adjust the weight of each part, the Jaccard similarity between the global feature and the first k nearest neighbors of the local feature is used as the weight .

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for refining pseudo labels for pedestrian re-identification based on an attention mechanism according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to perform the method for refining pseudo labels for pedestrian re-identification based on the attention mechanism as described in any one of claims 1 to 7.

Images