A pedestrian re-identification method
By introducing an attribute-guided attention mechanism and an LSTM network into the pedestrian re-identification method, the dependency relationship between attributes is established, which solves the problem of the failure to effectively utilize attribute features in the existing technology and improves the accuracy of pedestrian re-identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHENGZHOU XINDA ADVANCED TECH RES INST
- Filing Date
- 2020-10-21
- Publication Date
- 2026-05-01
AI Technical Summary
Existing pedestrian re-identification methods fail to effectively utilize pedestrian attribute features, especially neglecting the dependencies between attributes, resulting in low recognition accuracy.
An attribute-guided attention mechanism is adopted, which extracts the feature map of each attribute through convolutional kernels, establishes the dependency relationship between attributes using an LSTM network, and combines the cross-entropy loss function for training to improve the accuracy of attribute recognition.
By employing attribute-guided attention mechanisms and dependency modeling, the accuracy of pedestrian re-identification is significantly improved, especially in the ability to identify pedestrians in complex environments.
Smart Images
Figure CN114445672B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a pedestrian re-identification method, belonging to the field of image recognition and processing technology. Background Technology
[0002] Pedestrian re-identification studies the tracking of specific pedestrians across cameras in a non-overlapping perspective. This method can use a large number of online interconnected cameras to achieve cross-domain tracking of specific pedestrians for events, suspect tracking, behavior analysis, etc. It is a cloud-based video information analysis technology that can provide security for daily life.
[0003] In real-world scenarios, the accuracy of pedestrian identification is affected by many factors, such as environmental factors and pedestrian characteristics, which have varying impacts on pedestrian re-identification. Because environmental factors are complex and diverse, finding models adapted to different environments is challenging. However, pedestrian characteristics, such as gender, hair length, clothing color, and clothing type, are relatively constant over a certain period, making them highly effective for tracking specific pedestrians. Therefore, pedestrian attribute recognition is a common method for pedestrian re-identification and analysis, representing a fine-grained approach with broad application prospects in intelligent video surveillance. As a local feature, pedestrian attribute features can be combined with pedestrian ID information in pedestrian re-identification to improve accuracy.
[0004] However, in existing methods for attribute learning research, the attention mechanism uses a uniform feature distribution and does not apply different attention to each attribute. In other words, all attributes correspond to the same feature space, and vectors in this feature space are directly projected onto each attribute classification branch to obtain the confidence score for that attribute. This feature space is not specific to each attribute but rather to the entire pedestrian image, and therefore lacks the characteristics of attribute classes. Furthermore, pedestrian attribute features are not well utilized for representing attributes in specific regions, and the dependencies between pedestrian attributes are not adequately considered, resulting in relatively low recognition accuracy. Summary of the Invention
[0005] The purpose of this invention is to provide a pedestrian re-identification method to solve the problem of low recognition accuracy in current pedestrian re-identification methods due to the failure to consider the dependencies between attributes.
[0006] The present invention provides an identification method for solving the above-mentioned technical problems, comprising the following steps:
[0007] 1) Perform feature recognition on the acquired image to obtain a high-level feature map of the image;
[0008] 2) Use the same number of convolutional kernels as the pedestrian attributes to extract features from the high-level feature maps, and obtain the feature map for each attribute;
[0009] 3) Establish the dependency relationship for the feature vector corresponding to each attribute based on the feature map of each attribute;
[0010] 4) The image to be identified is identified by utilizing the dependency relationship between the feature vectors corresponding to each attribute.
[0011] This invention proposes attribute-guided attention, providing attribute-related attention guidance for each attribute. By using convolutional kernels with the same number of pedestrian attributes to extract features from high-level feature maps, feature maps for each attribute are obtained, thereby improving the semantic representation of attribute feature vectors. Then, dependencies between attributes are established based on the feature vectors of each attribute. Through these dependencies, the recognition accuracy of one attribute assists the recognition of another attribute, thus improving the overall attribute recognition accuracy.
[0012] Furthermore, in order to accurately obtain the high-level features of the image, step 1) uses a pre-trained convolutional neural network with the last pooling layer and fully connected layer removed for feature recognition.
[0013] Furthermore, the size of the convolution kernel in step 2) is 3*3.
[0014] Furthermore, in order to accurately establish the dependency relationship between the feature vectors corresponding to each attribute, step 3) uses an LSTM network to establish the dependency relationship between the feature vectors corresponding to each attribute. The LSTM network includes a forward LSTM layer and a backward LSTM layer. Assuming there are L pedestrian attributes, both the forward LSTM layer and the backward LSTM layer include L LSTM units. Each attribute feature is input into the corresponding LSTM unit in the forward LSTM layer and the backward LSTM layer, respectively.
[0015] Furthermore, the convolutional neural network described herein employs ResNet-50 with the final global average pooling layer and fully connected layer removed.
[0016] Furthermore, to ensure the accuracy of model training, step 4) uses the sum of the losses from attribute classification and pedestrian ID classification for training during the recognition process.
[0017] Furthermore, the cross-entropy loss function used for attribute classification loss is:
[0018]
[0019] Where p(y i,j ) represents the predicted probability of the j-th value of the i-th attribute, q(y i,j ) represents the true probability of the j-th value of the i-th attribute.
[0020] Furthermore, the cross-entropy loss function used for pedestrian ID classification is:
[0021]
[0022] Where p(y k Let q(k) represent the predicted probability of the k-th ID category, and let q(k) represent the true probability of the k-th ID category. Attached Figure Description
[0023] Figure 1 This is a schematic diagram of the network structure used in the pedestrian re-identification method of the present invention;
[0024] Figure 2-a This is a schematic diagram of the ResNet-50 network structure used in the convolutional feature extraction module of this invention;
[0025] Figure 2-b This is a schematic diagram of the Conv Block structure in the ResNet-50 structure used in this invention;
[0026] Figure 2-c This is a schematic diagram of the Identity Block structure in the ResNet-50 structure used in this invention;
[0027] Figure 3 This is a schematic diagram of the network structure of the attribute-guided attention module of the present invention;
[0028] Figure 4 This is a schematic diagram illustrating the calculation process of attribute feature vectors by a single LSTM unit in this invention. Detailed Implementation
[0029] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0030] The pedestrian re-identification method of the present invention starts from the perspective of fine-grained localization of pedestrian attribute features and attribute dependency modeling. It introduces an attribute-guided attention mechanism to learn discriminative features for each pedestrian attribute; it explicitly models the dependency relationship between each pedestrian attribute, makes full use of the co-occurrence relationship of pedestrian attributes to improve the accuracy of pedestrian attribute recognition; and it adopts an end-to-end training method for network training.
[0031] like Figure 1 As shown, the network structure used in the pedestrian re-identification method of the present invention includes a convolutional feature extraction module, an attribute-guided attention module, and an attribute-dependent learning module. The convolutional feature extraction module is used to extract features from the input image to obtain a high-level feature map; the attribute-guided attention module is used to provide an adaptive attention mechanism related to pedestrian attribute categories; and the attribute-dependent learning module is used to model the relationship between various pedestrian attributes.
[0032] Specifically, the convolutional feature extraction module of this invention uses a pre-trained CNN, removes the last fully connected layer of the CNN, and uses the CNN with the last fully connected layer removed to extract features from the input image to obtain the highest-level feature map. The CNN structure can be arbitrary, such as AlexNet, VGGNet-16, ResNet-50, etc. In this embodiment, ResNet-50 is used, and its structure is as follows... Figure 2-a As shown in the diagram. The first convolutional layer, Conv,64@7×7, indicates that this layer contains 64 convolutional kernels of size 7*7, resulting in a 64-channel output. The second layer, MPool,64@3×3, uses max pooling on the channels, with a pooling window size of 3*3. Subsequently, Conv Block and Identity Block represent specially designed residual modules in the residual network, with their specific structures as shown in the diagram. Figure 2-b and Figure 2-c As shown. The Identity Block is reused continuously; ×2 indicates that this layer is repeated twice. Since the goal is only to obtain the highest-level feature map, the final global average pooling layer and fully connected layer of the residual network have been removed.
[0033] Pedestrian re-identification differs from other classification problems. In typical classification problems, the class numbers in the training and test sets are the same; for example, the training set might have 0-9 classes, and the test set might also have 0-9 classes. However, in pedestrian re-identification, taking Market-1501 as an example, this dataset contains 1501 categories of pedestrian IDs. The training set contains 751 of these, while the test set contains the other 750. Therefore, the class numbers in the training and test sets do not overlap, which is the biggest difference between this and ordinary classification problems. Therefore, the training and testing of the pedestrian re-identification model in this invention are actually divided into two stages. The first stage is the training stage: first, according to the above... Figure 1 The network structure shown uses both ID loss and attribute classification loss to train a model that performs well on both the training and validation sets. The second stage is the testing stage, where the trained model is used for person re-identification. This is done by inputting an image into the trained model and extracting only the feature vector corresponding to the person recognition branch. This vector contains rich semantic information and can serve as a representative of the input image. It is then compared with the feature vector corresponding to each image in the candidate image library (which is also extracted using the same method). A simple Euclidean distance is typically used to measure the distance between the two vectors. The images are then sorted according to the comparison results, with the smallest distance representing the most similar image.
[0034] Simple pedestrian re-identification models use only one ID loss during training, i.e. Figure 1The lower branch of the model classifies 751 categories. However, since pedestrians generally look similar, this ID loss function can only determine pedestrian identity based on overall appearance, which is not very effective and cannot make good use of the detailed features of different pedestrians' bodies. Therefore, later research labeled the dataset with 27 attributes (Market-1501), and each pedestrian was labeled with these 27 attributes. In this way, classifying both pedestrian ID (lower branch) and pedestrian attributes (upper branch) simultaneously has better generalization performance. That is to say, although attribute classification is performed here, it is only added to the pedestrian re-identification model to generate more classifiers. The ultimate goal is still pedestrian re-identification, and the backbone network used is still the same network (ResNet-50).
[0035] Assuming the input image is x, after preprocessing, the resulting image has a size of 224*224. After feature extraction using the ResNet-50 network in this embodiment, the following results are obtained:
[0036] A = CNN ResNet-50 (x;θ)
[0037] Where A is the output feature map; θ is the input parameter.
[0038] The final convolutional layer (conv5_x) outputs a feature map A (the highest layer feature map, without going through the Pool5 pooling layer), with a size of 2048*7*7, where the number of channels is 2048 and the length and width are 7.
[0039] Attribute-guided attention modules, such as Figure 3 As shown, the system includes convolutional kernels with the same number of pedestrian attributes. Each kernel performs convolution on its corresponding attribute branch, merging channels of the multi-channel feature map from the previous layer. The parameters of the convolutional kernels are equivalent to the channel attention parameters during channel merging. Therefore, the convolutional kernel used by each attribute branch is a convolution performed on each attribute, specifically for channel merging to generate an attention map corresponding to that attribute; hence, it is called attribute-guided attention. "Attribute-guided" means that since each attribute branch ultimately maps to its respective attribute label, and the prediction loss is obtained from this label and updated in reverse, the parameter updates of each branch are guided by the corresponding attribute categories.
[0040] Each convolutional kernel is represented as:
[0041] M l =A*f l
[0042] Among them, f lLet L represent the l-th convolutional kernel, where l = 1, 2, ..., L. The number of convolutional kernels L is the same as the number of attributes. Thus, during the convolution process, each filter extracts features for one attribute, resulting in a feature map M for each attribute. l ∈R H×W Because feature extraction is performed on attributes, the feature map corresponding to the attribute is directly related to it, and the semantic information within it corresponds to that attribute. After L convolutions, the feature map corresponding to the attribute is obtained, as follows:
[0043] M = (M 1 M 2 ,...,M L )
[0044] The dimensions of M are L×H×W.
[0045] There are potential interdependencies among pedestrian attributes. For example, "male" often co-occurs with "short hair," and "female" often co-occurs with "dress." Analysis reveals significant co-occurrence relationships among some attributes. This invention uses an attribute dependency learning module to model the dependencies of various pedestrian attributes obtained by an attribute-guided attention module. This module employs an LSTM network to model the dependencies of the feature vectors corresponding to each attribute. LSTM stands for Long Short-Term Memory Network. A unidirectional LSTM can only iterate along one direction of the sequence, thus only utilizing sequence information from a single direction. Therefore, this invention uses a bidirectional LSTM, including a forward and a backward LSTM. Bidirectional LSTM is also a standard practice in the industry, using both forward and backward iterations simultaneously to extract sequence features from both directions. Finally, the input vectors from the two directions are fused in the next layer, effectively fusing features from both directions. Figure 4 As shown, the structure of the LSTM network used in this embodiment is as follows. Figure 1 As shown in the dashed box, it includes a forward LSTM layer and a backward LSTM layer. Assuming there are L pedestrian attributes, both the forward LSTM layer and the backward LSTM layer include L LSTM units. Each attribute feature is input into the corresponding LSTM unit in the forward LSTM layer and the backward LSTM layer, respectively.
[0046] For each LSTM unit, the process from the input attribute attention map to the output vector is as follows: Figure 1 As shown, the process is formally represented as follows:
[0047] i t=σ(W iv V t +W ih h t-1 +b i )
[0048] f t =σ(W fv V t +W fh h t-1 +b f )
[0049] o t =σ(W ov V t +W oh h t-1 +b o )
[0050] c t =f t ⊙c t-1 +i t ⊙tanh(W cv V t +W ch h t-1 +b c )
[0051] h t =o t ⊙c t
[0052] Where σ represents the sigmoid activation function; tanh(·) is the tanh function; ⊙ represents element-wise multiplication; matrix W and vector b are input parameters; f t i t ,o t ,c t These represent the states of the forget gate, input gate, output gate, and memory cell, respectively; h t Let represent the hidden state of the decoder, t represent the current time point, and t-1 represent the previous time point. The above equation describes the computation process of the LSTM network. For simplicity, this computation process is denoted as a function, representing the nonlinear transformation process of the LSTM network on the input vector, with the aim of learning the relationship between the input vectors at different times. To transform the feature map corresponding to the attribute, M = (M... 1 M 2 ,...,M L The input is fed into an LSTM, which stretches it from a two-dimensional matrix into a one-dimensional vector form, resulting in V = (V 1 V 2 ,...,V L ), at this time V l The dimensions are 1×H·W.
[0053] h i =LSTM(V t ,h i-1 )
[0054] Similarly, for the inverse LSTM layer, the inverse hidden unit vector h′ is obtained. i The forward and reverse hidden state vectors are merged to obtain the output vector corresponding to the i-th attribute:
[0055] y i =W s [h i ,h′ i ]+b s
[0056] We use the softmax function to normalize the data and obtain the probability of each attribute value corresponding to the i-th attribute vector:
[0057]
[0058] During training, the cross-entropy loss function is used to calculate the loss for pedestrian attribute classification and ID classification. Assume that there are K pedestrian IDs in the training set, and each pedestrian has L classes of attributes, where the l-th class of attributes has m different attribute values.
[0059] For the attribute, the cross-entropy loss function is:
[0060]
[0061] Where p(y i,j ) represents the predicted probability of the j-th value of the i-th attribute, q(y i,j ) represents the true probability of the j-th value of the i-th attribute, i.e., the 0-1 probability.
[0062] For pedestrian ID, the cross-entropy loss function is:
[0063]
[0064] Where p(y k Let q(k) represent the predicted probability of the k-th ID category, and let q(k) represent the true probability of the k-th ID category.
[0065] The final loss function is the sum of the pedestrian ID loss and the attribute loss:
[0066]
[0067] Here, the parameter λ represents the trade-off between the two losses. The value of λ affects the training effect based on its performance on the validation set, and the best value is selected as the final value.
[0068] This invention improves the semantic representation of attribute feature vectors by adding an attribute-guided attention module to provide attribute-related attention guidance for each attribute. There are obvious co-occurrence relationships between attributes, and the dependency relationship between attributes can use the recognition accuracy of one attribute to assist the recognition of another attribute. LSTM is used to model the dependency relationship of the feature vector corresponding to each attribute to achieve the effect of assisting the attribute and improving the overall attribute recognition accuracy.
Claims
1. A pedestrian re-identification method, characterized in that, The identification method includes the following steps: 1) Perform feature recognition on the acquired image to obtain a high-level feature map of the image; 2) The obtained high-level feature map is processed by an attribute-guided attention module. The attribute-guided attention module includes convolutional kernels with the same number of pedestrian attributes. Each convolutional kernel will perform convolution in its corresponding attribute branch. The multi-channel feature map obtained in step 1) is merged to obtain the feature map of each attribute. The parameter update of each branch is guided by the update of its corresponding attribute category. 3) Based on the feature maps of each attribute, establish the dependency relationship of the feature vector corresponding to each attribute. Use an LSTM network to establish the dependency relationship of the feature vector corresponding to each attribute. The LSTM network includes a forward LSTM layer and a backward LSTM layer. Assuming there are L pedestrian attributes, both the forward LSTM layer and the backward LSTM layer include L LSTM units. Each attribute feature is input into the corresponding LSTM unit in the forward LSTM layer and the backward LSTM layer respectively, and the forward hidden state vector and the backward hidden state vector of each attribute are obtained respectively. The forward and backward hidden state vectors are merged to obtain the output vector corresponding to each attribute. 4) The image to be identified is identified by utilizing the dependency relationship between the feature vectors corresponding to each attribute.
2. The pedestrian re-identification method according to claim 1, characterized in that, Step 1) uses a pre-trained convolutional neural network with the last pooling layer and fully connected layer removed for feature recognition.
3. The pedestrian re-identification method according to claim 1, characterized in that, The size of the convolution kernel in step 2) is 3.
3.
4. The pedestrian re-identification method according to claim 2, characterized in that, The convolutional neural network described uses ResNet-50 with the final global average pooling layer and fully connected layer removed.
5. The pedestrian re-identification method according to any one of claims 1-4, characterized in that, Step 4) In the recognition process, the sum of the losses of attribute classification and pedestrian ID classification is used for training.
6. The pedestrian re-identification method according to claim 5, characterized in that, The cross-entropy loss function used for attribute classification loss is: ； in, This represents the predicted probability of the j-th value of the i-th attribute. This represents the true probability of the j-th value of the i-th attribute.
7. The pedestrian re-identification method according to claim 5, characterized in that, The cross-entropy loss function used for pedestrian ID classification is: ； in, Indicates the first Predicted probabilities for each ID category Indicates the first The true probability of each ID category.