Multi-granularity pedestrian re-identification method fusing attention mechanism

Abstract

The invention discloses a multi-granularity pedestrian re-identification method fusing an attention mechanism. The method comprises the following specific steps: carrying out image enhancement on a disclosed standard pedestrian re-identification data set; resNet50 is adopted as a backbone network, a multi-branch structure is introduced, and a feature map containing different scale information is extracted; in combination with a multi-granularity segmentation module and an attention mechanism, local discrimination information of the feature map is further extracted, diversified feature representations are acquired, and coordination and unification of features are realized; supervised training is carried out on the model in a combined learning mode, and more comprehensive feature description is obtained; and extracting features of the pedestrian image and the image library image by using the trained network and carrying out similarity matching. The invention provides a multi-granularity pedestrian re-identification method fused with an attention mechanism, and aims to guide a neural network algorithm to pay attention to pedestrian discriminative features and improve the accuracy of pedestrian identification.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and computer vision technology, and specifically relates to a multi-granularity pedestrian re-identification method integrating an attention mechanism. Background Art

[0002] Pedestrian re-identification is a technology that uses computer vision technology to retrieve specific pedestrians from images or videos taken by surveillance cameras. Pedestrian images are difficult to identify in surveillance videos because of large changes in viewing angle and lighting, inconsistent pedestrian postures, and partial occlusion of pedestrians. This makes pedestrian re-identification technology a research hotspot in the current fields of computer vision and machine learning technology, and has important application value in fields such as intelligent visual monitoring and driverless cars. In recent years, with the development of convolutional neural networks (CNNs), pedestrian re-identification methods based on deep learning have made significant progress. Compared with traditional manual feature extraction methods, they have more powerful feature extraction capabilities and better automatic learning capabilities.

[0003] The current deep learning pedestrian re-identification model mainly adopts global feature representation learning and local feature representation learning. The global feature model is fast in the learning and reasoning stage, but it often ignores the detailed features of pedestrians. On the contrary, the local feature representation learning method can focus on details such as pedestrian skeleton, posture, and body parts, making it more robust in complex scenes. Although these methods have achieved some results and can deal with problems such as pedestrian posture misalignment and pedestrian occlusion to a certain extent, local feature representation learning tends to ignore the correlation between adjacent features, and some small-scale information is also easily ignored as background noise.

[0004] Therefore, how to effectively learn the semantic relationship between adjacent feature information and at the same time make the network focus on multi-scale image information to improve the accuracy of pedestrian re-identification is an urgent problem that researchers in the field of pedestrian re-identification need to solve. Summary of the invention

[0005] In order to address the shortcomings of existing pedestrian re-identification technology, this paper proposes a multi-granularity pedestrian re-identification method that integrates attention mechanism, aiming to obtain richer detail information in pedestrian images through multi-granularity structure and attention mechanism, and at the same time adopts joint loss function to jointly supervise optimization and improve the learning ability of the network.

[0006] The technical means used in this article are as follows:

[0007] The multi-granularity person re-identification method integrating the attention mechanism includes the following steps:

[0008] Step 1: Obtain original pedestrian image data, build a pedestrian re-identification dataset and preprocess the dataset;

[0009] Step 2: Use ResNet50 as the backbone network for person re-identification, design a multi-scale feature extraction module, and obtain the multi-scale features of pedestrians;

[0010] Step 3: Design a multi-granularity segmentation module to focus on the feature information between adjacent features of pedestrians;

[0011] Step 4: Use the attention mechanism to extract the discriminative local information of pedestrians;

[0012] Step 5: Construct loss functions for different branch networks corresponding to different granularity features, and then jointly train different branch networks;

[0013] Step 6: Use the network trained in step 5 to perform similarity matching between the pedestrian image to be queried and the pedestrian library image, and use the cumulative matching characteristic curve and query average curve to quantify the model performance.

[0014] Furthermore, the data preprocessing in step 1 includes the following steps:

[0015] Step 1.1: Convert the image resolution in the person re-identification dataset to 384×128;

[0016] Step 1.2: Perform image augmentation using random flipping, random cropping, and random erasing.

[0017] Furthermore, the step 2 comprises the following steps:

[0018] Step 2.1: Using the convolutional neural network ResNet50 as the network basis, input the pedestrian image into the ResNet50 network;

[0019] Step 2.2: Modify the downsampling phase of ResNet50 and divide the output after the third layer of ResNet50 into three branches. Each branch obtains feature maps of different scales to obtain pedestrian information of different scales.

[0020] Furthermore, the step 3 comprises the following steps:

[0021] Step 3.1: Perform multi-granularity segmentation on the feature maps extracted by the three branches in step 2. Given the current input feature map size is X h1 ∈R C×H×W , where C, H, and W represent the number of channels, length, and width of the feature map, respectively;

[0022] Step 3.2: Assume that the number of divisions is n, and perform n equal divisions on the feature map at the spatial level. The size of the finest feature map is X h2 ∈R C×(H / n)×W ;

[0023] Step 3.3: In order to obtain multi-granularity feature representation, the number of splits is reset to n-1, and the feature map is re-split to a size of X h3 ∈R C×(H / (n-1))×W ;

[0024] And so on, until the number of splits n=1, at which time no splitting operation is performed, which means that a feature map containing features from coarse-grained to fine-grained has been obtained.

[0025] Furthermore, step 4 includes the following steps:

[0026] Step 4.1: The attention mechanism used is composed of a spatial attention mechanism and a channel attention mechanism in parallel. The parallel structure enables the model to focus on both the channel and spatial levels while avoiding the increase in computational cost caused by the series connection of the attention mechanisms.

[0027] Step 4.2: Construct a spatial attention mechanism to capture important information of features in the spatial dimension. The specific process is as follows:

[0028] Step 4.2.1: Input feature X h ∈R C×H×W After linear transformation, we get Q∈R (hW)×C , K∈R (hW)×C , and V∈R (hW)×C ;

[0029] Step 4.2.2: Transform the feature Q to obtain Q r ∈R C×(hW) , and Q r Perform matrix multiplication with K and normalize through a Softmax layer to obtain the attention score F representing spatial information. s ∈R (hw)×(hw) , the expression is:

[0030] F s =Softmax(Q r ×K);

[0031] Step 4.2.3: After collecting the attention scores, use them to perform weighted summation with the feature V and obtain D∈R (hW)×C , to increase the weight of discriminative features and learn spatial attention information. Finally, in order to embed these features into the subsequent network, D is linearly transformed and adjusted back to the input shape to obtain the feature O with spatial attention. S ∈R C×h×W .

[0032] Step 4.3: Construct a channel attention mechanism to focus on channels with stronger discriminative features and enhance the feature expression of these channels. The specific process is as follows;

[0033] Step 4.3.1: Input feature X h ∈R C×H×W Perform average pooling and maximum pooling operations respectively, compress the height and width of the feature to 1, and flatten the feature map in each channel into a one-dimensional vector, that is, the features after average pooling and maximum pooling are X avg ∈R C×1×1 , X max ∈R C×1×1 ;

[0034] Step 4.3.2: Pass the obtained features through the fully connected layer, ReLu activation layer, fully connected layer, and Sigmoid activation layer to obtain the weight parameter F with channel attention mechanism c ∈R C×1×1 , the expression is:

[0035] F c =S[L2(R(L1(x avg )))+L2(R(L1(x max )))]

[0036] Where S is the Sigmoid activation function, L1 and L2 are fully connected layers of different sizes, and R is the ReLu activation function;

[0037] Step 4.3.3: Expand the weight parameter F in both height and width dimensions c , and the input feature X h Perform element-by-element multiplication to obtain the feature O with channel attention c ∈R C×h×W .

[0038] Furthermore, the step 5 comprises the following steps:

[0039] Step 5.1: Use average pooling and maximum pooling operations to process all multi-granularity feature maps with attention mechanism applied, and use Add operation to fuse the two pooled features. After 1x1 convolution layer, normalization and ReLu activation function, reduce the dimension of feature map to obtain feature vector for training;

[0040] Step 5.2: Design the cross entropy loss L cls , respectively learn the features extracted from feature vectors of different granularities, compare the relationship between network recognition results and real results, and help the recognition network to correctly extract pedestrian features. The cross entropy loss L cls The calculation formula is:

[0041]

[0042] Among them, M is the number of feature vectors, N is the total number of pedestrians in the training sample, and k i is the i-th pedestrian feature vector, q(k i ) is the predicted probability of pedestrian identity, p′(k i ) To improve the distribution of pedestrian image labels, label smoothing is used to prevent overfitting;

[0043] Furthermore, p′(k i ) is:

[0044]

[0045] Among them, p(k i ) is the original pedestrian image label distribution, ε is a hyperparameter, and y is the true label of the pedestrian image;

[0046] Step 5.3: Design the hard sample triplet loss L cls , respectively learn the features extracted from feature vectors of different granularities, measure the relationship between features and discriminative features, and encourage the network to focus on the discriminative features of pedestrians. The loss function L for the difficult sample triplet tri The calculation formula is:

[0047]

[0048] Among them, α is a hyperparameter, M is the number of feature vectors, P is the number of labels of pedestrian images, K is the number of pedestrian images corresponding to each label, and F a 、F p 、F n They are the feature vector representations of the original sample, positive sample, and negative sample respectively;

[0049] Step 5.4: The joint function expression of cross entropy loss and hard sample triple loss is:

[0050] L=L cls +L tri ;

[0051] Furthermore, the step 6 comprises the following steps:

[0052] Step 6.1: Input the query pedestrian image and the gallery pedestrian image into the trained network for recognition test;

[0053] Step 6.2: For the query image, during the test, feature vectors of different granularities of different branches are fused in the channel dimension to obtain the features of the query image;

[0054] Step 6.3: For the gallery image, during the test, the feature vectors of different granularities of different branches are fused in the channel dimension to obtain the features of the gallery image;

[0055] Step 6.4: Calculate the similarity between the query image and the gallery image, using the cumulative matching characteristic curve (CMC) and query average precision (mAP) as evaluation criteria, and return the matching result.

[0056] The advantages of the present invention are:

[0057] (1) From the perspective of multi-granularity, the multi-granularity segmentation method of the present invention avoids the influence of horizontal segmentation on the local features of pedestrians to a certain extent compared with other feature extraction methods of local feature segmentation, and ensures that the information contained in the obtained pedestrian features is more comprehensive; (2) By introducing the attention mechanism, the influence of pedestrian posture misalignment and pedestrian occlusion resulting in the loss of pedestrian discriminant features in pedestrian images is reduced, thereby improving the accuracy of re-identification; (3) From the perspectives of representation learning and metric learning, the joint loss function is used to supervise the network learning, so that the network can better extract the discriminant features of pedestrians. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a network architecture diagram of a multi-granularity person re-identification method that integrates an attention mechanism in an embodiment of the present invention.

[0059] Figure 2 This is a diagram of the multi-scale feature extraction network architecture in an embodiment of the present invention.

[0060] Figure 3 This is a diagram of a multi-granularity segmentation network architecture in an embodiment of the present invention. Specific implementation methods

[0062] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention, but it should be understood that the protection scope of the present invention is not limited by the specific implementation methods. According to the embodiments of the present invention, all other embodiments obtained by professionals in this field without performing innovative work should be included in the protection scope of the present invention.

[0063] like Figure 1 As shown, this embodiment provides a multi-granularity person re-identification method integrating an attention mechanism, including the following steps:

[0064] Step 1: Obtain original pedestrian image data, build a pedestrian re-identification dataset and preprocess the dataset;

[0065] Step 2: Use ResNet50 as the backbone network for person re-identification, design a multi-scale feature extraction module, and obtain the multi-scale features of pedestrians;

[0066] Step 3: Design a multi-granularity segmentation module, combine contextual semantic clues, and focus on the important information between adjacent features of pedestrians;

[0067] Step 4: Design an attention mechanism module to extract the discriminative local information of pedestrians;

[0068] Step 5: Construct loss functions for different branch networks corresponding to different granularity features, and then jointly train different branch networks;

[0069] Step 6: Use the network trained in step 5 to perform similarity matching between the pedestrian image to be queried and the pedestrian library image. Use the cumulative matching characteristic curve and the query average curve to quantify the model performance. Sort the library images matched by the model from large to small according to similarity. Use Rank-n to represent the probability of the correct matching result in the first n images.

[0070] Furthermore, the data preprocessing in step 1 includes the following steps:

[0071] Step 1.1: Convert the image resolution of the original person re-identification dataset to 384×128;

[0072] Step 1.2 uses random flipping, random cropping and random erasing to enhance the image;

[0073] Furthermore, if Figure 2 As shown, step 2 includes the following steps:

[0074] Step 2.1: Using the convolutional neural network ResNet50 as the network basis, input the pedestrian image into the ResNet50 network to extract the high-dimensional semantic feature information of the pedestrian image;

[0075] Step 2.2: Modify the downsampling link of ResNet50, divide the output after the third layer of ResNet50 into three branches, each branch obtains feature maps of different scales. The first branch uses the network structure of ResNet50, directly learns the global feature representation, and obtains a small-size feature map; the second branch deletes the fifth-layer downsampling of ResNet50, retains the fourth-layer downsampling link, and obtains a medium-sized feature map. The third branch deletes the fourth and fifth-layer downsampling links to obtain a large-size feature map, which retains more spatial information than the medium-sized feature map.

[0076] Furthermore, if Figure 2 As shown, step 3 includes the following steps:

[0077] Step 3.1: Perform multi-granularity segmentation on the feature maps extracted by the three branches in step 2. Given the current input feature map size is X h1 ∈RC×H×W , where C, H, and W represent the number of channels, length, and width of the feature map, respectively;

[0078] Step 3.2: Assume that the number of divisions is n, and perform n equal divisions on the feature map at the spatial level. The size of the finest feature map is X h2 ∈R C×(H / n)×W .

[0079] Step 3.3: In order to obtain multi-granularity feature representation, the number of splits is reset to n-1, and the feature map is re-split to a size of X h3 ∈R C×(H / (n-1))×W ;

[0080] And so on, until the number of splits n = 1, at which point no splitting operation is performed, which means that a feature map containing features from coarse-grained to fine-grained has been obtained;

[0081] For the small-size feature map branch that directly uses the ResNet50 structure, n is set to 1, and no slicing operation is performed to retain the global feature information of the backbone network; for the branches of medium-size feature maps and large-size feature maps, multi-granularity slicing methods of n=2 and n=3 are used respectively, which can not only extract finer-grained features, but also avoid introducing a large amount of background noise due to too many slicing numbers.

[0082] Furthermore, step 4 includes the following steps:

[0083] Step 4.1: Input the multi-granularity feature map obtained in step 3 into the attention mechanism module. The attention mechanism module is composed of a spatial attention mechanism and a channel attention mechanism in parallel. The parallel structure enables the model to pay attention to both the channel and spatial levels while avoiding the increase in computational cost caused by the series connection of the attention mechanism.

[0084] Step 4.2: Construct a spatial attention mechanism to capture important information of features in the spatial dimension;

[0085] Step 4.2.1: Input feature X h ∈R C×H×W After linear transformation, we get Q∈R (hW)×C , K∈R (hW)×C , and V∈R (hW)×C ;

[0086] Step 4.2.2: Transform the feature Q to obtain Q r ∈R C×(hW) , and Q r Perform matrix multiplication with K and normalize through a Softmax layer to obtain the attention score F representing spatial information. s ∈R (hw)×(hw) , the expression is:

[0087] F s =Softmax(Q r ×K);

[0088] Step 4.2.3: After collecting the attention scores, use them to perform weighted summation with the feature V and obtain D∈R (hW)×C , to increase the weight of discriminative features and learn spatial attention information. Finally, in order to embed these features into the subsequent network, D is linearly transformed and adjusted back to the input shape to obtain the feature O with spatial attention. S ∈R C×h×W .

[0089] Step 4.3: Construct a channel attention mechanism to focus on channels with stronger discriminative features and enhance the feature expression of these channels;

[0090] Step 4.3.1: Input feature X h ∈R C×H×W Perform average pooling and maximum pooling operations respectively, compress the height and width of the feature to 1, and flatten the feature map in each channel into a one-dimensional vector, that is, the features after average pooling and maximum pooling are X avg ∈R C×1×1 , X max ∈R C×1×1 ;

[0091] Step 4.3.2: Pass the obtained features through the fully connected layer, ReLu activation layer, fully connected layer, and Sigmoid activation layer to obtain the weight parameter F with channel attention mechanism c ∈R C×1×1 , the expression is:

[0092] F c =S[L2(R(L1(x avg )))+L2(R(L1(x max )))];

[0093] Where S is the Sigmoid activation function, L1 and L2 are fully connected layers of different sizes, and R is the ReLu activation function;

[0094] Step 4.3.3: Expand the weight parameter F in both height and width dimensions c , and the input feature X h Perform element-by-element multiplication to obtain the feature O with channel attention c ∈R C×h×W .

[0095] Furthermore, the step 5 comprises the following steps:

[0096] Step 5.1: Use average pooling and maximum pooling operations to process all multi-granularity feature maps with attention mechanism applied, and use Add operation to fuse the two pooled features. After 1x1 convolution layer, normalization and ReLu activation function, reduce the dimension of feature map to obtain feature vector for training;

[0097] Step 5.2: Design the cross entropy loss L cls , respectively learn the features extracted from feature vectors of different granularities, compare the relationship between network recognition results and real results, and help the recognition network to correctly extract pedestrian features. The cross entropy loss L cls The calculation formula is:

[0098]

[0099] Among them, M is the number of feature vectors, N is the total number of pedestrians in the training sample, and k i is the i-th pedestrian feature vector, q(k i ) is the predicted probability of pedestrian identity, p′(k i ) To improve the distribution of pedestrian image labels, label smoothing is used to prevent overfitting;

[0100] Furthermore, p′(k i ) is:

[0101]

[0102] Among them, p(k i ) is the original pedestrian image label distribution, ε is a hyperparameter, and y is the true label of the pedestrian image;

[0103] Step 5.3: Design the hard sample triplet loss L cls , respectively learn the features extracted from feature vectors of different granularities, measure the relationship between features and discriminative features, and encourage the network to focus on the discriminative features of pedestrians. The loss function L for the difficult sample triplet tri The calculation formula is:

[0104]

[0105] Among them, α is a hyperparameter, M is the number of feature vectors, P is the number of labels of pedestrian images, K is the number of pedestrian images corresponding to each label, and F a 、F p 、F n They are the feature vector representations of the original sample, positive sample, and negative sample respectively;

[0106] Step 5.4: The joint function expression of cross entropy loss and hard sample triple loss is:

[0107] L=L cls +L tri .

[0108] Furthermore, the step 6 comprises the following steps:

[0109] Step 6.1: Input the query pedestrian image and the gallery pedestrian image into the trained network for recognition test;

[0110] Step 6.2: For the query image, during the test, feature vectors of different granularities of different branches are fused in the channel dimension to obtain the features of the query image;

[0111] Step 6.3: For the gallery image, during the test, the feature vectors of different granularities of different branches are fused in the channel dimension to obtain the features of the gallery image;

[0112] Step 6.4: Calculate the similarity between the query image and the gallery image, use the cumulative matching characteristic curve (CMC) and query average precision (mAP) as evaluation criteria, and report the matching results in the form of mAP and Rank-1.

[0113] The mAP values ​​of this embodiment on the Market-1501, DukeMTMC-ReID and CUHK-03 test datasets reached 88.42%, 78.86% and 76.70% respectively, and the Rank-1 accuracy reached 95.82%, 88.83% and 78.94% respectively, showing excellent performance. It can be seen that the multi-granularity segmentation algorithm and the method of using the channel and spatial attention mechanism in parallel proposed by the present invention effectively improve the ability of pedestrian feature extraction, which fully verifies the robustness and superiority of this embodiment.

[0114] Parts or structures not specifically described in the present invention may adopt existing technologies or existing products and will not be described in detail here.

[0115] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-granularity person re-identification method integrating attention mechanism, characterized by: The following steps are involved: Step 1: Obtain original pedestrian image data, build a pedestrian re-identification dataset and preprocess the dataset; Step 2: Use ResNet50 as the backbone network for person re-identification, design a multi-scale feature extraction module, and obtain the multi-scale features of pedestrians; Step 3: Design a multi-granularity segmentation module to focus on the feature information between adjacent features of pedestrians; Step 4: Use the attention mechanism to extract the discriminative local information of pedestrians; Step 6: Use the network trained in step 5 to perform similarity matching between the pedestrian image to be queried and the pedestrian library image, and use the cumulative matching characteristic curve and query average curve to quantify the model performance.

2. The multi-granularity person re-identification method integrating attention mechanism according to claim 1 is characterized in that: The step 1 comprises the following steps: Step 1.1: Convert the image resolution in the person re-identification dataset to 384×128; Step 1.2: Perform image augmentation using random flipping, random cropping, and random erasing.

3. The multi-granularity person re-identification method integrating attention mechanism according to claim 1 is characterized in that: The step 2 comprises the following steps: Step 2.1: Using the convolutional neural network ResNet50 as the network basis, input the pedestrian image into the Res Net50 network; Step 2.2: Modify the downsampling phase of ResNet50 and divide the output after the third layer of ResNet50 into three branches. Each branch obtains feature maps of different scales to obtain pedestrian information of different scales.

4. The multi-granularity person re-identification method integrating attention mechanism according to claim 1 is characterized in that: The step 3 comprises the following steps: Step 3.1: Perform multi-granularity segmentation on the feature maps extracted by the three branches in step 2. Given the current input feature map size is X h1 ∈R C×H×W , where C, H, and W represent the number of channels, length, and width of the feature map, respectively; Step 3.2: Assume that the number of divisions is n, and perform n equal divisions on the feature map at the spatial level. The size of the finest feature map is X h2 ∈R C×(H / n)×W ; Step 3.3: In order to obtain multi-granularity feature representation, the number of splits is reset to n-1, and the feature map is re-split to a size of X h3 ∈R C×(H / (n-1))×W ; And so on, until the number of splits n=1, at which time no splitting operation is performed, which means that a feature map containing features from coarse-grained to fine-grained has been obtained.

5. The multi-granularity person re-identification method integrating attention mechanism according to claim 1 is characterized in that: The step 4 comprises the following steps: Step 4.1: Input the multi-granularity feature map obtained in step 3 into the attention mechanism module. The attention mechanism used is composed of the spatial attention mechanism and the channel attention mechanism in parallel. The parallel structure enables the model to pay attention to the channel and spatial levels at the same time, while avoiding the increase in computational cost caused by the series connection of the attention mechanism; Step 4.2: Construct a spatial attention mechanism to capture important information of features in the spatial dimension. The specific process is as follows: Step 4.2.1: Input feature X h ∈R C×H×W After linear transformation, we get Q∈R (hW)×C , K∈R (hW)×C , and V∈R (hW)×C ; Step 4.2.2: Transform the feature Q to obtain Q r ∈R C×(hW) , and Q r Perform matrix multiplication with K and normalize through a Softmax layer to obtain the attention score F representing spatial information. s ∈R (hw)×(hw) , the expression is: F s =Softmax(Q r ×K); Step 4.2.3: After collecting the attention scores, use them to perform weighted summation with the feature V and obtain D∈R (hW)×C , to increase the weight of discriminative features and learn spatial attention information. Finally, in order to embed these features into the subsequent network, D is linearly transformed and adjusted back to the input shape to obtain the feature O with spatial attention. S ∈R C×h×W . Step 4.3: Construct a channel attention mechanism to focus on channels with stronger discriminative features and enhance the feature expression of these channels. The specific process is as follows: Step 4.3.1: Input feature X h ∈R C×H×W Perform average pooling and maximum pooling operations respectively, compress the height and width of the feature to 1, and flatten the feature map in each channel into a one-dimensional vector, that is, the features after average pooling and maximum pooling are X avg ∈R C×1×1 , X max ∈R C×1×1 ; Step 4.3.2: Pass the obtained features through the fully connected layer, ReLu activation layer, fully connected layer, and Sigmoid activation layer to obtain the weight parameter F with channel attention mechanism c ∈R C×1×1 , the expression is: F c =S[L2(R(L1(x avg )))+L2(R(L1(x max )))]; Among them, S is the Sigmoid activation function, L1 and L2 are fully connected layers of different sizes, and R is the ReLu activation function; Step 4.3.3: Expand the weight parameter F in both height and width dimensions c , and the input feature X h Perform element-by-element multiplication to obtain the feature O with channel attention c ∈R C×h×W .

6. The multi-granularity person re-identification method integrating attention mechanism according to claim 1 is characterized in that: The step 5 comprises the following steps: Step 5.1: Use average pooling and maximum pooling operations to process all multi-granularity feature maps with attention mechanism applied, and use Add operation to fuse the two pooled features. After 1x1 convolution layer, normalization and ReLu activation function, reduce the dimension of feature map to obtain feature vector for training; Step 5.2: Design the cross entropy loss L cls , respectively learn the features extracted from feature vectors of different granularities, compare the relationship between network recognition results and real results, and help the recognition network to correctly extract pedestrian features. The cross entropy loss L cls The calculation formula is: Among them, M is the number of feature vectors, N is the total number of pedestrians in the training sample, and k i is the i-th pedestrian feature vector, q(k i ) is the predicted probability of pedestrian identity, p′(k i ) To improve the distribution of pedestrian image labels, label smoothing is used to prevent overfitting; Furthermore, p′(k i ) is: Among them, p(k i ) is the original pedestrian image label distribution, ε is a hyperparameter, and y is the true label of the pedestrian image; Step 5.3: Design the hard sample triplet loss L cls , respectively learn the features extracted from feature vectors of different granularities, measure the relationship between features and discriminative features, and encourage the network to focus on the discriminative features of pedestrians. The loss function L for the difficult sample triplet tri The calculation formula is: Among them, α is a hyperparameter, M is the number of feature vectors, P is the number of labels of pedestrian images, K is the number of pedestrian images corresponding to each label, and F a 、F p 、F n They are the feature vector representations of the original sample, positive sample, and negative sample respectively; Step 5.4: The joint function expression of cross entropy loss and hard sample triple loss is: L=L cls +L tri .

7. The multi-granularity person re-identification method integrating attention mechanism according to claim 1 is characterized in that: The step 6 comprises the following steps: Step 6.1: Input the query pedestrian image and the gallery pedestrian image into the trained network for recognition test; Step 6.2: For the query image, during the test, feature vectors of different granularities of different branches are fused in the channel dimension to obtain the features of the query image; Step 6.3: For the gallery image, during the test, the feature vectors of different granularities of different branches are fused in the channel dimension to obtain the features of the gallery image; Step 6.4: Calculate the similarity between the query image and the gallery image, using the cumulative matching characteristic curve and the query average precision as evaluation criteria, and return the matching result.

Images