Pedestrian re-identification method based on Transform and space attention mechanism

Abstract

The invention relates to the technical field of pedestrian re-identification, in particular to a pedestrian re-identification method based on Transform and a space attention mechanism. The method provided by the invention comprises the following steps of S1, selecting a Transform encoder feature extractor; s2, four space attention modules are enhanced; four space attention modules are introduced for the head, the trunk, the lower limbs and the feet, and features of the global situation, the head, the upper limbs, the lower limbs and the feet are defined; and S3, improving a training method and a loss function. According to the method, the precision and robustness of pedestrian re-identification are improved by combining the strong feature extraction capability of Transform and a space attention mechanism; compared with the prior art, the method provided by the invention has better recognition accuracy on mAPs and Rank-1 on a Market1501 data set, a DukeMTMC data set and an OccludDuke data set, and has a certain application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian re-identification, and in particular to a pedestrian re-identification method based on Transformer and spatial attention mechanism. Background Art

[0002] In the field of pedestrian re-identification in occluded scenes, the main challenges currently faced include: it is difficult to effectively align pedestrian images, the noise interference caused by occluders is significant, a large amount of pedestrian feature information is lost due to occluded areas, and the problem of handling inconsistent image sizes. In order to cope with these technical bottlenecks, industry researchers are actively exploring solutions. Taking the Chinese patent CN113705348A as an example, the patent proposes an innovative pedestrian re-identification method that cleverly integrates multiple attention mechanisms. Specifically, it adopts a stacked multi-layer convolutional neural network architecture and integrates channel attention and spatial attention mechanisms to finely capture the multi-dimensional features of pedestrians. Through deep training and learning, this method can extract richer and more efficient high-dimensional feature representations from complex pedestrian images, thereby improving the accuracy of pedestrian re-identification to a certain extent. However, although this method has shown certain potential, it still faces the problem of high computational complexity and the need to improve recognition accuracy in some cases. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a pedestrian re-identification method based on Transformer and spatial attention mechanism.

[0004] The technical solution adopted by the present invention is as follows: A pedestrian re-identification method based on Transformer and spatial attention mechanism includes the following steps: S1. Select the Transformer encoder feature extractor, including the following steps: S11. First, divide the image into H×W image blocks x. The step length of each image block is S and the size is P. Then the formula for the number of image blocks N is: (1) S12, the image block is flattened into a two-dimensional matrix, and after linear mapping, it is input into the Transformer encoder to obtain the feature F. The feature F formula is: (2) in, They are numbered 0, 1, ..., N image blocks respectively; S13, feature F is added through cls_token, and a set of position embedding and edge information embedding SIE is added to obtain a new feature , The formula is: (3) in, Position embedding is used to ensure that the encoder can recognize the input image blocks in the correct order; The image block added to cls_token is used to ensure the accuracy of the classification task; SIE is edge information embedding, which is used to mark different cameras; S14. Use multi-head attention mechanism to process features , to capture the features The dependencies in are: (4) (5) (6) (7) in, is the multi-head attention mechanism function; Q is the query matrix, K is the key matrix, and V is the value matrix; d is the dimension of the key matrix, which is used to scale the dot product to stabilize function; , , They are the transformation matrices of the query, key, and value of the feature respectively; S15. Introduce the feedforward neural network FFN to process the output of the multi-head attention mechanism to obtain the feature , The formula is: (8); S2: Strengthen a global attention and four spatial attention modules, including the following steps: S21. First, a global attention module is introduced in the global branch. At the same time, four spatial attention modules are introduced for the head, torso, lower limbs, and feet. The features of the global, head, upper limbs, lower limbs, and feet are defined as x respectively. global , x head , x upper ,x lower , x shoes ; S22, using convolutional layers and average pooling layers to generate global and four local area masks of the pedestrian, the corresponding masks are global_mask, head_mask, upper_mask, lower_mask, shoes_mask, so as to highlight the unobstructed parts of the pedestrian and minimize noise interference; S23, the matrix output by Transformer encoder Each element in is multiplied by each element in the five mask matrices to obtain the expressions of the five features, which are as follows: (9) (10) S3. Improve the training method and loss function, including the following steps: S31. Improve the training method by splicing the global feature with the four local features after global mask processing to obtain the local splicing function , The formula is: (11) Output five mask functions simultaneously , The formula is: (12) S32. Use triplet loss and ID loss to adjust the parameters of the model, denoted as L global , the formula is: (13) in, The loss is triplet loss, and the ID loss is cross entropy loss; S33, using BCELoss as the semantic loss to train the mask function of the global and four local features, denoted as L mask , the formula is: (14) in, The loss is semantic loss; S34. Let the total loss function be L, and the formula is: (15).

[0005] This technical solution improves the accuracy and robustness of pedestrian re-identification by combining the powerful feature extraction capability of Transformer and the spatial attention mechanism. Specifically, the Encoder part of Transformer is used to extract image features; the image is divided into multiple image blocks (patches), each of which is an element in a sequence, and the Transformer's ability to process sequence data is used to process the image; the image is evenly divided into multiple image blocks, and the size and step size of each image block are fixed to ensure the uniform distribution and efficient processing of image information; the image blocks are flattened into a two-dimensional matrix and converted into a dimension suitable for Transformer processing through linear mapping (usually a fully connected layer); cls_token, position embedding (pos_embedding) and edge information embedding (SIE) are added to enable Transformer to process sequence order information (position embedding), classification tasks (cls_token) and cross-camera information (edge ​​information embedding); through the multi-head attention mechanism, Transformer can capture the complex dependencies between image blocks for understanding contextual information in the image; the feedforward neural network (FFN) further processes the output of the multi-head attention mechanism to enhance the model's nonlinear representation ability; the spatial attention mechanism is introduced to improve the quality of feature representation by focusing on key areas in pedestrian images (such as head, torso, lower limbs, feet); The output of the encoder is multiplied by the mask, which is actually a spatial weighting on the feature map, strengthening the information of important areas and weakening unimportant areas; the model performance is optimized by improving the training method and loss function; the global feature and the four local features are spliced ​​together, and the global and local information are used at the same time to improve the expressiveness of the features; the model is trained using triplet loss and ID loss. The triplet loss is used to learn the relative distance between features, making the features of samples of the same type closer and the features of samples of different types farther apart; the ID loss is used for classification tasks, and the model learns the unique features of each pedestrian; the mask function is trained using BCELoss (binary cross entropy loss) to ensure that the generated mask can accurately reflect the important areas in the image; the above loss functions are combined into a total loss function, and the model parameters are optimized by minimizing this loss function.

[0006] In addition, the pedestrian re-identification method based on Transformer and spatial attention mechanism proposed in the present invention also has the following additional technical features: According to an embodiment of the present invention, in step S2, the global attention module includes a channel attention module and a spatial attention module, and the feature It is first processed by a channel attention module and then by a spatial attention module.

[0007] In this technical solution, the channel attention module and the spatial attention module are used in series, so that the model can perform feature enhancement while considering both the channel dimension and the spatial dimension. First, the channel attention module is used to screen out channels that are more critical to the task; then, in the spatial attention module, the key areas in the image are further screened for these key channels. The combined use significantly improves the model's feature extraction capability and task performance.

[0008] According to one embodiment of the present invention, in step S3, the improved training method and loss function are run in the training environment of the NIVIDA 3090Ti GPU equipped with the pytorch framework, and the model is trained using the gradient descent SGD optimizer; the mask of the training set is obtained through JPPNet, and the hole convolution module and ResNet-101 are used as feature extractors to extract image features, and a hole pooling pyramid module ASPP is introduced as the output layer for segmenting pedestrian images.

[0009] In this technical solution, PyTorch is a widely used deep learning library that provides rich APIs to build and train neural networks. The dynamic graph feature of PyTorch makes model development and debugging more flexible; NVIDIA3090 Ti GPU: It has powerful computing power and can accelerate the training process of neural networks. Through libraries such as CUDA and cuDNN, PyTorch makes full use of the parallel computing power of GPU; SGD is a commonly used optimization method used to update the weights of the model during training; By calculating the gradient of the loss function with respect to the model parameters and updating the parameters in the opposite direction of the gradient to minimize the loss function, SGD has the advantages of high computational efficiency and easy implementation, but it also leads to fluctuations and slow convergence during training; SGD combines strategies such as momentum and learning rate decay to improve its performance, such as SGD with Optimizers such as Momentum or Adam are used to make the model converge to the optimal solution faster and reduce fluctuations during training; JPPNet is a neural network specifically used to generate image masks (such as segmentation masks) to guide the model's training process, especially in tasks that require attention to specific areas in the image (such as pedestrians). Masks are used for a variety of purposes, such as data enhancement, attention guidance, or calculation of loss functions. In the pedestrian re-identification task, masks are used to highlight pedestrian areas, thereby helping the model to better learn the characteristics of pedestrians; the hole convolution module increases the receptive field by inserting "holes" between the elements of the convolution kernel while keeping the number of parameters and the amount of computation unchanged, which is used to help a wider range of contextual information; ResNet-101 solves the problem of gradient disappearance or gradient explosion in deep network training by introducing residual connections. ResNet-101 is a variant in the ResNet series with 101 convolutional layers that can extract rich image features; the hole pooling pyramid module (ASPP) applies Use dilated convolutions and global average pooling with different sampling rates to capture multi-scale contextual information, which are merged together through concatenation operations. The output of ASPP is used to generate the final segmentation result. The loss function needs to consider multiple aspects, such as the discriminability of feature representation, intra-class compactness, and inter-class separability. In addition to the common triplet loss and cross entropy loss, other loss functions (such as semantic loss, mask loss, etc.) are introduced to optimize the performance of the model. During the training process, the model will update its parameters according to the gradient of the loss function. By continuously iterating the samples on the training set, the model gradually learns how to map the input image to the target output (such as the feature representation of pedestrians or the segmentation mask).

[0010] According to one embodiment of the present invention, in step S32, ID loss is used for the model to learn the identity features of pedestrians, and Triplet loss is used to help the model process similarities and differences between pedestrians; By jointly optimizing these two loss functions, a more accurate and robust person re-identification model is obtained; The size of each batch is set to 64, and 120 epochs are trained on all three datasets; Each input image is resized to 256×128; The feature extractor uses 12 Transformer encoders and the weight decay is set to 1e -4 ; The hyperparameters in the loss function are set to 1; Get the relationship curve between the loss function value and epoch during the training process.

[0011] By optimizing ID loss and Triplet loss simultaneously, this technical solution enables the model to learn how to effectively measure the similarities and differences between identity features while learning identity features, so that the model can more accurately identify pedestrians in complex and changing scenes; each batch is set to 64, which means that in each training iteration, the model will process 64 pictures at the same time; 120 epochs are trained on all three datasets, which means that the entire dataset will be fully traversed 120 times; the input image is resized to 256×128, which is a relatively small size used to reduce the computational workload and memory requirements of the model, and also requires the model to make more effective use of limited pixel information; 12 Transformer encoders are used as feature extractors, indicating that the model uses the powerful ability of Transformer to process image data; Transformer was originally designed for natural language processing tasks and is used by the model to capture long-distance dependencies in images, thereby improving the quality of feature representation; weight decay is set to 1e -4It is a regularization strategy used to prevent model overfitting and reduce the complexity of the model by penalizing weights. The hyperparameter in the loss function is set to 1, which is the margin value in the Triplet loss or other parameters related to the loss function. During training, the loss function value usually decreases gradually with the increase of epochs. However, due to the joint optimization of ID loss and Triplet loss, the changing trends of the two loss function values ​​are not completely consistent. At some stages, the value of one loss function will temporarily rise in exchange for a larger decrease in the value of another loss function. The loss function value should tend to be stable or fluctuate within a small range, indicating that the model has converged to a relatively good solution. However, if the loss function value continues to decrease without obvious signs of convergence, it is necessary to consider adjusting the training parameters or optimization strategy to prevent overfitting.

[0012] According to one embodiment of the present invention, the method further comprises the following post-processing steps: S4. Datasets and evaluation metrics: S41, two benchmark datasets Market-1501, DukeMTMC-reID and occlusion dataset OccludedDuke are used as datasets, where: The pedestrian images in the Market1501 dataset come from 6 cameras on the Tsinghua University campus and contain 1,501 different pedestrian IDs; The images in the DukeMTMC-reID dataset come from 8 cameras on the Duke University campus. It is a subset of the DukeMTMC dataset and contains a total of 36,411 images of 1,812 pedestrians. The OccludedDuke dataset is an occluded dataset generated based on the DukeMTMC-reID dataset; S42. The mAP and Rank-1 evaluation indicators are used to measure the accuracy of the method.

[0013] This technical solution comprehensively evaluates the performance of the method in different scenarios by selecting data sets with different characteristics (such as the number of cameras, the number of images, the diversity of pedestrians, occlusion conditions, etc.). This is used to discover the strengths and weaknesses of the method and guide the further improvement of the method; the combined use of mAP and Rank-1 accuracy not only considers the average performance of the method in the global range, but also highlights its accuracy in a single retrieval, which is used to more comprehensively understand the performance of the method and provide strong support for its performance in practical applications; by applying these evaluation indicators on these benchmark data sets, a comprehensive, objective and quantitative evaluation of the pedestrian re-identification method is carried out, which is used for researchers to understand the performance bottlenecks and room for improvement of the method, and is also used to promote the development of the entire pedestrian re-identification field.

[0014] According to one embodiment of the present invention, in step S42, the unoccluded dataset Market1501 and the occluded dataset OccludedDuke are selected, and all attention mechanisms are removed, including global, head, upper limb, lower limb, and foot attention, and then the same dataset is used to test mAP and Rank-1 under the same conditions; The mAP on the market1501 dataset increased by 3% and the Rank-1 increased by 1%, while on the occluded Duke dataset, the mAP increased by 1.6% and the Rank-1 increased by 3%; The mAP on the market1501 dataset is 91.6, and the Rank-1 is 96.0. The mAP and Rank-1 on the DukeMTMC dataset are 78.7 and 87.2, respectively. The mAP and Rank-1 on the OccludedDuke dataset are 58.7 and 67.2, respectively. Therefore, the mAP and Rank-1 evaluation indicators on the dataset are significantly better than the Transformer framework without the attention mechanism.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention improves the accuracy and robustness of pedestrian re-identification by combining the powerful feature extraction capability and spatial attention mechanism of Transformer. The mAP and Rank-1 of the method proposed in the present invention on the market1501 dataset are 91.6 and 96.0, the mAP and Rank-1 on the DukeMTMC dataset are 78.7 and 87.2 respectively, and the mAP and Rank-1 on the OccludedDuke dataset are 58.7 and 67.2 respectively. The present invention has achieved good recognition accuracy on mainstream datasets and has certain application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a process principle framework diagram of the present invention.

[0017] Figure 2 It is the structural diagram of the spatial attention module.

[0018] Figure 3 This is the structure diagram of the global attention module.

[0019] Figure 4(a) is one of the segmentation effect diagrams of JPPNet.

[0020] Figure 4(b) is the second segmentation effect diagram of JPPNet.

[0021] Figure 4(c) is the third segmentation effect diagram of JPPNet.

[0022] Figure 4(d) is the fourth segmentation effect diagram of JPPNet.

[0023] Figure 5(a) is the Loss-epoch curve on market1501.

[0024] Figure 5(b) is the Loss-epoch curve on OccludedDuke. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Example 1 like Figure 1 As shown, this embodiment provides a pedestrian re-identification method based on Transformer and spatial attention mechanism, including the following steps: S1. Select the Transformer encoder feature extractor, including the following steps: S11. First, divide the image into H×W image blocks x. The step length of each image block is S and the size is P. Then the formula for the number of image blocks N is: (1) S12, the image block is flattened into a two-dimensional matrix, and after linear mapping, it is input into the Transformer encoder to obtain the feature F. The feature F formula is: (2) in, They are numbered 0, 1, ..., N image blocks respectively; S13, feature F is added through cls_token, and a set of position embedding and edge information embedding SIE is added to obtain a new feature , The formula is: (3) in, Position embedding is used to ensure that the encoder can recognize the input image blocks in the correct order; The image block added to cls_token is used to ensure the accuracy of the classification task; SIE is edge information embedding, which is used to mark different cameras; S14. Use multi-head attention mechanism to process features , to capture the features The dependencies in are: (4) (5) (6) (7) in, is the multi-head attention mechanism function; Q is the query matrix, K is the key matrix, and V is the value matrix; d is the dimension of the key matrix, which is used to scale the dot product to stabilize function; , , They are the transformation matrices of the query, key, and value of the feature respectively; S15. Introduce the feedforward neural network FFN to process the output of the multi-head attention mechanism to obtain the feature , The formula is: (8); S2: Strengthen a global attention and four spatial attention modules, including the following steps: S21. First, a global attention module is introduced in the global branch. At the same time, four spatial attention modules are introduced for the head, torso, lower limbs, and feet. The features of the global, head, upper limbs, lower limbs, and feet are defined as x respectively. global , x head , x upper ,x lower , x shoes ; S22, using convolutional layers and average pooling layers to generate global and four local area masks of the pedestrian, the corresponding masks are global_mask, head_mask, upper_mask, lower_mask, shoes_mask, so as to highlight the unobstructed parts of the pedestrian and minimize noise interference; S23, the matrix output by Transformer encoder Each element in is multiplied by each element in the five mask matrices to obtain the expressions of the five features, which are as follows: (9) (10) S3, improve the training method and loss function, such as Figure 2 As shown, it includes the following steps: S31. Improve the training method by splicing the global feature with the four local features after global mask processing to obtain the local splicing function , The formula is: (11) Output five mask functions simultaneously , The formula is: (12) S32. Use triplet loss and ID loss to adjust the parameters of the model, denoted as L global , the formula is: (13) in, The loss is triplet loss, and the ID loss is cross entropy loss; S33, using BCELoss as the semantic loss to train the mask function of the global and four local features, denoted as L mask , the formula is: (14) in, The loss is semantic loss; S34. Let the total loss function be L, and the formula is: (15).

[0027] This technical solution improves the accuracy and robustness of pedestrian re-identification by combining the powerful feature extraction capability of Transformer and the spatial attention mechanism. Specifically, the Encoder part of Transformer is used to extract image features; the image is divided into multiple image blocks (patches), each of which is an element in a sequence, and the Transformer's ability to process sequence data is used to process the image; the image is evenly divided into multiple image blocks, and the size and step size of each image block are fixed to ensure the uniform distribution and efficient processing of image information; the image blocks are flattened into a two-dimensional matrix and converted into a dimension suitable for Transformer processing through linear mapping (usually a fully connected layer); cls_token, position embedding (pos_embedding) and edge information embedding (SIE) are added to enable Transformer to process sequence order information (position embedding), classification tasks (cls_token) and cross-camera information (edge ​​information embedding); through the multi-head attention mechanism, Transformer can capture the complex dependencies between image blocks for understanding contextual information in the image; the feedforward neural network (FFN) further processes the output of the multi-head attention mechanism to enhance the model's nonlinear representation ability; the spatial attention mechanism is introduced to improve the quality of feature representation by focusing on key areas in pedestrian images (such as head, torso, lower limbs, feet); The output of the encoder is multiplied by the mask, which is actually a spatial weighting on the feature map, strengthening the information of important areas and weakening unimportant areas; the model performance is optimized by improving the training method and loss function; the global feature and the four local features are spliced ​​together, and the global and local information are used at the same time to improve the expressiveness of the features; the model is trained using triplet loss and ID loss. The triplet loss is used to learn the relative distance between features, making the features of samples of the same type closer and the features of samples of different types farther apart; the ID loss is used for classification tasks, and the model learns the unique features of each pedestrian; the mask function is trained using BCELoss (binary cross entropy loss) to ensure that the generated mask can accurately reflect the important areas in the image; the above loss functions are combined into a total loss function, and the model parameters are optimized by minimizing this loss function.

[0028] In addition, the pedestrian re-identification method based on Transformer and spatial attention mechanism proposed in the present invention also has the following additional technical features: According to an embodiment of the present invention, in step S2, the global attention module includes a channel attention module and a spatial attention module, and the feature It is first processed by a channel attention module and then by a spatial attention module.

[0029] In this technical solution, the channel attention module and the spatial attention module are used in series, so that the model can perform feature enhancement while considering both the channel dimension and the spatial dimension. First, the channel attention module is used to screen out channels that are more critical to the task; then, in the spatial attention module, the key areas in the image are further screened for these key channels. The combined use significantly improves the model's feature extraction capability and task performance.

[0030] According to one embodiment of the present invention, in step S3, the improved training method and loss function are run in the training environment of the NIVIDA 3090Ti GPU equipped with the pytorch framework, and the model is trained using the gradient descent SGD optimizer; the mask of the training set is obtained through JPPNet, and the hole convolution module and ResNet-101 are used as feature extractors to extract image features, and a hole pooling pyramid module ASPP is introduced as the output layer for segmenting pedestrian images.

[0031] Example 2 Based on Example 1, in order to better verify the effectiveness and reproducibility of the method proposed in the present invention, two benchmark datasets Market-1501 and DukeMTMC-reID and an occluded dataset OccludedDuke are used for experiments. The introduction of each dataset used in the present invention is as follows.

[0032] The pedestrian images in the Market1501 dataset come from 6 cameras on the Tsinghua University campus, containing 1501 different pedestrian IDs. The images in DukeMTMC-reID come from 8 cameras on the Duke University campus. It is a subset of the DukeMTMC dataset and contains 36411 images of 1812 pedestrians. OccludedDuke is an occluded dataset generated based on the DukeMTMC-reID dataset. This dataset is used in the present invention to test the accuracy of the method.

[0033] In this invention, mAP and Rank-1 are used to measure the accuracy of the method. In this invention, the mAP and Rank-1 obtained when the method is tested on each data set are as follows: Figure 3 shown.

[0034] When training the method proposed in the present invention, a NIVIDA 3090Ti GPU equipped with the pytorch framework is used as the training environment, and the gradient descent (SGD) optimizer is used to train the model. The mask of the training set is obtained by JPPNet. The network first uses a dilated convolution module and ResNet-101 as a feature extractor to extract image features, and then introduces a dilated pooling pyramid module (ASPP) as the output layer for segmenting pedestrian images. The results are shown in Figure 2. Figure 4(a) to Figure 4(d) shown.

[0035] During the training process, ID Loss and Triplet Loss are used as loss functions. ID Loss is used for the model to learn the identity features of pedestrians, while Triplet Loss helps the model better handle the similarities and differences between pedestrians. Through the joint optimization of these two loss functions, a more accurate and robust pedestrian re-identification model is obtained. The size of each batch is set to 64, and 120 epochs are trained on all three datasets. The size of each input image is resized to 256×128. The feature extractor uses 12 Transformer encoders and the weight decay is set to 1e -4 , the hyperparameter in the loss function is set to 1, and the relationship curve between the loss function value and epoch during training is as follows Figure 5(a) to Figure 5(b) shown.

[0036] In order to better quantify the comparative analysis, the recognition results of the proposed method are compared with those of other pedestrian re-identification methods such as PAT, PGFA, and HOReID in terms of mAP and Rank-1 to prove the superior performance of the proposed method.

[0037] On the market1501 dataset, the mAP of the proposed method is 91.6% and the Rank-1 is 96%. On the DukeMTMC-reID dataset, the mAP of the proposed method is 81.8% and the Rank-1 is 89.8%. The experimental results are shown in Table 1.

[0038] Table 1. Comparison of method performance on two benchmark datasets

[0039] On the occlusion dataset OccludedDuke, the proposed method also has excellent performance, with mAP reaching 58.7% and Rank-1 reaching 67.2%. The experimental results are shown in Table 2.

[0040] Table 2 Algorithm performance comparison on the occlusion dataset OccludedDuke

[0041] Example 3 On the basis of Example 2, in order to further illustrate the effectiveness of the method proposed in the present invention, a set of ablation experiments was designed in the research process to verify the role of the five attention modules in the pedestrian re-identification process; in this experiment, the present invention selected the unoccluded dataset Market1501 and the occluded dataset OccludedDuke, and removed all attention mechanisms at the same time, including global, head, upper limbs, lower limbs, and feet attention, and then used the same dataset to test mAP, Rank-1 and other related indicators under the same conditions; the experimental results are shown in Table 3.

[0042] Table 3 Ablation experiment results

[0043] As can be seen from the table, compared with the baseline, the method proposed in this invention increases mAP by 3% and Rank-1 by 1% on the market1501 dataset, and increases mAP by 1.6% and Rank-1 by 3% on the occluded Duke dataset; According to the comparative experimental results of Example 2 and the ablation experimental results of Example 3, it can be seen that the mAP and Rank-1 indicators of the method proposed in the present invention on various data sets are significantly better than those of the Transformer framework when the attention mechanism is not adopted. Therefore, the attention mechanism added by the present invention achieves certain effects.

[0044] Aiming at the problem of noise interference caused by occlusion in pedestrian re-identification under occlusion scenes, this paper proposes a pedestrian re-identification method based on Transformer encoder and attention mechanism. The key of this method is to firstly The encoder performs overall feature extraction, and then introduces a global attention module and four local attention modules to generate masks for the global, head, torso, lower limbs, and feet respectively, so as to reduce the noise interference of occluders and enhance the accuracy of overall recognition; based on this method, the three datasets of market1501, DukeMTMC-ReID, and OccludedDuke are used to carry out comparison and verification experiments on the two indicators of mAP and Rank-1; during the experimental training process, BCELoss is used to train the mask; the experimental results show that the mAP and Rank-1 of the method proposed in the present invention on the market1501 dataset are 91.6, and the Rank-1 is 96.0, the mAP and Rank-1 on the DukeMTMC dataset are 78.7 and 87.2 respectively, and the mAP and Rank-1 on the OccludedDuke dataset are 58.7 and 67.2 respectively. It can be seen that the method has achieved better recognition accuracy on mainstream datasets and has certain application prospects.

[0045] Although the present invention is described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A person re-identification method based on Transformer and spatial attention mechanism, characterized in that: The steps include: S1. Select the Transformer encoder feature extractor, including the following steps: S11. First, divide the image into H×W image blocks x. The step length of each image block is S and the size is P. Then the formula for the number of image blocks N is: (1) S12, the image block is flattened into a two-dimensional matrix, and after linear mapping, it is input into the Transformer encoder to obtain the feature F. The feature F formula is: (2) in, They are numbered 0, 1, ..., N image blocks respectively; S13, feature F is added through cls_token, and a set of position embedding and edge information embedding SIE is added to obtain a new feature , The formula is: (3) in, Position embedding is used to ensure that the encoder can recognize the input image blocks in the correct order; The image block added to cls_token is used to ensure the accuracy of the classification task; SIE is edge information embedding, which is used to mark different cameras; S14. Use multi-head attention mechanism to process features , to capture the features The dependencies in are: (4) (5) (6) (7) in, is the multi-head attention mechanism function; Q is the query matrix, K is the key matrix, and V is the value matrix; d is the dimension of the key matrix, which is used to scale the dot product to stabilize function; , , They are the transformation matrices of the query, key, and value of the feature respectively; S15. Introduce the feedforward neural network FFN to process the output of the multi-head attention mechanism to obtain the feature , The formula is: (8)； S2: Strengthen a global attention and four spatial attention modules, including the following steps: S21. First, a global attention module is introduced in the global branch. At the same time, four spatial attention modules are introduced for the head, torso, lower limbs, and feet. The features of the global, head, upper limbs, lower limbs, and feet are defined as x respectively. global , x head , x upper ,x lower , x shoes ; S22, using convolutional layers and average pooling layers to generate global and four local area masks of the pedestrian, the corresponding masks are global_mask, head_mask, upper_mask, lower_mask, shoes_mask, so as to highlight the unobstructed parts of the pedestrian and minimize noise interference; S23, the matrix output by Transformer encoder Each element in is multiplied by each element in the five mask matrices to obtain the expressions of the five features, which are as follows: (9) (10) S3. Improve the training method and loss function, including the following steps: S31. Improve the training method by splicing the global feature with the four local features after global mask processing to obtain the local splicing function , The formula is: (11) Output five mask functions simultaneously , The formula is: (12) S32. Use triplet loss and ID loss to adjust the parameters of the model, denoted as L global , the formula is: (13) in, The loss is triplet loss, and the ID loss is cross entropy loss; S33, using BCELoss as the semantic loss to train the mask function of the global and four local features, denoted as L mask , the formula is: (14) in, The loss is semantic loss; S34. Let the total loss function be L, and the formula is: (15)。 2. The pedestrian re-identification method based on Transformer and spatial attention mechanism as claimed in claim 1, characterized in that: In step S2, the global attention module includes a channel attention module and a spatial attention module. It is first processed by a channel attention module and then by a spatial attention module.

3. The pedestrian re-identification method based on Transformer and spatial attention mechanism as claimed in claim 1, characterized in that: In step S3, the improved training method and loss function are run in the training environment of NIVIDA 3090TiGPU equipped with the pytorch framework, and the model is trained using the gradient descent SGD optimizer; the mask of the training set is obtained through JPPNet, and the hole convolution module and ResNet-101 are used as feature extractors to extract image features, and a hole pooling pyramid module ASPP is introduced as the output layer for segmenting pedestrian images.

4. The pedestrian re-identification method based on Transformer and spatial attention mechanism as claimed in claim 1, characterized in that: In step S32, ID loss is used for the model to learn the identity characteristics of pedestrians, and Triplet loss is used to help the model process the similarities and differences between pedestrians; By jointly optimizing these two loss functions, a more accurate and robust person re-identification model is obtained; The size of each batch is set to 64, and 120 epochs are trained on all three datasets; Each input image is resized to 256×128; The feature extractor uses 12 Transformer encoders and the weight decay is set to 1e -4 ; The hyperparameters in the loss function are set to 1; Get the relationship curve between the loss function value and epoch during the training process.

5. The pedestrian re-identification method based on Transformer and spatial attention mechanism as claimed in claim 1, characterized in that: The method also includes the following post-processing steps: S4. Datasets and evaluation metrics: S41, two benchmark datasets Market-1501, DukeMTMC-reID and occlusion dataset OccludedDuke are used as datasets, where: The pedestrian images in the Market1501 dataset come from 6 cameras on the Tsinghua University campus and contain 1,501 different pedestrian IDs; The images in the DukeMTMC-reID dataset come from 8 cameras on the Duke University campus. It is a subset of the DukeMTMC dataset and contains a total of 36,411 images of 1,812 pedestrians. The OccludedDuke dataset is an occluded dataset generated based on the DukeMTMC-reID dataset; S42. The mAP and Rank-1 evaluation indicators are used to measure the accuracy of the method.

6. The pedestrian re-identification method based on Transformer and spatial attention mechanism as described in claim 5, characterized in that: In step S42, the unoccluded dataset Market1501 and the occluded dataset OccludedDuke are selected, and all attention mechanisms are removed, including the global, head, upper limb, lower limb, and foot attention. Then, the same dataset is used to test mAP and Rank-1 under the same conditions. The mAP on the market1501 dataset increased by 3% and the Rank-1 increased by 1%, while on the occluded Duke dataset, the mAP increased by 1.6% and the Rank-1 increased by 3%; The mAP on the market1501 dataset is 91.6, and the Rank-1 is 96.

0. The mAP and Rank-1 on the DukeMTMC dataset are 78.7 and 87.2, respectively. The mAP and Rank-1 on the OccludedDuke dataset are 58.7 and 67.2, respectively. Therefore, the mAP and Rank-1 evaluation indicators on the dataset are significantly better than the Transformer framework without the attention mechanism.

Images