A clothes-changing pedestrian re-identification method based on fine-grained interaction network

Abstract

The application provides a clothes-changing pedestrian re-identification method based on a fine-grained interaction network, S1, image preprocessing of a clothes-changing pedestrian data set, and generation of a clothes-changing image through pixel sampling; S2, ResNet50 is used as a backbone network, initial images and generated images are input into the backbone network, feature vectors F of the initial images and the generated images are extracted, and a spatial attention module is introduced to obtain corresponding fine-grained features; S3, a pair-wise interaction network is constructed, fine-grained features are interactively embedded into a feature space and F, similar context information is adaptively fused through a non-local module, a discriminative feature region is obtained, and the feature region is used for pedestrian identity retrieval; and S4, pedestrian features are optimized through multiple loss functions. The application focuses on fine-grained spatial features outside the clothing region by introducing an attention algorithm, and a pair-wise interaction network is designed to further suppress the influence of the clothing region on the model and improve the discrimination ability of the model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision pedestrian re-identification, and particularly relates to a clothes-changing pedestrian re-identification method based on a fine-grained interaction network. BACKGROUND

[0002] Pedestrian re-identification refers to capturing pedestrians with the same identity as a given target under different cameras, and this task has been widely studied and made great progress in recent years. However, existing research always assumes that the target image and the matching query image have the same clothing appearance, that is, the target pedestrian does not change his clothes when appearing under different cameras, which obviously does not conform to the real application scenario. For example, people will frequently change their clothes when it is hot; criminals may also change their clothes as camouflage to escape tracking in a short time. The change of clothes brings more complex challenges to the research of pedestrian re-identification.

[0003] In recent years, researchers have been committed to mining discriminative identity information for clothes-changing pedestrian re-identification tasks, mainly including two directions of pedestrian motion features and appearance features that do not change with clothes. JIN et al. extract gait features from a single image, and obtain continuous motion clues through sequence prediction. ZHAO et al. explore robust gait features from gait contours and posture heat maps through information complementation and effective fusion mechanism. However, due to the problems of information loss in the image, pedestrian body part occlusion and angle, etc., it is difficult to infer motion features based on images, which makes it difficult to guarantee the reliability of the inferred motion features, and the current model performance cannot meet the expected effect.

[0004] Although the above method captures some gait features, it ignores other useful appearance features outside the clothing area, and more researchers are digging texture features that do not change with clothing from the pedestrians themselves. WAN et al. directly extract facial features, and further enhance the facial features. SHI et al. focus on the whole head, and assist in judging the identity of the pedestrian by increasing the head feature flow of the human body. However, these methods cannot well solve the problem of face and head occlusion. YANG et al. introduce a learning-based spatial polar transform (SPT) mode and a multi-flow model for capturing pedestrian contour features. QIAN et al. propose a similar model that uses key points as prior information to extract shape information from the initial image. These methods are limited by limited viewing angles and can only capture one-sided shape representation and ignore detailed visual clues (such as face, hair, skin color, etc.). HONG et al. propose a two-flow framework that learns from each other through dense interaction for collaborative training, transfers shape knowledge from shape flow to appearance flow, and enables appearance flow to be independently deployed in reasoning. CHEN et al. use contour feature learning as regularization to mine more effective shape perception features to maximize the mutual information between color appearance features and contour shape features. However, this introduces clothing color information to some extent. XU et al. separate the features of the pedestrian's identity and clothing through a generative adversarial network. YAGHOUBI et al. introduce a short-term embedded convolutional neural network to identify clothing style and other features, and then separate the invariant features and clothing features through a cosine similarity loss function. However, due to the difficulty of feature separation, the above models cannot achieve ideal performance. SUMMARY

[0005] Existing pedestrian re-identification methods rely on the appearance of the pedestrian's clothing to a certain extent for retrieval in a short period of time, and do not consider the case that a pedestrian is likely to change his / her clothes over a long period of time. In view of the problems of great interference of clothing information on the model in the clothing-changing pedestrian re-identification task and difficulty in extracting effective information of the pedestrian, a clothing-changing pedestrian re-identification method based on a fine-grained interaction network is proposed to reduce the interference of the pedestrian's clothing information on the network model, so as to further extract more robust pedestrian features.

[0006] The present application selects Resnet50 as the backbone network of the model. First, the importance of a specific channel is highlighted by introducing a spatial attention module, and valuable fine-grained features are selected to reduce the influence of clothing factors on the performance of the model. Then, a pair interaction network is designed to capture the similarity of the associated regions in the pedestrian image pair with the same identity and different clothing, and to learn the consistency features of the pedestrian before and after changing clothes. On this basis, more significant pedestrian features are obtained through attention pooling, and the features are constrained by the mean square error and the triplet loss function.

[0007] To achieve the above object, the present application provides the following technical scheme: a clothes-changing pedestrian re-identification method based on fine-grained interaction network, which mainly reduces the influence of clothing factors on the model and optimizes the extraction of effective features.

[0008] Inspired by people's observation habits, the present application determines whether they are the same identity by capturing the similarity correlation area in the image pair before and after clothes-changing and learning consistent features. For the same pedestrian, when embedding its effective features into the image features under different clothes, the features should still be able to serve as the criterion for determining whether they are the same pedestrian. The present application introduces a spatial attention module to select valuable fine-grained features and thus reduce the influence of clothing factors on the model performance, and then captures the similarity correlation area in the image pair through a pair-wise interaction network and learns it. The pedestrian features are optimized through multiple loss functions, improving the generalization ability of the model.

[0009] The above scheme includes the following steps:

[0010] S1, pre-processing the images of the clothes-changing pedestrian dataset, generating the images after clothes-changing through pixel sampling;

[0011] S2, using ResNet50 as the backbone network, inputting the initial image and the generated image into the backbone network to extract their feature vectors F and introducing a spatial attention module to obtain the corresponding fine-grained features;

[0012] S3, constructing a pair-wise interaction network, embedding the fine-grained features into the feature space and F through the non-local module to adaptively fuse similar context information and obtain the discriminative feature area for pedestrian identity retrieval;

[0013] S4, optimizing the pedestrian features through multiple loss functions.

[0014] Further, step S1 includes:

[0015] To address the issue of limited images in pedestrian re-identification datasets involving clothing changes, new samples are generated through pixel sampling. This involves randomly selecting pixels from the input image and replacing them with new pixel values ​​to obtain a new image. Repeating this process multiple times generates several different new images to meet the training data requirements of the pedestrian re-identification task. The main idea is to generate the changed image by altering the pixels in the clothing region of the input sample. To achieve this, a pre-trained human parsing model, SCHP, is first used to divide the input image into 20 semantic parts, including the background, various types of clothing, and different parts of the human body. Based on this, to adapt to the pedestrian re-identification task involving clothing changes, these 20 semantic parts are merged back into three semantic parts: background, clothing, and body. New images are generated by swapping the pixels in the clothing region of the original image.

[0016] Let the input small sample be denoted as X = [x1, x2, ..., x]. i , ..., x B ], where x represents the input image and B represents the size of the input sample. This invention represents all pixel vectors in the sample as P. X = [p1, p2, ..., p c1 , ..., p cN , ..., p M ], p represents the pixel vector, M represents the total number of vectors, [p c1 , ..., p cN The pixels in question belong to the clothing category and correspond to the semantic part of clothing in the input image. Considering the scenario of clothing variations, a small sample X′ = [x′1, x′2, ..., x′] is obtained by randomly shuffling X. i ,...,x′ B After shuffling, the pixel vectors in the small sample also change. The new pixel vectors corresponding to the clothing part are shown in the following formula: P′ cloth ={p′ j |p′ j =x′ i [s′ i =1]}, j∈[1,N]

[0017] Where N represents the total number of clothing pixel vectors in the mini-batch samples, which typically changes during iteration. j represents the index of the j-th pixel vector, p′ j This represents a pixel vector. `i` represents the index of the `i`th image, and `x′`... i This represents an image sample. s′ i This represents the semantic segmentation result corresponding to the sample, including three semantic parts: background, clothing, and body. Here, 1 represents the pixel index corresponding to the clothing part, and "==" indicates equality. The transformed pixel vector P′cloth For changing the pixel value of the clothing part in the original small sample X, the pixel vector in the small sample X is changed, and the following formula is shown: X =[p1,p2,...p′ c1 ,...,p′ cN ,...,P M ],[p′ c1 ,...,p′ cN ]∈P′ cloth .

[0018] Further, the step S2 comprises:

[0019] Since the clothing occupies a large proportion in the pedestrian image, the visual features such as color and texture related to the clothing also have a large proportion in the overall features, and the clothing factor has a great influence on the model. In view of this problem, the application introduces a spatial attention module to select valuable fine-grained features. The importance of the spatial features corresponding to each channel is inferred by weighting the channels during the running process, so as to generate fine-grained spatial features. Specifically, the input image is first generated through a series of convolution, activation and pooling operations to generate a feature map F;Then, the spatial feature map F is converted into a one-dimensional vector through a global average pooling layer, and the operation process is as follows:

[0020]

[0021] Wherein, w and h represent the width and height of the input feature space respectively, and i and j represent the index corresponding to the feature.

[0022] Then, an activation function is used to weigh different channels to obtain a weight vector S c , which reflects the importance of each channel. Finally, a weighted sum is used to simulate a confusion attention heat map, and the heat map is activated through a sigmoid activation function, which is used to represent the importance of different spatial positions, and the calculation process is as follows:

[0023]

[0024]

[0025] Wherein, i represents the index of the channel, c represents the channel, n represents the total number of channels, F is the spatial feature vector of the picture, and F p represents the preprocessed spatial feature vector. Since the elements in F can be negative, F needs to be preprocessed before M Atten is calculated, and the pre-processing calculation process is as follows: wherein, F min represents the minimum element in F.

[0026] F p = F - F min

[0027] Further, step S3 comprises:

[0028] In order to obtain the similarity correlation region in the image pair before and after changing clothes, the initial image and the generated image are input into the backbone network, the feature vectors F and F of the initial image and the generated image are extracted, and the spatial attention module is introduced to obtain the corresponding fine-grained features. Then the fine-grained features are embedded into the feature space and F, the similar context information is adaptively fused through the non-local module, and the discriminative feature region is obtained for pedestrian identity retrieval.

[0029] Taking the initial image as an example, first, the feature vector F of the initial image is extracted through the backbone network, then the fine-grained feature M Atten of the initial image is obtained through the spatial attention module, then the feature is embedded into the feature space of the generated image, the spatial context is introduced through the non-local module, and finally the fine-grained feature is highlighted through attention pooling; the fine-grained embedding feature is shown in the following formula:

[0030]

[0031] The spatial embedding feature M e is transmitted through the non-local module. The key-value pair (K-V) is used to capture the dependency in the feature space. The query (Q) and the key (K) vector dot product, and the weight matrix W is obtained through the softmax function activation, which assigns weights to the key-value pair, and gives more weights to the values similar to the context. Then the weighted sum is performed on each value (V) vector, and the 1x1 convolution is performed to make it have the same dimension as the input tensor, and the calculation process is shown in the following formula:

[0032]

[0033]

[0034] V = reshape (relu (δ 1×1 (M e ))

[0035] wherein, denotes L2 normalization of the vector, reshape denotes conversion of the matrix dimension, relu and softmax denote ReLU activation function and softmax activation function respectively, θ and δ denote 1x1 convolution, T denotes vector transposition, and * denotes multiplication of tensors.

[0036] Then, a weighted sum and a 1×1 convolution are performed on each value vector V, and the resulting vector is added to M. e In the process, we obtain the context-aware feature M. c Furthermore, through M c Weighted spatial average pooling is used to highlight fine-grained features M Atten The result is M ap The calculation process is shown in the following formula:

[0037]

[0038]

[0039] Where β represents a 1×1 convolution, and reshape represents the transformation of the matrix dimension. The expression represents tensor multiplication, w and h represent the width and height of the feature space, respectively, and i and j represent feature indices.

[0040] Finally based on M ap Pedestrian identification is predicted by batch normalizing and passing the input image through a fully connected layer. The output of the fully connected layer is recorded as Y. Simultaneously, fine-grained features of the generated image are extracted and embedded into the feature space F of the original image, and used together for pedestrian identification prediction. The result is recorded as... In theory, the predictions of the two networks should be consistent.

[0041] Furthermore, step S4 includes:

[0042] Because the pixel count of the clothing area changes before and after the clothing change, the generated image has a different clothing appearance than the initial image. To reduce the interference of clothing area features on the model, this invention uses a mean squared error function to constrain the learned features. The formula for calculating the mean squared error function is as follows:

[0043]

[0044] in, This is represented as L2 normalization of the vector, where i represents the feature index, and f... i f represents the features of the initial image. i ′ represents the feature of the generated image.

[0045] Furthermore, this invention also employs both the cross-entropy loss function and the triplet loss function for training the person re-identification model. For the input image x... i Its corresponding label is y i This means that the probability of the model predicting correctly is p(x). i The formula for calculating the loss caused by model prediction, i.e., cross-entropy loss, is as follows:

[0046] Lce = -y i • log p(x i )

[0047] The triplet loss function is based on the concept of triplets, minimizing the distance between the target image and the positive sample with the same identity image and the negative sample with different identity, which is calculated as shown in the following formula:

[0048]

[0049] wherein, and respectively represent the maximum distance of the positive sample pair and the minimum distance of the negative sample pair, f i represents the feature of the initial image. During the training process, m is set to 0.3.

[0050] The present application optimizes the model by combining the calculation of the above loss functions, thereby improving its generalization ability and prediction accuracy. B represents the size of the input small sample, and 2B represents the size of the sample, that is, the total number of input small samples and generated small samples. The total loss is shown in the following formula:

[0051]

[0052] Compared with the prior art, the present application has the following beneficial effects:

[0053] 1) The present application introduces an attention algorithm focusing on the fine-grained spatial features outside the clothing area, reducing the influence of clothing factors on the model, and the attention mechanism module has a positive guiding effect on the pair interaction network;

[0054] 2) The present application designs a pair interaction network, taking a pair of pictures of a person before and after changing clothes as input, obtaining consistent person features before and after changing clothes to further suppress the influence of the clothing area on the model and improve the discrimination ability of the model;

[0055] 3) The present application evaluates the proposed method on three standard clothing-changing person re-identification data sets PRCC, Celeb-reID and Celeb-reID-light, and extensive experimental results show that the proposed method is superior to existing clothing-changing person re-identification methods. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a flowchart of the clothing-changing person re-identification method based on the fine-grained interaction network of the present application;

[0057] Figure 2 is a whole framework diagram of the present application;

[0058] Figure 3 is a spatial attention diagram of the present application;

[0059] Figure 4 Non-local diagram of the present application;

[0060] Figure 5 Cumulative characteristic curve comparison diagram of the present application in inserting different attention modules;

[0061] Figure 6 Cumulative characteristic curve comparison diagram of ablation experiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. The specific examples described herein are only used to explain the technical scheme of the present application, and are not limited to the present application.

[0063] As shown in Figure 1 , a clothes-changing pedestrian re-identification method based on fine-grained interaction network includes the following steps:

[0064] s1: generating a clothes-changing image through pixel sampling;

[0065] s2: constructing a clothes-changing pedestrian re-identification model based on fine-grained pairwise interaction network;

[0066] s3: optimizing the loss function;

[0067] s4: training the constructed clothes-changing pedestrian re-identification model based on fine-grained pairwise interaction network;

[0068] s5: verifying the model performance.

[0069] s1: generating a clothes-changing image through pixel sampling as Figure 2 shown in the left, by randomly selecting clothing region pixels from the input image, and then replacing these pixels with new pixel values, thereby obtaining a new image of the pedestrian after clothing change;

[0070] s2.1: using ResNet50 as the backbone network to extract features;

[0071] s2.2: the overall framework proposed by the present application as Figure 2 shown, which is committed to reducing the influence of clothing factors on the model and optimizing the extraction of effective features. The clothes-changing pedestrian re-identification method based on fine-grained interaction network proposed by the present application includes:

[0072] ResNet50 is used as the feature extraction network, with images before and after clothing changes taken as input. A spatial attention module is introduced to select valuable fine-grained features, thereby reducing the impact of clothing factors on model performance. Then, a pairwise interaction network is used to capture and learn similar regions in image pairs. Multiple loss functions are jointly used to optimize pedestrian features, improving the model's generalization ability.

[0073] Valuable fine-grained features are selected through the spatial attention module, such as... Figure 3 As shown;

[0074] By weighting the channels during operation, the importance of spatial features corresponding to each channel is inferred, thereby generating fine-grained spatial features. Specifically, the input image is first processed through a series of convolution, activation, and pooling operations to generate a feature map F; then, a global average pooling layer transforms the spatial feature map F into a one-dimensional vector, the operation of which is shown in the following formula:

[0075]

[0076] Where w and h represent the width and height of the input feature space, respectively, and i and j represent the indices of the features.

[0077] Then, an activation function is used to weigh the different channels, resulting in a weight vector S. c This vector reflects the importance of each channel. Finally, a weighted sum is used to simulate a confounding attention heatmap, which is then activated by a sigmoid activation function. This heatmap represents the importance of different spatial locations, and the calculation process is shown in the following formula:

[0078]

[0079]

[0080] Where i represents the channel index, c represents the channel, n represents the total number of channels, and F is the spatial feature vector of the image. p This represents the preprocessed spatial feature vector. Since elements in F can be negative, this is important when calculating M. Atten Previously, F needed to be preprocessed. The preprocessing calculation process is shown in the following formula, where F min Let F represent the smallest element in F.

[0081] F p =FF min

[0082] like Figure 2 As shown on the right, similarity association regions in image pairs are captured and learned through pairwise interactive networks.

[0083] The initial image and the generated image are input into the backbone network, and the feature vectors F and The spatial attention module is introduced to obtain the corresponding fine-grained features. Then the fine-grained features are interactively embedded into the feature space and F, and the similar context information is adaptively fused through the non-local module to obtain the discriminative feature region for pedestrian identity retrieval.

[0084] Taking the initial image as an example, the feature vector F of the initial image is first extracted through the backbone network, and then the fine-grained feature M Atten of the initial image is obtained through the spatial attention module. The spatial context is introduced through the non-local module, and finally the fine-grained feature is highlighted through the attention pooling. The fine-grained embedding feature is shown in the following formula:

[0085]

[0086] Figure 4 The structure of the non-local module is shown. The spatial embedding feature M e is transmitted through the non-local module. The key-value pair (K-V) is used to capture the dependency in the feature space. The query (Q) and the key (K) vector dot product, and the weight matrix W is obtained through the softmax function activation, which assigns weights to the key-value pair, and gives more weights to the values similar to the context. Then the weighted sum is performed on each value (V) vector, and the 1x1 convolution is performed to make it have the same dimension as the input tensor, and the calculation process is shown in the following formula:

[0087]

[0088]

[0089] V = reshape (relu (δ 1×1 (M e ))

[0090] wherein, represents the L2 normalization of the vector, reshape represents the conversion of the matrix dimension, relu and softmax represent the ReLU activation function and the softmax activation function respectively, θ and δ represent the 1x1 convolution, T represents the vector transpose, represents the multiplication of tensors.

[0091] Then the weighted sum and the 1x1 convolution are performed on each value vector V, and the obtained vector is added to M e to obtain the context-aware feature M c . In addition, the Mc Weighted spatial average pooling is used to highlight fine-grained features M Atten The result is M ap The calculation process is shown in the following formula:

[0092]

[0093]

[0094] Where β represents a 1×1 convolution, and reshape represents the transformation of the matrix dimension. The expression represents tensor multiplication, w and h represent the width and height of the feature space, respectively, and i and j represent feature indices.

[0095] Finally based on M ap Pedestrian identification is predicted by batch normalizing and passing the input image through a fully connected layer. The output of the fully connected layer is recorded as Y. Simultaneously, fine-grained features of the generated image are extracted and embedded into the feature space F of the original image, and used together for pedestrian identification prediction. The result is recorded as... In theory, the predictions of the two networks should be consistent.

[0096] S3: Because the pixel count of the clothing area changes before and after the clothing change, the generated image has a different clothing appearance than the initial image. To reduce the interference of clothing area features on the model, this invention uses a mean squared error function to constrain the learned features. The formula for calculating the mean squared error function is as follows:

[0097]

[0098] in, This is represented as L2 normalization of the vector, where i represents the feature index, and f... i f represents the features of the initial image. i ′ represents the feature of the generated image.

[0099] Furthermore, this invention also employs both the cross-entropy loss function and the triplet loss function for training the person re-identification model. For the input image x... i Its corresponding label is y i This means that the probability of the model predicting correctly is p(x). i The formula for calculating the loss caused by model prediction, i.e., cross-entropy loss, is as follows:

[0100] L ce =-y i ·logp(x i )

[0101] The triplet loss function is based on the concept of triplets, which minimizes the distance between the target image and the positive sample with the same identity image and the negative sample with different identity, and its calculation method is shown in the following formula:

[0102]

[0103] wherein, and respectively represent the maximum distance of the positive sample pair and the minimum distance of the negative sample pair, and f i represents the feature of the initial image. During the training process, m is set to 0.3.

[0104] The model is optimized by combining the calculation of the above loss functions, so as to improve its generalization ability and prediction accuracy. B represents the size of the input small sample, and 2B represents the size of the sample, that is, the total number of input small samples and generated small samples. Its total loss is shown in the following formula:

[0105]

[0106] s4.1: Set the training parameters, and adjust the image size to 256x128. The initial learning rate is set to 3.5x10 -3 . Set the small batch sample to 32, randomly extract 8 identities from the training data set, and then randomly extract 4 instances for each identity for clothes changing operation.

[0107] S4.2: Put the clothes changing pedestrian data set into the clothes changing pedestrian re-identification model based on the fine-grained pair-wise interaction network for training;

[0108] s4.3: During the training period, random flipping, random cropping and random erasing are used for data enhancement.

[0109] s5: According to the trained clothes changing pedestrian re-identification model based on the fine-grained pair-wise interaction network, evaluate the performance of the model from the average precision mean (mAP) and cumulative matching characteristics (CMC) Rank-1 / 5 precision.

[0110] To confirm the performance of our model, we compare our method with some recent state-of-the-art methods on the PRCC dataset and show in Table 1. On the PRCC dataset, “Cross clothes” means the scenario where the pedestrian’s clothes change, and the image captured from camera C is set as the query image, and the images captured from camera A are set as the gallery set. Our method achieves 69.5% Rank-1 accuracy and 61.8% mAP. Compared with Shu et al., our method improves the Rank-1 accuracy by 3.7% and the mAP by 0.6%. The mAP is slightly lower than Pos-Neg and SPANet, but the Rank-1 accuracy is higher by 14.63% and 14.74%, respectively. “Same clothes” means the scenario where the pedestrian’s clothes do not change, and the image captured from camera B is set as the query image, and the images captured from camera A are set as the gallery set. Our proposed method achieves 99.7% Rank-1 accuracy and 97.6% mAP, close to the current advanced cross-dressing pedestrian re-identification method.

[0111] Table 1 Comparison with existing theories on the dataset PRCC (%)

[0112]

[0113] To verify the effectiveness of the spatial attention module, we compare it with the Self-Attention and CBAM two attention mechanism modules. The Self-Attention module directly calculates the whole image, calculates the relevance between each position in the image, and locks the image area worth paying attention to. The CBAM module integrates channel attention and spatial attention, and represents the sequential way of using channel attention first and then using spatial attention can achieve the best effect. As shown in Figure 5 , under the same experimental settings, the CMC accuracy of our method on the PRCC dataset is better than that of inserting other attention modules.

[0114] To analyze the effectiveness of each module in the pair-wise interaction network, we conduct extensive experiments on the PRCC dataset. Under the same experimental settings, the CMC accuracy of different modules is shown in Figure 6 According to the experimental results, we can conclude that the non-local module, attention pooling module and pair-wise interaction experiment we added have improved the performance of the model.

[0115] The above merely expresses the preferred embodiments of the present application, which are described in more detail and in a more specific manner, but should not be understood as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications, improvements and substitutions can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for re-identifying pedestrians changing clothes based on fine-grained interactive networks, characterized in that: Includes the following steps: S1, preprocess the images of the pedestrians changing clothes dataset, and generate images after changing clothes by pixel sampling; S2 uses ResNet50 as the backbone network, inputting both the initial and generated images into the backbone network to extract their feature vectors. and A spatial attention module is introduced to obtain its corresponding fine-grained features; S3 constructs a pairwise interaction network, embedding fine-grained feature interactions into the feature space. and By adaptively fusing similar contextual information through a non-local module, discriminative feature regions are obtained for pedestrian identity retrieval. S4 optimizes pedestrian features by jointly using multiple loss functions; Step S3 includes: First, feature vectors of the initial image are extracted using the backbone network. Then, its fine-grained features are obtained through the spatial attention module. Then, this feature is embedded into the feature space of the generated image. Spatial context is introduced through a non-local module, and fine-grained features are highlighted through attention pooling. The fine-grained embedding features are shown in the following formula: ， Spatial embedding features are passed through the non-local module. ; Capture dependencies in the feature space using key-value pairs (KV); Query the dot product of Q and key-K vectors, through The activation function yields a weight matrix W, which is used to assign weights to key-value pairs, giving more weight to values ​​with similar contexts. Then, a weighted sum is applied to each value vector V, followed by a 1x1 convolution to make it the same dimension as the input tensor. The calculation process is shown in the following formula: ， ， ， in, This indicates L2 normalization of the vector, and `reshape` indicates a transformation of the matrix dimension. and Representing the ReLU activation function and Activation function and This represents a 1×1 convolution, and T represents the vector transpose. Represents tensor multiplication; Then, a weighted sum and a 1×1 convolution are performed on each value vector V, and the resulting vector is added to... In this process, context-aware features are obtained. Furthermore, through the analysis of Weighted spatial average pooling is used to highlight fine-grained features. The result is The calculation process is shown in the following formula: ， ， in, This represents a 1×1 convolution, and `reshape` represents the transformation of the matrix dimension. Represents tensor multiplication. and Let represent the width and height of the feature space, respectively. and Indicates feature index; Finally based on To predict pedestrian behavior, the input is batch-processed, normalized, and passed through a fully connected layer. The output of the fully connected layer is then recorded. Simultaneously, fine-grained features of the generated image are extracted and embedded into the feature space of the original image. This information is used together to predict pedestrian identities, and the results are recorded as follows: Theoretically, the predictions of the two networks should be consistent.

2. The method for re-identifying pedestrians changing clothes based on fine-grained interactive networks according to claim 1, characterized in that: Step S2 includes: The input image is first processed through a series of convolution, activation, and pooling operations to generate feature maps. Then, spatial features are mapped using a global average pooling layer. Converting it to a one-dimensional vector, the operation process is shown in the following formula: ， in, and Let represent the width and height of the input feature space, respectively. and Indicates the index corresponding to the feature; Then, an activation function is used to weigh the different channels, resulting in a weight vector. This vector reflects the importance of each channel; finally, a weighted sum is used to simulate a confounding attention heatmap, which is then activated by a sigmoid activation function. This heatmap represents the importance of different spatial locations, and the calculation process is shown in the following formula: ， ， in, Indicates the index of the channel. Indicates a channel. Indicates the total number of channels. This is the spatial feature vector of the image. This represents the preprocessed spatial feature vector; in the calculation Previously, for Preprocessing is performed, and the preprocessing calculation process is shown in the following formula, where, express The smallest element in; 。 3. The method for re-identifying pedestrians changing clothes based on fine-grained interactive networks according to claim 1, characterized in that: Step S4 includes: The learned features are constrained by the mean squared error function, which is calculated using the following formula: ， in, This can be represented as L2 normalization of the vector. Indicates feature index, Features representing the initial image, Features representing the generated image; Furthermore, the cross-entropy loss function and the triplet loss function were also used together for training the person re-identification model; for the input image Its corresponding tag is This means that the probability of the model predicting correctly is... The formula for calculating the loss from model prediction, i.e., cross-entropy loss, is as follows: ， The triplet loss function is based on the concept of triples and minimizes the distance between the target image and positive samples with the same identity and negative samples with different identities. Its calculation method is shown in the following formula: ， in, and These represent the maximum distance between positive sample pairs and the minimum distance between negative sample pairs, respectively. Represents the features of the initial image; during training, Set to 0.3; The model is optimized by combining the calculations of the above loss functions, where B represents the size of the small input sample, and 2B represents the sample size, i.e., the total number of small input samples and generated small samples; the total loss is shown in the following formula: 。

Images