Pedestrian re-identification method and device based on hybrid attention decoupled re-identification network

Abstract

The application provides a pedestrian re-identification method and equipment based on a mixed attention decoupling re-identification network, so as to enhance the discrimination ability of field-invariant pedestrian features, thereby forming reliable class boundaries and learning the in-class semantic diversity. The design of the mixed attention module in the method is to strengthen the field-invariant feature expression in the form of attention weight decoupling from the spatial and channel perspectives, which forces the network to automatically use the image regions and attribute clues conducive to cross-domain re-identification. In addition, based on the enhanced field-invariant feature expression, a multi-difficult sample memory learning strategy is proposed to improve the in-class diversity of target domain samples. The application optimizes the feature learning process by updating the reliable sample memory library and multiple difficult sample memory libraries, and by considering the relationship between samples in the same class, the application can be used to capture significant in-class semantic changes, and can positively affect the accuracy of pseudo labels.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a pedestrian re-identification method and device based on a hybrid attention-decoupled re-identification network. Background Technology

[0002] Pedestrian re-identification is a challenging task that aims to re-identify a given query image from a library of non-overlapping camera-captured images. In recent years, this direction has seen significant progress in supervised methods, which have been widely applied to image applications, including criminal investigations, multi-camera tracking, and public safety. However, these re-identification models trained on a single dataset are often affected by domain shift, exhibiting significant performance degradation when applied to target application scenarios where imaging conditions differ from the training data. While manually collecting and annotating training samples in the target domain can mitigate the domain shift problem, this is extremely expensive and time-consuming.

[0003] Unsupervised domain adaptation techniques can bridge the gap between domains by transferring knowledge learned from a labeled source domain to an unlabeled target domain. Some re-identification models use unsupervised image-to-image transformation based on generative adversarial networks to study this problem. Labeled source domain samples are transformed into target styles and used to train the target re-identification model without target data labels. However, the re-identification performance of this approach is highly dependent on the quality of image generation and is significantly affected by large variations in camera angle or pedestrian pose.

[0004] Currently, mainstream methods employ pseudo-label-based approaches. Compared to generative adversarial network (GAN)-based methods, pseudo-label-based methods can directly train the target domain model by generating pseudo-labels. These methods adaptively adjust the cross-domain feature distribution through a three-step process: (1) learning a person re-identification model using labeled source domain training samples; (2) clustering the target domain training samples and generating pseudo-labels using features extracted from the re-identification model trained in the source domain; and (3) optimizing the model using a memory bank that mixes target domain data with pseudo-labels and source domain samples with real labels. However, most pseudo-labeling methods ignore class boundary samples to reduce the impact of erroneous pseudo-labels, inevitably sacrificing semantic diversity within classes. Although some methods employ hard sample memory banks to discover relationships between samples and describe the diversity of samples within classes, they are prone to increasing the risk of generating incorrect pseudo-labels.

[0005] Therefore, intra-class diversity and pseudo-label accuracy are contradictory. Increasing intra-class diversity may lead to more erroneous pseudo-labels, while increasing the number of high-confidence pseudo-label training samples may affect intra-class diversity. Balancing intra-class diversity and pseudo-label accuracy is a challenging problem in cross-domain person re-identification. Therefore, researching a method that can improve the discriminative power of person re-identification features and learn intra-class semantic diversity is of great significance for person re-identification tasks. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a pedestrian re-identification method based on a hybrid attention-decoupled re-identification network. Its purpose is to solve the problems in pedestrian re-identification such as the inability to fully learn intra-class semantic diversity and the inability to extract discriminative feature representations, thereby enhancing the discriminative ability of domain-invariant pedestrian features, thus forming reliable category boundaries and learning intra-class semantic diversity.

[0007] To achieve the above objectives, this invention provides a pedestrian re-identification method based on a hybrid attention-decoupled re-identification network, applicable to unsupervised adaptive pedestrian re-identification. The method includes:

[0008] Construct a hybrid attention module in the pedestrian re-identification network model, including a decoupled spatial attention mechanism and a channel attention mechanism;

[0009] Source domain data D s and target domain data D t The initial feature f is obtained by feeding it into the initial feature extraction network;

[0010] The initial features are fed into the hybrid attention module to obtain the domain-invariant features f. di and domain-specific features f ds , as well as domain classification loss and orthogonality loss;

[0011] The domain-invariant feature f di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain.

[0012] Based on the multi-difficulty sample memory learning strategy, initialize multiple sample memories;

[0013] The domain-invariant feature f di The similarity between the memory and multiple sample memories is calculated to obtain the contrast loss for each memory.

[0014] The network parameters are updated by using contrastive loss, domain classification loss, and orthogonal loss for each memory location. At the same time, the memory features are updated based on the input query samples to learn the relationships between samples. When the network model converges, the training is complete. The trained network model is then used for pedestrian re-identification to obtain the recognition result.

[0015] Furthermore, the hybrid attention module in the pedestrian re-identification network model includes:

[0016] Construct a hybrid attention module consisting of a decoupled spatial attention mechanism and a channel attention mechanism;

[0017] This hybrid attention module is used to extract domain-invariant features for subsequent recognition and classification tasks.

[0018] Furthermore, the source domain data D s and target domain data D t The data is fed into the initial feature extraction network to obtain initial features, including:

[0019] The ResNet-50 network with five stages was used as the initial feature extraction network. The convolution stride of the fifth stage was changed from 2 to 1. The ResNet-50 network had been pre-trained on ImageNet.

[0020] The source domain data and target domain data are fed into the pre-trained ResNet-50 network to obtain initial features.

[0021] Furthermore, the initial features are fed into the hybrid attention module to obtain the neighborhood-invariant features f. di and domain-specific features f ds In addition, there are domain classification loss and orthogonality loss, including:

[0022] The initial features are fed into the hybrid attention module. First, the initial features pass through a decoupling module to obtain the domain-invariant weights w. i Domain-specific weights w s ;

[0023] Based on the principle of vector orthogonality, orthogonal loss is used to ensure that the neighborhood-invariant weights w are maintained. i Domain-specific weights w s They are mutually orthogonal, thus making domain-invariant features and domain-specific features independent of each other, and the orthogonal loss L I for:

[0024]

[0025]

[0026] in and It is a domain-invariant weight w i Domain-specific weights w s After average pooling, || || 2 L2 regularization is represented by ||, which represents the absolute value operation. I[i,j] indicates that the value at position (i,j) belongs to L2 regularization. The value of b represents the number of samples in each batch, and c represents the number of channels;

[0027] Based on the domain-invariant weight w i Domain-specific weights w s Obtain the domain-invariant feature f di and domain-specific features f ds Its formula is:

[0028] W i =E dm (f), W s =1-W i

[0029]

[0030] in E represents element-wise multiplication. dm For decoupling modules;

[0031] Utilizing domain-specific features f ds Computational domain classification loss, such that f ds Includes more domain-specific feature information, domain classification loss L dom for:

[0032] L dom =-[D L logP+(1-D L log(1-P)]

[0033] P = E dc (f ds )

[0034] Where D L These are domain labels; the source domain's domain label is 0, and the target domain's domain label is 1. dc Let P be the domain classification module, where P is the domain classification probability.

[0035] Furthermore, the neighborhood-invariant feature f di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain, including:

[0036] Before each round of training, clustering was performed using DBSCAN. For the parameters in DBSCAN, the parameter k in k-distance was set to 30, the minimum number of neighbors n was set to 4, the distance threshold between sample pairs was set to 0.6 for the DukeMTMC-reID and Market-1501 datasets, and the distance threshold was 0.7 for the MSMT17 dataset.

[0037] During each training round, the pseudo-labels for the data in the target domain are the results of the previous round of model optimization.

[0038] Furthermore, the initialization of multiple sample memories according to the multi-hard sample memory learning strategy includes:

[0039] The multi-difficulty memory learning strategy consists of reliable sample memory and multiple difficult sample memory.

[0040] The reliable sample memory and multiple hard sample memory use the same initialization strategy, which is initialized at the beginning of each training round and initialized to the average features of all instance images of that class, treating each class feature in memory as a sample prototype.

[0041] Furthermore, the neighborhood-invariant feature f di Calculate the similarity with multiple sample memories to obtain the contrastive loss for each memory location, including:

[0042] In the reliable sample memory, contrastive learning is performed by narrowing the distance between positive samples and widening the distance between negative samples. The contrastive loss is calculated within the domain, and the contrastive loss in the reliable sample memory is as follows:

[0043]

[0044]

[0045] in, This represents the contrast loss from samples in the source domain. n represents the contrastive loss from samples in the target domain. s and n t Let q represent the number of categories in the source and target domains, respectively. s and q t c represents query samples from the source domain and the target domain, respectively. + It is the prototype of the query with the same label in the memory library. and This represents the negative prototype sample corresponding to the query sample, where τ is a hyperparameter set to 0.05;

[0046] In the hard sample memory, two hard sample memories are used. By calculating the similarity within each hard sample memory and adopting the same learning strategy as the reliable sample memory, the contrastive loss in the hard sample memory is obtained as follows:

[0047]

[0048] in, Let h represent the loss function for the memory of the nth hard sample, q be the query sample from the target domain, and h be the loss function for the memory of the hard sample. + h represents the hard prototype of pseudo-labels in the memory database that are identical to the query sample. i This represents the negative prototype sample corresponding to the query sample.

[0049] Furthermore, the network parameters are updated using contrastive loss, domain classification loss, and orthogonality loss for each memory location. Simultaneously, memory features are updated based on the input query samples to learn relationships between samples. When the network model converges, training is complete. Pedestrian re-identification is then performed using the trained network model to obtain the identification results, including:

[0050] The overall framework loss of the pedestrian re-identification network model includes: reliable sample in-memory comparison loss, multiple hard sample in-memory comparison loss, orthogonality loss, and domain classification loss. Each mini-batch contains n source domain queries. and queries for n target domains The overall loss function is as follows:

[0051]

[0052] here and These are the training losses from the source and target domains, respectively, in the memory of reliable samples. The loss is the amount of memory from the hard samples, where n is the number of hard samples in memory, and L is the value of the memory. I For orthogonal loss, L dom For the domain classification loss, μ and θ are hyperparameters, with μ set to 0.5 and θ set to 4.

[0053] In addition to optimizing the network model parameters through the overall loss function, the memory prototype is also updated by querying samples;

[0054] For updating the hard sample in-memory prototype, the prototype in each hard sample in-memory library is updated by hard query features. These features share the same pseudo-labels in mini-batches and have low similarity in similarity ranking combinations. The similarity is sorted from smallest to largest using the argsort function, and query features are selected as follows:

[0055]

[0056] Where K is the number of instance features belonging to the same pseudo-label cluster in each mini-batch, and r i It is the prototype of the i-th query sample in the reliable sample memory database. For each mini-batch, the k-th query sample belongs to the instance features of the i-th pseudo-label cluster. and r i They belong to the same pseudo-label; the positive prototypes in the reliable sample memory library are used to represent the cluster center points, and the similarity between the query features and the positive prototypes in the reliable sample memory library is calculated to obtain the similarity permutation and combination;

[0057] The hard sample memory prototype update strategy is as follows:

[0058]

[0059] in, This represents the prototype of the difficulty of the i-th query in the n-th difficulty sample memory database. This represents the nth query sample in the similarity permutation index, where α is the momentum coefficient;

[0060] During the reliable sample memory update process, reliable samples from the target domain are used to update the reliable sample memory prototype. The similarity between the query sample and the corresponding positive prototype in the reliable sample memory is determined by using similarity permutations and combinations. Difficult samples with low similarity are identified and removed in subsequent reliable memory updates. All samples from the source domain are marked as reliable and suitable for immediate use. The reliable sample memory prototype update strategy is as follows:

[0061]

[0062] In the formula, c i This represents the prototype sample of the i-th query sample in the source domain. This represents the average value of the instance samples of the i-th query in the mini-batch; within the target domain, This represents the average value of the i-th query after removing difficult samples;

[0063] As backpropagation optimizes and updates the network model parameters and the memory prototype, the network acquires more discriminative features and fully learns the intra-class diversity of the data. When the network model converges, training is complete. Pedestrian re-identification is then performed using the trained network model to obtain the recognition results.

[0064] In addition, to achieve the above objectives, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the pedestrian re-identification method.

[0065] In addition, to achieve the above objectives, the present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the pedestrian re-identification method described above.

[0066] This invention addresses the problems in unsupervised adaptive person re-identification, namely the inability to fully learn intra-class semantic diversity and extract discriminative feature representations. It proposes a person re-identification method based on a hybrid attention-decoupled re-identification network to enhance the discriminative power of domain-invariant person features, thereby forming reliable class boundaries and learning intra-class semantic diversity. By designing a hybrid attention module, domain-invariant feature representations can be strengthened from spatial and channel perspectives through attention weight decoupling. This forces the network to automatically utilize image regions and attribute cues that are beneficial for cross-domain re-identification. Furthermore, based on the enhanced domain-invariant feature representations, a multi-difficulty sample in-memory learning strategy is proposed to improve the intra-class diversity of target domain samples. This invention optimizes the feature learning process by updating the reliable sample memory library and multiple difficult sample memory libraries. By considering the relationships between samples within the same class, it can capture significant intra-class semantic changes and positively impact the accuracy of pseudo-labels. Attached Figure Description

[0067] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0068] Figure 1 This is a flowchart illustrating the execution of a pedestrian re-identification method based on a hybrid attention-decoupled re-identification network proposed in this invention.

[0069] Figure 2 This is a framework diagram of a pedestrian re-identification method based on a hybrid attention-decoupled re-identification network proposed in this invention;

[0070] Figure 3 This is a diagram of a spatial attention structure based on decoupling in an embodiment of the present invention;

[0071] Figure 4 This is a structural diagram of the channel attention mechanism in an embodiment of the present invention;

[0072] Figure 5 This is a schematic diagram of the structure of an electronic device according to the present invention. Detailed Implementation

[0073] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0074] Please refer to Figure 1This embodiment provides a pedestrian re-identification method based on a hybrid attention-decoupled re-identification network, which mainly includes the following steps:

[0075] S1: Construct a hybrid attention module in the pedestrian re-identification network model, including a decoupled spatial attention mechanism and a channel attention mechanism;

[0076] S2: Transfer source domain data D s and target domain data D t The initial feature f is obtained by feeding it into the initial feature extraction network;

[0077] S3: Feed the initial features into the hybrid attention module to obtain the domain-invariant features f. di and domain-specific features f ds , as well as domain classification loss and orthogonality loss;

[0078] S4: Apply the domain-invariant feature f di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain.

[0079] S5: Initialize multiple sample memories based on the multi-difficulty sample memory learning strategy;

[0080] S6: Apply the domain-invariant feature f di The similarity between the memory and multiple sample memories is calculated to obtain the contrast loss for each memory.

[0081] S7: Update the network parameters by using the contrast loss, domain classification loss and orthogonality loss of each memory. At the same time, update the memory features according to the input query samples to learn the relationship between samples. When the network model converges, the training is completed. Pedestrian re-identification is performed through the trained network model to obtain the recognition result.

[0082] The steps in the above embodiments are described in detail below.

[0083] Based on, but not limited to, the above embodiments, in step S1, a hybrid attention module is constructed, such as... Figure 2 As shown, the network consists of a decoupled spatial attention mechanism and a channel attention mechanism, and its specific steps are as follows:

[0084] S101: Construct a module consisting of a decoupled spatial attention mechanism and a channel attention mechanism;

[0085] S102: This module is mainly used to extract domain-invariant features for subsequent classification tasks.

[0086] Based on, but not limited to, the above embodiments, step S2 includes the following steps:

[0087] S201: The initial feature extraction network uses a five-stage ResNet-50, where the stride of the fifth stage is changed from 2 to 1, which preserves more image detail. The ResNet-50 network has been pre-trained on ImageNet.

[0088] S202: Feed the source domain data and target domain data into the pre-trained ResNet-50 network to obtain initial features.

[0089] Based on, but not limited to, the above embodiments, step S3 specifically includes:

[0090] S301: The initial features are fed into the hybrid attention module. First, the initial features pass through the decoupling module to obtain the domain-invariant weights w. i Domain-specific weights w s ;

[0091] S302: Based on the principle of vector orthogonality, orthogonal loss is used to ensure that the neighborhood-invariant weights w are maintained. i Domain-specific weights w s They are mutually orthogonal, thus making domain-invariant features and domain-specific features independent of each other, and the orthogonal loss L I for:

[0092]

[0093]

[0094] in and It is a domain-invariant weight w i Domain-specific weights w s After average pooling, || || 2 L2 regularization is represented by ||, which represents the absolute value operation. I[i,j] indicates that the value at position (i,j) belongs to L2 regularization. The value of b represents the number of samples in each batch, and c represents the number of channels;

[0095] S303: Based on the domain-invariant weight w i Domain-specific weights w s Obtain the domain-invariant feature f di and domain-specific features f ds Its formula is:

[0096] W i =E dm (f), W s =1-W i (3)

[0097]

[0098] in E represents element-wise multiplication. dm For decoupling modules;

[0099] S304: Utilizing domain-specific features f ds Computational domain classification loss, such that f ds Includes more domain-specific feature information, domain classification loss L dom for:

[0100] L dom =-[D L logP+(1-D L )log(1-P)] (5)

[0101] P = E dc (f ds (6)

[0102] Where D L These are domain labels; the source domain's domain label is 0, and the target domain's domain label is 1. dc Let P be the domain classification module, where P is the domain classification probability.

[0103] Based on, but not limited to, the above embodiments, S4 specifically includes:

[0104] S401: Before each round of training, clustering is performed using DBSCAN. The preferred parameters in DBSCAN are as follows: the parameter k in k-distance is set to 30, the minimum number of neighbors n is set to 4, the distance threshold between sample pairs is set to 0.6 for the DukeMTMC-reID and Market-1501 datasets, and the distance threshold is 0.7 for the MSMT17 dataset.

[0105] It should be noted that the specific values ​​of each parameter in this embodiment are preferred implementations. In other embodiments, adjustments can be made according to the actual situation.

[0106] S402: During each training round, the pseudo-labels for the data in the target domain are the results of the previous round of model optimization.

[0107] Based on, but not limited to, the above embodiments, S5 specifically includes:

[0108] S501: The multi-difficulty memory learning strategy consists of reliable sample memory and multiple difficult sample memory;

[0109] S502: The reliable sample memory and multiple difficult sample memories use the same initialization strategy. Initialization is performed at the beginning of each training round, and each memory is initialized with the average features of all instance images of that class, treating each class feature in memory as a sample prototype.

[0110] Based on, but not limited to, the above embodiments, S6 specifically includes:

[0111] S601: In the reliable sample memory, contrastive learning is performed by narrowing the distance between positive samples and widening the distance between negative samples. This invention calculates the contrastive loss only within the domain. The contrastive loss in the reliable sample memory is as follows:

[0112]

[0113]

[0114] in, This represents the contrast loss from samples in the source domain. n represents the contrastive loss from samples in the target domain. s and n t Let q represent the number of categories in the source and target domains, respectively. s and q t c represents query samples from the source domain and the target domain, respectively. + It is the prototype of the query with the same label in the memory library. and This represents the negative prototype sample corresponding to the query sample, where τ is a hyperparameter, preferably set to 0.05;

[0115] S602: In the hard sample memory, this invention uses two hard sample memories. By calculating the similarity within each hard sample memory and adopting a learning strategy consistent with the reliable sample memory, the contrastive loss in the hard sample memory is obtained as follows:

[0116]

[0117] in, Let h represent the loss function for the memory of the nth hard sample, q be the query sample from the target domain, and h be the loss function for the memory of the hard sample. + h represents the hard prototype of pseudo-labels in the memory database that are identical to the query sample. i This represents the negative prototype sample corresponding to the query sample.

[0118] Based on, but not limited to, the above embodiments, step S7 specifically includes:

[0119] S701: The overall framework loss of the pedestrian re-identification network model includes: reliable sample in-memory comparison loss, multiple difficult sample in-memory comparison loss, orthogonality loss, and domain classification loss. Each mini-batch contains n source domain queries. and queries for n target domains The overall loss function is as follows:

[0120]

[0121] here and These are the training losses from the source and target domains, respectively, in the memory of reliable samples. The loss is the amount of memory from the hard samples, where n is the number of hard samples in memory, and L is the value of the memory. I For orthogonal loss, L dom For the domain classification loss, μ and θ are hyperparameters, with μ set to 0.5 and θ set to 4.

[0122] S702: In addition to optimizing the network model parameters through the overall loss function, the in-memory prototype is also updated using query samples. For updating the hard sample in-memory prototype, the prototype in each hard sample's in-memory library is updated by hard query features that share the same pseudo-labels in mini-batches and have low similarity in the similarity ranking combination. Here, similarity is measured using cosine similarity; the smaller the value, the lower the similarity. Then, the query features are selected by sorting the similarities from smallest to largest using the argsort function as follows:

[0123]

[0124] Where K is the number of instance features belonging to the same pseudo-label cluster in each mini-batch, and r i It is the prototype of the i-th query sample in the reliable sample memory database. For each mini-batch, the k-th query sample belongs to the instance features of the i-th pseudo-label cluster. and r i They belong to the same pseudo-label; considering that using positive prototypes from the reliable sample memory database to represent cluster centers is more reliable than using positive prototypes from the hard sample memory database, the similarity between the query feature and the positive prototypes from the reliable sample memory database is calculated to obtain similarity permutations and combinations. The hard sample memory prototype update strategy is as follows:

[0125]

[0126] in, This represents the prototype of the difficulty of the i-th query in the n-th difficulty sample memory database. This represents the nth query sample in the similarity permutation index, where α is the momentum coefficient;

[0127] S703: During the reliable sample memory update process, only reliable samples from the target domain are used to update the reliable sample memory prototype to avoid damage from potentially incorrect pseudo-labeled samples. By using similarity permutations to determine the similarity between the query sample and the corresponding positive prototype in the reliable sample memory, difficult samples with low similarity can be identified and excluded in subsequent reliable memory updates. Furthermore, all samples in the source domain are labeled as reliable and suitable for immediate use. The reliable sample memory prototype update strategy is as follows:

[0128]

[0129] In the formula, c i This represents the prototype sample of the i-th query sample in the source domain. This represents the average value of the instance samples of the i-th query in the mini-batch; within the target domain, This represents the average value of the i-th query after removing difficult samples;

[0130] S704: As backpropagation optimizes and updates the network model parameters and the memory prototype, the network acquires more discriminative features and fully learns the intra-class diversity of the data. When the network model converges, training is complete. Pedestrian re-identification is performed using the trained network model to obtain the recognition result.

[0131] The following provides a detailed explanation of the relevant aspects of the method:

[0132] (1) The following is a detailed explanation of the decoupled spatial attention module.

[0133] This method attempts to obtain domain-invariant and domain-specific spatial attention weights from features extracted from the ResNet-50 model by decoupling representation learning. The two attention weight vectors are decomposed by applying vector decomposition to make the independent factors mutually orthogonal.

[0134] like Figure 3 As shown, given an input image x, the feature map f is generated by the feature extractor E. res Obtained. Once from the decomposition module E dm From w i Domain-invariant feature f di The element-wise multiplication operation will then be used to separate w from f. Specifically, in this embodiment, w is considered... s It is 1 and w i The difference between them, and the corresponding domain-specific features, can be expressed as formulas 3 and 4.

[0135] (2) The channel attention mechanism is explained in detail as follows.

[0136] This method adds a channel attention mechanism after the third layer of the ResNet50 backbone network, so that the model can pay more attention to the important information of pedestrian images in the channels.

[0137] To efficiently compute channel attention, the input feature map is compressed in spatial dimension. This is achieved by applying max pooling and average pooling to the feature map f. in Pooling is performed to aggregate the spatial information of the feature maps, and corresponding pooled features f are generated respectively. max and f avg The features from both pools are forwarded to a shared multi-layer convolutional layer M. c Then the obtained feature M c (f max ) and M c (f avg The sum of the two samples is then passed through a sigmoid function and finally added to the input feature map f. in Multiply to obtain the channel attention map. like Figure 4 As shown. In short, the method for calculating channel attention is:

[0138] f max =MaxPool(f in ), f avg =AvgPool(f in (14)

[0139]

[0140] Where σ is the sigmoid function, f in It is the input feature map.

[0141] (3) The following is a detailed explanation of the in-memory learning strategy for multiple difficult samples:

[0142] After feature decomposition, the domain-invariant feature representation can be used for cross-domain feature clustering. By dividing cluster samples into reliable and hard samples, the discriminative feature representation of the model and the relationships between samples can be learned. This invention allocates one memory space for reliable samples and two memory spaces for hard samples. The size of the reliable sample memory is set to (n... s +n t )×C,n s It is the number of source domain categories, n t Where is the number of clusters in the target domain, and C is the number of feature channels. For hard samples, the memory size is set to n×n. t ×C, where n is the amount of memory for difficult samples.

[0143] (4) The following is a detailed explanation of the prototype update in memory for difficult samples:

[0144] After obtaining the similarity permutations using Formula 11, the first n query features are considered as hard samples, and their number equals the size of the hard sample memory. The first hard sample memory is updated using the sample with the lowest similarity in the permutation; the second sample with the lowest similarity is used to update the second hard sample memory, and so on, until all hard sample memorys have been updated. The remaining query features are treated as reliable samples and used to update the reliable sample memory.

[0145] During training, the hard sample memory is updated after each iteration. The commonly used PK sampling method is used to sample the P classes that make up each mini-batch and the K queries within each class. Each iteration changes the instance features corresponding to all K identities in the mini-batch to maintain consistency in memory updates.

[0146] To verify the effectiveness of the method of this invention, experiments were conducted on three mainstream person re-identification datasets, including Market-1501, DukeMTMC-reID, and MSMT17:

[0147] Market-1501 contains 32,668 images from 1,501 pedestrians, taken by 6 different cameras. The dataset is divided into 12,936 images from 751 randomly sampled pedestrians for training and 19,732 images from the remaining 750 pedestrians for testing, including 3,368 query images.

[0148] DukeMTMC-reID is a dataset consisting of 36,411 images of 1,812 pedestrians, collected from 8 cameras. The majority of the dataset comprises 1,404 pedestrians appearing in multiple cameras. Of these, 16,522 images of 702 pedestrians were used for training, while the remaining 2,228 query images of the 702 pedestrians and 17,611 gallery images were used for testing.

[0149] MSMT17 consists of 126,441 images of 4,101 identities captured by 15 cameras from different angles. The training set contains 32,621 images, including 1,041 pedestrians. The test set contains 3,060 pedestrians and 93,820 images, including 11,659 query images and 82,161 gallery images.

[0150] In this embodiment, the average precision (mAP) and the rank-1 / 5 / 10 of the cumulative matching features (CMC) are used to evaluate the performance of the proposed method in the target domain. The specific evaluation of the experimental performance is shown in Tables 1 and 2:

[0151] Table 1: Evaluation of experimental performance on Duke-to-Market and Market-to-Duke.

[0152]

[0153]

[0154] Table 2: Evaluation of experimental performance on Duke-to-MSMT and Market-to-MSMT

[0155]

[0156] The experimental results in Tables 1 and 2 show that our method is significantly better than existing methods, demonstrating its superiority.

[0157] like Figure 5 The diagram illustrates the physical structure of an electronic device, which may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640. The processor 610, communication interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the steps of the aforementioned person re-identification method, specifically including: constructing a hybrid attention module in the person re-identification network model, including a decoupled spatial attention mechanism and a channel attention mechanism; and processing source domain data D. s and target domain data D t The initial features are fed into the initial feature extraction network to obtain the initial features f; the initial features are then fed into the hybrid attention module to obtain the domain-invariant features f. di and domain-specific features f ds In addition, there are domain classification loss and orthogonal loss; the domain-invariant feature f is applied. di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain. Multiple sample memories are initialized according to the multi-hard sample memory learning strategy. The domain-invariant feature f is then... di The similarity between the network and multiple sample memory is calculated to obtain the contrast loss of each memory. The network parameters are updated by the contrast loss of each memory, the domain classification loss and the orthogonality loss. At the same time, the memory features are updated according to the input query sample to learn the relationship between samples. When the network model converges, the training is completed. The trained network model is used to perform pedestrian re-identification to obtain the recognition result.

[0158] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] In another aspect, embodiments of the present invention also provide a storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the aforementioned person re-identification method, specifically including: constructing a hybrid attention module in a person re-identification network model, including a decoupled spatial attention mechanism and a channel attention mechanism; and processing source domain data D... s and target domain data D t The initial features are fed into the initial feature extraction network to obtain the initial features f; the initial features are then fed into the hybrid attention module to obtain the domain-invariant features f. di and domain-specific features f ds In addition, there are domain classification loss and orthogonal loss; the domain-invariant feature f is applied. di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain. Multiple sample memories are initialized according to the multi-hard sample memory learning strategy. The domain-invariant feature f is then... di The similarity between the network and multiple sample memory is calculated to obtain the contrast loss of each memory. The network parameters are updated by the contrast loss of each memory, the domain classification loss and the orthogonality loss. At the same time, the memory features are updated according to the input query sample to learn the relationship between samples. When the network model converges, the training is completed. The trained network model is used to perform pedestrian re-identification to obtain the recognition result.

[0160] The beneficial effects of implementing this invention are:

[0161] (1) This invention proposes a hybrid attention-based decoupled re-identification network for unsupervised adaptive pedestrian re-identification. Domain-invariant features are separated from domain-specific features for use in pedestrian re-identification tasks. The hybrid attention module is designed to automatically select discriminative image regions and attribute cues to improve the accuracy of target sample clusters.

[0162] (2) This invention proposes a multi-difficulty sample in-memory learning strategy, providing reliable samples and a multi-difficulty sample in-memory library. By updating different in-memory libraries, the boundaries of clusters are identified, enabling the person re-identification network model to model the distribution of intra-class diversity.

[0163] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0164] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as identifiers.

[0165] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A pedestrian re-identification method based on a hybrid attention-decoupled re-identification network, characterized in that, Includes the following steps: Construct a hybrid attention module in the pedestrian re-identification network model, including a decoupled spatial attention mechanism and a channel attention mechanism; Source domain data D s and target domain data D t The initial feature f is obtained by feeding it into the initial feature extraction network; The initial features are fed into the hybrid attention module to obtain the domain-invariant features f. di and domain-specific features f ds , as well as domain classification loss and orthogonality loss; The domain-invariant feature f di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain. Based on the multi-difficulty sample memory learning strategy, initialize multiple sample memories; The domain-invariant feature f di The similarity between the memory and multiple sample memories is calculated to obtain the contrast loss for each memory. The network parameters are updated by using contrastive loss, domain classification loss and orthogonality loss for each memory location. At the same time, the memory features are updated based on the input query samples to learn the relationships between samples. When the network model converges, the training is complete. The trained network model is then used for pedestrian re-identification to obtain the recognition results. The initial features are fed into the hybrid attention module to obtain the domain-invariant features f. di and domain-specific features f ds In addition, there are domain classification loss and orthogonality loss, including: The initial features are fed into the hybrid attention module. First, the initial features pass through a decoupling module to obtain the domain-invariant weights w. i Domain-specific weights w s ; Based on the principle of vector orthogonality, orthogonal loss is used to ensure that the neighborhood-invariant weights w are maintained. i Domain-specific weights w s They are mutually orthogonal, thus making domain-invariant features and domain-specific features independent of each other, and the orthogonal loss L I for: in and It is a domain-invariant weight w i Domain-specific weights w s After average pooling, ||| 2 L2 regularization is represented by ||, the absolute value operation is represented by I[i,j], and the value at position (i,j) belongs to the L2 regularization. The value of b represents the number of samples in each batch, and c represents the number of channels; Based on the domain-invariant weight w i Domain-specific weights w s Obtain the domain-invariant feature f di and domain-specific features f ds Its formula is: W i =E dm (f),W s =1-W i in E represents element-wise multiplication. dm For decoupling modules; Utilizing domain-specific features f ds Computational domain classification loss, such that f ds Includes more domain-specific feature information, domain classification loss L dom for: L dom =-[D L logP+(1-D L )log(1-P)] P=E dc (f ds ) Where D L These are domain labels; the source domain's domain label is 0, and the target domain's domain label is 1. dc For the domain classification module, P is the domain classification probability; The domain-invariant feature f di Calculate the similarity with multiple sample memories to obtain the contrastive loss for each memory location, including: In the reliable sample memory, contrastive learning is performed by narrowing the distance between positive samples and widening the distance between negative samples. The contrastive loss is calculated within the domain, and the contrastive loss in the reliable sample memory is as follows: in, This represents the contrast loss from samples in the source domain. n represents the contrastive loss from samples in the target domain. s and n t Let q represent the number of categories in the source and target domains, respectively. s and q t c represents query samples from the source domain and the target domain, respectively. + It is the prototype of the query with the same label in the memory library. and This represents the negative prototype sample corresponding to the query sample, where τ is a hyperparameter set to 0.05; In the hard sample memory, two hard sample memories are used. By calculating the similarity within each hard sample memory and adopting the same learning strategy as the reliable sample memory, the contrastive loss in the hard sample memory is obtained as follows: in, Let h represent the loss function for the memory of the nth hard sample, q be the query sample from the target domain, and h be the loss function for the memory of the nth hard sample. + h represents the hard prototype of pseudo-labels in the memory database that are identical to the query sample. i This represents the negative prototype sample corresponding to the query sample; The network parameters are updated using contrastive loss, domain classification loss, and orthogonality loss for each memory location. Simultaneously, memory features are updated based on the input query samples to learn relationships between samples. When the network model converges, training is complete. The trained network model is then used for pedestrian re-identification to obtain the identification results, including: The overall framework loss of the pedestrian re-identification network model includes: reliable sample in-memory comparison loss, multiple hard sample in-memory comparison loss, orthogonality loss, and domain classification loss. Each mini-batch contains n source domain queries. and queries for n target domains The overall loss function is as follows: here and These are the training losses from the source and target domains, respectively, in the memory of reliable samples. The loss is the amount of memory from the hard samples, where n is the number of hard samples in memory, and L is the value of the memory. I For orthogonal loss, L dom For the domain classification loss, μ and θ are hyperparameters, with μ set to 0.5 and θ set to 4; In addition to optimizing the network model parameters through the overall loss function, the in-memory prototype is also updated by querying samples; For updating the hard sample in-memory prototype, the prototype in each hard sample in-memory library is updated by hard query features. These features share the same pseudo-labels in mini-batches and have low similarity in similarity ranking combinations. The similarity is sorted from smallest to largest using the argsort function, and query features are selected as follows: Where k is the number of instance features belonging to the same pseudo-label cluster in each mini-batch, and r i It is the prototype of the i-th query sample in the reliable sample memory database. For each mini-batch, the k-th query sample belongs to the instance features of the i-th pseudo-label cluster. and r i They belong to the same pseudo-label; the positive prototypes in the reliable sample memory library are used to represent the cluster center points, and the similarity between the query features and the positive prototypes in the reliable sample memory library is calculated to obtain the similarity permutation and combination; The hard sample memory prototype update strategy is as follows: in, This represents the prototype of the difficulty of the i-th query in the n-th difficulty sample memory database. This represents the nth query sample in the similarity permutation index, where α is the momentum coefficient; During the reliable sample memory update process, reliable samples from the target domain are used to update the reliable sample memory prototype. The similarity between the query sample and the corresponding positive prototype in the reliable sample memory is determined by using similarity permutations and combinations. Difficult samples with low similarity are identified and removed in subsequent reliable memory updates. All samples from the source domain are marked as reliable and suitable for immediate use. The reliable sample memory prototype update strategy is as follows: In the formula, c i This represents the prototype sample of the i-th query sample in the source domain. This represents the average value of the instance samples of the i-th query in the mini-batch; within the target domain, This represents the average value of the i-th query after removing difficult samples; As backpropagation optimizes and updates the network model parameters and the memory prototype, the network acquires more discriminative features and fully learns the intra-class diversity of the data. When the network model converges, training is complete. Pedestrian re-identification is then performed using the trained network model to obtain the recognition results.

2. The pedestrian re-identification method as described in claim 1, characterized in that, The hybrid attention module in the pedestrian re-identification network model includes: Construct a hybrid attention module consisting of a decoupled spatial attention mechanism and a channel attention mechanism; This hybrid attention module is used to extract domain-invariant features for subsequent recognition and classification tasks.

3. The pedestrian re-identification method as described in claim 1, characterized in that, The source domain data D s and target domain data D t The data is fed into the initial feature extraction network to obtain initial features, including: The ResNet-50 network with five stages was used as the initial feature extraction network. The convolution stride of the fifth stage was changed from 2 to 1. The ResNet-50 network had been pre-trained on ImageNet. The source domain data and target domain data are fed into the pre-trained ResNet-50 network to obtain initial features.

4. The pedestrian re-identification method as described in claim 1, characterized in that, The domain-invariant feature f di The DBSCAN clustering method is used to cluster samples in the target domain, and pseudo-labels are obtained for supervised training in the target domain, including: Before each round of training, clustering was performed using DBSCAN. For the parameters in DBSCAN, the parameter k in k-distance was set to 30, the minimum number of neighbors n was set to 4, the distance threshold between sample pairs was set to 0.6 for the DukeMTMC-reID and Market-1501 datasets, and the distance threshold was 0.7 for the MSMT17 dataset. During each training round, the pseudo-labels for the data in the target domain are the results of the previous round of model optimization.

5. The pedestrian re-identification method as described in claim 1, characterized in that, The initialization of multiple sample memories based on the multi-difficulty sample memory learning strategy includes: The multi-difficulty memory learning strategy consists of reliable sample memory and multiple difficult sample memory. The reliable sample memory and multiple hard sample memory use the same initialization strategy, which is initialized at the beginning of each training round and initialized to the average features of all instance images of that class, treating each class feature in memory as a sample prototype.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the pedestrian re-identification method as described in any one of claims 1-5.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the pedestrian re-identification method as described in any one of claims 1-5.

Images