An unsupervised pedestrian retrieval method based on part-guided fusion transformation network

Abstract

The application relates to the technical field of computer vision, and specifically discloses an unsupervised pedestrian retrieval method based on a component-guided fusion transformation network, which extracts global and component features of pedestrians through a component-guided fusion transformation network; adopts a component-guided fusion strategy to generate component-global fusion features for component-guided contrast learning; performs average aggregation on the component features to obtain component aggregation features; obtains corresponding clustering centers based on the global features and the component aggregation features through a clustering algorithm, and constructs a double-flow dynamic memory library; further constructs a component-guided regularization module to constrain the consistency between the global features and the component aggregation features through a dynamic weighting coefficient; and finally iteratively optimizes the parameters of the component-guided fusion transformation network and the component-guided regularization module through a total loss calculation module. The application optimizes feature distribution through unsupervised learning, reduces the dependence on labeled data, and improves the pedestrian feature representation capability and the model generalization performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly to a non-supervised pedestrian retrieval method based on component guided fusion transformation network. BACKGROUND

[0002] Pedestrian retrieval, as a key technology in the field of computer vision, aims to achieve accurate matching of pedestrian identities across camera scenes, and has important application value in intelligent security, video surveillance and other scenarios. With the development of deep learning, supervised learning-based pedestrian retrieval methods have made significant progress by training deep models with large-scale labeled data. However, such methods face challenges such as high labeling cost and weak cross-domain generalization ability in practical applications. In recent years, researchers have attempted to introduce the Transformer model into the pedestrian retrieval task, using its global attention mechanism to model long-range dependencies. For example, TransReID proposed by He et al. learns discriminative representations of pedestrian features through a pure Transformer architecture, and DC-Former designed by Li et al. enhances feature expression ability by constructing multiple embedding subspaces. In addition, some methods embed the Transformer module into the convolutional neural network to fuse local and global features to enhance model performance. Although these supervised learning methods perform well on specific datasets, their high dependence on labeled data leads to high deployment costs, and models are easily affected by data distribution differences in cross-domain scenarios, resulting in significant decline in generalization performance.

[0003] To reduce the dependence on labeled data, unsupervised pedestrian retrieval methods have become a research hotspot, mainly divided into two categories: unsupervised domain adaptation (UDA) and pure unsupervised learning (USL). Unsupervised domain adaptation methods pre-train the model in the source domain and then transfer it to the target domain for optimization. Typical methods include style transfer technology based on generative adversarial networks (GAN) and dynamic optimization strategy based on pseudo-labels. For example, some studies use GAN to generate pedestrian images in the target domain style to reduce the domain difference, while other methods generate pseudo-labels through clustering to guide the training of the target domain model. However, such methods require the data distribution of the source domain and the target domain to be highly related, while in actual cross-scene applications, the domain difference often leads to a sharp decline in model performance. Pure unsupervised learning methods rely entirely on unlabeled data in the target domain, and improve feature discriminability through pseudo-label iteration optimization and dynamic memory update. For example, the SpCL framework distinguishes between source domain classes and target domain clusters through self-contrast learning, and the HCM model combines pedestrian identity-level and image-level contrast learning to mine difficult sample information. Recently, researchers have tried to introduce the global interaction mechanism of Transformer into the unsupervised framework, such as enhancing pedestrian expression by aggregating global features. However, existing methods focus on global feature learning and lack fine-grained modeling of local regions of pedestrians (such as body parts and clothing details), resulting in insufficient feature discriminability in complex scenes such as occlusion and pose changes.

[0004] At the same time, as the core technology of unsupervised learning, contrastive learning optimizes feature space distribution by constructing positive and negative sample pairs. Existing methods such as cluster contrast learning (C-Contrast) improve discriminability by mining difficult samples, and camera contrast learning (CCL) uses camera information to construct proxy tasks to optimize feature expression. In addition, the AICES scheme enhances fine-grained knowledge learning by aggregating instance relationships perceived by the camera. However, existing contrastive learning methods have significant limitations: first, they mainly rely on global feature contrast, ignoring the modeling of local region (such as head, torso, and stripes) relevance, resulting in loss of local semantic information; second, the feature optimization process lacks explicit constraints on local feature consistency, making it vulnerable to background interference when cross-domain matching; third, the memory update strategy is mainly based on global feature mean, which fails to effectively capture the spatiotemporal dynamic association of local features, limiting the fine-grained discriminability. For example, in pedestrian images, the local features of the shirt stripes and the pants pattern may have strong discriminability, but existing methods are difficult to model and optimize such local relationships.

[0005] The deficiencies of the prior art can be attributed to the following core problems: the application of the Transformer model in pedestrian retrieval is mostly limited to global feature interaction, lacking explicit modeling of local region structured relationships, resulting in insufficient utilization of detailed information; the unsupervised contrast learning framework relies too much on global feature similarity measurement, without building contrast relationships between local regions, making it difficult to cope with pose and view changes in complex scenes; the semantic consistency constraint between global and local features is missing in the feature learning process, making it difficult to optimize multi-granularity features collaboratively, affecting the robustness of the model. SUMMARY

[0006] The purpose of the present application is to provide a non-supervised pedestrian retrieval method based on a component-guided fusion transformation network, which realizes the bidirectional interaction of global and local features by designing a component-guided fusion transformation network, strengthens the local discriminativeness through the dual constraints of identity attributes and component semantics by developing a component-guided contrast loss, and ensures the consistency of multi-granularity features from the two dimensions of feature space and clustering distribution through component-guided regularization, thereby significantly improving the representation ability of pedestrian features and the generalization performance of the model in the non-supervised pedestrian retrieval task.

[0007] To achieve the above purpose, the present application provides a non-supervised pedestrian retrieval method based on a component-guided fusion transformation network, comprising the following steps:

[0008] S1, constructing a component-guided fusion transformation network based on a pre-trained visual Transformer model to extract global features and component features of pedestrians;

[0009] S2, generating component-global fusion features using a component-guided fusion strategy for component-guided contrast learning; performing average aggregation processing on the component features of pedestrians to obtain component aggregation features;

[0010] S3, based on the global features and the component aggregation features, using a clustering algorithm to obtain corresponding clustering centers, and constructing a double-flow dynamic memory bank;

[0011] S4, constructing a component-guided regularization module to constrain the consistency between the global features and the component aggregation features by introducing a dynamic weighting coefficient;

[0012] S5, constructing a total loss calculation module to calculate a total loss value, and iteratively optimizing the parameters of the component-guided fusion transformation network and the component-guided regularization module based on the total loss value.

[0013] Preferably, in S1, the constructed component-guided fusion transformation network comprises one transformation layer, one global encoder and one component encoder, based on a pre-trained visual Transformer model containing The visual Transformer model of the transformation layer, i.e., the ViT model, initializes parameters of the part-guided fusion transformation network, and the initialization process is specifically as follows:

[0014] The parameters of the first layer of the ViT model correspond to the parameters of the first layer of the part-guided fusion transformation network. The parameters of the second layer of the ViT model correspond to the parameters of the second layer of the part-guided fusion transformation network. The parameters of the third layer of the ViT model correspond to the parameters of the global encoder and the part encoder in the part-guided fusion transformation network. The parameters of the fourth layer of the ViT model correspond to the parameters of the global encoder and the part encoder in the part-guided fusion transformation network.

[0015] Preferably, in S1, the pedestrian image is scaled and normalized to obtain a preprocessed pedestrian image ; the preprocessed pedestrian image is input into the initialized part-guided fusion transformation network to obtain global features and part features of the pedestrian.

[0016] Preferably, S2 includes the following steps:

[0017] S21, the global features and the part features of the pedestrian are fused by using a part-guided fusion strategy to generate part-global fusion features The fusion manner of the global features and the part features is as follows:

[0018] ;

[0019] wherein, is a coupling factor, represents the i-th part-global fusion feature of the j-th pedestrian image, represents the i-th part feature of the j-th pedestrian image, represents the number of parts; The part-guided contrastive loss in the part-guided contrastive learning is as follows:

[0020]

[0021] ;

[0022] wherein, , and respectively represent the cosine similarity between positive sample parts, the cosine similarity between negative sample parts, and the cosine similarity between weak positive sample parts; is a smoothing factor, and is defined as:

[0023] ;

[0024] wherein,​​​​​ , , and , denotes taking an absolute value;

[0025] S22, performing average aggregation processing on the part features of the pedestrian to obtain part aggregation features , and the expression is:

[0026] ;

[0027] wherein, denotes the i-th part feature of the j-th pedestrian image, denotes the i-th part feature of the j-th pedestrian image, denotes the i-th part feature of the j-th pedestrian image, denotes the number of parts.

[0028] Preferably, in S21, for each part corresponding part-global fusion features in the pedestrian image, the positive sample part is defined as the same part of the same pedestrian image, the negative sample part is defined as the part of the different pedestrian image, and the weak positive sample part is defined as the other part of the same pedestrian image and the non-corresponding part of the same pedestrian image.

[0029] Preferably, S3 comprises:

[0030] The clustering algorithm is used to cluster the global features and the part aggregation features respectively, to obtain the corresponding clustering centers to construct a double-flow dynamic memory library, and the features stored in the double-flow dynamic memory library are randomly initialized;

[0031] The features stored in the double-flow dynamic memory library are updated by momentum in the following way:

[0032] ;

[0033] wherein, is a momentum coefficient, and denote the global features and the part aggregation features, and are the i-th clustering center obtained by clustering the global features and the part aggregation features, and denote the sample set under the same identity corresponding to the global features and the part aggregation features, denotes the number of samples in the set.

[0034] Preferably, S3 further comprises:

[0035] Unsupervised feature learning is performed on the batch data and the clustering centers in the double-flow dynamic memory library, and the global loss and the part loss​ The form expression is as follows:

[0036] ;

[0037] ;

[0038] wherein, and represent global features and negative pair similarities between the cluster centers in the dual-flow dynamic memory bank, and represent component aggregation features and negative pair similarities between the cluster centers in the dual-flow dynamic memory bank, is a temperature coefficient, is the number size of batch data.

[0039] Preferably, in S4, the global features and the component aggregation features are first mapped to the same feature space by using a fully connected layer; then a component-guided contrastive loss is designed to constrain the consistency between the global features and the component aggregation features, and the form is as follows:

[0040] ;

[0041] ;

[0042] wherein, represents a fully connected layer, represents a norm, is a dynamic weighting coefficient, represents the number of samples in the set, is an adjustment factor, and represent component aggregation features and the corresponding cluster clusters of global features .

[0043] Preferably, in S5, the total loss value is obtained by calculating the weighted sum of the component-guided contrastive loss , the global loss , the component loss , and the component-guided regular loss :

[0044] ;

[0045] wherein, is a weight factor;

[0046] Based on the total loss value, the parameters of the part-guided fusion transformation network and the part-guided regularization module are iteratively updated by gradient back propagation, and the form is as follows:

[0047] ;

[0048] wherein, denotes the learning rate, is the parameter of the current round in optimization, is the parameter of the previous round.

[0049] Therefore, the unsupervised pedestrian retrieval method based on the part-guided fusion transformation network has the following beneficial effects:

[0050] The part-guided fusion transformation network is designed to realize bidirectional interaction of global and local features, the part-guided contrastive loss is developed to strengthen local discriminativeness through double constraints of identity attribute and part semantics, and the part-guided regularization is used to guarantee consistency of multi-granularity features from two dimensions of feature space and clustering distribution, so that the representation ability of pedestrian features and the generalization performance of the model are significantly improved in the unsupervised pedestrian retrieval task.

[0051] The technical solutions of the present application will be further described in detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is the overall flowchart of an embodiment of the unsupervised pedestrian retrieval method based on the part-guided fusion transformation network of the present application. DETAILED DESCRIPTION

[0053] The technical solutions of the present application will be further described in detail below with the aid of the accompanying drawings and examples.

[0054] Unless otherwise defined, the technical terms or scientific terms used in the present application should be understood as the usual meaning understood by those skilled in the art in the field to which the present application belongs.

[0055] As Figure 1 shown, an unsupervised pedestrian retrieval method based on a part-guided fusion transformation network comprises the following steps:

[0056] S1, based on a pre-trained visual Transformer model, a part-guided fusion transformation network is constructed to extract global features and part features of pedestrians.

[0057] In this embodiment, the part-guided fusion transformation network constructed includes transform layers, a global encoder and a part encoder, based on a pre-trained visual Transformer model containing The visual Transformer model (ViT model) of each transformation layer initializes the parameters of the component-guided fusion transformation network. The initialization process is as follows:

[0058] ViT model before The parameters of the layer correspond to the components guiding the fusion transformation network. The parameters of the layer, the ViT model The parameters of the layer correspond to the parameters of the global encoder and the component encoder in the component-guided fusion transform network, respectively. In this embodiment... .

[0059] The pedestrian image is scaled and normalized to obtain the preprocessed pedestrian image. In this embodiment, Indicates the first Using the publicly available MSMT17 dataset as the training set, pedestrian images are scaled to a predefined scale. The size is adjusted, and all pixel values ​​of the pedestrian image are normalized, that is, reduced to between 0 and 1.

[0060] Preprocessed pedestrian images The initialized components guide the fusion transform network to obtain the global features of the pedestrian. and component features In this embodiment, the outputs of the global encoder and the component encoder correspond to global features, respectively. and component features ,Pick Indicates the number of parts.

[0061] S2. A component-global fusion feature is generated using a component-guided fusion strategy for component-guided comparative learning. Simultaneously, the pedestrian's component features are averaged and aggregated to obtain aggregated component features, including the following steps:

[0062] S21. Generate component-global fused features by fusing the global features and component features of pedestrians. Furthermore, the component-global fused features are used as input for component-guided contrastive learning. The fusion method of global features and component features is as follows:

[0063] ;

[0064] in, It is a coupling factor. Indicates the first The first pedestrian image In this embodiment, the component-global fusion feature is described. .

[0065] Component-guided contrastive loss in component-guided contrastive learning Designed as follows:

[0066] ;

[0067] in, , and These represent the cosine similarity between positive sample components, between negative sample components, and between weakly positive sample components, respectively. For the component-global fusion feature corresponding to each component in a pedestrian image, a positive sample component is defined as the same component in a pedestrian image with the same identity, a negative sample component is defined as a component in a pedestrian image with different identities, and a weakly positive sample component is defined as other components in a pedestrian image with the same identity and non-corresponding components in a pedestrian image with the same identity. The smoothing factor is defined as:

[0068] ;

[0069] in, , ,and , Indicates taking the absolute value;

[0070] S22. Perform average aggregation processing on the pedestrian component features to obtain component aggregate features. Its expression is:

[0071] ;

[0072] in, Indicates the first The first pedestrian image Individual component features, Indicates the number of parts.

[0073] S3. Based on global features and component aggregation features, clustering algorithms are used to obtain corresponding cluster centers, and a dual-stream dynamic memory library is constructed, including:

[0074] Clustering algorithms are used to cluster global features and component aggregation features respectively, and corresponding cluster centers are obtained to construct a dual-stream dynamic memory library. The features stored in the dual-stream dynamic memory library are randomly initialized. In this embodiment, the clustering algorithm used is the DBSCAN algorithm, but other clustering algorithms such as K-means can also be selected.

[0075] During iterative training, the features stored in the dual-stream dynamic memory database are updated with momentum in the following manner:

[0076] ;

[0077] in, For the momentum coefficient, in the embodiment, and represent global features and component aggregation features respectively, and are the first cluster centers obtained by clustering using global features and component aggregation features respectively, and represent sample sets under the same identity corresponding to global features and component aggregation features respectively, represents the number of samples in the set.

[0078] Next, unsupervised feature learning is performed using batch data and cluster centers in the dual-flow dynamic memory bank. The global loss and component loss in the unsupervised feature learning are expressed as follows:

[0079]

[0080]

[0081] wherein, and represent positive and negative pair similarities between global features and cluster centers in the dual-flow dynamic memory bank, and represent positive and negative pair similarities between component aggregation features and cluster centers in the dual-flow dynamic memory bank, is a temperature coefficient, is the size of the batch data; in the embodiment, , .

[0082] S4, a component-guided regularization module is constructed to constrain the consistency between global features and component aggregation features by introducing a dynamic weighting coefficient. The specific process is as follows:

[0083] First, the global features and component aggregation features are mapped to the same feature space using a fully connected layer; then, a component-guided regularization loss is designed to constrain the consistency between global features and component aggregation features, which is expressed as follows:

[0084]

[0085]

[0086] wherein, represents a fully connected layer, ​​​​​denotes norm, is a dynamic weighting coefficient, denotes the number of samples in the set, is an adjustment factor, . and denotes the component aggregation feature corresponding to the global feature of the cluster cluster.

[0087] S5, the total loss calculation module calculates the total loss value, and iteratively optimizes the parameters of the component guided fusion transformation network and the component guided regularization module based on the total loss value. Specifically:

[0088] The total loss value is obtained by calculating the weighted sum of the component guided contrast loss , the global loss , the component loss , and the component guided regularization loss :

[0089] ;

[0090] wherein, is a weight factor;

[0091] Based on the total loss value, the parameters of the component guided fusion transformation network and the component guided regularization module are iteratively updated by gradient backpropagation, and the form is as follows:

[0092] ;

[0093] wherein, denotes the learning rate, is the parameter of the current round in the optimization, is the parameter of the previous round.

[0094] In the preferred embodiment of the present application, , the SGD optimizer is used as the training optimizer of the model, the initial learning rate is set to 0.01, the cosine decay strategy is used for learning rate adjustment in iteration, and the total iteration round of training is 80.

[0095] It is worth noting that in the preferred embodiment of the present application, the training method of the network and the parameter configuration of the learning rate optimization strategy are a kind of non-restrictive optimal selection. The skilled person in the art can select the training method of the model and the parameter configuration according to various indicators such as classification accuracy and efficiency.

[0096] ​The pedestrian retrieval model obtained by training in the preferred embodiment of the present application is tested on the test set of the MSMT17 dataset, and the results are shown in Table 1.

[0097] Table 1 Comparison of results of models on the MSMT17 dataset

[0098] ;

[0099] In Table 1, the baseline method represents using the ViT model for global feature extraction, and the component guided method represents using the component guided fusion transformation network for global feature and component feature extraction.

[0100] As can be seen from Table 1, the average precision and ranking accuracy of the pedestrian retrieval model in the present embodiment reached 46.3% and 71.5%, which were increased by 6.8% and 4.9% compared with 39.5% and 66.6% of the baseline method. At the same time, the introduction of the component guided contrast loss and the component guided regularization can enhance the discriminability of the features, thereby further improving the performance of retrieval.

[0101] Therefore, the present application adopts the above-mentioned unsupervised pedestrian retrieval method based on the component guided fusion transformation network, realizes the bidirectional interaction of global and local features by designing the component guided fusion transformation network, develops the component guided contrast loss to strengthen the local discriminability through the dual constraints of identity attribute and component semantics, and guarantees the consistency of multi-granularity features from the two dimensions of feature space and clustering distribution through the component guided regularization, thereby significantly improving the representation ability of pedestrian features and the generalization performance of the model in the unsupervised pedestrian retrieval task.

[0102] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application but not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the present application, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A non-supervised pedestrian retrieval method based on part-guided fusion transformation network, characterized in that, The method comprises the following steps: S1, constructing a part-guided fusion transformation network based on a pre-trained visual Transformer model to extract global features and part features of pedestrians; S2, generating part-global fusion features by using a part-guided fusion strategy, and using the part-global fusion features for part-guided contrast learning; performing average aggregation processing on the part features of the pedestrians to obtain part aggregation features; S3, obtaining corresponding cluster centers by using a clustering algorithm based on the global features and the part aggregation features, and constructing a double-flow dynamic memory bank; S4, constructing a part-guided regularization module, and introducing a dynamic weighting coefficient to constrain the consistency between the global features and the part aggregation features; S5, constructing a total loss calculation module to calculate a total loss value, and iteratively optimizing parameters of the part-guided fusion transformation network and the part-guided regularization module based on the total loss value; S2 comprises the following steps: S21, adopt the component-guided fusion strategy to fuse the global feature and the component feature of the pedestrian, and generate component-global fusion feature The fusion manner of the global feature and the component feature is as follows: ； wherein, is a coupling factor, denotes the component-global fusion feature of the person image, denotes the component feature of the person image, denotes the number of components; Component guided contrastive learning in component guided contrastive loss For: ； wherein, , and respectively represent the cosine similarity between positive sample parts, between negative sample parts, and between weak positive sample parts; is a smoothing factor, defined as: ； wherein , , and , denotes taking the absolute value; S22, average aggregation processing is performed on the part features of the pedestrian to obtain part aggregated features The expression is: ； wherein, represents the number of component features of the th pedestrian image, th component feature, represents the number of components; In S21, for the part-global fusion features corresponding to each part in the pedestrian image, a positive sample part is defined as a same part of a same-identity pedestrian image, a negative sample part is defined as a part of a different-identity pedestrian image, and a weak positive sample part is defined as other parts of the same pedestrian image and non-corresponding parts of the same-identity pedestrian image; S3 comprises: performing clustering on the global features and the part aggregation features respectively by using a clustering algorithm to obtain corresponding cluster centers and construct a double-flow dynamic memory bank, and randomly initializing the features stored in the double-flow dynamic memory bank; The features stored in the double-flow dynamic memory bank are updated by momentum in the following manner: ； wherein, is a momentum coefficient, and denote global features and part-aggregated features, respectively, and are the first cluster centers obtained by clustering with global features and part-aggregated features, respectively, and denote sample sets under the same identity corresponding to global features and part-aggregated features, respectively, denotes the number of samples in the set. S3 further comprises: unsupervised feature learning with batch data and cluster centers in a dual-flow dynamic memory bank, a global loss and a component loss The form expressions are as follows: ； ； wherein, and represent global features positive and negative pair similarities between the component and the cluster centers in the dual-flow dynamic memory bank, and represent component aggregate features positive and negative pair similarities between the component and the cluster centers in the dual-flow dynamic memory bank, is a temperature coefficient, is a quantity size of the batch data. 2.The unsupervised pedestrian retrieval method based on part-guided fusion transformer network according to claim 1, wherein, In S1, the constructed part-guided fusion transformation network comprises one global encoder and one part encoder, and the part-guided fusion transformation network is initialized based on a visual Transformer model (ViT model) comprising one transformation layer, which is pre-trained on an ImageNet dataset, and the initialization process is specifically as follows: Before the ViT model The parameters of the layer correspond to the parameters of the part-guided fusion transformer network before The parameters of the layer correspond to the parameters of the part-guided fusion transformer network before The parameters of the layer correspond to the parameters of the part-guided fusion transformer network before 3.The unsupervised pedestrian retrieval method based on part-guided fusion transformer network according to claim 2, characterized in that, In S1, the pedestrian image is scaled and normalized to obtain a preprocessed pedestrian image ; The preprocessed pedestrian image is input into the initialized part-guided fusion transformation network to obtain global features of the pedestrian The initialized part-guided fusion transformation network is input to obtain global features of the pedestrian and part features . 4.The unsupervised pedestrian retrieval method based on part-guided fusion transformer network according to claim 3, wherein, In S4, firstly, the global feature and the part aggregated feature are mapped to the same feature space by using a fully connected layer; then a part-guided regularization loss is designed The consistency between the global feature and the part aggregated feature is constrained, which is in the following form: ； ； wherein, represents a fully connected layer, represents a norm, is a dynamic weighting coefficient, represents the number of samples in a set, is an adjustment factor, and represents component aggregated features corresponding to a cluster cluster of global features .

5. The unsupervised pedestrian retrieval method based on part-guided fusion transformer network according to claim 4, characterized in that, In S5, the component-guided contrast loss is calculated. Global loss Component loss and component-guided regularization loss The weighted sum gives the total loss value. : ； wherein is a weight factor; iteratively updating the parameters of the part-guided fusion transformation network and the part-guided regularization module by gradient backpropagation based on the total loss value, and the form is as follows: ； wherein, denotes a learning rate, is a parameter of the current round in the optimization, is a parameter of the previous round.

Images