Block-based stepwise transformer-based target re-identification method and system

Abstract

The application relates to the technical field of computer vision, and particularly discloses a target re-identification method and system based on inter-block ladder-shaped Transformer, which proposes a novel feature extraction network, namely an inter-block ladder-shaped Transformer (IBLSFormer) based on multi-subspace feature adjustment (MSFA), for target re-identification. The core of the IBLSFormer is an MSFA module. The IBLSFormer can dynamically adjust features in multiple subspaces such as an Euclidean distance subspace, a cosine distance subspace and a KL divergence subspace through class-patch interaction, enhance significant patch labels and weaken non-significant patch labels. In addition, a ladder-shaped constraint from narrow to wide is embedded in the internal encoder block MSFA to optimize feature learning. A large number of experiments show that the IBLSFormer is superior to other methods in learning discriminative and robust representations for target re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a target re-identification method and system based on inter-block ladder Transformer. BACKGROUND

[0002] Target re-identification is an important challenge in the field of computer vision, and has important significance in various applications such as video surveillance, vehicle-road cooperative systems and intelligent transportation systems. Its goal is to match targets under different cameras, so as to achieve seamless tracking and identification in different spatial and temporal contexts. In recent years, the development of computer technology has significantly promoted the progress of target re-identification. Although target re-identification has made great progress, it still faces many inherent challenges in real-world scenarios. These challenges mainly come from complex factors such as view angle changes, lighting conditions, occlusions and cluttered backgrounds. Traditional methods are difficult to cope with these problems, resulting in poor performance and limited applicability in the real world.

[0003] In recent years, with the rise of deep learning technology, target re-identification has made significant progress. Before the appearance of visual Transformer (ViT), convolutional neural networks (CNN) were considered as the cornerstone of target re-identification methods, which performed well in extracting discriminative features. CNN analyzes local regions of images through convolution operations, captures local attention, and can see more extensive regions as the network depth increases. Related research shows great interest in CNN-based methods. However, due to the limitations of convolution and pooling operations, CNN has obvious shortcomings in capturing long-distance dependencies. Transformer, as the dominant method in natural language processing, has achieved remarkable success. Researchers gradually extend Transformer to image tasks. In Transformer, each patch token is first converted into a vector, and the interaction between these patch tokens is represented through self-attention mechanisms. This enables the model to effectively learn the dependencies between patch tokens at different positions in the sequence. As an effective model, Transformer can aggregate features from a global perspective, facilitating the exploration of diverse internal features, far exceeding the capabilities of traditional CNN. Especially in the task of target re-identification, the attention mechanism performs particularly well in capturing long-distance dependencies. Therefore, researchers have introduced different Transformer-based methods, which have achieved significant improvements in target re-identification.

[0004] However, most existing research directly uses the encoder of the visual Transformer as the feature extraction module, and few researches improve the internal structure of the visual Transformer. First, compared with the Transformer using additional branches or image attributes, the optimization of the internal structure brings lower computational cost. In addition, the existing Transformer-based target re-identification method usually directly inputs the sequence feature into the encoder block, ignoring the significant differences between different patches. SUMMARY

[0005] The application provides a target re-identification method and system based on an inter-block ladder TRANSFORMER, which solves the technical problem of how to improve the discriminability and robustness of target re-identification.

[0006] To solve the above technical problems, the application provides a target re-identification method based on an inter-block ladder TRANSFORMER, which includes the following steps:

[0007] Constructing a target re-identification network;

[0008] The target re-identification network uses an inter-block ladder TRANSFORMER as a feature extraction network; the inter-block ladder TRANSFORMER includes multiple levels of encoder block groups with increasing depth, each level of encoder block group has a plurality of encoder blocks stacked in the same number, and a plurality of subspace feature adjustment modules are arranged in the first layer of each encoder block. The constraints of the plurality of subspace feature adjustment modules of the multiple levels of encoder block groups from shallow to deep are narrow and wide; the plurality of subspace feature adjustment modules enhance patch labels with stronger class label association and weaken patch labels with weaker class label association through class patch interaction, and the class patch interaction is performed in multiple subspaces including Euclidean distance subspace, cosine distance subspace and KL divergence subspace;

[0009] Training and testing the target re-identification network;

[0010] Applying the completed target re-identification network to target re-identification.

[0011] Further, the processing flow of the plurality of subspace feature adjustment modules includes the following steps:

[0012] For the input feature sequence, the distances D Euc , D Cos and D KL of the class label and the patch label in the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace are calculated, respectively;

[0013] According to D Euc , D Cos and D KLcalculating a saliency vector of each patch in the feature sequence;

[0014] normalizing the saliency vector to obtain a normalized saliency vector of each patch;

[0015] multiplying the label of each patch with the normalized saliency vector corresponding to the patch to generate a new patch label containing saliency features;

[0016] connecting the class label with the new patch label to generate a weighted label sequence, and obtaining a weighted feature sequence.

[0017] Further, the saliency vector is normalized, specifically:

[0018] The saliency vector is normalized by using maximum-minimum normalization with scale and boundary constraints;

[0019] The normalized saliency vector of the jth patch is calculated as j = 1, 2, …, N, N is the total number of patches; S j is the saliency vector of the jth patch, S is a vector composed of the saliency vectors of N patches, min(·) represents taking the minimum, max(·) represents taking the maximum, B left and B right are the left and right boundaries of normalization, Euc, Cos, KL correspond to Euclidean distance subspace, cosine distance subspace and KL divergence subspace respectively.

[0020] Further, the negative value of the distance D Euc , D Cos and D KL between the class label and the patch label is used as the saliency vector of the patch in the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace.

[0021] Further, the distance D Euc is calculated as cls i and patch i represent the vector values of the ith component of the class label and the patch label respectively;

[0022] The distance D Cos is calculated as

[0023] The distance D KL is calculated as

[0024] Furthermore, the inter-block ladder-type TRANSFORMER includes T-level encoder block groups with increasing depth, and the constraints of the multi-subspace feature adjustment module in the encoder block in the t-th level encoder block group are set as follows: G t =[α t ,β t ], α t , β t They represent the left and right boundaries of the t-th level constraint, and satisfy α1<α2<…<α T <1, β T >β T-1 >…>β1>1.

[0025] Furthermore, before IBLSFomer performs feature extraction, the target image is segmented into N patches using a homogeneous sliding window. And processed by linear mapping, an additional class marker Indicates that a learnable class is attached to the input, and D represents the dimension of the class;

[0026] The feature representation of the input inter-block step-wise TRANSFORMER is: where F(·) represents the linear mapping function and P is the position embedding.

[0027] Furthermore, during the training process, the cross entropy loss L CE and triplet loss L TRI The sum is the total loss L, cross entropy loss Tuple loss q i is the target probability, p i is the predicted probability, f a 、f p and f n They represent anchor points, positive samples, and negative samples respectively.

[0028] Furthermore, the target re-identification network is based on the network architecture of TransReID, which changes the feature extraction network to an inter-block stepped TRANSFORMER on the basis of the network architecture of TransReID;

[0029] The datasets used to train and test the target re-identification network are the benchmark datasets for pedestrian re-identification and vehicle re-identification.

[0030] The application further provides a target re-identification system based on an inter-block ladder TRANSFORMER, which is characterized in that the system is provided with an intelligent agent, which is used to realize: constructing a target re-identification network, in which an inter-block ladder TRANSFORMER is used as a feature extraction network; training and testing the target re-identification network; and applying the completed target re-identification network to target re-identification.

[0031] The application provides a target re-identification method and system based on an inter-block ladder TRANSFORMER, which proposes a novel feature extraction network, referred to as an inter-block ladder Transformer (IBLSFormer) based on multi-subspace feature adjustment (MSFA), for target re-identification. The core of the IBLSFormer is an MSFA module. The module can dynamically adjust features in multiple subspaces such as Euclidean distance subspaces, cosine distance subspaces and KL divergence subspaces through class-patch interaction, enhance significant patch labels and weaken non-significant patch labels. In addition, a ladder constraint from narrow to wide is embedded in the MSFA of the internal encoder block to optimize feature learning. A large number of experiments show that the IBLSFormer is superior to other methods in learning discriminative and robust representations for target re-identification. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a flowchart of the target re-identification method based on the inter-block ladder TRANSFORMER provided by the embodiment of the application;

[0033] Figure 2 is a processing flowchart of the MSFA module provided by the embodiment of the application;

[0034] Figure 3 is a block ladder boundary constraint principle diagram in the IBLSFormer provided by the embodiment of the application;

[0035] Figure 4 is a multi-subspace feature fusion analysis result diagram provided by the embodiment of the application;

[0036] Figure 5 is a boundary constraint analysis result diagram provided by the embodiment of the application. DETAILED DESCRIPTION

[0037] The embodiments of the application will be described in detail below with reference to the accompanying drawings. The embodiments are given only for illustrative purposes and cannot be understood as limiting the application, including the accompanying drawings for reference and illustration only, and do not constitute a limitation on the scope of patent protection of the application, because many changes can be made to the application without departing from the spirit and scope of the application.

[0038] The present application aims to optimize the internal structure of visual Transformer, and thus proposes a lightweight but effective improved Transformer architecture for target re-identification. Figure 1 The flowchart is shown in the figure, which includes the steps of:

[0039] A target re-identification network is constructed, and the inter-block ladder TRANSFORMER is used as the feature extraction network in the target re-identification network.

[0040] The target re-identification network is trained and tested.

[0041] The target re-identification network is applied to the target re-identification.

[0042] The target re-identification network here is based on the network architecture of TransReID, and the feature extraction network is changed to the specially designed inter-block ladder TRANSFORMER on the network architecture of TransReID. Compared with the existing visual Transformer of TransReID, the inter-block ladder TRANSFORMER constructed by the present application has two improvements:

[0043] 1. Feature dynamic adjustment design: a multi-subspace feature adjustment (MSFA) module is set at the first layer of each encoder block, which enhances the patch labels associated with stronger class labels and weakens the patch labels associated with weaker class labels through class patch interaction in multiple subspaces including Euclidean distance subspace, cosine distance subspace and KL divergence subspace.

[0044] 2. Inter-block ladder design: the inter-block ladder TRANSFORMER includes multiple levels of encoder block groups from shallow to deep, and each level of encoder block group is stacked with the same number of multiple encoder blocks, and the constraint of the multi-subspace feature adjustment module of the multiple levels of encoder block groups from shallow to deep is from narrow to wide.

[0045] The present application proposes a novel feature extraction network, called inter-block ladder TRANSFORMER (IBLSFormer). Figure 1 The overall architecture of IBLSFormer is shown. First, the model designs a multi-subspace feature adjustment (MSFA) module at the input of each encoder block, which is used to enhance the significant patch labels associated with higher class labels and weaken the non-significant patch labels associated with lower class labels. Then, a ladder constraint from narrow to wide is embedded in the multi-level encoder block group to optimize feature learning.

[0046] Before feature extraction in IBLSFormer, the target image is segmented into patches and processed by linear mapping, and the results are used as the input of the multi-level encoder block group. The feature extraction process of IBLSFormer is as follows: the linear mapping input stack of the multi-level encoder block group (divided into different groups according to the embedding depth); in each encoder block of each level encoder block group, the input needs to pass through the class-patch interaction in the MSFA module to dynamically adjust the features, and the patch label associated with the class label is enhanced, while the patch label associated with the class label is weakened. To achieve more comprehensive adjustment, the interaction is carried out in multiple subspaces including Euclidean distance subspace, cosine distance subspace and KL divergence subspace. The features processed by the MSFA module are passed to the subsequent layer for deeper extraction. Multiple consecutive encoder blocks are connected to form a complete feature extraction pipeline. The MSFA configurations of different encoder blocks are different. The step-by-step constraints from narrow to wide are embedded in the internal encoder blocks. Finally, the feature representations f E uc, f C os, f KL For target re-identification.

[0047] The target image is segmented into N fixed size patches by the target re-identification baseline in this embodiment After being converted into tensors, the patches are then linearly mapped to a low-dimensional space. Each output is used as the input of the subsequent layer, and an additional class label (D represents the dimension of the class) represents a learnable class, which is attached to the input to better reflect the global features. In order to capture spatial information, the model combines position embedding. The feature representation of the input encoder is as follows:

[0048]

[0049] Where P is the position embedding, and N patches are linearly mapped to a D-dimensional space by function F.

[0050] Unlike traditional visual Transformers, which divide images into non-overlapping patches, the baseline model generates N patches with overlapping pixels using a sliding window:

[0051]

[0052] Where N H represents the number of patches generated by sliding in the height direction, N W represents the number of patches generated by sliding in the width direction, H and W are the height and width of the target image, respectively, is a floor function, and R is the sliding step. By reducing the step, more overlapping patches can be generated, thus improving performance, but also increasing the computational cost.

[0053] In the optimization process, the cross-entropy loss L CE and the triplet loss L TRI are adopted without label smoothing. In order to enhance the combination of losses, a batch normalization layer is also added, and the total loss is represented as:

[0054] L = L CE + L TRI (3)

[0055]

[0056] where q i is the target probability, p i is the predicted probability, f a , f p and f n represent the anchor, positive and negative samples, respectively.

[0057] Existing Transformer-based target re-identification methods usually directly input features into the encoder block and optimize these features between the encoder blocks. However, these features are not pre-processed before being input into the encoder block, and the weight of each patch feature is also considered to be the same. This mechanism has defects, even if the subsequent attention mechanism can dynamically adjust the features, the computational cost is still high. Therefore, the present example proposes a multi-subspace feature adjustment (MSFA) module, which distinguishes the saliency of different patch labels through the interaction between class labels and patch labels, and the processing flow of the MSFA module is shown in Figure 2 For the input feature sequence, the interaction between class labels and patch labels is carried out in multiple subspaces. The essence of class-patch interaction is to compare the similarity between class labels and patch labels. In the visual Transformer, class labels are used to represent global information, while patch labels represent local information. By analyzing the similarity between class labels and patch labels, the saliency of patch labels can be determined. Specifically, the higher the similarity with the class label, the greater the contribution of the patch label to the final representation, indicating that the patch belongs to a more discriminative region. Based on this, patch labels that are more relevant to the class label have stronger saliency, and therefore their features need to be enhanced. Conversely, patch labels that are less relevant to the class label have weaker saliency, and therefore their features need to be weakened. In order to achieve more systematic interaction, multiple subspaces including the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace are introduced.

[0058] Euclidean distance is a common metric used to calculate the straight-line distance between class labels and patch labels, quantifying the difference in Euclidean space. It has wide applications in image processing, pattern recognition, and machine learning. The distance between class labels and patch labels in the Euclidean distance subspace is calculated as follows:

[0059]

[0060] where cls i and patch i represent the vector values of the i-th component of the class label and patch label, respectively.

[0061] Euclidean distance can measure the relative position relationship between class labels and patch labels in geometric space. However, it cannot directly consider directional information and only focuses on the distance between class labels and patch labels. In addition, Euclidean distance is very sensitive to the scale of vectors. When there are outliers in different dimensions, the calculation result may be dominated by this outlier, affecting the accuracy of the distance. To solve the above problems, this example embeds a cosine distance subspace in the subspace, which measures the similarity between class labels and patch labels by calculating the angle between vectors. The distance between labels in the cosine distance subspace is calculated as follows:

[0062]

[0063] Cosine distance can measure directional similarity and quantify the collinearity between class labels and patch labels, so it focuses more on directional relationships. However, it only uses directional information between labels, ignoring the impact of absolute scale. Therefore, the advantages of the Euclidean distance subspace can complement the shortcomings of the cosine distance subspace.

[0064] In addition to measuring geometric distance in the Euclidean distance subspace and directional difference in the cosine distance subspace, it is also important to compare the differences in probability distribution between class labels and patch labels. The smaller the probability difference, the more similar the patch label is to the class label, and the more important the corresponding local information. Based on this, this example introduces a KL divergence subspace to compare the probability distribution difference of patch labels on class labels. Traditional KL divergence calculation requires the probability distribution of the vector to evaluate the similarity by the distribution difference. However, class labels and patch labels usually have high-dimensional features, and calculating the probability distribution will consume a lot of time and resources. To solve this problem, this example adopts an intuitive simplification strategy, considering the corresponding feature value as a pseudo-probability. In class labels and patch labels, features are materialized as a vector, where each dimension value corresponds to the probability value of the relevant element. The distance between class labels and patch labels in the KL divergence subspace is calculated as follows:

[0065]

[0066] After computing the distance metrics in multiple subspaces, MSFA module needs to generate the saliency of patch labels according to the distance metrics. The smaller the distance between patch labels, the greater the contribution of the patch to the global representation, indicating that the saliency is stronger. Therefore, this example directly uses the negative value of the distance between class label and patch label as the saliency vector of the patch, and the formula is as follows:

[0067]

[0068] Euc, Cos, KL correspond to Euclidean distance subspace, cosine distance subspace and KL divergence subspace respectively.

[0069] For a sequence containing one class label and N patch labels, a saliency vector S of length N can be generated through similarity analysis. Each element corresponds to the similarity between the patch label and the class label. In order to improve the performance of the algorithm, the saliency vector is normalized. In the normalization process, considering that patch labels with lower relevance to the class label should be assigned smaller saliency (weight less than 1), and patch labels with higher relevance should be assigned larger saliency (weight greater than 1), this example uses maximum-minimum normalization with scale and boundary constraints. The calculation method is as follows:

[0070]

[0071] where B left and B right are the left and right boundaries of normalization, denotes the normalized saliency vector for the jth patch,

[0072] By multiplying the patch labels in the original feature sequence with the normalized saliency vector, new patch labels containing saliency features are generated. Finally, the class label is connected with the new patch label to generate a weighted label sequence:

[0073]

[0074] where, is the class label in the subspace, is the patch label in the subspace,

[0075] Performing similar operations in three subspaces, adjusting features through class-patch interaction, embedding saliency into weight features, and obtaining weighted feature sequence.

[0076] In the task of target re-identification, input features are sequentially propagated through stacked internal encoder blocks to obtain the final representation. Similar to convolutional neural networks, encoder blocks close to the input can only extract low-level features, while deeper encoder blocks can extract high-level features. On the one hand, low-level features usually include edge, color, etc. information, which is original, general, and less relevant to specific tasks. On the other hand, high-level features usually include object shape, local parts, and semantic information. Due to multiple layers of nonlinear transformation, high-level features are more focused on specific tasks and have stronger discriminability.

[0077] Existing Transformer-based methods rarely consider the difference between low-level and high-level features. They usually directly pass the features output by the previous encoder block to the next encoder block. To highlight the difference between different levels of features, this example proposes an inter-block ladder Transformer (IBLSFormer) based on MSFA. The main idea of IBLSFormer is to set different boundary constraints for MSFA modules between different encoder blocks, as shown in Figure 3

[0078] For low-level image features, since these features are usually general information that is irrelevant to specific tasks, a narrower constraint range is used. The narrower boundary constraint can concentrate the significance weight of the patch label around 1. This means that the patch label has a smaller range of variation compared to the original visual Transformer. Conversely, for high-level features, since these features are usually exclusive information for specific tasks, a wider constraint range is used. The wider boundary constraint allows the significance of the patch label to have a larger dispersion range, which means that the patch label has a larger range of variation compared to the original visual Transformer. This mechanism can provide the patch label with higher degrees of freedom, thereby extracting more discriminative features. IBLSFormer divides the stacked encoder blocks in the original baseline into T groups according to depth, T ≥ 2. Each group contains the same number of encoder blocks. In the shallowest part of the network, a relatively narrow constraint is deployed; in the middle depth part, a relatively balanced constraint is deployed. In the deepest part of the encoder block, the widest constraint is embedded. This ladder constraint pattern can be summarized as:

[0079] G t = [α t , β t ], t ∈ [1, T] (12)

[0080] where α t , β t represent the left boundary and right boundary of the t-th level constraint, respectively, and satisfy α1<α2<…<α T <1, β T >β​T-1 >…>β1>1.

[0081] As a preferred embodiment, the embodiment is provided with 3-level encoder block groups, each level of the encoder block group is provided with M encoder modules, M≥1, as shown in Figure 3 In order to directly represent the height (depth) of the level, in Figure 3 , α H , α M , α L represent the left boundary of the high, medium and low level encoder block groups, and β H , β M , β L represent the right boundary of the high, medium and low level encoder block groups, and α H <α M <α L <1, β H >β M >β L >1.

[0082] By applying the stepwise constraint in the internal encoder block, stronger narrow constraints can be imposed on the low-level encoder block. This allows the features to be weighted smoothly without affecting the generality. In the high-level encoder block, weaker wide constraints are imposed, which can further enhance the specificity and freedom of patch labels. By processing in three subspaces, the IBLSFormer can generate three corresponding subspace features. These features are concatenated to generate the final image representation, which is then input into the later structure of the encoder block.

[0083] Based on the above-mentioned target re-identification method based on the inter-block stepwise TRANSFORMER, the embodiment further provides a target re-identification system based on the inter-block stepwise TRANSFORMER, which is provided with an intelligent agent. The intelligent agent is used to realize: constructing a target re-identification network, and using the inter-block stepwise TRANSFORMER as a feature extraction network in the target re-identification network; training and testing the target re-identification network; and applying the completed target re-identification network to perform target re-identification. The intelligent agent (Agent) herein refers to an agent that can perceive the environment and take actions to achieve a specific goal. It can be software, hardware or a system, and has autonomy, adaptability and interaction ability. The intelligent agent perceives changes in the environment (such as through sensors or data input), makes judgments and decisions according to the knowledge and algorithms learned by itself, and then performs actions to affect the environment or achieve the predetermined goal.

[0084] The embodiment provides a target re-identification method and system based on inter-block ladder TRANSFORMER, which proposes a novel inter-block ladder TRANSFORMER (IBLSFormer) model as a feature extraction network for the target re-identification task. Unlike most methods that directly use visual TRANSFORMER as a feature extractor, IBLSFormer focuses on optimizing the internal structure of TRANSFORMER to enhance its feature expression capability. Specifically, the proposed multi-subspace feature adjustment (MSFA) module dynamically adjusts the feature weight in the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace through class-patch interaction, thereby effectively distinguishing between significant patch labels and non-significant patch labels. In addition, IBLSFormer further optimizes the hierarchy of feature learning by grouping the encoder blocks and embedding the narrow-medium-wide ladder constraint strategy.

[0085] To verify the superiority of IBLSFormer, the embodiment performs experiments on benchmark datasets of pedestrian re-identification and vehicle re-identification. The former includes Market-1501 and DukeMTMC-reID, and the latter includes VeRi-776 and VehicleID. Market-1501 is a large-scale pedestrian re-identification benchmark dataset collected by 6 cameras. The training set contains 12936 images of 751 identities, and the test set contains 19732 images of 750 identities. DukeMTMC-reID is derived from the Duke dataset, which collects 36411 images from 8 cameras, involving 1812 identities. Among them, the training set contains 16522 images of 702 identities, and the test set contains 2228 query images and 17661 gallery images. VeRi-776 is the first publicly available large-scale vehicle re-identification dataset, which is composed of images collected by 20 cameras, involving 776 identities. There are 37778 images of 576 identities in the training set, the gallery contains 11579 images of 200 identities, and the query set contains 1678 images of 200 identities. VehicleID is a vehicle image dataset collected in multiple locations and cameras. A total of 221763 images of 26267 vehicles are included, the training set contains 110178 images of 13134 vehicles, and the test set contains 111585 images of 13133 vehicles.

[0086] The performance is evaluated using Cumulative Matching Characteristics (CMC), mean Average Precision (mAP), and Rank-1 to Rank-5 accuracy. CMC ranks the accuracy by calculating the proportion of correct answers in the top n returned results, while mAP measures the overall accuracy by calculating the area under the precision-recall curve. Rank-1 accuracy is the most intuitive measure, which measures the percentage of times the highest probability class predicted by the model is the same as the true label. Rank-2 to Rank-5 accuracy is an extension of Rank-1 accuracy, which considers the top 2 to 5 highest probability classes output by the model.

[0087] In this example, TransReID pre-trained on ImageNet is used as the baseline. Pedestrian images are resized to 256x128, and vehicle images are resized to 256x256. The batch size is 64, and each batch contains 16 identities. The training images are subjected to random horizontal flipping, cropping, and erasing data augmentation. The SGD optimizer with momentum 0.9 and weight decay 0.0001 is used. The initial learning rate is 0.008, and cosine-based learning rate decay is used. IBLSFormer is trained and tested on a deep learning platform with PyTorch = 1.7.1, torchvision = 0.8.2, and CUDA Version = 11.2. The hardware configuration is Ubuntu 18.04 operating system and NVIDIA GeForce RTX 3090Ti GPU.

[0088] To verify the feasibility of IBLSFormer, this example compares it with the latest methods on pedestrian re-identification and vehicle re-identification benchmark datasets, as shown in Tables 1 and 2, respectively.

[0089] As shown in Table 1, on the Market-1501 dataset, the mAP / Rank-1 of IBLSFormer reached 88.7% / 95.3%, compared with the best performing ResNet-based method ResT-ReID and the Transformer-based method DCAL, the mAP of IBLSFormer improved by 0.5% and 1.2%, respectively, and the Rank-1 improved by 0.6%. Compared with the baseline Baseline, IBLSFormer improved by 0.9% and 0.6% in mAP / Rank-1, respectively. On the DukeMTMC-reID dataset, the mAP / Rank-1 of IBLSFormer were 81.4% / 90.4%, respectively, compared with ResT-ReID and DCAL, which achieved a gain of 0.8% / 0.4% and 1.3% / 1.4% in mAP / Rank-1, respectively. Compared with the baseline Baseline, IBLSFormer improved by 1.1% and 0.6% in mAP / Rank-1, respectively.

[0090] Table 1. Comparison with other pedestrian re-identification methods

[0091]

[0092] As shown in Table 2, the mAP / Rank-1 of IBLSFormer on VeRi-776 and VehicleID datasets are 80.0% / 97.1% and 89.4% / 83.6%, respectively. Compared with the best ResNet-based method CAMNet, the mAP / Rank-1 are improved by 0.4% / 0.5% and 0.8% / 1.1%, respectively. Compared with the best Transformer-based SSBVER, the mAP / Rank-1 are improved by 2.6% / 1.1% and 0.3% / 0.7%, respectively. Compared with the baseline, the IBLSFormer improves the mAP / Rank-1 by 1.8% / 0.6% and 2.3% / 1.3%, respectively.

[0093] Table 2. Comparison with other vehicle re-identification methods

[0094]

[0095] The multi-subspace feature adjustment (MSFA) module adjusts patch labels by performing saliency analysis in Euclidean distance subspace, cosine distance subspace, and KL divergence subspace. Correspondingly, IBLSFormer generates features in three subspaces, which are concatenated to form the final vector. To verify whether this combination can improve performance, this section conducts corresponding analysis experiments. First, the three single subspace features are analyzed, and their re-identification results are shown in Tables 3 and 4. From the results in Tables 3 and 4, it can be seen that the feature based on the Euclidean distance subspace can achieve the highest performance compared to the features based on the cosine distance subspace and the KL divergence subspace. This indicates that the Euclidean distance can accurately reflect the similarity between class labels and patch labels. In addition, the feature based on the KL divergence subspace has relatively low performance, which may be because the KL divergence is not a strictly defined distance metric. However, feature adjustment based on any single subspace can achieve higher accuracy than the baseline.

[0096] Table 3. Single subspace feature analysis on pedestrian re-identification datasets

[0097]

[0098] Table 4. Single subspace feature analysis on vehicle re-identification datasets

[0099]

[0100] In addition to comparing the single subspace feature, the performance improvement obtained by fusing the subspace features is further analyzed in this example. The experimental results are shown in Figure 4 Figure 4 (a), (b), (c), (d) are experimental results for data sets Market-1501, DukeMTMC-reID, VeRi-776, VehicleID, respectively, F Euc , F Cos , F KL represent the features of the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace, respectively. The performance of the feature based on the Euclidean distance subspace is selected as the baseline. After introducing the cosine distance subspace, the mAP / Rank-1 is improved by 0.4% / 0.2%, 1.0% / 0.3%, 0.7% / 0.2% and 0.2% / 0.3% on the four data sets, respectively. This shows that the cosine distance subspace can effectively supplement the similarity description between the class label and the patch label when considering the direction similarity between features. In addition, after introducing the KL divergence subspace, the mAP / Rank-1 is improved by 0.6% / 0.3%, 0.6% / 0.5%, 0.9% / 0.3% and 1.1% / 0.5% compared to the original Euclidean distance subspace feature. This shows that considering the feature distribution between the class label and the patch label can further supplement the similarity information. These data verify the feasibility and effectiveness of the multi-subspace feature fusion mechanism.

[0101] In the process of realizing the significance weighting of different patch labels based on class-patch interaction, the boundaries of the significance map are constrained by normalization in this example. In order to verify the feasibility of the boundary constraint strategy (BCS), comparative experiments are carried out to compare the performance of with BCS and w / o BCS, and the experimental results are shown in Figure 5 Figure 5 ​​The (a), (b), (c), (d) are experimental results for data sets Market-1501, DukeMTMC-reID, VeRi-776, VehicleID respectively. Compared with the case without boundary constraint strategy, after introducing the boundary constraint mechanism to constrain the saliency mapping, the performance of the model is improved in different indicators. Specifically, the boundary constraint improves the mAP / Rank-1 on the four benchmark data sets by 0.5% / 0.3%, 1.0% / 1.1%, 0.9% / 0.3% and 0.8% / 0.5% respectively. This is because the boundary constraint strategy limits the left boundary to less than 1, while limiting the right boundary to greater than 1. Through this mechanism, class-patch interaction can dynamically weaken and enhance patch labels, thereby improving the discriminability of features. It should be noted that although the re-identification performance without boundary constraint condition has declined compared to the optimal case, compared with the baseline model, it still obtains 1.4%, 1.1%, 0.9% and 1.5% improvement in mAP, and 0.3%, 0.5%, 0.3% and 0.8% improvement in Rank-1, respectively. This shows that the method of weighting patch labels through similarity comparison is feasible. Although this rough mechanism cannot achieve fine-grained weakening or enhancement of features, it can still selectively distinguish patch labels.

[0102] The experimental results show that IBLSFormer outperforms existing methods on multiple benchmark datasets for pedestrian and vehicle re-identification, fully verifying its effectiveness.

[0103] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods, and shall be included in the protection scope of the present application.

Claims

1. The target re-identification method based on inter-block step-by-step TRANSFORMER is characterized by: Including steps: Build a target re-identification network; The target re-identification network adopts an inter-block ladder-type TRANSFORMER as a feature extraction network; the inter-block ladder-type TRANSFORMER includes a multi-level encoder block group with increasing depth, wherein each level of the encoder block group is stacked with a plurality of encoder blocks of the same number, and a multi-subspace feature adjustment module is set at the first layer of each encoder block, and the constraints of the multi-subspace feature adjustment module of the multi-level encoder block group from shallow to deep are narrowed to widened; the multi-subspace feature adjustment module enhances the patch labels with stronger correlation with the class label and weakens the patch labels with weaker correlation with the class label through class patch interaction, and the class patch interaction is performed in multiple subspaces including the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace; Before feature extraction in inter-block staircase TRANSFORMER, the target image is segmented into N patches using a homogeneous sliding window. And processed by linear mapping, an additional class marker Indicates that a learnable class is attached to the input, and D represents the dimension of the class; The feature representation of the input inter-block step-wise TRANSFORMER is: ,in represents a linear mapping function, is the position embedding; The processing flow of the multi-subspace feature adjustment module includes the following steps: For the input feature sequence, the distance between its class label and patch label in the Euclidean distance subspace, cosine distance subspace and KL divergence subspace is calculated respectively. 、 and ; according to 、 and Calculate the saliency vector of each patch in the feature sequence; Normalize the saliency vector to obtain the normalized saliency vector of each patch; Multiply the label of each patch by the normalized saliency vector corresponding to the patch to generate a new patch label containing saliency features; Connect the class label with the new patch label to generate a weighted label sequence and obtain a weighted feature sequence; Train and test the object re-identification network; Apply the tested target re-identification network to perform target re-identification.

2. The object re-identification method based on inter-block step-by-step TRANSFORMER according to claim 1, characterized in that: Normalize the saliency vector as follows: The saliency vector is normalized using the maximum-minimum normalization with scale and boundary constraints; No. Normalized saliency vector of patches Calculated as , , is the total number of patches; For the The saliency vector of the patch, for The vector of saliency vectors of patches, Indicates taking the minimum, Indicates taking the maximum value, and are the normalized left and right boundaries, , They correspond to the Euclidean distance subspace, cosine distance subspace and KL divergence subspace respectively.

3. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 2, characterized in that: Utilize the distance between class label and patch label 、 and The negative value of is used as the saliency vector of the patch in the Euclidean distance subspace, cosine distance subspace and KL divergence subspace.

4. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 3, characterized in that: distance Calculated as , and denote the class label and the patch label, respectively. A vector value of components; distance Calculated as ; distance Calculated as .

5. The object re-identification method based on inter-block stepwise TRANSFORMER according to any one of claims 1 to 4, characterized in that: The inter-block ladder TRANSFORMER includes T-level encoder block groups with increasing depth, wherein the constraints of the multi-subspace feature adjustment module in the encoder block in the t-th level encoder block group are set as follows: , Represent the left and right boundaries of the t-th level constraint, and satisfy , .

6. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 1, characterized in that: During training, the cross entropy loss and triplet loss The sum of , cross entropy loss , tuple loss , is the target probability, is the predicted probability, 、 and They represent anchor points, positive samples, and negative samples respectively.

7. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 1, characterized in that: The target re-identification network is based on the network architecture of TransReID, which changes the feature extraction network to an inter-block stepped TRANSFORMER on the basis of the network architecture of TransReID; The datasets used to train and test the target re-identification network are the benchmark datasets for pedestrian re-identification and vehicle re-identification.

8. The object re-identification system based on inter-block ladder TRANSFORMER is characterized by: The system is based on the target re-identification method based on inter-block ladder-type TRANSFORMER according to any one of claims 1 to 7, and is provided with an intelligent agent, which is used to implement: constructing a target re-identification network, in which the inter-block ladder-type TRANSFORMER is used as a feature extraction network; Train and test the target re-identification network; apply the tested target re-identification network to perform target re-identification.

Images