Target re-identification method and system based on inter-block stepped TRANSFORMER

Abstract

The invention relates to the technical field of computer vision, and particularly discloses an inter-block stepped Transformer-based target re-identification method and system, and the system proposes a novel feature extraction network, which is called as an inter-block stepped Transformer (IBLSformer) based on multi-subspace feature adjustment, and is used for target re-identification. The core of the IBLSFormer is a multi-subspace feature adjustment (MSFA) module. Characteristics can be dynamically adjusted through class-patch interaction in a plurality of subspaces such as an Euclidean distance subspace, a cosine distance subspace and a KL divergence subspace, significant patch marks are enhanced, and non-significant patch marks are weakened. In addition, a narrow-to-wide stepped constraint is embedded in the internal encoder block MSFA to optimize feature learning. A large number of experiments show that the IBLSFormer is superior to other methods in the aspect of learning discrimination and robustness representation for target re-identification.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target re-identification method and system based on an inter-block ladder Transformer. Background Art

[0002] Object re-identification is an important challenge in the field of computer vision and is of great significance in many applications such as video surveillance, vehicle-road cooperative systems, and intelligent transportation systems. Its goal is to match objects under different cameras, thereby achieving seamless tracking and recognition in different spatial and temporal contexts. In recent years, the development of computer technology has significantly promoted the progress of object re-identification. Although object re-identification has made great progress, it still faces many inherent challenges in real-world scenarios. These challenges mainly come from complex factors such as viewpoint changes, lighting conditions, occlusions, and background clutter. Traditional methods have difficulty in dealing with these problems, resulting in poor performance and limited applicability in the real world.

[0003] In recent years, with the rise of deep learning technology, object re-identification has made significant progress. Before the emergence of Visual Transformer (ViT), Convolutional Neural Network (CNN) was regarded as the cornerstone of object re-identification methods, which performed well in extracting discriminative features. CNN analyzes local areas of the image through convolution operations, captures local attention, and can see wider areas as the depth of the network increases. Related research has shown great interest in CNN-based methods. However, due to the limitations of convolution and pooling operations, CNN has obvious shortcomings in capturing long-range dependencies. Transformer, as the dominant method in natural language processing, has achieved remarkable success. Researchers have gradually extended Transformer to image tasks. In Transformer, each patch marker is first converted into a vector, and the interaction relationship between these patch markers is represented by the self-attention mechanism. This enables the model to effectively learn the dependencies between patch markers at different positions in the sequence. As an effective model, Transformer can aggregate features from a global perspective and promote the exploration of diverse internal features, far exceeding the capabilities of traditional CNN. Especially in the task of object re-identification, the attention mechanism is particularly outstanding in capturing long-range dependencies. Therefore, researchers introduced different Transformer methods, which have achieved significant improvements in target re-identification.

[0004] However, most existing studies directly use the encoder of the visual transformer as a feature extraction module, and few studies have improved the internal structure of the visual transformer. First, compared with the transformer using additional branches or image attributes, the computational cost of optimizing the internal structure is lower. In addition, existing transformer-based object re-identification methods usually directly input sequence features into the encoder block, ignoring the significant differences between different patches. Summary of the invention

[0005] The present invention provides a target re-identification method and system based on inter-block stepped TRANSFORMER, and solves the technical problem of how to improve the discrimination and robustness of target re-identification.

[0006] In order to solve the above technical problems, the present invention provides a target re-identification method based on inter-block step-by-step TRANSFORMER, comprising the steps of:

[0007] Build a target re-identification network;

[0008] The target re-identification network adopts an inter-block ladder TRANSFORMER as a feature extraction network; the inter-block ladder TRANSFORMER includes a multi-level encoder block group with increasing depth, and each level of the encoder block group is stacked with a plurality of encoder blocks of the same number, and a multi-subspace feature adjustment module is set at the first layer of each encoder block, and the constraints of the multi-subspace feature adjustment module of the multi-level encoder block group from shallow to deep are narrower and wider; the multi-subspace feature adjustment module enhances the patch mark with a stronger association with the class mark and weakens the patch mark with a weaker association with the class mark through class patch interaction, and the class patch interaction is performed in multiple subspaces including the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace;

[0009] Train and test the object re-identification network;

[0010] Apply the tested target re-identification network to perform target re-identification.

[0011] Furthermore, the processing flow of the multi-subspace feature adjustment module includes the steps of:

[0012] For the input feature sequence, the distance D between its class label and patch label in the Euclidean distance subspace, cosine distance subspace and KL divergence subspace is calculated respectively. Euc , D Cos and D KL ;

[0013] According to D Euc , D Cos and D KLCalculate the saliency vector for each patch in the feature sequence;

[0014] Normalize the saliency vector to obtain the normalized saliency vector of each patch;

[0015] Multiply the label of each patch by the normalized saliency vector corresponding to the patch to generate a new patch label containing salient features;

[0016] The class labels are concatenated with the new patch labels to generate a weighted label sequence and obtain a weighted feature sequence.

[0017] Furthermore, the saliency vector is normalized as follows:

[0018] The saliency vector is normalized using the maximum-minimum normalization with scale and boundary constraints;

[0019] Normalized saliency vector of the jth patch Calculated as j=1,2,…,N, N is the total number of patches; S j is the saliency vector of the jth patch, S is the vector composed of the saliency vectors of N patches, min(·) means taking the minimum, max(·) means taking the maximum, B left and B right are the normalized left and right boundaries, Euc, Cos, and KL correspond to the Euclidean distance subspace, cosine distance subspace, and KL divergence subspace, respectively.

[0020] Furthermore, the distance D between the class label and the patch label is used Euc , D Cos and D KL The negative value of is used as the saliency vector of the patch in the Euclidean distance subspace, cosine distance subspace, and KL divergence subspace.

[0021] Furthermore, the distance D Euc Calculated as cls i and patch i The vector values ​​representing the i-th component of the class label and patch label respectively;

[0022] Distance D Cos Calculated as

[0023] Distance D KL Calculated as

[0024] Furthermore, the inter-block ladder TRANSFORMER includes T-level encoder block groups with increasing depth, and the constraints of the multi-subspace feature adjustment module in the encoder block in the t-th level encoder block group are set as follows: G t =[α t ,β t ], α t , β t They represent the left and right boundaries of the t-th level constraint, respectively, and satisfy α1<α2<…<α T <1, β T >β T-1 >…>β1>1.

[0025] Furthermore, before IBLSFomer performs feature extraction, the target image is segmented into N patches using a homogeneous sliding window. And processed by linear mapping, an additional class marker Indicates that a learnable class is attached to the input, and D represents the dimension of the class;

[0026] The characteristic representation of the input inter-block ladder TRANSFORMER is Where F(·) represents the linear mapping function and P is the position embedding.

[0027] Furthermore, during the training process, the cross entropy loss L CE and triplet loss L TRI The sum is taken as the total loss L, the cross entropy loss Tuple loss q i is the target probability, p i is the predicted probability, f a 、f p and f n They represent anchor points, positive samples, and negative samples respectively.

[0028] Furthermore, the target re-identification network is based on the network architecture of TransReID, which changes the feature extraction network into an inter-block stepped TRANSFORMER on the network architecture of TransReID;

[0029] The datasets used for training and testing the target re-identification network are benchmark datasets for pedestrian re-identification and vehicle re-identification.

[0030] The present invention also provides a target re-identification system based on inter-block ladder TRANSFORMER, the key of which is that the system is provided with an intelligent agent, which is used to realize: constructing a target re-identification network, in which the inter-block ladder TRANSFORMER is used as a feature extraction network; training and testing the target re-identification network; and applying the tested target re-identification network to perform target re-identification.

[0031] The object re-identification method and system based on inter-block ladder-type TRANSFORMER provided by the present invention propose a novel feature extraction network, called inter-block ladder-type Transformer based on multi-subspace feature adjustment (IBLSFormer), for object re-identification. The core of IBLSFormer is the multi-subspace feature adjustment (MSFA) module. It can dynamically adjust features through class-patch interaction in multiple subspaces such as Euclidean distance subspace, cosine distance subspace and KL divergence subspace, enhance significant patch marks and weaken non-significant patch marks. In addition, a ladder-type constraint from narrow to wide is embedded in the internal encoder block MSFA to optimize feature learning. A large number of experiments show that IBLSFormer is superior to other methods in learning discriminative and robust representations for object re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a target re-identification method based on inter-block stepped TRANSFORMER provided by an embodiment of the present invention;

[0033] Figure 2 is a processing flow chart of the MSFA module provided in an embodiment of the present invention;

[0034] Figure 3 It is a schematic diagram of the inter-block stepped boundary constraint in IBLSFomer provided by an embodiment of the present invention;

[0035] Figure 4 is a multi-subspace feature fusion analysis result diagram provided by an embodiment of the present invention;

[0036] Figure 5 This is a boundary constraint analysis result diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0038] The present invention aims to optimize the internal structure of the visual Transformer, and thus this example proposes a lightweight but effective improved Transformer architecture for object re-identification. The architecture can dynamically weight the features of the input encoder, thereby enhancing the importance of salient features. Specifically, the embodiment of the present invention provides an object re-identification method based on inter-block ladder TRANSFORMER, such as Figure 1 As shown in the flowchart, it includes the steps of:

[0039] Construct a target re-identification network, in which the inter-block ladder TRANSFORMER is used as the feature extraction network;

[0040] Train and test the object re-identification network;

[0041] Apply the tested target re-identification network to perform target re-identification.

[0042] The target re-identification network here is based on the network architecture of TransReID. In this example, the feature extraction network is changed to a specially designed inter-block ladder TRANSFORMER based on the network architecture of TransReID. Compared with the existing visual Transformer of TransReID, the inter-block ladder TRANSFORMER constructed by the present invention has two improvements:

[0043] 1. Design of dynamic feature adjustment: A multi-subspace feature adjustment (MSFA) module is set at the first layer of each encoder block. The multi-subspace feature adjustment (MSFA) module enhances the patch labels that are more strongly associated with the class labels and weakens the patch labels that are less associated with the class labels through class patch interactions. The class patch interactions are carried out in multiple subspaces including the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace.

[0044] 2. Inter-block stepped design: The inter-block stepped TRANSFORMER includes a multi-level encoder block group with increasing depth. Each level of the encoder block group is stacked with multiple encoder blocks of the same number. The constraints of the multi-subspace feature adjustment modules of the multi-level encoder block groups from shallow to deep increase from narrow to wide.

[0045] This paper proposes a novel feature extraction network called Inter-Block Ladder-Structured Transformer (IBLSFormer). Figure 1 The overall architecture of IBLSFomer is presented. First, a multi-subspace feature adjustment (MSFA) module is designed at the input of each encoder block to enhance the salient patch labels with high correlation with the class labels and weaken the non-salient patch labels with low correlation with the class labels. Then, a step-by-step constraint from narrow to wide is embedded in the multi-level encoder block group to optimize feature learning.

[0046] Before feature extraction in IBLSFormer, the target image is segmented into patches and processed by linear mapping, and these results are used as inputs to a multi-level encoder block group. The feature extraction process of IBLSFormer is as follows: a multi-level encoder block group (divided into different groups according to the embedding depth) with linear mapping input stacked; in each encoder block of each level of encoder block group, the input needs to go through the class-patch interaction in the MSFA module to dynamically adjust the features, and the patch labels with stronger association with the class label are enhanced, while the patch labels with weaker association with the class label are weakened. To achieve more comprehensive adjustment, the interaction is carried out in multiple subspaces including the Euclidean distance subspace, the cosine distance subspace, and the KL divergence subspace. The features processed by the MSFA module are passed to subsequent layers for deeper extraction. Multiple consecutive encoder blocks are connected together to form a complete feature extraction pipeline. The MSFA configuration of different encoder blocks is different. A stepped constraint from narrow to wide is embedded in the internal encoder block. Finally, the feature representations f of these three subspaces are generated. E uc,f C os, f KL Used for target re-identification.

[0047] This example uses the target re-identification baseline to segment the target image into N patches of fixed size. After being converted into tensors, these patches are then linearly mapped onto a lower-dimensional space. Each output is used as input to subsequent layers, with an additional class label (D represents the dimension of the class) represents a learnable class that is attached to the input to better reflect the global features. To capture spatial information, the model incorporates position embedding. The feature representation of the input encoder is as follows:

[0048]

[0049] Where P is the position embedding, and the N patches are linearly mapped into the D-dimensional space through the function F.

[0050] Different from the traditional visual Transformer that divides the image into non-overlapping patches, the baseline model uses a sliding window to generate N patches with overlapping pixels:

[0051]

[0052] Among them, N H Indicates the number of patches generated by sliding in the height direction, N W represents the number of patches generated by sliding in the width direction, H and W are the height and width of the target image respectively, is the floor function, and R is the sliding step size. By reducing the step size, more overlapping patches can be generated, thereby improving performance, but it also increases the computational cost.

[0053] During the optimization process, the cross entropy loss L without label smoothing is used. CE and triplet loss L TRI In order to enhance the combination of losses, a batch normalization layer is added, and the total loss is expressed as:

[0054] L=L CE +L TRI (3)

[0055]

[0056] Among them, q i is the target probability, p i is the predicted probability, f a 、f p and f n They represent anchor points, positive samples, and negative samples respectively.

[0057] Existing Transformer-based object re-identification methods usually input features directly into the encoder block and optimize these features between encoder blocks. However, these features are not preprocessed before being input into the encoder block, and the weights of each patch feature are also considered the same. This mechanism has defects. Even if the subsequent attention mechanism can dynamically adjust the features, its computational cost is still high. Therefore, this example proposes a multi-subspace feature adjustment (MSFA) module to distinguish the saliency of different patch labels through the interaction between class labels and patch labels. The processing flow of the MSFA module is as follows: Figure 2 As shown. For the input feature sequence, the interaction between the class label and the patch label is carried out in multiple subspaces. The essence of class-patch interaction lies in comparing the similarity between the class label and the patch label. In the visual Transformer, the class label is used to represent global information, while the patch label represents local information. By analyzing the similarity between the class label and the patch label, the saliency of the patch label can be determined. Specifically, the patch label with higher similarity to the class label has a greater contribution to the final representation, indicating that the patch belongs to a more discriminative region. Based on this, the patch label that is more relevant to the class label has a stronger saliency, so its features need to be enhanced. Conversely, the patch label with lower correlation with the class label has weaker saliency, so its features need to be weakened. In order to achieve a more systematic interaction, this example introduces multiple subspaces including the Euclidean distance subspace, the cosine distance subspace, and the KL divergence subspace.

[0058] Euclidean distance is a common metric that calculates the straight-line distance between class labels and patch labels to quantify the difference in Euclidean space. Euclidean distance has a wide range of applications in image processing, pattern recognition, and machine learning. The distance between class labels and patch labels in the Euclidean distance subspace is calculated as follows:

[0059]

[0060] Among them, cls i and patch i The vector values ​​of the i-th component representing the class label and patch label respectively.

[0061] Euclidean distance can measure the relative position relationship between class markers and patch markers in geometric space. However, it cannot directly consider directional information and only focuses on the distance between class markers and patch markers. In addition, Euclidean distance is very sensitive to the scale of the vector. When there are outliers in different dimensions, the calculation results may be dominated by the outliers, thereby affecting the accuracy of the distance. To solve the above problems, this example embeds a cosine distance subspace in the subspace to measure the similarity between class markers and patch markers by calculating the angle between vectors. The distance calculation formula between markers in the cosine distance subspace is as follows:

[0062]

[0063] Cosine distance can measure directional similarity and quantify the collinearity between class labels and patch labels, so it pays more attention to directional relationships. However, it only uses the directional information between labels and ignores the influence of absolute scale. Therefore, the advantages of Euclidean distance subspace can complement the shortcomings of cosine distance subspace.

[0064] In addition to measuring the geometric distance in the Euclidean distance subspace and the directional difference in the cosine distance subspace, it is also important to compare the difference in probability distribution between the class label and the patch label. The smaller the probability difference, the more similar the patch label is to the class label, and the more important the corresponding local information is. Based on this, this example introduces the KL divergence subspace to compare the probability distribution difference of the patch label on the class label. Traditional KL divergence calculation requires the probability distribution of statistical vectors to evaluate the similarity through distribution differences. However, class labels and patch labels usually have high-dimensional features, and calculating the probability distribution consumes a lot of time and resources. To solve this problem, this example adopts an intuitive simplification strategy to regard the corresponding feature values ​​as pseudo-probabilities. In class labels and patch labels, the features are concretized as a vector, in which the value of each dimension corresponds to the probability value of the relevant element. The distance calculation formula between the class label and the patch label in the KL divergence subspace is as follows:

[0065]

[0066] After calculating the distance index in multiple subspaces, the MSFA module needs to generate the saliency of the patch mark based on the distance index. The smaller the distance between patch marks, the greater the contribution of the patch to the global representation, indicating that its saliency is stronger. Therefore, this example directly uses the negative value of the distance between the class mark and the patch mark as the saliency vector of the patch, and the formula is as follows:

[0067]

[0068] Euc, Cos, and KL correspond to the Euclidean distance subspace, cosine distance subspace, and KL divergence subspace, respectively.

[0069] For a sequence containing a class label and N patch labels, a saliency vector S of length N can be generated by similarity analysis. Each element corresponds to the similarity between the patch label and the class label. In order to improve the performance of the algorithm, this example normalizes the saliency vector. During the normalization process, considering that patch labels with lower correlation with the class label should be assigned smaller significance (weight less than 1), while patch labels with higher correlation should be assigned larger significance (weight greater than 1), this example uses maximum-minimum normalization with scale and boundary constraints. The calculation method is as follows:

[0070]

[0071] Among them, B left and B right are the normalized left and right boundaries, represents the normalized saliency vector for the jth patch,

[0072] By multiplying the patch labels in the original feature sequence with the normalized saliency vector, a new patch label containing salient features is generated. Finally, the class label is concatenated with the new patch label to generate a weighted label sequence:

[0073]

[0074] in, is the class label in the subspace, is the patch label in the subspace,

[0075] Similar operations are performed in three subspaces to adjust features through class-patch interactions and embed saliency into weighted features to obtain a weighted feature sequence.

[0076] In the object re-identification task, the input features are propagated sequentially through the stacked internal encoder blocks to obtain the final representation. Similar to convolutional neural networks, the encoder blocks close to the input can usually only extract low-level features, while the deeper encoder blocks can extract high-level features. On the one hand, low-level features usually include information such as edges and colors, which are primitive, general, and less relevant to specific tasks. On the other hand, high-level features usually include the shape, local parts, and semantic information of objects. Due to the multi-layer nonlinear transformation, high-level features are more focused on specific tasks and more discriminative.

[0077] Existing Transformer-based methods rarely consider the difference between low-level and high-level features. They usually directly pass the features output by the previous encoder block to the next encoder block. In order to highlight the differences between features at different levels, this example proposes an inter-block ladder-like Transformer (IBLSFormer) based on MSFA. The main idea of ​​IBLSFormer is to set different boundary constraints for the MSFA module between different encoder blocks, such as Figure 3 shown.

[0078] For low-level image features, since these features are usually general information that is not related to a specific task, a narrower constraint range is used. Narrower boundary constraints can concentrate the saliency weights of patch markers around 1. This means that the patch markers have a smaller range of variation compared to the original visual Transformer. On the contrary, for high-level features, since these features are usually task-specific information, a wider constraint range is used. Wider boundary constraints allow the saliency of patch markers to have a larger dispersion range, which means that the patch markers have a larger range of variation compared to the original visual Transformer. This mechanism can provide higher degrees of freedom for patch markers, thereby extracting more discriminative features. IBLSFormer divides the stacked encoder blocks in the original baseline into T groups according to depth, T ≥ 2. Each group contains the same number of encoder blocks. In the shallowest part of the network, relatively narrow constraints are deployed; in the middle depth part, relatively balanced constraints are deployed. In the deepest part of the encoder block, the widest constraints are embedded. This step-by-step constraint pattern can be summarized as:

[0079] G t =[α t ,β t ],t∈[1,T] (12)

[0080] Among them, α t , β t They represent the left and right boundaries of the t-th level constraint, respectively, and satisfy α1<α2<…<α T <1, β T >βT-1 >…>β1>1.

[0081] As a preferred implementation, this embodiment is provided with 3 levels of encoder block groups, each level of encoder block group is provided with M encoder modules, M ≥ 1, such as Figure 3 In order to directly indicate the height (depth) of the level, Figure 3 α H , α M , α L to denote the left boundaries of the high-level, mid-level, and low-level encoder block groups, and β H , β M , β L to denote the right boundaries of the high-level, mid-level, and low-level encoder block groups, and α H <α M <α L <1, β H >β M >β L >1.

[0082] By applying stepped constraints in the inner encoder blocks, stronger narrow constraints can be imposed on the low-level encoder blocks. This allows the features to be weighted smoothly without compromising generality. In the high-level encoder blocks, weaker wide constraints are imposed to further enhance the specificity and freedom of patch labeling. By processing in three subspaces, IBLSFormer is able to generate three corresponding subspace features. These features are cascaded to generate the final image representation, which is then input into the later stages of the encoder block.

[0083] Based on the above-mentioned target re-identification method based on inter-block ladder TRANSFORMER, this embodiment also provides a target re-identification system based on inter-block ladder TRANSFORMER, which is provided with an intelligent agent, which is used to achieve: constructing a target re-identification network, in which the inter-block ladder TRANSFORMER is used as a feature extraction network; training and testing the target re-identification network; and applying the target re-identification network after the test to perform target re-identification. The agent here refers to an agent that can perceive the environment and take actions to achieve specific goals. It can be software, hardware or a system with autonomy, adaptability and interaction capabilities. The agent perceives changes in the environment (such as through sensors or data input), makes judgments and decisions based on the knowledge and algorithms learned by itself, and then performs actions to affect the environment or achieve predetermined goals.

[0084] This embodiment provides a target re-identification method and system based on inter-block ladder TRANSFORMER, which proposes a novel inter-block ladder Transformer (IBLSFormer) model as a feature extraction network for target re-identification tasks. Unlike most methods that directly use visual Transformer as a feature extractor, IBLSFomer focuses on optimizing the internal structure of Transformer to enhance its feature expression ability. Specifically, the proposed multi-subspace feature adjustment (MSFA) module dynamically adjusts feature weights in Euclidean distance subspace, cosine distance subspace, and KL divergence subspace through class-patch interaction, thereby effectively distinguishing between significant patch markers and non-significant patch markers. In addition, IBLSFomer further optimizes the hierarchy of feature learning by grouping encoder blocks and embedding a narrow-medium-wide ladder constraint strategy.

[0085] In order to verify the superiority of IBLSFomer, this example conducts experiments on the benchmark datasets of pedestrian re-identification and vehicle re-identification. The former includes Market-1501 and DukeMTMC-reID, and the latter includes VeRi-776 and VehicleID. Market-1501 is a large-scale pedestrian re-identification benchmark dataset collected by 6 cameras. The training set contains 12,936 images of 751 identities, and the test set contains 19,732 images of 750 identities. DukeMTMC-reID comes from the Duke dataset, which contains 36,411 images collected by 8 cameras, involving 1,812 identities. Among them, the training set contains 16,522 images of 702 identities, and the test set contains 2,228 query images and 17,661 gallery images. VeRi-776 is the first publicly available large-scale vehicle re-identification dataset, consisting of images collected by 20 cameras, involving 776 identities. There are 37778 images of 576 identities in the training set, 11579 images of 200 identities in the gallery, and 1678 images of 200 identities in the query set. VehicleID is a vehicle image dataset collected from a variety of locations and cameras. It contains a total of 221763 images of 26267 vehicles, 110178 images of 13134 vehicles in the training set, and 111585 images of 13133 vehicles in the test set.

[0086] Performance is measured using Cumulative Matching Features (CMC), Mean Average Precision (mAP), and Rank-1 to Rank-5 accuracy. CMC ranks accuracy by calculating the proportion of correct answers in the top few returned results, while mAP measures overall accuracy by calculating the area under the precision-recall curve. Rank-1 accuracy is the most intuitive metric, which measures the percentage of times the highest probability category predicted by the model is the same as the true label in the total number of tests. Rank-2 to Rank-5 accuracy is an extension of Rank-1 accuracy, which takes into account the 2 to 5 highest probability categories output by the model.

[0087] This example uses TransReID pre-trained on ImageNet as a baseline. Pedestrian images are resized to 256×128 and vehicle images are resized to 256×256. The batch size is 64, and each batch contains 16 identities. The training images are augmented with random horizontal flipping, cropping, and erasing. The SGD optimizer with momentum 0.9 and weight decay 0.0001 is used. The initial learning rate is 0.008, and a cosine-based learning rate decay is used. IBLSFormer is trained and tested on the deep learning platform with PyTorch=1.7.1, torchvision=0.8.2, and CUDAVersion=11.2. The hardware configuration is Ubuntu 18.04 operating system and NVIDIA GeForce RTX 3090Ti GPU.

[0088] To verify the feasibility of IBLSFomer, this example compares with the latest methods on the pedestrian re-identification and vehicle re-identification benchmark datasets, as shown in Table 1 and Table 2 respectively.

[0089] As shown in Table 1, on the Market-1501 dataset, IBLSFormer's mAP / Rank-1 reached 88.7% / 95.3%, which is 0.5% and 1.2% higher than the best ResNet-based method ResT-ReID and Transformer-based method DCAL, and 0.6% higher in Rank-1. Compared with the baseline, IBLSFormer improved by 0.9% and 0.6% in mAP / Rank-1. On the DukeMTMC-reID dataset, IBLSFormer's mAP / Rank-1 was 81.4% / 90.4%, which was 0.8% / 0.4% and 1.3% / 1.4% higher than ResT-ReID and DCAL, respectively. Compared with the baseline, IBLSFormer improved by 1.1% and 0.6% in mAP / Rank-1.

[0090] Table 1. Comparison with other pedestrian re-identification methods

[0091]

[0092] As shown in Table 2, on the VeRi-776 and VehicleID datasets, IBLSFormer achieves mAP / Rank-1 of 80.0% / 97.1% and 89.4% / 83.6%, respectively. Compared with the best ResNet-based method CAMNet, mAP / Rank-1 is improved by 0.4% / 0.5% and 0.8% / 1.1%, respectively. Compared with the best Transformer-based SSBVER, mAP / Rank-1 is improved by 2.6% / 1.1% and 0.3% / 0.7%, respectively. Compared with the baseline, IBLSFormer improves mAP / Rank-1 by 1.8% / 0.6% and 2.3% / 1.3%, respectively.

[0093] Table 2. Comparison with other vehicle re-identification methods

[0094]

[0095] The multi-subspace feature adjustment (MSFA) module adjusts the patch labels by performing saliency analysis in the Euclidean distance subspace, cosine distance subspace, and KL divergence subspace. Accordingly, IBLSFomer generates features in three subspaces, which are concatenated to form the final vector. In order to verify whether this combination can improve the performance, corresponding analysis experiments are conducted in this section. First, the three single subspace features are analyzed, and their re-identification results are shown in Tables 3 and 4. From the results in Tables 3 and 4, it can be seen that the features based on the Euclidean distance subspace can achieve the highest performance compared with the features based on the cosine distance subspace and the KL divergence subspace. This shows that the Euclidean distance can accurately reflect the similarity between the class label and the patch label. In addition, the performance of the features based on the KL divergence subspace is relatively low, which may be because the KL divergence is not a strictly defined distance metric. However, feature adjustment based on any single subspace can achieve higher accuracy than the baseline.

[0096] Table 3. Analysis of single subspace features on the pedestrian re-identification dataset

[0097]

[0098] Table 4. Analysis of single subspace features on vehicle re-identification dataset

[0099]

[0100] In addition to comparing single subspace features, this example further analyzes the performance improvement obtained by fusing subspace features. Figure 4 As shown, Figure 4 (a), (b), (c), and (d) are the experimental results for the datasets Market-1501, DukeMTMC-reID, VeRi-776, and VehicleID, respectively. Euc 、F Cos 、F KL Represent the features of Euclidean distance subspace, cosine distance subspace and KL divergence subspace respectively. The feature performance based on Euclidean distance subspace is selected as the baseline. After the introduction of cosine distance subspace, mAP / Rank-1 improvements of 0.4% / 0.2%, 1.0% / 0.3%, 0.7% / 0.2% and 0.2% / 0.3% were obtained on the four datasets respectively. This shows that the cosine distance subspace can effectively supplement the similarity description between class labels and patch labels when considering the directional similarity between features. In addition, after the introduction of KL divergence subspace, compared with the original Euclidean distance subspace features, mAP / Rank-1 increased by 0.6% / 0.3%, 0.6% / 0.5%, 0.9% / 0.3% and 1.1% / 0.5%. This shows that considering the feature distribution between class labels and patch labels can further supplement the similarity information. These data verify the feasibility and effectiveness of the multi-subspace feature fusion mechanism.

[0101] In the process of implementing saliency weighting of different patch labels based on class-patch interaction, this example constrains the boundaries of the saliency map by normalization. In order to verify the feasibility of the boundary constraint strategy (BCS), this example conducted a comparative experiment, specifically comparing the performance of the two cases with boundary constraints (with BCS) and without boundary constraints (w / o BCS). The experimental results are shown in Figure 2. Figure 5 As shown, Figure 5(a), (b), (c), and (d) are experimental results for the datasets Market-1501, DukeMTMC-reID, VeRi-776, and VehicleID, respectively. Compared with the case without boundary constraint strategy, after introducing the boundary constraint mechanism to constrain the saliency map, the performance of the model is improved in different indicators. Specifically, the mAP / Rank-1 improvement of boundary constraint on the four benchmark datasets is 0.5% / 0.3%, 1.0% / 1.1%, 0.9% / 0.3%, and 0.8% / 0.5%, respectively. This is because the boundary constraint strategy restricts the left boundary to a range less than 1, while restricting the right boundary to a range greater than 1. Through this mechanism, the class-patch interaction can dynamically weaken and enhance the patch mark, thereby improving the discriminability of the feature. It should be noted that although the re-identification performance without boundary constraints has decreased compared to the optimal case, it still achieved 1.4%, 1.1%, 0.9% and 1.5% improvements in mAP, and 0.3%, 0.5%, 0.3% and 0.8% improvements in Rank-1, respectively, compared to the baseline model. This shows that the method of weighting patch markers by similarity comparison is feasible. Although this coarse mechanism cannot achieve fine-grained weakening or enhancement of features, it can still selectively distinguish patch markers.

[0102] Experimental results show that IBLSFormer outperforms existing methods on multiple benchmark datasets of pedestrian and vehicle re-identification, fully verifying its effectiveness.

[0103] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. The target re-identification method based on inter-block ladder TRANSFORMER is characterized by: Includes steps: Build a target re-identification network; The target re-identification network adopts an inter-block ladder TRANSFORMER as a feature extraction network; the inter-block ladder TRANSFORMER includes a multi-level encoder block group with increasing depth, and each level of the encoder block group is stacked with a plurality of encoder blocks of the same number, and a multi-subspace feature adjustment module is set at the first layer of each encoder block, and the constraints of the multi-subspace feature adjustment module of the multi-level encoder block group from shallow to deep are narrower and wider; the multi-subspace feature adjustment module enhances the patch mark with a stronger association with the class mark and weakens the patch mark with a weaker association with the class mark through class patch interaction, and the class patch interaction is performed in multiple subspaces including the Euclidean distance subspace, the cosine distance subspace and the KL divergence subspace; Train and test the object re-identification network; Apply the tested target re-identification network to perform target re-identification.

2. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 1, characterized in that: The processing flow of the multi-subspace feature adjustment module includes the following steps: For the input feature sequence, the distance D between its class label and patch label in the Euclidean distance subspace, cosine distance subspace and KL divergence subspace is calculated respectively. Euc , D Cos and D KL ; According to D Euc , D Cos and D KL Calculate the saliency vector for each patch in the feature sequence; Normalize the saliency vector to obtain the normalized saliency vector of each patch; Multiply the label of each patch by the normalized saliency vector corresponding to the patch to generate a new patch label containing saliency features; The class labels are concatenated with the new patch labels to generate a weighted label sequence and obtain a weighted feature sequence.

3. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 2, characterized in that: Normalize the saliency vector as follows: The saliency vector is normalized using the maximum-minimum normalization with scale and boundary constraints; Normalized saliency vector of the jth patch Calculated as N is the total number of patches; S j is the saliency vector of the jth patch, S is the vector composed of the saliency vectors of N patches, min(·) means taking the minimum, max(·) means taking the maximum, B left and B right are the normalized left and right boundaries, Euc, Cos, and KL correspond to the Euclidean distance subspace, cosine distance subspace, and KL divergence subspace, respectively.

4. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 3, characterized in that: Using the distance D between the class label and the patch label Euc , D Cos and D KL The negative value of is used as the saliency vector of the patch in the Euclidean distance subspace, cosine distance subspace, and KL divergence subspace.

5. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 4, characterized in that: Distance D Euc Calculated as cls i and patch i The vector values ​​representing the i-th component of the class label and patch label respectively; Distance D Cos Calculated as Distance D KL Calculated as 6. The object re-identification method based on inter-block stepwise TRANSFORMER according to any one of claims 1 to 5, characterized in that: The inter-block ladder TRANSFORMER includes T-level encoder block groups with increasing depth, and the constraints of the multi-subspace feature adjustment module in the encoder block in the t-th level encoder block group are set as follows: G t =[α t ,β t ], α t , β t They represent the left and right boundaries of the t-th level constraint, respectively, and satisfy α1<α2<…<α T <1, β T >β T-1 >…>β1>1.

7. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 1, characterized in that: Before IBLSFomer performs feature extraction, the target image is segmented into N patches using a homogeneous sliding window. And processed by linear mapping, an additional class marker Indicates that a learnable class is attached to the input, and D represents the dimension of the class; The characteristic representation of the input inter-block ladder TRANSFORMER is Where F(·) represents the linear mapping function and P is the position embedding.

8. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 7, characterized in that: During training, the cross entropy loss L CE and triplet loss L TRI The sum is taken as the total loss L, the cross entropy loss Tuple Loss q i is the target probability, p i is the predicted probability, f a 、f p and f n They represent anchor points, positive samples, and negative samples respectively.

9. The object re-identification method based on inter-block stepwise TRANSFORMER according to claim 1, characterized in that: The target re-identification network is based on the network architecture of TransReID, which changes the feature extraction network into an inter-block stepped TRANSFORMER on the network architecture of TransReID; The datasets used for training and testing the target re-identification network are benchmark datasets for pedestrian re-identification and vehicle re-identification.

10. The object re-identification system based on inter-block ladder TRANSFORMER is characterized by: The system is provided with an intelligent agent, which is used to realize: constructing a target re-identification network, in which an inter-block ladder-type TRANSFORMER is used as a feature extraction network; Train and test the target re-identification network; apply the tested target re-identification network to perform target re-identification.

Images