A pedestrian re-identification method and system combining air-ground perspectives

Abstract

The present application relates to the technical field of computer vision, and discloses a pedestrian re-identification method and system combining aerial and ground perspectives, which learns two branches of learnable global feature tokens, one branch obtains original global feature tokens for comparison after transformer learning, and the other branch reorganizes patch feature sequences to construct reorganized sequences between the aerial and ground perspectives, and inputs the reorganized sequences into the transformer for learning to obtain reorganized global feature tokens effectively combining pedestrian information of the aerial and ground perspectives, aligns the reorganized global feature tokens with the original global feature tokens and calculates alignment loss to optimize the identification model, and since the reorganized global feature tokens contain dramatic perspective change information of pedestrians under the unmanned aerial vehicle and the ground camera, can effectively fuse the pedestrian information of the aerial and ground perspectives, and improve the accuracy of aerial and ground pedestrian re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a pedestrian re-identification method and system combining aerial and ground perspectives. BACKGROUND

[0002] Pedestrian re-identification technology aims to retrieve a pedestrian of interest from multiple cameras with non-overlapping views. With the continuous development of deep learning, researchers have gradually deepened their exploration of pedestrian re-identification, including pedestrian re-identification technology based on ground cameras and pedestrian re-identification technology from the perspective of aerial drones.

[0003] Pedestrian re-identification technology based on ground cameras has achieved good performance, most of which are based on deep learning methods. These methods can be mainly divided into two categories. One category is based on convolutional neural networks, such as BOT, SBS, OSNet, AGW, and PCB, etc. The other category is based on Transformer, such as TransReID and DC-Former. These methods have achieved satisfactory performance on conventional ground datasets such as Market-1501 and MSMT17.

[0004] For aerial drone perspective pedestrian re-identification technology, Zhang et al. proposed using subspace pooling of convolutional feature maps to represent input pedestrian images to better learn aerial pedestrian discriminative features. Chen et al. proposed a new rotation-invariant visual transformer in 2022 to address the rotation problem in aerial scenarios. These methods have achieved good performance in aerial pedestrian re-identification tasks and have achieved high accuracy on PRAI-1581 and UAV-Human datasets.

[0005] However, existing pedestrian re-identification methods based on ground cameras cannot effectively handle aerial perspective pedestrian correlation. When simultaneously correlating pedestrian pictures obtained by ground cameras and pedestrian pictures taken by drones, the retrieval effect of the above methods is poor, because aerial perspective pedestrian pictures and ground camera pedestrian pictures have a large difference in perspective. In aerial perspective pedestrian pictures, the position of the pedestrian body part has a large uncertainty, which leads to the inability to correlate aerial and ground perspective pedestrian re-identification.

[0006] Therefore, it is of great value to research a pedestrian re-identification method that can combine aerial and ground perspective pedestrian information. SUMMARY

[0007] The present application aims to provide a pedestrian re-identification method and system combining aerial and ground perspectives to solve the problem of effectively combining aerial and ground perspective pedestrian information in existing pedestrian re-identification methods.

[0008] In order to solve the above technical problems, the first technical solution of the present invention provides a pedestrian re-identification method combining an air-ground perspective, comprising the following steps:

[0009] Training recognition models;

[0010] Using the trained recognition model to recognize images from different perspectives to obtain recognition results;

[0011] The training process of the recognition model is as follows:

[0012] Generate patch feature embedding sequences based on images from different perspectives;

[0013] Adding a learnable global feature token to the patch feature embedding sequence to obtain an output sequence;

[0014] Input the output sequence into the Transformer layer to obtain a preliminary processed feature sequence;

[0015] Inputting the preliminary processed feature sequence into two branches respectively;

[0016] The preliminarily processed feature sequence in one branch is learned through the final Transformer layer to obtain the original global feature token;

[0017] The preliminary processed feature sequence in the other branch is recombined to generate a recombined sequence, which is input into the final Transformer layer for learning to obtain a recombined global feature token;

[0018] Performing feature alignment on the original global feature token and the recombined global feature token to obtain an aligned global feature token, and calculating an alignment loss;

[0019] constructing a total loss using the alignment loss;

[0020] The recognition model is optimized by using the total loss training to obtain the trained recognition model.

[0021] In some embodiments of the first technical solution, the preliminary processed characteristic sequence in the other branch is subjected to a recombination process to generate a recombined sequence. This process specifically includes the following steps:

[0022] performing separation processing on the preliminary processed feature sequence to obtain a separated patch feature embedding sequence and a separated global feature token;

[0023] performing a shift process on the separated patch feature embedding sequence to obtain a shifted patch feature embedding sequence;

[0024] sequentially grouping the shifted patch feature embedding sequence, constructing a grouping matrix, transposing the grouping matrix, and flattening the transposed grouping matrix to obtain a reorganized patch feature embedding sequence;

[0025] adding the separated global feature token to the reorganized patch feature embedding sequence to obtain the reorganized sequence.

[0026] In some embodiments of the first technical solution, in the process of shifting the separated patch feature embedding sequence, the following steps are specifically performed:

[0027] moving the first s patches of the separated patch feature embedding sequence to the end of the separated patch feature embedding sequence, and moving the remaining patches of the separated patch feature embedding sequence to the beginning of the separated patch feature embedding sequence.

[0028] In some embodiments of the first technical solution, in the process of sequentially grouping the shifted patch feature embedding sequence and constructing a grouping matrix, the following steps are specifically performed:

[0029] dividing the shifted patch feature embedding sequence into M groups, each group containing N patch feature embeddings;

[0030] constructing the M groups of patch feature embeddings into a grouping matrix;

[0031] wherein the product of M and N is l, and l is the number of patch blocks.

[0032] In some embodiments of the first technical solution, in the process of flattening the transposed grouping matrix to obtain a reorganized patch feature embedding sequence, the following steps are specifically performed:

[0033] traversing the patch feature embeddings row by row to flatten the transposed grouping matrix to obtain a reorganized patch feature embedding sequence.

[0034] In some embodiments of the first technical solution, in the step of generating a patch feature embedding sequence based on different view images, the following steps are specifically included:

[0035] obtaining different view images, and performing block processing on the different view images to obtain a plurality of patch blocks of a predetermined length;

[0036] flattening each of the patch blocks and performing projection processing to generate the patch feature embedding sequence.

[0037] In some embodiments of the first technical solution,

[0038] The function of the alignment loss is as follows:

[0039]

[0040] In the formula, t' represents a recombined global feature token, and t'0 represents an original global feature token. p In the formula, t' represents a recombined global feature token, and t'0 represents an original global feature token.

[0041] In some embodiments of the first technical solution, the total loss further includes a score loss, and the score loss is calculated as follows:

[0042] The classifier is used to calculate a predicted score of the original global feature token and the recombined global feature token, respectively.

[0043] The score loss is calculated, and the score loss and the alignment loss are combined to obtain the total loss.

[0044] In some embodiments of the first technical solution, the total loss further includes a cross-entropy loss, and the cross-entropy loss is calculated as follows:

[0045] The cross-entropy loss of the branch of the original global feature token is calculated, and the cross-entropy loss and the alignment loss are combined to obtain the total loss.

[0046] The second technical solution of the present application provides a pedestrian re-identification system combining aerial and ground perspectives, which applies the pedestrian re-identification method combining aerial and ground perspectives in the first technical solution, and includes:

[0047] The training module is configured to generate a patch feature embedding sequence based on different perspective pictures, add a learnable global feature token to the patch feature embedding sequence to obtain an output sequence, input the output sequence into a multi-layer Transformer layer to obtain a preliminary processing feature sequence, and input the preliminary processing feature sequence into two branches respectively. The preliminary processing feature sequence in one branch is learned through a final Transformer layer to obtain an original global feature token, and the preliminary processing feature sequence in the other branch is recombined to generate a recombined sequence, which is input into a final Transformer layer to obtain a recombined global feature token. The original global feature token and the recombined global feature token are aligned to obtain an aligned global feature token, and an alignment loss is calculated. The recognition model is trained and optimized using the alignment loss to obtain the trained recognition model.

[0048] The recognition module is configured to identify different perspective pictures using the trained recognition model to obtain an identification result.

[0049] The present application has the following advantages:

[0050] The application introduces learnable global feature tokens, divides the global feature tokens into two branches for learning, one branch obtains original global feature tokens for comparison after learning by a transformer, and the other branch constructs a reorganized sequence between the spatial domain and the geographical domain by reorganizing the patch feature sequence, and inputs the reorganized sequence into the transformer for learning to obtain reorganized global feature tokens containing effective combination of spatial and geographical perspective pedestrian information, aligns the reorganized global feature tokens and the original global feature tokens and calculates alignment loss to optimize the recognition model, and since the reorganized global feature tokens contain the dramatic perspective change information of pedestrians under the unmanned aerial vehicle and the ground camera, the spatial and geographical perspective pedestrian information can be effectively fused to improve the accuracy of spatial and geographical pedestrian re-identification. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0052] Fig. 1 is the overall process step flowchart provided by the preferred embodiment of the present application;

[0053] Fig. 2 is the training step flowchart of the recognition model provided by the preferred embodiment of the present application;

[0054] Fig. 3 is the framework diagram of the recognition model provided by the preferred embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application.

[0056] The existing pedestrian re-identification method based on ground cameras cannot well handle the correlation of pedestrians in the air perspective, while the pedestrian re-identification method based on spatial perspective can effectively alleviate this problem, but when we need to simultaneously perform cross-domain correlation on the pedestrian pictures obtained by the ground camera and the pedestrian pictures taken by the unmanned aerial vehicle, the retrieval effect of the above method is poor, which is because the pedestrian pictures in the spatial perspective and the pedestrian pictures of the ground camera have a very large difference in perspective, and the position of the pedestrian body part in the pedestrian picture in the spatial perspective has a large uncertainty, which is a major challenge for the correlation between the spatial domain and the geographical domain.

[0057] In order to solve the above problems, the present application provides a pedestrian re-identification method which can realize the combination of aerial view and ground view information, and improve the accuracy of pedestrian re-identification in the cross-air and ground domains, and specifically comprises the following steps:

[0058] S1, training an identification model, please refer to Figs. 1 to 3 .

[0059] Among them, step S1 specifically comprises the following steps, as shown in Fig. 2 and Fig. 3 :

[0060] S10, based on different view pictures, generate patch feature embedding sequence, and the generated patch feature embedding sequence is used for input into the Transformer layer for subsequent processing.

[0061] Preferably, step S10 comprises the following steps:

[0062] S100, acquiring different view pictures, and performing block processing on the different view pictures to obtain a plurality of patch blocks with a preset length, the preset length being a fixed length, and the fixed length patch is to adapt the feature representation of the patch block to the input form of the Transformer model.

[0063] Specifically, for the processing process of the picture taken by the ground camera and the RGB picture taken by the unmanned aerial vehicle (the channel number C is 3): first, the picture is adjusted to H times W size by the bilinear interpolation method, and H is set to 256 and W is set to 128, for the input pedestrian picture p∈R^(H×W×C) with a height of H, a width of W and a channel number of C; then, the visual Transformer is used to divide it into l patch blocks, each patch block has a size of 16 times 16, and there are a total of 16 times 8 patches.

[0064] Among them, each patch has a size of 16 times 16, and such fixed length patch feature embedding is to adapt the feature representation of the patch block to the input form of the Transformer model, and it should be understood that the patch size in the present scheme is only preferred, as long as the fixed length patch that can adapt to the input form of the Transformer model can be selected by those skilled in the art.

[0065] S101, flattening each patch block and performing projection processing to generate a patch feature embedding sequence.

[0066] Preferably, after each patch is flattened into a one-dimensional vector, it is processed by a linear projection layer to generate a patch feature embedding with a dimension of D.

[0067] S11, adding a learnable global feature token to the patch feature embedding sequence to obtain an output sequence.

[0068] Preferably, the learnable global feature token is added to the prefix of the patch feature embedding sequence as a representation of the global feature, and finally, the output sequence p e R (l+1)×D , where l represents that the patch feature embedding sequence has D dimensions, and 1 represents that the learnable global feature token has only 1 dimension.

[0069] S12, input the output sequence into the Transformer layer to obtain a preliminary processing feature sequence, which is to aggregate the global information and local information of the pedestrian, and enhance the understanding and representation ability of the model to the data. The global feature token is learnable, which can be adjusted during the training process to better obtain the overall features of the pedestrian.

[0070] Preferably, the output sequence is received by the Lth Transformer, and after passing through the L-1th Transformer, the preliminary processing feature sequence is obtained.

[0071] S13, input the preliminary processing feature sequence into two branches respectively.

[0072] a, the preliminary processing feature sequence in one branch a is learned by the final Transformer layer to obtain the original global feature token t'0.

[0073] Wherein, one branch a is composed of only a single Transformer, which essentially acts as an original Transformer and mainly utilizes the self-attention mechanism of the Transformer for processing.

[0074] b, the preliminary processing feature sequence in the other branch b is reorganized to generate a reorganized sequence, and the reorganized sequence is input into the final Transformer layer to learn to obtain a reorganized global feature token.

[0075] After adopting this setting mode, the learnable global feature token is divided into two branches for learning. One branch learns through the transformer to obtain the original global feature token for comparison. The other branch processes the patch feature sequence to generate a reorganized sequence, and inputs the reorganized sequence into the transformer to learn to obtain a reorganized global feature token containing the effective combination of aerial and ground perspective pedestrian information. The reorganized global feature token and the original global feature token are aligned and the alignment loss is calculated to optimize the recognition model. Since the reorganized global feature token contains the dramatic perspective change information of pedestrians under the unmanned aerial vehicle and the ground camera, it can effectively fuse the aerial and ground perspective pedestrian information, and improve the accuracy of aerial and ground pedestrian re-identification.

[0076] Preferably, the processing of the preliminary processed feature sequence in another branch b comprises the following steps:

[0077] b1, performing separate processing on the preliminary processed feature sequence to obtain a separate patch feature embedding sequence and a separate global feature token, after this step, the global information and the local information of the pedestrian can be effectively explored, and the discriminative features of different pedestrians can be better obtained.

[0078] The feature represented by the L-1 layer of the Transformer is represented as [t p ;F P ], where t p ∈R 1×D represents the separate global feature token, F P ∈R l×D represents the separate patch feature embedding sequence, and for the i P th patch in F th , it is represented as Therefore, we have: It should be understood that since F P needs to be processed first in the next step, this separate processing is needed.

[0079] b2, performing shift processing on the separate patch feature embedding sequence to obtain a shifted patch feature embedding sequence, after this step, the shift can make the patch features at the front end also be processed, and subsequent transposition and regular rearrangement can also be performed, if there is no shift, the patch features at the front end cannot be rearranged.

[0080] Preferably, step b2 comprises the following steps:

[0081] b20, moving the first s patches of the separate patch feature embedding sequence to the end of the separate patch feature embedding sequence, and moving the remaining patches of the separate patch feature embedding sequence to the beginning of the separate patch feature embedding sequence, that is, simultaneously moving the remaining patches forward.

[0082] It should be pointed out that for the movement of the patches, for example, the sequence A, B, C, D, E, F, G, we need to shift 2, then we move the objects A, B to the tail, and get: C, D, E, F, G, A, B.

[0083] After shifting, F P is represented as F′ p , and each shifted patch is represented as g i :

[0084]

[0085] It should be noted that in the present embodiment, the preset condition is preferably that moving the first s patches embedding the separated patch features to the end of the separated patch feature embedding sequence can be stopped.

[0086] b3, grouping the shifted patch feature embedding sequence in order, constructing a grouping matrix, transposing the grouping matrix, and flattening the transposed grouping matrix to obtain a reorganized patch feature embedding sequence. After this step, the position uncertainty of each body part of the pedestrian under the perspective of the unmanned aerial vehicle can be effectively simulated, which helps the model better align the air domain.

[0087] Preferably, step b3 comprises the following steps:

[0088] b30, first flattening the shifted patch feature embedding sequence F′ p into M groups, each group containing N patch feature embeddings; then, constructing M groups of patch feature embeddings into a grouping matrix G P ;

[0089]

[0090] wherein the product of M and N is l, and l is the number of patch blocks.

[0091] b31, transposing the grouping matrix, i.e. exchanging the dimensions M and N, to obtain the transposed result H P of G P : P T .

[0092] b32, flattening the transposed grouping matrix by traversing the patch feature embeddings row by row to obtain a reorganized patch feature embedding sequence I P :

[0093] I P = [g1, g N+1 ,…, g l-N+1 ,g2, g N+2 ,…, g l ] T

[0094] b4, adding a separated global feature token to the reorganized patch feature embedding sequence to obtain a reorganized sequence.

[0095] Preferably, the separated global feature token t p is added to the front of the reorganized patch feature embedding sequence I P to obtain a reorganized sequence I′ p :

[0096] I′ p = [t pg1, g N+1 ..., g l-N+1 g2, g N+2 ..., g l ] T

[0097] b5, input the reorganization sequence into the final Transformer layer learning to obtain reorganized global feature tokens t' p After this step, the model can be better optimized to obtain discriminative features of pedestrians at the global level.

[0098] Summarize the process of branch b, which obtains reorganized patch feature embedding sequences I by shifting, global transposition and regular reorganization P The reorganization embedded in this branch can efficiently construct reorganized patch feature embedding sequences between the air space and the region, effectively handle the dramatic changes in the visual angle of pedestrians under unmanned aerial vehicles and ground cameras, and thus improve the accuracy of air-ground pedestrian recognition.

[0099] S14, align the original global feature tokens t'0 and the reorganized global feature tokens t' p to obtain aligned global feature tokens, and calculate the alignment loss, that is, combine this t' p with the global feature tokens t'0 from another branch to obtain pedestrian robust feature representations of different branches. Two different branches can obtain different feature representations of pedestrians. Since the calculation method is based on a supervised method, each pedestrian has a corresponding identity label.

[0100] Wherein, the alignment loss refers to the smooth p loss between the tokens t' 0 obtained by two different branches. In order to reduce the differences between the air space and the region, the alignment loss is used to guide the optimization of the model.

[0101] The alignment loss function is as follows:

[0102]

[0103] In the above formula, t' p represents the reorganized global feature tokens, and t'0 represents the original global feature tokens.

[0104] S15, construct the total loss using the alignment loss As shown in Fig. 3 , the total loss includes alignment loss score loss triplet loss and cross-entropy loss The total loss is utilized to optimize the model. The total loss is utilized to optimize the model.

[0105] wherein, Fig. 3 The right side feature alignment in the total loss contains an alignment loss and a score loss Both of them can play the role of empty region feature alignment.

[0106] In order to make the predictions of different global feature tokens of the same identity consistent, the model introduces a score loss for optimization. For the score loss, first, the classifier is used to calculate the prediction scores of the original global feature token and the reorganized global feature token respectively; then, the score loss is calculated, and the score loss and the alignment loss are combined. After using the score loss, for the reorganized global feature token t' p and the original global feature token t'0, the model makes the predictions of different global feature tokens of the same identity consistent.

[0107] The score loss function is as follows:

[0108]

[0109] In the above formula, s' p is the prediction score of the reorganized global feature token calculated by the classifier, and s'0 is the prediction score of the original global feature token calculated by the classifier.

[0110] In order to minimize the distance between pedestrians of the same identity and maximize the distance between pedestrians of different identities, the model introduces a triplet loss, calculates the triplet loss of the branch of the original global feature token, and combines the alignment loss and the triplet loss to optimize the recognition model by using the triplet loss.

[0111] For the triplet loss , the triplet loss function is as follows:

[0112]

[0113] wherein {Φ,Φ + ,Φ -} is a triplet, Φ is an anchor sample in the triplet, Φ + is a positive sample, and Φ - is a negative sample.

[0114] Specifically, for the pedestrian feature samples in the first branch, we find the corresponding positive sample pedestrian features and negative sample pedestrian features through the label, and form a triplet with the three pedestrian features. After adopting the triplet loss, the triplet loss can minimize the distance between pedestrians with the same identity and maximize the distance between pedestrians with different identities, which helps the model to learn more distinctive features.

[0115] Since the above method is a supervised method, each pedestrian has a corresponding identity label. Therefore, in order to improve the calculation accuracy in branch a, the model introduces cross-entropy loss for supervised optimization. For the cross-entropy loss, the calculation process is as follows: the cross-entropy loss of the branch of the original global feature token is calculated, the alignment loss and the cross-entropy loss are combined, and the cross-entropy loss is used to optimize the recognition model.

[0116] Cross-entropy loss The function is as follows:

[0117]

[0118] Where K represents the number of all pedestrian categories, and the indicator function y i * is equal to 1 when the ith pedestrian belongs to the ith category, and y i is the prediction value of the Transformer.

[0119] Finally, the total loss of our model is:

[0120]

[0121] In the above formula, alpha and beta are used to balance the alignment loss and the score loss

[0122] The total loss is used to optimize the recognition model to obtain the final trained recognition model. After combining the alignment loss and the score loss , both of them are used to optimize the application of patch feature embedding sequence, which significantly reduces the difference between the two domains of empty land, and further enhances the comprehensive performance of the model.

[0123] S16, the total loss is used to train and optimize the recognition model to obtain the trained recognition model.

[0124] S2, the trained recognition model is used to recognize different perspective pictures to obtain the recognition result.

[0125] In the specific implementation, our model is implemented using the PyTorch framework, pre-trained on ImageNet, and run on an NVIDIA GeForce RTX 3090 graphics card. During training, we set the batch size to 64, sample 4 pedestrian images per identity, and adjust the input image size to 256x128; for data augmentation, we apply random flipping, padding, and random cropping strategies to the training data, and do not use the enhancement strategy in the test phase; and use stochastic gradient descent as the optimizer, with a base learning rate of 8e-3, a minimum learning rate of 1.6e-5, and a weight decay of 1e-4. In the test phase, we use the Rank1, Rank5, and Rank10 of the Cumulative Matching Characteristic (CMC) curve and the mean precision (mAP) indicators.

[0126] To illustrate the matching accuracy and effect of the present scheme, the method is applied to the existing AG-ReID data set containing unmanned aerial vehicle and ground camera perspective and the self-constructed air-ground multi-source data set containing only unmanned aerial vehicle perspective PRAI-1581, and compared with previous methods such as MGN, BOT, AGW and TransReID, the method of the present scheme achieves better performance, as shown in Tables 1 and 2.

[0127] The mAP (mean average precision) indicator in Tables 1 and 2 measures the retrieval performance of the model on all queries, and Rank k (k = 1, 5, 10,...) represents the probability of correct matching appearing in the top k retrieval results.

[0128] Table 1 Experimental performance of the method on the self-constructed air-ground multi-source data set

[0129]

[0130]

[0131] The self-constructed air-ground multi-source data set contains 66,242 pedestrian images from 2,032 different volunteers.

[0132] Table 2 Experimental performance of the method on AG-ReID and PRAI-1581 data sets

[0133]

[0134] From the above, the basic processing process of the present scheme will be given below, which is a pedestrian re-identification system combining air-ground perspectives, specifically including a training module and a recognition module.

[0135] For the training module, the training module is configured to generate a patch feature embedding sequence based on the different-view pictures; add a learnable global feature token to the patch feature embedding sequence to obtain an output sequence; input the output sequence into a multi-layer Transformer layer to obtain a preliminary processing feature sequence; input the preliminary processing feature sequence into two branches respectively; the preliminary processing feature sequence in one branch is learned by a final Transformer layer to obtain an original global feature token; the preliminary processing feature sequence in the other branch is reorganized to generate a reorganized sequence, and the reorganized sequence is input into a final Transformer layer for learning to obtain a reorganized global feature token; the original global feature token and the reorganized global feature token are aligned in feature to obtain an aligned global feature token, and an alignment loss is calculated; the trained recognition model is obtained by training and optimizing the recognition model using the alignment loss.

[0136] For the recognition module, the recognition module is configured to recognize the different-view pictures using the trained recognition model to obtain a recognition result.

[0137] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements are also considered within the scope of protection of the present application.

Claims

1. A pedestrian re-identification method combining open-ground perspective, characterized in that: The following steps are involved: Training recognition models; Using the trained recognition model to recognize images from different perspectives to obtain recognition results; The training process of the recognition model is as follows: Generate patch feature embedding sequences based on images from different perspectives; Adding a learnable global feature token to the patch feature embedding sequence to obtain an output sequence; Input the output sequence into the Transformer layer to obtain a preliminary processed feature sequence; Inputting the preliminary processed feature sequence into two branches respectively; The preliminarily processed feature sequence in one branch is learned through the final Transformer layer to obtain the original global feature token; The preliminary processed feature sequence in the other branch is recombined to generate a recombined sequence, which is input into the final Transformer layer for learning to obtain a recombined global feature token; Performing feature alignment on the original global feature token and the recombined global feature token to obtain an aligned global feature token, and calculating an alignment loss; constructing a total loss using the alignment loss; The recognition model is optimized by using the total loss training to obtain the trained recognition model.

2. The pedestrian re-identification method combining air-ground perspective according to claim 1 is characterized in that: The preliminary processed characteristic sequence in the other branch is subjected to a recombination process to generate a recombined sequence. This process specifically includes the following steps: performing separation processing on the preliminary processed feature sequence to obtain a separated patch feature embedding sequence and a separated global feature token; performing a shift process on the separated patch feature embedding sequence to obtain a shifted patch feature embedding sequence; Sequentially grouping the shifted patch feature embedding sequence and constructing a grouping matrix, transposing the grouping matrix, and flattening the transposed grouping matrix to obtain a recombined patch feature embedding sequence; The separated global feature token is added to the recombined patch feature embedding sequence to obtain the recombined sequence.

3. The pedestrian re-identification method combining air-ground perspective according to claim 2 is characterized in that: In the process of shifting the separated patch feature embedding sequence, the specific steps are as follows: The first s patches of the separated patch feature embedding sequence are moved to the end of the separated patch feature embedding sequence, and the remaining patches of the separated patch feature embedding sequence are moved to the beginning of the separated patch feature embedding sequence.

4. The pedestrian re-identification method combining air-ground perspective according to claim 2 is characterized in that: The shifted patch feature embedding sequence is grouped in sequence and a grouping matrix is ​​constructed. The specific steps are as follows: Dividing the shifted patch feature embedding sequence into M groups, each group containing N patch feature embeddings; Construct M groups of patch feature embeddings as a grouping matrix; The product of M and N is l, where l is the number of patches.

5. The pedestrian re-identification method combining air-ground perspective according to claim 2 is characterized in that: The recombined patch feature embedding sequence is obtained from the flattened and transposed grouping matrix. The specific process is as follows: The patch feature embeddings are traversed row by row to flatten the transposed grouping matrix and obtain a reorganized patch feature embedding sequence.

6. The pedestrian re-identification method combining air-ground perspective according to claim 1 is characterized in that: In the step of generating patch feature embedding sequences based on images from different perspectives, the following steps are specifically included: Obtaining pictures from different viewing angles, dividing the pictures from different viewing angles into blocks to obtain multiple patch blocks of preset lengths; Each of the patch blocks is flattened and projected to generate the patch feature embedding sequence.

7. The pedestrian re-identification method combining air-ground perspective according to claim 1 is characterized in that: The function of the alignment loss is as follows: In the above formula, t′ p represents the recombined global feature token, and t′0 represents the original global feature token.

8. The pedestrian re-identification method combining air-ground perspective according to claim 1 is characterized in that: The total loss also includes a fractional loss, and the fractional loss calculation process is as follows: Calculating prediction scores of the original global feature token and the recombined global feature token respectively using a classifier; The score loss is calculated, and the score loss and the alignment loss are combined to construct the total loss.

9. The pedestrian re-identification method combining air-ground perspective according to claim 1, characterized in that: The total loss also includes cross entropy loss, and the cross entropy loss is calculated as follows: The cross entropy loss of the branch of the original global feature token is calculated, and the cross entropy loss and the alignment loss are combined to construct the total loss.

10. A pedestrian re-identification system combining air-ground perspective, characterized in that: The pedestrian re-identification method combining an air-ground perspective according to any one of claims 1 to 9 is applied, comprising: A training module, wherein the training module is used to generate a patch feature embedding sequence based on images of different viewpoints; add a learnable global feature token to the patch feature embedding sequence to obtain an output sequence; input the output sequence into a multi-layer Transformer layer to obtain a preliminary processed feature sequence; input the preliminary processed feature sequence into two branches respectively; the preliminary processed feature sequence in one branch is learned through a final Transformer layer to obtain an original global feature token; the preliminary processed feature sequence in the other branch is recombined to generate a recombined sequence, and the recombined sequence is input into a final Transformer layer to learn to obtain a recombined global feature token; feature alignment is performed on the original global feature token and the recombined global feature token to obtain an aligned global feature token, and alignment loss is calculated; the recognition model is trained and optimized using the alignment loss to obtain the trained recognition model; The recognition module is used to use the trained recognition model to recognize pictures from different perspectives to obtain recognition results.

Images