Shielding pedestrian re-identification method based on combination of species perception and cross-layer information

Abstract

The invention provides an occluded pedestrian re-identification method based on combination of species perception and cross-layer information, and belongs to the field of pedestrian re-identification in multimedia information processing. According to the method, pedestrian re-identification is carried out on an occluded pedestrian image by training an occluded pedestrian re-identification network which is composed of two 12-layer twinning Vision Transform backbone networks sharing parameters, respective puzzle branches of the twinning Vision Transform backbone networks and respective cross-layer information aggregation modules of the twinning Vision Transform backbone networks. The method comprises the following steps of: firstly, converting training data into a required size, and carrying out forward propagation on a complete branch of the training data in a twin Vision Transform backbone network; and then, the output of the last but one Transform layer is used for forward propagation on a puzzle branch. The method comprises the following steps of: randomly erasing a picture, and performing forward propagation in an erasing branch and a jigsaw branch in a twin Vision Transform backbone network by using the erased data. And finally, collecting the similarity matrixes of all branches, collecting the cross-layer information of the two branches, which is oriented by the species discrimination characteristics, by using a self-cross-layer information aggregation module, and calculating the loss of the cross-layer information. And finally, training the whole network through back propagation. In the test process, test data pedestrian images are input into the sheltered pedestrian re-identification network to obtain pedestrian features, and pedestrian re-identification is implemented. Compared with other methods, the method has the advantage that the accuracy of re-identification of the shielded pedestrians is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of pedestrian re-identification in computer vision, and specifically relates to an occluded pedestrian re-identification method based on the combination of species perception and cross-layer information. Background Art

[0002] Person re-identification is a retrieval task to find target pedestrians with the same identity in different environments. With the popularization of smart cities and Skynet systems, it has attracted more and more attention. The task needs to overcome many difficulties, such as interference from the environment where the target pedestrian is located, inherent differences in images taken by different devices, and the appearance of random obstacles. The occlusion in the target pedestrian image is the most important factor affecting the accuracy of the pedestrian re-identification task.

[0003] At present, deep learning-based methods often use attention mechanisms for end-to-end detection. However, such methods usually make the model focus on more discriminative local features, but cannot determine whether the feature actually comes from the target object. Specifically, the network tends to focus on the most recognizable parts of the image, such as a bright background, while other parts of the pedestrian's body are suppressed, that is, the network will have shortcut learning and overfitting problems. Therefore, a pedestrian re-identification technology that can overcome both shortcut learning and the influence of occlusions is needed to improve the performance of pedestrian re-identification in extreme environments such as occlusion. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method for re-identifying occluded pedestrians based on the combination of species perception and cross-layer information. The method comprises the following steps:

[0005] The training data U and the test data are collected from the surveillance camera device. The training data U includes the occluded pedestrian re-identification pictures, and the test data only includes the original occluded pedestrian re-identification pictures and pedestrian numbers.

[0006] First, the complete target feature diffusion occlusion pedestrian re-identification network is trained to convergence using the training data U. Then, the test data is sent to the complete branch of the complete target feature diffusion occlusion pedestrian re-identification network, and the class_token output by the last vision transformer layer in the complete branch is obtained as the feature of each image. Finally, for each query image, the feature distance is sorted from small to large, and the 10 closest library images are taken as the query result of the query image. After the accuracy is counted, a good pedestrian recognition accuracy can be obtained.

[0007] The complete target feature diffusion occluded pedestrian re-identification network is characterized by being composed of two 12-layer twin Vision Transformer backbone networks with shared parameters, their respective puzzle branches and their respective cross-layer information aggregation modules. The training process of the occluded pedestrian re-identification network is as follows:

[0008] The occluded pedestrian re-identification network uses the twin Vision Transformer backbone network for feature extraction. The twin Vision Transformer backbone network means that two Vision Transformers share model parameters but process different data. The twin Vision Transformer backbone network inputs an image X with a height of H, a width of W, and a dimension of C, and then uses convolution to split the image X into N pieces of size P. H ×P W The image block, P H With P W Represent the height and width of the image block respectively. When the horizontal step size and vertical step size of the convolution are S W With S H When , N can be expressed using the following formula:

[0009]

[0010] in Indicates rounding down. In the next step, all image patches will be flattened to 1D to become patches, and a learnable class_token will be added before the patches. The resulting vector will be added to the learnable position embedding to help the visiontransformer model the spatial information of cls_token and different patches. The final vector input to the visiontransformer can be expressed as:

[0011]

[0012] Among them, represents the embedding input to the first Vision Transformer layer, Represents the class_token input to the first layer, Indicates the *th patch obtained by converting the image block. represents position embedding, and D is the dimension after the image is converted into patches;

[0013] When there are l Vision Transformerl layers in total, represents the features output by the penultimate Vision Transformer layer. We randomly shuffle the rest of the layers except class_token and divide them into k groups. The features after random shuffling can be expressed as After shuffling, the class_token is inserted at the beginning of each group. When k=3, the feature representation obtained after random shuffling and feature insertion is as follows:

[0014]

[0015] in It represents the input of the puzzle branch in the complete branch. In the puzzle branch, the features of k groups are input into three Vision Transformer layers with shared parameters to obtain the local features of k groups. The loss function of the puzzle branch is as follows:

[0016]

[0017] Among them, L tri is the triplet loss, L ID is the ID loss, The class_token represents the *th group of local features, where the triple loss function formula is as follows:

[0018]

[0019] In the above formula, B is the number of training mini-batch samples, x represents the input feature, Represents a positive sample with the same category as the input feature but different images, represents negative samples of different categories from the input features, α represents the set training interval, which is set to 0.2, and f(x) is the class_token of the sample image x;

[0020] The L ID The expression is as follows:

[0021]

[0022] In the above formula, x represents the input feature, y represents the true label value of the input feature, Z represents the number of pedestrian identities in the dataset, and the ε value is a constant set to 0.1;

[0023] Represents the output feature of the penultimate Vision Transformer layer, which is input to the last Vision Transformer layer and the output is use The class_token calculates the triple loss and ID loss to get the general loss of the complete branch, and the formula is as follows:

[0024]

[0025] In summary, the final loss function of the complete branch is as follows:

[0026] L complete =L u +L p

[0027] Furthermore, the occluded person re-identification network uses an erasing branch to assist in comparative learning. The erased branch structure is exactly the same as the full branch structure. The difference between the two is that the full branch directly trains the pictures in the data U. For the erased branch of the twin VisionTransformer backbone network, the randomly erased pictures come from the pictures used by the full branch. The picture is obtained by erasing a rectangular area with a random position and random aspect ratio. The erasing method is to fill each channel of all pixels in the randomly selected area with a random number that conforms to the normal distribution. Therefore, the loss function of the erased branch is as follows:

[0028] L re =L u ′+L′ p

[0029] where L′ p With L u ′ represents the puzzle branch loss and general loss of the erased branch, which is calculated in exactly the same way as the intact branch.

[0030] Furthermore, the occluded pedestrian re-identification network uses a cross-layer information aggregation module for auxiliary training, and the auxiliary training process is as follows:

[0031] Given the input feature sequence of layer l Its feature sequence after the self-attention operation of the lth transformer layer can be expressed as follows:

[0032]

[0033] in Represents the three linear layer parameters of the self-attention operation in the lth transformer layer, (*) T represents the transpose operation, Represents the self-attention weight in the lth transformer layer. After 12 transformer blocks, we will get 12 self-attention weight matrices The cross-layer attention weights obtained after aggregating different blocks can be expressed as:

[0034]

[0035] in E represents the identity matrix, Norm(*) represents row normalization, and Γ(*) means that row normalization is performed every two consecutive multiplications;

[0036] Given the output feature sequence of the penultimate Vision Transformer layer Except for the class_token part After concatenation, transposition and reshape, we get We use the convolution operation to convert p l-1 Converted to a compact target species representation, the convolution can be expressed as:

[0037]

[0038] Where * represents the convolution operation, is the species-aware mapping of category t of the target species, represents the convolution kernel, k t,d It represents the 3×3 convolution kernel indexed by the category index t and the feature map index d in the tensor.

[0039] The above-mentioned species-aware mapping is combined with the cross-layer attention weights to obtain cross-layer information guided by species-discriminative features. The combination is as follows:

[0040]

[0041] in represents the cross-layer position information guided by species-discriminative features, Indicates from A * Extract the attention weight of class_token for all patches and reshape it to N H ×N W ⊙ represents the Hadamard product, ψ represents the normalization of the calculation results to the species perception mapping M t The numerical range of ;

[0042] Since the twin Vision Transformer backbone network has two branches, the complete branch and the erased branch, the above process is applied to the two branches to obtain the cross-layer information of the complete branch and the erased branch guided by the species discrimination features. and The cross-layer information constraint formula guided by species discrimination features is obtained as follows:

[0043]

[0044] Where E m The erase mask matrix obtained for the erase branch, where the pixel value corresponding to the erased position is 0, and the rest are 1;

[0045] The total loss function of the occluded pedestrian re-identification network is expressed as:

[0046]

[0047] After all networks are trained until convergence, the test data is input into the complete branch of the complete target feature diffusion occluded pedestrian re-identification network. The feature representation of each image is extracted through the class_token output by the last Vision Transformer layer of this branch to obtain a feature matrix. According to the feature matrix, the cosine similarity matrix between the query image and the library image is calculated. For each query image, it is sorted in descending order according to its feature similarity, and the ten library images with the largest cosine similarity are selected as the query results. The library images with the same identity label as the query image are regarded as correct matching results. Then, the average accuracy of each query image is calculated, and the average accuracy of all query images is summarized to obtain the overall average accuracy on the data set. At the same time, it is also necessary to calculate the matching accuracy of the closest library images among all query images to finally obtain the first hit rate. The present invention provides an occluded pedestrian re-identification method based on the combination of species perception and cross-layer information, which has the following advantages:

[0048] (1) The cross-layer information aggregation module of the method effectively describes the attention position of the model by aggregating the attention position information of multiple layers. Through dual-branch contrast learning, it explicitly provides supervision signals to avoid the overfitting problem caused by excessive concentration of attention, thus reflecting the advantage of attention diffusion.

[0049] (2) The method uses a shared parameter method to construct a dual-branch twin structure network based on Vision Transformer, which effectively avoids the shortcut learning problem without significantly increasing parameters.

[0050] (3) The method introduces category-related semantic information into cross-layer information that is irrelevant to category information by taking species-discriminative features as a guide, so that the aggregated information can more accurately describe the model's focus area on the image, significantly enhancing the accuracy and effectiveness of the supervision signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 It is a flow chart of a method for re-identifying an occluded pedestrian based on the combination of species perception and cross-layer information provided by the present invention;

[0053] Figure 2 is a schematic diagram of the complete target feature diffusion occlusion pedestrian re-identification network structure, (1), (2), and (3) are the overall structure diagram of the present invention, the twin Vision Transformer single branch structure diagram, and the cross-layer information aggregation module structure diagram respectively;

[0054] Figure 3 It is a complete target feature diffusion occlusion pedestrian re-identification network attention diffusion effect diagram. The two parts of each group of pictures are the test data input original picture and the model attention distribution map obtained by the present invention; DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0056] Exemplary Methods

[0057] like Figure 1 The present invention provides a flow chart of a method for re-identifying an occluded pedestrian based on the combination of species perception and cross-layer information. The method steps are as follows:

[0058] Step S110: First, all the test set images in the data set are retained, and the training data U are converted to 384 pixels in height and 128 pixels in width.

[0059] Step S120: Use the training data U to perform forward propagation on the complete branch of the twin vision transformer backbone network and the puzzle branches it contains and calculate the loss function.

[0060] In this paper, we use a twin Vision Transformer backbone network with a single branch and 12 layers. We input an image X with a height of H, a width of W, and a dimension of C, and then use convolution to split the image X into N pieces of size P. H ×PW , that is, a 16×16 image block, P H With P W Represent the height and width of the image block respectively. When the horizontal step size and vertical step size of the convolution are S W With S H When , N can be expressed using the following formula:

[0061]

[0062] in In the present invention, S W With S H The same is 16, and N is 192. Next, all image patches are flattened to 1 dimension to become patches, and a learnable class_token is added before the patches. The resulting vector is added to the learnable position embedding to help the vision transformer model the spatial information of cls_token and different patches. The final vector input to the vision transformer can be expressed as:

[0063]

[0064] Among them, represents the embedding input to the first Vision Transformer layer, Represents the class_token input to the first layer, Indicates the *th patch obtained by converting the image block. represents position embedding, D is the dimension of the image after being converted into patches, which is 768 in this invention.

[0065] When there are l Vision Transformerl layers in total, represents the features output by the penultimate Vision Transformer layer. We randomly shuffle the rest of the layers except class_token and divide them into k groups, which are divided into 3 groups in this invention. The features after random shuffling can be expressed as After shuffling, the class_token is inserted at the beginning of each group. When k=3, the feature representation obtained after random shuffling and feature insertion is as follows:

[0066]

[0067] in It represents the input of the puzzle branch in the complete branch. In the puzzle branch, the features of k groups are input into three Vision Transformer layers with shared parameters to obtain the local features of k groups. The loss function of the puzzle branch is as follows:

[0068]

[0069] Among them, L tri is the triplet loss, L ID is the ID loss, The class_token represents the *th group of local features, where the triple loss function formula is as follows:

[0070]

[0071] In the above formula, B is the number of training mini-batch samples, x represents the input feature, Represents a positive sample with the same category as the input feature but different images, represents negative samples of different categories from the input features, α represents the set training interval, which is set to 0.2, and f(x) is the class_token of the sample image x;

[0072] The L ID The expression is as follows:

[0073]

[0074] In the above formula, x represents the input feature, y represents the true label value of the input feature, Z represents the number of pedestrian identities in the dataset, and the ε value is a constant set to 0.1;

[0075] Represents the output feature of the penultimate Vision Transformer layer, which is input to the last Vision Transformer layer and the output is use The class_token calculates the triple loss and ID loss to get the general loss of the complete branch, and the formula is as follows:

[0076]

[0077] In summary, the final loss function of the complete branch is as follows:

[0078] L complete =L u +L p

[0079] Step S130: Use the training data U to perform forward propagation in the erasing branch and the puzzle branch it contains in the twin vision transformer backbone network and calculate the loss function.

[0080] The erased branch structure is exactly the same as the complete branch structure. The difference between the two is that the complete branch directly trains the pictures in the data U. For the erased branch of the twin VisionTransformer backbone network, the randomly erased pictures come from the pictures used by the complete branch. The picture is erased by erasing a rectangular area with a random position and random aspect ratio. The erasing method is to fill each channel of all pixels in the randomly selected area with a random number that conforms to the normal distribution. After obtaining the erased picture, the picture is input into the erased branch with the same structure as the complete branch. Therefore, the loss function of the erased branch is as follows:

[0081] L re =L u ′+L′ p

[0082] where L′ p With L u ′ represents the puzzle branch loss and general loss of the erased branch, which is calculated in exactly the same way as the complete branch;

[0083] Step S140: Collect all attention weight matrices of the complete branch and the erased branch and the output feature sequence of the penultimate VisionTransformer layer, and use a cross-layer information aggregation module to assist in training.

[0084] Given the input feature sequence of layer l Its feature sequence after the self-attention operation of the lth transformer layer can be expressed as follows:

[0085]

[0086] in Represents the three linear layer parameters of the self-attention operation in the lth transformer layer, (*) T represents the transpose operation, Represents the self-attention weight in the lth transformer layer. After 12 transformer blocks, we will get 12 self-attention weight matrices The cross-layer attention weights obtained after aggregating different blocks can be expressed as:

[0087]

[0088] in E represents the identity matrix, Norm(*) represents row normalization, and Γ(*) means that row normalization is performed every two consecutive multiplications;

[0089] Given the output feature sequence of the penultimate Vision Transformer layer Except for the class_token part After concatenation, transposition and reshape, we get We use the convolution operation to convert p l-1 Converted to a compact target species representation, the convolution can be expressed as:

[0090]

[0091] Where * represents the convolution operation, is the species-aware mapping of category t of the target species, represents the convolution kernel, k t,d It represents the 3×3 convolution kernel indexed by the category index t and the feature map index d in the tensor;

[0092] The above-mentioned species-aware mapping is combined with the cross-layer attention weights to obtain cross-layer information guided by species-discriminative features. The combination is as follows:

[0093]

[0094] in represents the cross-layer information guided by species-discriminative features, Indicates from A * Extract the attention weight of class_token for all patches and reshape it to N H ×N W ⊙ represents the Hadamard product, ψ represents the normalization of the calculation results to the species perception mapping M t The numerical range of ;

[0095] Since the twin Vision Transformer backbone network has two branches, the complete branch and the erased branch, the above process is applied to the two branches to obtain the cross-layer position information of the complete branch and the erased branch guided by the species discrimination features. and The cross-layer information constraint formula guided by species discrimination features is obtained as follows:

[0096]

[0097] Where E m The erase mask matrix obtained for the erase branch, where the pixel value corresponding to the erased position is 0, and the rest are 1;

[0098] Step S150: Calculate the total loss function of the occluded pedestrian re-identification network, which can be expressed as:

[0099]

[0100] We used the stochastic gradient descent optimizer and set the momentum to 0.9 and the weight decay to 0.0001. The initial learning rate was set to 0.008, and as the training progressed, the learning rate gradually decreased according to the cosine decay strategy. A total of 90 epochs were trained. After the 50th epoch, if the current loss value was the lowest at the end of each epoch, the current parameters were saved.

[0101] Step S160: Test the training model parameters obtained in step S140 using the test data. Input the test data into the complete branch of the complete target feature diffusion occluded pedestrian re-identification network. Extract the feature representation of each image through the class_token output by the last Vision Transformer layer of the branch to obtain a feature matrix. Calculate the cosine similarity matrix between the query image and the library image according to the feature matrix. For each query image, sort it in descending order according to its feature similarity, and select the ten library images with the largest cosine similarity as the query result. The library images with the same identity label as the query image are regarded as correct matching results. Then, calculate the average accuracy of each query image, and summarize the average accuracy of all query images to obtain the overall average accuracy on the data set. At the same time, it is also necessary to calculate the matching accuracy of the closest library images among all query images to finally obtain the first hit rate. According to the calculation method, calculate the pedestrian re-identification accuracy of the complete target feature diffusion occluded pedestrian re-identification network on the occluded pedestrian re-identification data sets Occluded-DukeMTMC and Occluded-REID.

[0102] Through this implementation, the dataset U is used to perform forward propagation in the complete branch, the erasure branch, and the cross-layer information aggregation module, and back propagation is performed after calculating the total loss function. After the model is trained to convergence, the test image is input into the complete branch of the complete target feature diffusion occlusion pedestrian re-identification network, and the feature representation of each image is extracted through the class_token output by the last Vision Transformer layer of the branch to obtain the feature matrix. The similarity matrix between the query image and the library image is calculated through the feature matrix and the classification result predicted by the network is obtained.

[0103] To further illustrate, assuming that an occluded pedestrian re-identification dataset is classified according to this implementation, a classification result with higher accuracy than most methods will be obtained.

[0104] Detailed description of the invention Results

[0105] This implementation uses the publicly available occluded person re-identification datasets Occluded-DukeMTMC and Occluded-REID. The details of the datasets are as follows:

[0106] The Occluded-DukeMTMC dataset is derived from the DukeMTMC-reID dataset. This dataset contains 15,618 training images covering 702 identities; 2,210 query images covering 519 identities and 17,661 library images covering 1,110 identities. 9%, 100% and 10% of the images in the training set, query set and library set of this dataset are affected by occlusion to varying degrees, respectively. This feature results in at least one image being occluded when calculating the pairwise distance between the query image and the library image. At the same time, the Occluded-DukeMTMC dataset also contains a variety of scenes with different shooting angles and occluded objects. The above factors make it one of the most challenging datasets in the field of occluded pedestrian re-identification.

[0107] The Occluded-REID dataset is an occluded person ReID dataset captured by a mobile camera. It contains 2000 images covering 200 identities. Each identity has 5 full-body images and 5 occluded pedestrian images with different perspectives and different types of severe occlusion. 150 identities in this dataset are used as training sets, and the occluded pedestrian images of the other 50 identities are used as query images, and the corresponding 50 full-body images of the identities are used as library images.

[0108] In order to verify the superiority of this implementation (Ours), this implementation is compared with several existing occluded pedestrian re-identification methods, including PGFA, FED, ViT base and other methods. The mean average accuracy (mAP) and the first hit rate (Rank-1) of these methods on the above public datasets will be compared. The specific data comparison is shown in Table 1.

[0109] Table 1 Accuracy of occlusion dataset (%)

[0110]

[0111] By comparing the data in the above table, we can clearly see that Ours achieves the best performance and significantly improves the accuracy of occluded pedestrian re-identification. The quantitative results fully demonstrate the superiority of Ours, because Ours can better overcome the network's shortcut learning and overfitting problems. Ours uses a combination of cross-layer information aggregation and category-aware mapping to explicitly provide supervision signals to limit the network's over-concentration of attention, allowing the network's attention to spread more widely to the overall structure of the target pedestrian. A large number of experiments show that this method is superior to existing methods.

[0112] This embodiment proposes an occluded pedestrian re-identification method based on the combination of species perception and cross-layer information, which is used to perform pedestrian re-identification tasks on occluded images.

[0113] By training an occluded pedestrian re-identification network consisting of two 12-layer twin Vision Transformer backbone networks with shared parameters, their respective puzzle branches and their respective cross-layer information aggregation modules, pedestrian re-identification is performed on occluded pedestrian images. The occluded pedestrian re-identification network uses a contrastive learning method combined with cross-layer information guided by species discriminant features, which effectively solves the problem of the network falling into shortcut learning, increases the effective attention range of the model, and makes the learned features more discriminative. The experimental results on the public datasets Occluded-DukeMTMC and Occluded-REID show that this implementation has higher accuracy and better superiority than other methods.

[0114] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A method for re-identifying occluded pedestrians based on combining species perception with cross-layer information, characterized in that: The method comprises the steps of: The training data U and test data are collected from the surveillance camera device. The training data U includes the pedestrian re-identification images with occlusion. The test data is divided into query images and library images. They only contain the original pedestrian re-identification images with occlusion and pedestrian numbers. Construct a complete target feature diffusion occluded pedestrian re-identification network, the occluded pedestrian re-identification network consists of two 12-layer twin Vision Transformer backbone networks with shared parameters, their respective puzzle branches and their respective cross-layer information aggregation modules, wherein the two twin Vision Transformer backbone networks with shared parameters are a complete branch and an erased branch, the complete branch and the erased branch extract global features in the pedestrian image and perform loss calculation and back propagation, the puzzle branches of the complete branch and the erased branch reorganize the features and perform local feature propagation, the puzzle branch outputs multiple local feature calculations for classification and calculates the loss to assist the complete branch and the erased branch in training, the cross-layer information aggregation module distributes the model's attention more widely by aggregating information from multiple Vision Transformer layers and comparing between the complete branch and the erased branch, and the occluded pedestrian re-identification network is trained to convergence using training data U; In the prediction process of the occluded pedestrian re-identification network, the test data is input into the complete branch of the complete target feature diffusion occluded pedestrian re-identification network, and the feature representation of each image is extracted through the class_token output by the last Vision Transformer layer of the branch to obtain a feature matrix, and the cosine similarity matrix between the query image and the library image is calculated according to the feature matrix. For each query image, it is sorted in descending order according to its feature similarity, and the ten library images with the largest cosine similarity are selected as query results. The library images with the same identity label as the query image are regarded as correct matching results. Then, the average accuracy of each query image is calculated, and the average accuracy of all query images is summarized to obtain the overall average accuracy on the data set; The training process of the complete branch and the jigsaw branch contained in the twin vision transformer backbone network is as follows: The occluded pedestrian re-identification network uses the twin Vision Transformer backbone network for feature extraction. The twin Vision Transformer backbone network means that two Vision Transformers share model parameters but process different data. The twin Vision Transformer backbone network inputs an image X with a height of H, a width of W, and a dimension of C, and then uses convolution to split the image X into N pieces of size P. H ×P W The image block, P H With P W Represent the height and width of the image block respectively. When the horizontal step size and vertical step size of the convolution are S W With S H When , N can be expressed using the following formula: in Indicates rounding down. In the next step, all image patches will be flattened to 1D to become patches, and a learnable class_token will be added before the patches. The resulting vector will be added to the learnable position embedding to help the visiontransformer model the spatial information of cls_token and different patches. The final vector input to the visiontransformer can be expressed as: Among them, represents the embedding input to the first Vision Transformer layer, Represents the class_token input to the first layer, Indicates the *th patch obtained by converting the image block. represents position embedding, and D is the dimension after the image is converted into patches; When there are l Vision Transformerl layers in total, represents the features output by the penultimate Vision Transformer layer. We randomly shuffle the rest of the layers except class_token and divide them into k groups. The features after random shuffling can be expressed as After shuffling, the class_token is inserted at the beginning of each group. When k=2, the feature representation obtained after random shuffling and feature insertion is as follows: in It represents the input of the puzzle branch in the complete branch. In the puzzle branch, the features of k groups are input into three Vision Transformer layers with shared parameters to obtain the local features of k groups. The loss function of the puzzle branch is as follows: Among them, L tri is the triplet loss, L ID is the ID loss, The class_token represents the *th group of local features, where the triple loss function formula is as follows: In the above formula, B is the number of training mini-batch samples, x represents the input feature, Represents a positive sample with the same category as the input feature but different images, represents negative samples of different categories from the input features, α represents the set training interval, which is set to 0.2, and f(x) is the class_token of the sample image x; The L ID The expression is as follows: In the above formula, x represents the input feature, y represents the true label value of the input feature, Z represents the number of pedestrian identities in the dataset, and the ε value is a constant set to 0.1; Represents the output feature of the penultimate Vision Transformer layer, which is input to the last Vision Transformer layer and the output is use The class_token calculates the triple loss and ID loss to get the general loss of the complete branch, and the formula is as follows: In summary, the final loss function of the complete branch is as follows: L complete =L u +L p 2. The method for re-identifying occluded pedestrians based on combining species perception and cross-layer information according to claim 1, characterized in that: The training process of the erase branch is as follows: The structure of the erasing branch is exactly the same as that of the full branch. The difference between the two is that the full branch directly trains the pictures in the data U. For the erasing branch of the twin VisionTransformer backbone network, the randomly erased pictures come from the pictures used by the full branch. The picture is obtained by erasing a rectangular area with a random position and random aspect ratio. The erasing method is to fill each channel of all pixels in the randomly selected area with a random number that conforms to the normal distribution. Therefore, the loss function of the erasing branch is as follows: L re =L′ u +L′ p where L′ p and L′ u The puzzle branch loss and general loss representing the erased branch are calculated exactly the same way as the full branch.

3. The method for re-identifying pedestrians under occlusion based on combining species perception and cross-layer information according to claim 1 is characterized in that The cross-layer information aggregation module is used to assist training. The auxiliary training process is as follows: Given the input feature sequence of layer l Its feature sequence after the self-attention operation of the lth transformer layer can be expressed as follows: in represents the three linear layer parameters of the self-attention operation in the lth transformer layer, represents the transpose operation, Represents the self-attention weight in the lth transformer layer. After 12 transformer blocks, we will get 12 self-attention weight matrices The cross-layer attention weights obtained after aggregating different blocks can be expressed as: in E represents the identity matrix, Norm(*) represents row normalization, and Γ(*) means that row normalization is performed every two consecutive multiplications; Given the output feature sequence of the penultimate Vision Transformer layer Except for the class_token part After concatenation, transposition and reshape, we get We use the convolution operation to convert p l-1 Converted to a compact target species representation, the convolution can be expressed as: Where * represents the convolution operation, is the species-aware mapping of category t of the target species, represents the convolution kernel, k t,d It represents the 3×3 convolution kernel indexed by the category index t and the feature map index d in the tensor; The above-mentioned species-aware mapping is combined with the cross-layer attention weights to obtain cross-layer information guided by species-discriminative features. The combination is as follows: in represents the cross-layer information guided by species-discriminative features, Indicates from A * Extract the attention weight of class_token for all patches and reshape it to N H ×N W ⊙ represents the Hadamard product, ψ represents the normalization of the calculation results to the species perception mapping M t The numerical range of ; Since the twin Vision Transformer backbone network has two branches, the complete branch and the erased branch, the above process is applied to the two branches to obtain the cross-layer information of the complete branch and the erased branch guided by the species discrimination features. and The cross-layer information constraint formula guided by species discrimination features is obtained as follows: Where E m The erase mask matrix obtained by the erase branch, where the pixel value corresponding to the erased position is 0, and the rest are 1; The total loss function of the occluded pedestrian re-identification network is expressed as:

4. The method for re-identifying occluded pedestrians based on combining species perception and cross-layer information according to claim 1, characterized in that: The prediction process of the complete target feature diffusion occlusion pedestrian re-identification network is as follows: After all networks are trained to convergence, the test data is input into the complete branch of the complete target feature diffusion occlusion pedestrian re-identification network. The feature representation of each image is extracted through the class_token output by the last Vision Transformer layer of this branch to obtain the feature matrix. The cosine similarity matrix between the query image and the library images is calculated based on the feature matrix. For each query image, it is sorted in descending order according to its feature similarity, and the ten library images with the largest cosine similarity are selected as the query results. The library images with the same identity label as the query image are considered to be correct matches. Next, the average accuracy of each query image is calculated, and the average accuracy of all query images is summarized to obtain the overall average accuracy on the data set. At the same time, the matching accuracy of the closest library images among all query images must also be calculated to finally obtain the first hit rate.

Images