A method and system for occluded person re-identification based on multi-branch joint learning

Abstract

This invention relates to a method and system for re-identifying occluded pedestrians based on multi-branch joint learning, belonging to the field of image recognition. The method includes: inputting a pedestrian image to be identified into a pedestrian re-identification model and outputting the pedestrian re-identification result; the pedestrian re-identification model is obtained by jointly training a teacher network and a student network based on a training set; both the teacher network and the student network are multi-branch networks, including a shared layer, a global branch, a local branch, and an output layer; the global branch performs global average pooling on the feature map; the local branch fuses multiple feature vectors of different scales; the output layer fuses the outputs of the global branch and the local branch to obtain a feature fusion vector; and the pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vectors of pedestrian images in the query database is used as the output of the pedestrian re-identification model. This invention improves the accuracy of occluded pedestrian re-identification.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method and system for re-identifying occluded pedestrians based on multi-branch joint learning. Background Technology

[0002] With the rapid development of artificial intelligence and the advancement of deep learning technology, deep learning-based pedestrian re-identification methods have received widespread attention. However, considering that pedestrian images captured by cameras in real-world scenarios often have a lot of occlusion, although many works have achieved good results in the field of pedestrian re-identification, they still fail to achieve satisfactory results when faced with a large number of occluded scenes.

[0003] Thanks to the rapid development of convolutional neural networks (CNNs), research on person re-identification based on CNNs has garnered widespread attention and achieved considerable results. The occluded person re-identification task aims to ensure that the model can still perform efficient and accurate person retrieval and matching even under large-scale occlusion. Compared to general person re-identification tasks, the occluded person re-identification task places higher demands on deep learning models.

[0004] While existing work has achieved some success in occluded pedestrian re-identification tasks, many problems remain. First, many occluded pedestrian re-identification methods rely entirely on local features. Although local features can effectively avoid interference from some occlusion information, this approach ignores global semantic information. Similarly, some occluded pedestrian re-identification methods based on global features struggle to focus on key local regions and are more susceptible to interference from occlusion factors. For occluded pedestrian re-identification tasks, both global and local features are important and complementary. Focusing on only one or ignoring the relationship between the two makes it difficult to obtain comprehensive and discriminative pedestrian features for pedestrian matching and retrieval. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for re-identifying occluded pedestrians based on multi-branch joint learning, which improves the accuracy of re-identifying occluded pedestrians.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] An occluded person re-identification method based on multi-branch joint learning includes:

[0008] Acquire images of the pedestrians to be identified;

[0009] The pedestrian image to be identified is input into the pedestrian re-identification model, and the pedestrian re-identification result is output; the pedestrian re-identification model is obtained by jointly learning and training the teacher network and the student network based on the training set;

[0010] Both the teacher network and the student network are multi-branch networks, each comprising a shared layer, a global branch, local branches, and an output layer. The shared layer extracts features from the pedestrian image to be identified and outputs a feature map. The global branch performs global average pooling after feature extraction from the feature map. The local branch performs region pooling after feature extraction from the feature map and fuses multiple feature vectors of different scales. The output layer fuses the outputs of the global and local branches to obtain a feature fusion vector. The pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vectors of pedestrian images in the query database is used as the output of the pedestrian re-identification model.

[0011] The sample images in the query database are occluded pedestrian images;

[0012] The training set stores pedestrian images with multiple pedestrian identities.

[0013] Optionally, the training process for joint learning training of teacher networks and student networks includes:

[0014] The feature vectors output by the local branches of the teacher network are used as the supervision information of the global branches of the student network, and the feature vectors output by the global branches of the teacher network are used as the supervision information of the local branches of the student network. The network parameters of the student network during multiple training rounds are averaged and used as the network parameters of the teacher network. The teacher network after training a set number of times is used as the pedestrian re-identification model.

[0015] Optionally, the overall loss function for joint learning training of the teacher network and the student network is:

[0016] L all =λ1L center +λ2L cam +λ3L ce +λ4L kl +λ5L tri ;

[0017] Among them, L all L represents the overall loss. center L represents the identity-level camera-level contrast loss. cam L represents camera-level contrast loss. ce L represents the cross-entropy loss. klL represents the KL divergence loss. tri This represents the soft triplet loss, where λ1, λ2, λ3, λ4, and λ5 are all weighting coefficients.

[0018]

[0019] Where q represents the sample feature q, k i k represents the class center feature of the i-th pedestrian identity in the query database. + The class center feature represents the pedestrian identity corresponding to the sample feature q, τ is the temperature coefficient of the first contrastive learning, and K represents the number of pedestrian identities in the training set;

[0020]

[0021] Among them, f ab The sample feature f represents the pedestrian's identity as a and the camera that took the picture as b. ab p ai Representing sample features f ab The original form, p j Representing sample features f ab The negative prototype, τ c For the second comparative learning temperature coefficient, N represents the number of camera-level comparison prototypes. neg To represent the number of negative samples in contrastive learning, P denotes the set of contrastive prototypes at the camera level, and C denotes the set of class center features;

[0022] L ce =CE(R) sl ,R tg )+CE(R sg ,R tl );

[0023] Where CE() represents the cross-entropy loss function, R sg R represents the similarity relationship between the feature vectors output by the global branch in the student network and the global class centers. tg R represents the similarity relationship between the feature vectors output by the global branch in the teacher network and the global class centers. sg R represents the similarity relationship between the feature vectors output by the local branches in the student network and the local class centers. tl This represents the similarity relationship between the feature vectors output by the local branches in the teacher network and the local class centers;

[0024] L kl =KL(W sl W tg )+KL(W sg W tl );

[0025] Where KL() represents the KL divergence loss function, W sl W represents the similarity between the feature vectors output by the local branches in the student network. tg W represents the similarity between the feature vectors output by the global branches in the teacher network. sg W represents the similarity between the feature vectors output by the global branches in the student network. tl This represents the similarity between the feature vectors output by the local branches in the teacher network;

[0026]

[0027] in, This represents the loss that uses the feature vectors output by the global branch of the teacher network to supervise the outputs of the local branches of the student network. This represents the loss that uses the feature vectors output by the local branches of the teacher network to supervise the output of the global branches of the student network;

[0028]

[0029] Among them, T tg T represents the positive and negative sample pair relation of triples obtained from the global branch in the teacher network. sl Let θ' represent the positive and negative sample pairs of triples obtained from the local branches in the student network, θ' represent the network parameters of the teacher network, and θ represent the network parameters of the student network. sl x represents the output of the student network. tg This represents the output of the teacher network.

[0030] Optionally, λ1 = 1, λ2 = 0.5, λ3 = 1, λ4 = 1, and λ5 = 10.

[0031] Optionally, the process of constructing the training set includes:

[0032] Obtain the Occluded-Duke dataset;

[0033] Each pedestrian image in the Occluded-Duke dataset is resized to a set size to obtain the resized dataset;

[0034] Each pedestrian image in the resized dataset is augmented using random flipping and random erasing, and the augmented dataset is used as the training set.

[0035] Optionally, the set size is 256×128.

[0036] This invention discloses an occluded pedestrian re-identification system based on multi-branch joint learning, comprising:

[0037] The pedestrian image acquisition module is used to acquire images of pedestrians to be identified.

[0038] The pedestrian re-identification module is used to input the pedestrian image to be identified into the pedestrian re-identification model and output the pedestrian re-identification result; the pedestrian re-identification model is obtained by jointly learning and training the teacher network and the student network based on the training set;

[0039] Both the teacher network and the student network are multi-branch networks, each comprising a shared layer, a global branch, local branches, and an output layer. The shared layer extracts features from the pedestrian image to be identified and outputs a feature map. The global branch performs global average pooling after feature extraction from the feature map. The local branch performs region pooling after feature extraction from the feature map and fuses multiple feature vectors of different scales. The output layer fuses the outputs of the global and local branches to obtain a feature fusion vector. The pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vectors of pedestrian images in the query database is used as the output of the pedestrian re-identification model.

[0040] The sample images in the query database are occluded pedestrian images;

[0041] The training set stores pedestrian images with multiple pedestrian identities.

[0042] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0043] This invention performs joint learning on teacher and student networks with multi-branch network structures based on the training set, taking into account both local and global semantic information, and improves the accuracy of pedestrian re-identification in scenarios with a large amount of occlusion. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the process of a pedestrian re-identification method based on multi-branch joint learning according to the present invention;

[0046] Figure 2This is a schematic diagram of the multi-branch network structure of the present invention;

[0047] Figure 3 This is a schematic diagram illustrating the joint learning between the teacher network and the student network of the present invention;

[0048] Figure 4 This is a schematic diagram of the structure of an occluded pedestrian re-identification system based on multi-branch joint learning according to the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] The purpose of this invention is to provide a method and system for re-identifying occluded pedestrians based on multi-branch joint learning, which improves the accuracy of re-identifying occluded pedestrians.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example 1

[0053] Figure 1 This is a schematic diagram of the process of a pedestrian re-identification method based on multi-branch joint learning according to the present invention, as shown below. Figure 1 As shown, an occluded pedestrian re-identification method based on multi-branch joint learning includes the following steps.

[0054] Step 101: Obtain the image of the pedestrian to be identified.

[0055] Step 102: Input the pedestrian image to be identified into the pedestrian re-identification model and output the pedestrian re-identification result; the pedestrian re-identification model is obtained by jointly learning and training the teacher network and the student network based on the training set.

[0056] To comprehensively consider both global and local features to adapt to the task of pedestrian re-identification under occlusion, this invention discloses, as follows: Figure 2 The multi-branch network shown.

[0057] Both the teacher network and the student network are multi-branch networks, which include a shared layer, global branches, local branches, and an output layer.

[0058] In a multi-branch network, the global branch, in relation to the network as a whole, considers the overall number of parameters. The global branch and local branches share the parameters of some layers of the convolutional neural network, and each branch has its own unique parameters. The feature map obtained from the shared layer by the global branch is used to obtain the final output feature map. Considering global semantic information, this invention uses global average pooling to obtain the final feature vector.

[0059] The local branch takes into account the scale differences of local key semantics in different samples, and then adopts a region pooling method to obtain feature vectors of multiple regions of different scales. Then, it uses 1×1 convolution to fuse these feature vectors to obtain the output of the local branch.

[0060] The first three layers of ResNet-50 are used as shared layers, and the fourth layer of ResNet is used to implement feature extraction functions in two independent branches, global and local.

[0061] The shared layer is used to extract features from the pedestrian image to be identified and output a feature map. The global branch and the local branch are both connected to the output of the shared layer, and the output layer is connected to the outputs of the global branch and the local branch respectively. The global branch is used to perform global average pooling after feature extraction from the feature map. The local branch is used to perform region pooling after feature extraction from the feature map and to fuse multiple feature vectors of different scales. The output layer fuses the outputs of the global branch and the local branch to obtain a feature fusion vector. The pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vector of pedestrian images in the query database is used as the output of the pedestrian re-identification model.

[0062] The sample images in the query database are pedestrian images with occlusions.

[0063] The training set stores pedestrian images with multiple pedestrian identities.

[0064] The training process for joint learning training of teacher networks and student networks includes:

[0065] The feature vectors output by the local branches of the teacher network are used as the supervision information of the global branches of the student network, and the feature vectors output by the global branches of the teacher network are used as the supervision information of the local branches of the student network. The network parameters of the student network during multiple training rounds are averaged and used as the network parameters of the teacher network. The teacher network after training a set number of times is used as the pedestrian re-identification model.

[0066] The overall loss function for joint learning training of the teacher network and the student network is:

[0067] L all =λ1L center +λ2L cam +λ3L ce +λ4L kl +λ5L tri ;

[0068] Among them, L all L represents the overall loss. center L represents the identity-level camera-level contrast loss. cam L represents camera-level contrast loss. ce L represents the cross-entropy loss. kl L represents the KL divergence loss. tri This represents the soft triplet loss, where λ1, λ2, λ3, λ4, and λ5 are all weighting coefficients.

[0069] KL divergence represents the degree of proximity between two probability distributions, and the KL divergence loss function is a loss function based on KL divergence.

[0070] Considering that pedestrian re-identification is an image retrieval problem, the key is to learn discriminative pedestrian representations. This invention employs a contrastive learning method to guide global and local branches in representation learning. First, for all image features with the same identity, their class centers (feature averages) are used as positive and negative samples for contrastive learning. That is, class centers with the same identity label are positive samples for contrastive learning, and vice versa. The identity-level and camera-level contrastive loss is as follows:

[0071]

[0072] Where q represents the sample feature, k i k represents the class center feature of the i-th pedestrian identity in the query database. + Let τ represent the class center feature corresponding to the pedestrian identity of the sample feature q, τ be the temperature coefficient of the first contrastive learning, and K represent the number of pedestrian identities in the training set. A class center identity-level camera-level contrastive loss is used to allow sample features to move closer to their own class center and further away from the centers of other classes.

[0073] Considering the difficulties that different cameras used to capture pedestrians bring to the pedestrian re-identification task, pedestrian images under each category are divided into several groups based on the camera used. The average feature value of each group is used to represent all samples in that group and is regarded as the prototype for contrastive learning. Furthermore, considering that the training data for some pedestrians may have been captured by the same camera, there may be overlap between the class center prototype and the camera prototype. In this case, the camera prototype should not participate in contrastive learning. Based on the prototype obtained from the camera labels, the following camera-level contrastive loss function is derived.

[0074]

[0075] Among them, f ab The sample feature f represents the pedestrian's identity as a and the camera that took the picture as b. ab p ai Representing sample features f ab The original form, p j Representing sample features f ab The negative prototype, τ c For the second comparative learning temperature coefficient, N represents the number of camera-level comparison prototypes. neg P represents the number of negative samples in contrastive learning, where P represents the set of contrastive prototypes at the camera level and C represents the set of class center features. By using a camera prototype-based contrastive learning method, the model can better adapt to changes in camera style and obtain more effective and discriminative features.

[0076] The average of the feature vectors of all pedestrian images with the same identity is the class center, which is the identity level comparison prototype. A class center (identity level comparison prototype) can be obtained for each identity in the training set, and these class centers form a set C.

[0077] Based on the different cameras used to capture the images, all images with the same identity can be divided into several parts. All images within each part share the same identity and were captured by the same camera. The average feature vector of all images in each part is the camera-level contrast prototype. Similarly, for all pedestrians with different identities in the training set, several camera-level contrast prototypes can be obtained, and their set is P.

[0078] Global and local branches acquire feature representations of the same pedestrian image in different ways, so the connection between them is significant. This invention proposes a cross-branch joint learning method based on a multi-branch structure to improve the quality of global and local features, such as... Figure 3 As shown. Firstly, this invention introduces a teacher-student network. The teacher network does not participate in backpropagation; its parameters are obtained by time-series averaging of the student network parameters. Therefore, the introduction of the teacher network does not increase computational cost. However, because the teacher network integrates the student network data from each training cycle, it often achieves better performance than the student network. The teacher network is updated as follows:

[0079]

[0080] in, This is the teacher model under the current training cycle. This represents the teacher model from the previous training cycle. Let α represent the student model in the current training cycle, and α represent the updated weights.

[0081] Considering that the teacher network can provide more supervisory information to the student network, and that there is a certain correlation and complementarity between local and global features, this invention adopts a cross-branch joint strategy between the teacher and student networks. Specifically, it utilizes the local features of the teacher network to provide supervisory information for the global branch of the student network, and similarly, it utilizes the global features of the teacher network to provide supervisory information for the local branch of the student network. Firstly, considering that the distance from a sample feature to each class center can be regarded as the probability that the sample belongs to a certain class, which can be considered a soft label for the sample, the cross-entropy loss function can be used:

[0082] L ce =CE(R) sl ,R tg )+CE(R sg ,R tl );

[0083] Where CE() represents the cross-entropy loss function, R sg R represents the similarity relationship between the feature vectors output by the global branch in the student network and the global class centers. tg R represents the similarity relationship between the feature vectors output by the global branch in the teacher network and the global class centers. sg R represents the similarity relationship between the feature vectors output by the local branches in the student network and the local class centers. tl This represents the similarity relationship between the feature vectors output by the local branches in the teacher network and the local class centers.

[0084] The global class center is the class center feature of all pedestrians with different identities in the training set.

[0085] The crossover loss function allows the student network to learn the similarity between samples and class centers in the teacher network. Considering the relationships between sample instances, the loss function used is as follows:

[0086] L kl =KL(W sl W tg )+KL(W sg W tl );

[0087] Where KL() represents the KL divergence loss function, W sl W represents the similarity between the feature vectors output by the local branches in the student network. tgW represents the similarity between the feature vectors output by the global branches in the teacher network. sg W represents the similarity between the feature vectors output by the global branches in the student network. tl This represents the similarity between the feature vectors output by the local branches in the teacher network.

[0088] By utilizing similarity distillation between instances, the student network can learn the similarity relationships between sample features in the teacher network, which is crucial for the person re-identification retrieval task. Finally, considering the distance relationship between positive and negative samples, this invention employs a soft triplet loss method to transfer knowledge between the teacher and student networks. The loss function used is as follows:

[0089]

[0090] Among them, T tg T represents the positive and negative sample pair relation of triples obtained from the global branch in the teacher network. sl Let θ' represent the positive and negative sample pairs of triples obtained from the local branches in the student network, θ' represent the network parameters of the teacher network, and θ represent the network parameters of the student network. sl x represents the output of the student network. tg This represents the output of the teacher network.

[0091] Considering the pairwise interactions between multiple branches of the teacher network and the student network, the loss is obtained:

[0092]

[0093] in, This represents the loss that uses the feature vectors output by the global branch of the teacher network to supervise the outputs of the local branches of the student network. This represents the loss that uses the feature vectors output by the local branches of the teacher network to supervise the output of the global branches of the student network.

[0094]

[0095] Among them, T tl T represents the positive and negative sample pair relation of triples obtained from the local branches in the teacher network. sg x represents the positive and negative sample pair relation of triples obtained from the global branch in the student network. t x represents the output of the teacher network. s This represents the output of the student network.

[0096] This invention employs a contrastive learning approach for representation learning on both the global and local branches of the student network. Furthermore, considering the connections and differences between global and local features, a multi-branch joint learning module is added between the teacher and student networks. The final overall loss function is obtained by combining these two parts.

[0097] The process of constructing the training set includes:

[0098] Obtain the Occluded-Duke dataset;

[0099] Each pedestrian image in the Occluded-Duke dataset is resized to a set size to obtain the resized dataset;

[0100] Each pedestrian image in the resized dataset is augmented using random flipping and random erasing, and the augmented dataset is used as the training set.

[0101] The specified dimensions are 256×128.

[0102] Example 2

[0103] Considering that this invention is a method for occluded pedestrian re-identification based on multi-branch joint learning, it uses the challenging occluded pedestrian re-identification dataset Occluded-Duke for training and testing. The Occluded-Duke dataset contains 35,489 pedestrian images sampled from 6 cameras, including 1,812 pedestrians with different identities, many of which contain occlusion information.

[0104] During training, the weights of the loss function are λ1 = 1, λ2 = 0.5, λ3 = 1, λ4 = 1, and λ5 = 10. The temporal average weight α of the teacher network is set to 0.999. In contrast, the temperature coefficients τ and τ0 are used during learning. c The values ​​were set to 0.5 and 0.07 respectively, and the number of negative samples N... neg Set it to 150.

[0105] The optimizer used is Adam, with a learning rate of 0.00035. A WarmUp learning rate increment method was employed, gradually increasing the learning rate from 0.000035 to 0.00035 over the first 10 training epochs. For the input data, each pedestrian image was set to 256×128 pixels. Data augmentation techniques such as random flipping and random erasure were used during training. This invention uses ResNet-50 as the encoder and employs weights pre-trained on ImageNet. The last fully connected layer in ResNet-50 was removed, and a new Batch Normalization (BN) layer was added. In the multi-branch structure, the global branch and local branches share the first three layers of ResNet-50.

[0106] During the testing process, the teacher network was only used in the final test. The image size during the test was also adjusted to 256×128, and no data augmentation was used. The output global feature vector and local feature vector were processed by L2 regularization and then fused and stitched together. The cosine similarity of the fused features between the query sample and all the samples to be retrieved was calculated and sorted to obtain the final retrieval results.

[0107] Table 1 compares the experimental results of this invention with existing methods on the public dataset Occluded-Duke. The experimental results demonstrate the superiority of this invention over other existing methods in the task of occluded person re-identification. The evaluation metrics are as follows:

[0108] (1) Mean Average Precision (mAp): mAP is the average value of the average precision (AP) of each query image. mAp reflects the ranking of all correctly queried images in the query database.

[0109] (2) Rank-k: Rank-k represents the proportion of the first k data in the query ranking table that contain the correct image. Rank-1 is an important evaluation metric in ReID, representing the first-rank hit rate.

[0110] Table 1 shows the performance of each method on the Octcluded-Duke dataset.

[0111]

[0112]

[0113] Table 1 lists existing methods for person re-identification, including: Local Person Re-identification Based on Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-Free Approach (DSR); Identifying visible parts via pose estimation for occluded person re-identification (PMFB); Multihead self-attention network for occluded person re-identification (MHSA-Net); Pose-guided feature alignment for occluded person re-identification (PGFA); Beyond Part Models: PersonRetrieval with Refined Part Pooling (PCB); and High-order information matters: Learning relation and topology for occluded person re-identification. As shown in Table 1, the occluded person re-identification method based on multi-branch joint learning proposed in this invention has good performance.

[0114] Example 3

[0115] Figure 4 This is a schematic diagram of the occluded pedestrian re-identification system based on multi-branch joint learning according to the present invention, as shown below. Figure 4 As shown, an occluded pedestrian re-identification system based on multi-branch joint learning includes:

[0116] The pedestrian image acquisition module 201 is used to acquire pedestrian images to be identified.

[0117] The pedestrian re-identification module 202 is used to input the pedestrian image to be identified into the pedestrian re-identification model and output the pedestrian re-identification result; the pedestrian re-identification model is obtained by jointly learning and training the teacher network and the student network based on the training set.

[0118] Both the teacher network and the student network are multi-branch networks, each comprising a shared layer, a global branch, local branches, and an output layer. The shared layer extracts features from the pedestrian image to be identified and outputs a feature map. The global branch and the local branch are both connected to the output of the shared layer, and the output layer is connected to the outputs of the global branch and the local branch, respectively. The global branch performs global average pooling after feature extraction from the feature map. The local branch performs region pooling after feature extraction from the feature map and fuses multiple feature vectors of different scales. The output layer fuses the outputs of the global branch and the local branch to obtain a feature fusion vector. The pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vector of pedestrian images in the query database is used as the output of the pedestrian re-identification model.

[0119] The sample images in the query database are pedestrian images with occlusions.

[0120] The training set stores pedestrian images with multiple pedestrian identities.

[0121] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0122] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for re-identifying occluded pedestrians based on multi-branch joint learning, characterized in that, include: Acquire images of the pedestrians to be identified; The pedestrian image to be identified is input into the pedestrian re-identification model, and the pedestrian re-identification result is output. The pedestrian re-identification model is obtained by jointly learning and training the teacher network and the student network based on the training set. Both the teacher network and the student network are multi-branch networks, each comprising a shared layer, a global branch, local branches, and an output layer. The shared layer extracts features from the pedestrian image to be identified and outputs a feature map. The global branch performs global average pooling after feature extraction from the feature map. The local branch performs region pooling after feature extraction from the feature map and fuses multiple feature vectors of different scales. The output layer fuses the outputs of the global and local branches to obtain a feature fusion vector. The pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vectors of pedestrian images in the query database is used as the output of the pedestrian re-identification model. The sample images in the query database are occluded pedestrian images; The training set stores pedestrian images with multiple pedestrian identities; The training process for joint learning training of teacher networks and student networks includes: The feature vectors output by the local branches of the teacher network are used as the supervision information of the global branches of the student network, and the feature vectors output by the global branches of the teacher network are used as the supervision information of the local branches of the student network. The network parameters of the student network during multiple training rounds are averaged and used as the network parameters of the teacher network. The teacher network after training a set number of times is used as the pedestrian re-identification model.

2. The occluded person re-identification method based on multi-branch joint learning according to claim 1, characterized in that, The overall loss function for joint learning training of the teacher network and the student network is: L all =λ1L center +λ2L cam +λ3L ce +λ4L kl +λ5L tri ; Among them, L all L represents the overall loss. center L represents the identity-level camera-level contrast loss. cam L represents camera-level contrast loss. ce L represents the cross-entropy loss. kl L represents the KL divergence loss. tri This represents the soft triplet loss, where λ1, λ2, λ3, λ4, and λ5 are all weighting coefficients. Where q represents the sample feature q, k i k represents the class center feature of the i-th pedestrian identity in the query database. + The class center feature represents the pedestrian identity corresponding to the sample feature q, τ is the temperature coefficient of the first contrastive learning, and K represents the number of pedestrian identities in the training set; Among them, f ab The sample feature f represents the pedestrian's identity as a and the camera that took the picture as b. ab p ai Representing sample features f ab The original form, p j Representing sample features f ab The negative prototype, τ c For the second comparative learning temperature coefficient, N represents the number of camera-level comparison prototypes. neg To represent the number of negative samples in contrastive learning, P denotes the set of contrastive prototypes at the camera level, and C denotes the set of class center features; IT ce =CE(R sl ,R tg )+CE(R sg ,R tl ); Where CE() represents the cross-entropy loss function, R sl R represents the similarity relationship between the feature vectors output by the global branch in the student network and the global class centers. tg R represents the similarity relationship between the feature vectors output by the global branch in the teacher network and the global class centers. sg R represents the similarity relationship between the feature vectors output by the local branches in the student network and the local class centers. tl This represents the similarity relationship between the feature vectors output by the local branches in the teacher network and the local class centers; L kl =KL(W sl ,W tg )+KL(W sg ,W tl ); Where KL() represents the KL divergence loss function, W sl W represents the similarity between the feature vectors output by the local branches in the student network. tg W represents the similarity between the feature vectors output by the global branches in the teacher network. sg W represents the similarity between the feature vectors output by the global branches in the student network. tl This represents the similarity between the feature vectors output by the local branches in the teacher network; in, This represents the loss that uses the feature vector output of the global branch of the teacher network to supervise the output of the local branch of the student network. This represents the loss that uses the feature vectors output by the local branches of the teacher network to supervise the output of the global branches of the student network; Among them, T tg T represents the positive and negative sample pair relation of triples obtained from the global branch in the teacher network. sl θ represents the positive and negative sample pair relation of triples obtained from the local branch in the student network. ' Let θ represent the network parameters of the teacher network, and let x represent the network parameters of the student network. sl x represents the output of the student network. tg This represents the output of the teacher network.

3. The occluded person re-identification method based on multi-branch joint learning according to claim 2, characterized in that, λ1=1, λ2=0.5, λ3=1, λ4=1, λ5=10.

4. The occluded pedestrian re-identification method based on multi-branch joint learning according to claim 1, characterized in that, The process of constructing the training set includes: Obtain the Occluded-Duke dataset; Each pedestrian image in the Occluded-Duke dataset is resized to a set size to obtain the resized dataset; Each pedestrian image in the resized dataset is augmented using random flipping and random erasing, and the augmented dataset is used as the training set.

5. The occluded person re-identification method based on multi-branch joint learning according to claim 4, characterized in that, The specified dimensions are 256×128.

6. An occluded pedestrian re-identification system based on multi-branch joint learning, characterized in that, The occluded pedestrian re-identification system based on multi-branch joint learning applies the occluded pedestrian re-identification method based on multi-branch joint learning as described in claim 1. The occluded pedestrian re-identification system based on multi-branch joint learning includes: The pedestrian image acquisition module is used to acquire images of pedestrians to be identified. The pedestrian re-identification module is used to input the pedestrian image to be identified into the pedestrian re-identification model and output the pedestrian re-identification result; the pedestrian re-identification model is obtained by jointly learning and training the teacher network and the student network based on the training set; Both the teacher network and the student network are multi-branch networks, each comprising a shared layer, a global branch, local branches, and an output layer. The shared layer extracts features from the pedestrian image to be identified and outputs a feature map. The global branch performs global average pooling after feature extraction from the feature map. The local branch performs region pooling after feature extraction from the feature map and fuses multiple feature vectors of different scales. The output layer fuses the outputs of the global and local branches to obtain a feature fusion vector. The pedestrian identity corresponding to the pedestrian image with the highest cosine similarity between the feature fusion vector and the feature vectors of pedestrian images in the query database is used as the output of the pedestrian re-identification model. The sample images in the query database are occluded pedestrian images; The training set stores pedestrian images with multiple pedestrian identities.

Images