A method for constructing a person re-identification learning network and a person re-identification method

Abstract

The present invention discloses a method for constructing a person re-identification learning network and a person re-identification method, which relates to the field of deep learning technology. The method comprises the following steps: obtaining a person re-identification dataset, preprocessing the person re-identification dataset to obtain a training dataset; inputting the training dataset into an attribute soft-sharing and contextual local network to extract global features, local features, and attribute features; inputting the global features, local features, and attribute features into a global identity classifier, a local identity classifier, and an attribute classifier, respectively, to obtain predicted probabilities of the pedestrian identity, local identity, and pedestrian attributes; calculating an overall loss based on the predicted probabilities of the pedestrian identity, local identity, and pedestrian attributes; and iteratively updating the model parameters of the attribute soft-sharing and contextual local network according to the overall loss through back-propagation until the model converges, thereby obtaining a person re-identification learning network. The present invention can extract more discriminative multi-level features and improve the retrieval performance of person re-identification.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and more particularly to a method for constructing a person re-identification learning network and a person re-identification method. Background Art

[0002] Pedestrian re-identification (PRED) has numerous applications in smart security, pedestrian detection, and other scenarios. With the increasing number of surveillance cameras in public places like schools, shopping malls, and streets, PRED has become a significant practical need and has attracted widespread attention from researchers. However, PRED still faces many challenges. Pedestrian images are often unclear and irregular due to the influence of the camera equipment, and pedestrian postures are complex and varied. Changes in clothing, occlusion, viewing angle, and lighting also complicate PRED. Therefore, PRED is one of the most challenging research topics in computer vision.

[0003] Changes in pedestrian posture and shooting angle are one of the main issues affecting pedestrian re-identification performance. Traditional methods are mainly based on body structure and manually construct pedestrian image features. However, manually designed features lack robustness and cannot adapt to practical scenarios. With the widespread application of deep learning technology, many researchers have begun to apply convolutional neural networks (CNNs) to the field of person re-identification. Among the convolutional neural network-based methods, global-based methods can extract global features from the entire human body image. However, pedestrian images obtained from surveillance cameras are usually very complex, and the number of images available for learning is limited. Therefore, global-based methods are often susceptible to irrelevant background information and target occlusion.

[0004] In recent years, local-based methods have been shown to improve the retrieval performance of person re-identification. Local-based methods usually segment a human body image into multiple body parts. Each body part contains partial identity features and excludes the interference of background information, which enables CNN to focus on discriminative body parts. The discriminability of aggregating multiple local features is usually better than that of global features. However, this is usually limited by the matching degree of the reference human body parts and the image quality. Assuming that the pedestrian image has incomplete detection, partial occlusion or incorrect alignment of human body parts, the performance of the local-based method will deteriorate. In addition, the existing local-based methods learn each part of the human body as a separate part. This learning method can improve robustness, but it will also lose some detailed features.

[0005] Attribute-based methods also offer a way to improve retrieval performance. Attributes can be used to describe local or global information about a person, such as hairstyle, clothing, or shoes. Therefore, using attribute information as the primary supervised representation for person re-identification is feasible, as demonstrated by numerous person attribute recognition tasks. Describing person characteristics at the attribute level and at the holistic level are complementary. Consequently, these two learning methods are often combined.

[0006] Therefore, how to extract more discriminative multi-level features and reduce the difficulty of retrieving pedestrian images with large intra-class variations is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0007] In view of this, the present invention provides a method for constructing a pedestrian re-identification learning network and a pedestrian re-identification method, which can extract more discriminative multi-level features and improve the retrieval performance of pedestrian re-identification.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for constructing a person re-identification learning network includes the following steps:

[0010] Obtain a person re-identification dataset, preprocess the person re-identification dataset, and obtain a training dataset;

[0011] The training dataset is input into the attribute soft sharing and context local network to extract global features, local features and attribute features;

[0012] The global features, local features and attribute features are input into the global identity classifier, local identity classifier and attribute classifier respectively to obtain the predicted probabilities of pedestrian identity, local identity and pedestrian attributes;

[0013] Calculate the overall loss based on the predicted probabilities of pedestrian identity, local identity, and pedestrian attributes;

[0014] The model parameters of attribute soft sharing and context local network are iteratively updated according to the overall loss back propagation until the model converges to obtain the pedestrian re-identification learning network.

[0015] Optionally, obtaining a training dataset includes the following steps:

[0016] We select datasets with attribute labels, including Market-1501 and DukeMTM-reID, as person re-identification datasets.

[0017] Each time, the same number of samples are randomly extracted from the pedestrian re-identification dataset, the image size of each sample is adjusted, and horizontal flipping and random erasing are performed with a preset probability to obtain batch samples as the training dataset.

[0018] Optionally, the attribute soft sharing and context local network includes: a backbone network, a global feature branch, a local feature branch, and an attribute feature branch;

[0019] The backbone network uses the ResNet-50 model, and the classifier in the ResNet-50 model is removed;

[0020] The global feature branch includes a global identity classifier, the local feature branch includes a contextual local module and a local identity classifier, and the attribute feature branch includes an attribute soft sharing module and an attribute classifier;

[0021] The global feature branch is used to provide global-level features, namely global features;

[0022] The local feature branch obtains local features by utilizing the potential spatial relationship and contextual information between the local subject and the whole through the designed contextual local module;

[0023] The attribute feature branch establishes the intrinsic connection between attributes and constructs attribute features by designing the attribute soft sharing module.

[0024] Optionally, extract global features, specifically:

[0025] In the global feature branch, generalized average pooling is introduced to aggregate global features; among them, the global feature F is obtained by generalized average pooling. g The calculation formula is:

[0026]

[0027] Where, Indicates F g The i-th channel of the feature to be aggregated in , X represents the feature map; q represents the aggregation parameter of generalized average pooling; when q tends to infinity, generalized average pooling degenerates to maximum pooling; when q = 1, generalized average pooling degenerates to average pooling.

[0028] Optionally, extract local features, specifically:

[0029] Two branches are designed in the context local module, namely context local maximum pooling and context local average pooling;

[0030] The spatial attention modules are introduced in the context local maximum pooling and the context local average pooling, which can be expressed by the following formula:

[0031] S(X)=σ(Conv3×3 ([Avg(X);Max(X)])) (2);

[0032] Where S represents the spatial attention module; [A; B] represents the connection of A and B along the channel; X represents the input feature map; σ represents the sigmoid function; Conv 3×3 Represents a 3×3 convolutional layer; Avg and Max represent the average pooling and maximum pooling along the channel, respectively, which generate their own feature attention maps for the feature map X;

[0033] Each pixel of the feature map X contains preliminary spatial context information after passing through the spatial attention module, so the feature map X containing spatial context information is obtained. m The formula is as follows:

[0034] X m =Conv(X⊙S(X)) (3);

[0035] Where Conv is a fully convolutional network consisting of a 1×1 convolutional layer, a batch normalization layer, and a rectified linear activation unit layer; ⊙ represents pixel-level multiplication;

[0036] The feature map X containing spatial context information m Input to the maximum pooling layer / average pooling layer to aggregate features and obtain P m 、P a And the i-th row is used as the local feature of the context local maximum pooling / context local average pooling

[0037] Will Connect along the channel to get p i , the specific formula is as follows:

[0038]

[0039] will p i Input into the fully connected layer, and use the output of the fully connected layer as the local feature of the current context local module

[0040] Optionally, extract attribute features, specifically:

[0041] Input the feature map X into a fully convolutional network to extract the low-level attribute feature map A i ,(i∈(1,2,…,r)), r represents the number of attribute groups;

[0042] For each attribute feature map A i , except A i All feature maps except A jPerform averaging, then feature maps from other types for:

[0043]

[0044] Attribute feature map A i Perform two soft sharing operations, the formula is as follows:

[0045]

[0046] Where, Attribute feature graph representing the shared information in the first stage;

[0047] Obtain attribute features through global average pooling Avg The number of channels is C a .

[0048] Optionally, calculate the overall loss, which includes the following steps:

[0049] Using the cross entropy loss function l ce And the triple loss function l tri , supervise the global feature branch and the local feature branch; adopt the triple loss function l tri And the weighted binary cross entropy loss function l bce , supervised attribute feature branch;

[0050] Cross entropy loss function l ce The formula is as follows:

[0051]

[0052]

[0053] Where N represents the number of images in each batch; w i represents the predicted probability of identity i; y i Indicates the identity of the character; K indicates the total number of identities;

[0054] Triple loss function l tri Contains anchor images Positive sample image and negative sample images The formula is as follows:

[0055]

[0056] Where N M and N K Represents the number of images and identities in each batch respectively; α represents the parameter that controls the distance between a set of positive and negative sample pairs; i and j represent the person identity index and image index respectively;

[0057] Weighted binary cross entropy loss function l bce The formula is as follows:

[0058]

[0059] Where, σ represents the sigmoid function; represents the probability prediction of the attributes of character i and j; Labels representing attributes of images i and j;

[0060] Introducing weight coefficient p j , the specific formula is as follows:

[0061]

[0062] Where q j Indicates the proportion of attribute j in the training data set;

[0063] The overall loss of attribute soft sharing and context local network is the sum of all branch losses, which is calculated as follows:

[0064] l t =l g +l l +l a (12);

[0065] l g =l tri +l ce (13);

[0066]

[0067]

[0068] Where, l g 、l l 、l a They represent the losses of the global feature branch, local feature branch, and attribute feature branch respectively; i and j represent the indexes of local features and attribute features respectively.

[0069] A person re-identification method, using the above-mentioned person re-identification learning network, includes the following steps:

[0070] Obtain a person re-identification dataset, preprocess the person re-identification dataset, and obtain pedestrian images;

[0071] Input the pedestrian image into the pedestrian re-identification learning network to extract global features, local features and attribute features;

[0072] The global features, local features and attribute features are input into the global identity classifier, local identity classifier and attribute classifier respectively to confirm the pedestrian identity, local identity and pedestrian attributes, and obtain the pedestrian re-identification results.

[0073] It can be seen from the above technical solution that compared with the existing technology, the present invention provides a method for constructing a pedestrian re-identification learning network and a pedestrian re-identification method. The attribute soft sharing and context local network are composed of three feature branches, and each branch extracts features at different levels. In addition, the context local module can learn the context information from a local body, and the attribute soft sharing module can learn the intrinsic connection between attributes. With the support of these two modules, the attribute soft sharing and context local network can extract more discriminative multi-level features to solve the problems existing in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0075] Figure 1 A flowchart of the method for constructing a person re-identification learning network provided by the present invention;

[0076] Figure 2 The overall architecture diagram of the attribute soft sharing and context local network provided by the present invention;

[0077] Figure 3 A schematic diagram of the overall structure of the context local module provided by the present invention;

[0078] Figure 4 This is a structural diagram of the attribute soft sharing module provided by the present invention. DETAILED DESCRIPTION

[0079] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0080] In person re-identification, the key to recovering the correct pedestrian image is to extract discriminative image features, and features at different levels have been shown to be complementary. In response to the problems existing in the background technology, the embodiment of the present invention discloses a method for constructing a person re-identification learning network, such as Figure 1 As shown, the following steps are included:

[0081] Obtain a person re-identification dataset, preprocess the person re-identification dataset, and obtain a training dataset;

[0082] The training dataset is input into the Attribute Soft-Sharing and Contextual Local Network (ASCLNet) to extract global features, local features, and attribute features.

[0083] The global features, local features and attribute features are input into the global identity classifier, local identity classifier and attribute classifier respectively to obtain the predicted probabilities of pedestrian identity, local identity and pedestrian attributes;

[0084] Calculate the overall loss based on the predicted probabilities of pedestrian identity, local identity, and pedestrian attributes;

[0085] The model parameters of attribute soft sharing and context local network are iteratively updated according to the overall loss back propagation until the model converges to obtain the pedestrian re-identification learning network.

[0086] Furthermore, obtaining a training data set specifically includes the following steps:

[0087] We select datasets with attribute labels, including Market-1501 and DukeMTM-reID, as person re-identification datasets.

[0088] Each time, the same number of samples are randomly extracted from the pedestrian re-identification dataset, the image size of each sample is adjusted, and horizontal flipping and random erasing are performed with a preset probability to obtain batch samples as the training dataset.

[0089] Further, refer to Figure 2 ,The attribute soft sharing and context local network includes: backbone network, global feature branch, local feature branch and attribute feature branch;

[0090] The backbone network uses the ResNet-50 model. In order to integrate branches, the classifier in the ResNet-50 model is removed;

[0091] The global feature branch includes a global identity classifier, the local feature branch includes a contextual local module and a local identity classifier, and the attribute feature branch includes an attribute soft sharing module and an attribute classifier;

[0092] The global feature branch is used to provide global-level features, namely global features;

[0093] The local feature branch obtains local features by utilizing the potential spatial relationship and contextual information between the local subject and the whole through the designed contextual local module;

[0094] The attribute feature branch establishes the intrinsic connection between attributes and constructs attribute features by designing the attribute soft sharing module.

[0095] Furthermore, global features are extracted, specifically:

[0096] In the global feature branch, generalized mean pooling (GMeP) is introduced to aggregate global features; the global feature F is obtained by generalized mean pooling. g The calculation formula is:

[0097]

[0098] Where, Indicates F g The i-th channel of the feature to be aggregated in , X represents the feature map; q represents the aggregation parameter of generalized average pooling; when q tends to infinity, generalized average pooling degenerates to global max pooling (GMP); when q = 1, generalized average pooling degenerates to global average pooling (GAP).

[0099] In past local-based methods, local features are usually independent of each other. For example, PCB (convolutional baseline network) divides the initial feature map X equally into six horizontal parts. The size of each horizontal part is H / 6×W×C. When this coarse division method encounters irregular pedestrian image detection, the local parts may not be aligned. This will lead to a decrease in retrieval performance. Therefore, this embodiment proposes a contextual local module (CLM), which establishes a contextual relationship between the local and the whole. PCB only focuses on local features that are independent of each other, however, in CLM, a contextual relationship is established for each local part, which makes the local features have better discrimination ability.

[0100] Specifically, refer to Figure 3 The overall structure of the context local module shown in Figure 1 extracts local features, specifically:

[0101] Two branches are designed in the contextual local module, namely contextual local max pooling (CMP) and contextual local average pooling (CAP). CAP learns global information from the entire person image, but it is more susceptible to interference from background and irrelevant objects; CMP aggregates features from the most discriminative body parts and is less susceptible to interference from irrelevant information, but it contains less information from the entire body than CAP. Therefore, this scheme combines CAP and CMP to improve performance.

[0102] The spatial attention module is introduced into the contextual local maximum pooling and the contextual local average pooling, which allows the CLM to learn which body parts are more discriminative. When the current part is occluded or misplaced, the module will pay more attention to other valid parts of the human body to compensate for the current local features. It can be expressed by the following formula:

[0103] S(X)=σ(Conv 3×3 ([Avg(X);Max(X)])) (2);

[0104] Where S represents the spatial attention module; [A; B] represents the connection of A and B along the channel; X represents the input feature map; σ represents the sigmoid function; Conv 3×3 Represents a 3×3 convolutional layer; Avg and Max represent the average pooling and maximum pooling along the channel, respectively, which generate their own feature attention maps for the feature map X;

[0105] Each pixel of the feature map X contains preliminary spatial context information after passing through the spatial attention module, so the feature map X containing spatial context information is obtained. m The formula is as follows:

[0106] X m =Conv(X⊙S(X)) (3);

[0107] Where Conv is a fully convolutional network consisting of a 1×1 convolution (conv) layer, a batch normalization (BN) layer, and a rectified linear activation unit (ReLu) layer; ⊙ represents pixel-level multiplication.

[0108] The feature map X containing spatial context information m Input to the maximum pooling layer / average pooling layer to aggregate features and obtain P m 、P aAnd the i-th row is used as the local feature of the context local maximum pooling / context local average pooling and discard the rest, which is the key for local features to obtain contextual information, which allows CLM to focus on the i-th part while learning the rest;

[0109] Will Connect along the channel to get p i , the specific formula is as follows:

[0110]

[0111] will p i Input into the fully connected layer, and use the output of the fully connected layer as the local feature of the current context local module

[0112] In most previous person re-identification methods, attributes are usually used as auxiliary information to improve retrieval performance. However, pedestrian attributes are generally unchanged, so it is feasible to reconstruct pedestrian features based on attributes. Therefore, this scheme regards attribute labels as information that is equally important as identity labels for ASCLNet to learn. In order to enable ASCLNet to learn the correlation between attributes, we refer to the relevant parameter sharing method in multi-task learning and apply it to ASM. Figure 4 As shown in the figure, dense connections are used to share attribute information, and attribute features are extracted as follows:

[0113] Input the feature map X into a fully convolutional network to extract the low-level attribute feature map A i ,(i∈(1,2,…,r)), r represents the number of attribute groups;

[0114] For each attribute feature map A i , except A i All feature maps except A j Perform averaging, then feature maps from other types for:

[0115]

[0116] Attribute feature map A i Perform two soft sharing operations, the formula is as follows:

[0117]

[0118] Where, Attribute feature graph representing the shared information in the first stage;

[0119] Obtain attribute features through global average pooling Avg The number of channels is Ca .

[0120] Regarding the calculation of losses, we can use To represent a person re-identification dataset with attribute labels. i and y i Represents the character image and identity respectively. K and n represent the total number of identities and the total number of images. As a set of attributes of a pedestrian identity i, represents the j-th attribute of identity i, and m represents the total number of attribute categories.

[0121] Specifically, calculating the overall loss includes the following steps:

[0122] Using the cross entropy loss function l ce And the triple loss function l tri , supervise the global feature branch and the local feature branch; adopt the triple loss function l tri And the weighted binary cross entropy loss function l bce , supervised attribute feature branch;

[0123] Cross entropy loss function l ce The formula is as follows:

[0124]

[0125]

[0126] Where N represents the number of images in each batch; w i represents the predicted probability of identity i; y i Indicates the identity of the character; K indicates the total number of identities;

[0127] Triple loss function l tri Contains anchor images Positive sample image and negative sample images The formula is as follows:

[0128]

[0129] Where N M and N K Represents the number of images and identities in each batch respectively; α represents the parameter that controls the distance between a set of positive and negative sample pairs; i and j represent the person identity index and image index respectively;

[0130] Weighted binary cross entropy loss function l bce The formula is as follows:

[0131]

[0132] Where, σ represents the sigmoid function; represents the probability prediction of the attributes of character i and j; Labels representing attributes of images i and j;

[0133] Introducing weight coefficient p j To deal with the imbalance between the frequency of attribute occurrences, the specific formula is as follows:

[0134]

[0135] Where q j Indicates the proportion of attribute j in the training data set;

[0136] The overall loss of attribute soft sharing and context local network is the sum of all branch losses, which is calculated as follows:

[0137] l t =l g +l l +l a (12);

[0138] l g =l tri +l ce (13);

[0139]

[0140]

[0141] Where, l g 、l l 、l a They represent the losses of the global feature branch, local feature branch, and attribute feature branch respectively; i and j represent the indexes of local features and attribute features respectively.

[0142] Furthermore, during the testing phase, all local features can be concatenated: Then concatenate all the attribute features: Then, they are combined with the global feature F g Connect to get the final feature F of a pedestrian image t . Calculate the Euclidean distance E between the features of two pedestrian images to measure the similarity. The formula is as follows:

[0143] F t =[F g ; F l ; F a ];

[0144]

[0145] Where i and j are two pedestrian images with different identities and indices in the dataset.

[0146] Furthermore, the performance of ASCLNet can be evaluated using the cumulative matching characteristic (CMC), mean inverse negative penalty (mINP), and mean average precision (mAP). CMC reflects the probability that the top n results in the retrieved image have correct results, mINP reflects the cost of retrieving the most difficult correct image, and mAP reflects the average retrieval accuracy of all identities. To demonstrate these advantages, the experimental results of ASCLNet were compared with existing methods through many experiments on several datasets. The comparative results show that ASCLNet has good performance. In addition, ablation experiments were performed to verify the effects of CLM and ASM. Finally, many visualizations and analyses were provided to explain the effectiveness of the network in this technology.

[0147] 1. Training details

[0148] The backbone networks used in this experiment were all pre-trained on the ImageNet dataset. The input images were resized to 384×128 and horizontally flipped and randomly erased with a probability of 50%. Adaptive gradient was used as the optimizer, with weight decay = 4e-4, β1 = 0.89, and β2 = 0.899. The aggregation parameter q of GMeP was 3. The initial learning rate was 3.6e-5, which increased linearly to 3.6e-4 at the 10th epoch, and decreased to 3.6e-5 and 3.6e-6 at the 50th and 80th epochs, respectively. The batch size of the image was 32, and the training was performed for 140 epochs. The size of the initial feature map X was 24×8×2048. C in the feature branch a and C p All are 512.

[0149] 2. Dataset

[0150] Experiments are primarily conducted on two attribute-labeled datasets, Market-1501 and DukeMTM-reID. Furthermore, to demonstrate ASCLNet's superior performance even without attribute label references, a series of experiments are conducted on two datasets, MSMT17 and CUHK03-NP, without attribute labels, after removing the attribute feature branch from ASCLNet.

[0151] Market-1501 contains 32,668 images of 1,501 people. All images are automatically generated using a deformable part model. The training set contains 12,936 images of 751 people with different identities. The gallery set and search set contain 19,732 and 3,368 images of 750 people with different identities, respectively. This dataset is annotated with pedestrian attribute labels.

[0152] DukeMTMC-reid is a subset of DukeMTMC. It contains 36,411 images of 1,404 people (excluding 408 distractors). All images are manually annotated. The training set contains 16,522 images of 702 different pedestrian identities. The gallery and retrieval sets contain 17,661 and 2,228 images of 702 different pedestrian identities, respectively. This dataset annotates the pedestrian attributes.

[0153] CUHK03-np is CUHK03 with a new partitioning scheme. It contains 14,097 images of 1,467 people. The person images were annotated using two methods: manual annotation and automatic annotation using a deformable part model. In the manually annotated set, the training set contains 7,368 images of 767 people. The gallery and retrieval sets contain 5,328 and 1,400 images, respectively, of 700 people of different identities. In the automatically annotated set, the training set contains 7,365 images of 767 people. The gallery and retrieval sets contain 5,332 and 1,400 images, respectively, of 700 people of different identities.

[0154] MSMT17 is a large-scale dataset that closely resembles real-world scenarios. It contains 126,441 images of 4,101 people. The images in the dataset come from multiple scenes and time periods. The training set contains 32,621 images of 1,401 people. The gallery set and retrieval set contain 93,820 and 11,659 images of 3,060 people, respectively.

[0155] 3. Comparison with other methods

[0156] The experiment mainly uses two performance indicators, average precision mAP (%) and cumulative matching features R=1 (%), for comparison.

[0157] Compare on Market-1501

[0158] As shown in Table 1, ASCLNet is compared with 20 other state-of-the-art methods on Market-1501 in terms of R=1 and mAP. The methods included in the comparison are divided into three categories: traditional methods, deep learning-based methods, and attribute-based methods. ASCLNet achieves an mAP score of 88.9% and an R=1 accuracy of 95.9%. It can be seen that our method significantly outperforms traditional methods. In comparison with deep learning-based methods, our method surpasses PCB+RPP by 2.1% and 7.3% in R=1 and mAP, respectively, and surpasses Pyramid by 0.2% and 0.7%. Compared with the latest attribute-based methods, ASCLNet surpasses AANet by 2.0% and 6.5% in R=1 and mAP, and APNet by 0.7% and 1.6% in R=1 and mAP.

[0159] Table 1 Comparison with the most advanced methods on Market-1501

[0160]

[0161]

[0162] Comparison on DukeMTMC-reID

[0163] As shown in Table 2, ASCLNet is compared with 20 state-of-the-art methods on DukeMTMC-reID for R=1 and mAP. ASCLNet achieves a mAP score of 80.2% and an accuracy of 89.8% for R=1. Our method outperforms PCB+RPP by 6.5% and 11.0% for R=1 and mAP, respectively, and ISP by 0.2% and 0.2%. Furthermore, ASCLNet achieves significant performance improvements over new attribute-based methods, surpassing AANet by 3.4% and 7.6% for R=1 and mAP, and APNet by 0.6% and 1.2% for R=1 and mAP, respectively.

[0164] Table 2 Comparison of DukeMTMC-reID with the most advanced methods

[0165]

[0166]

[0167] Comparison on Cuhk03-NP

[0168] CUHK03-NP does not have public attribute labels, so the attribute feature branch of ASCLNet was removed when conducting experiments on this dataset. As shown in Table 3, ASCLNet (without A) also performs significantly better on CUHK03-NP. Except for R=1 on CUHK03-NP (Detected), where it is 1.3% lower than Pyramid, it outperforms other methods in all other cases. ASCLNet achieves R=1 / mAP=80.9% / 78.7% on CUHK03-NP (Labeled) and R=1 / mAP=77.6% / 74.9% on CUHK03-NP (Detected).

[0169] Table 3 Comparison on Cuhk03-NP, where (W / OA) represents the removal of attribute feature branches

[0170]

[0171] Comparison on MSMT17

[0172] MSMT17 also lacks public attribute labels, so experiments on this dataset continue using ASCLNet after removing the attribute feature branch. As shown in Table 4, ASCLNet (without A) also performs significantly better on MSMT17, surpassing many leading methods. On MSMT17, our method achieves R=1 and mAP of 81.7% and 58.8%, respectively.

[0173] Table 4 compares on MSMT17, where (W / OA) indicates the removal of attribute feature branches

[0174]

[0175] Ablation experiments

[0176] This paper conducts a series of ablation experiments and visual explanations. The baseline network is the Attribute-Person Recognition Network (ARN) with the Attribute Re-weighting Module (ARM) removed. For the ablation experiments, the same training techniques and image preprocessing are applied to the baseline network and ASCLNet. In addition, two additional metrics, the cumulative matching feature R=10 and the inverse negative penalty rate (mINP), are included in the comparison.

[0177] The impact of different feature branches

[0178] As shown in Tables 5 and 6, ablation experiments were conducted on two attribute-labeled datasets using the two proposed modules. As can be seen from the second and third rows of the table, the use of CLM and ASM can improve retrieval performance. Taking Market-1501 as an example, the attribute feature branch provides performance improvements of 1.19%, 2.67%, and 3.09% on R=1, mAP, and mINP, respectively, while the local feature branch provides performance improvements of 1.78%, 3.47%, and 3.03% on R=1, mAP, and mINP, respectively. Meanwhile, removing any branch from CLM degrades performance, demonstrating that CAP and CMP have different focuses and can work together. On Market-1501, our method improves the baseline network's R=1, R=10, mAP, and mINP by 3.06%, 0.74%, 5.1%, and 6.07%, respectively. Compared to the baseline network, ASCLNet achieves higher retrieval performance, with each feature branch focusing on features at different levels. The global feature branch focuses more on overall features, the local feature branch mainly learns local features, and the attribute feature branch focuses on secondary features, which explains why combining these features can improve retrieval performance.

[0179] Table 5 shows the results of the ablation experiment on the modules applied to ASCLNet on Market-1501 (%). “W / OASM” means the attribute feature branch is removed, “W / O CLM” means the local feature branch is removed, “W / O CMP” means the contextual local maximum pooling is removed in the local feature branch, and “W / O CAP” means the contextual local average pooling is removed in the local feature branch.

[0180]

[0181] Table 6 Ablation experiment results (%) of the modules applied to ASCLNet on DukeMTMC-reID. “W / OASM” means the attribute feature branch is removed, “W / O CLM” means the local feature branch is removed, “W / O CMP” means the contextual local maximum pooling is removed in the local feature branch, and “W / O CAP” means the contextual local average pooling is removed in the local feature branch.

[0182]

[0183] The impact of different backbone networks

[0184] This experiment uses variants of well-known residual networks, such as ResNet-101 and ResNetNL-101, to replace the backbone network. Compared to ResNet-50, these networks either have deeper network structures or have added attention mechanisms. Overall, they perform better in most cases. As with the previous ablation experiments, experiments are conducted on two attribute-labeled datasets. The experimental results are shown in Tables 7 and 8. After replacing the well-performing backbone networks on both datasets, performance improves slightly, demonstrating the effectiveness and robustness of the modules proposed in this solution.

[0185] Table 7 Results of ablation study on backbone network in Market-1501 (%)

[0186]

[0187] Table 8 Results of ablation study on the backbone network on DukeMTMC-reID (%)

[0188]

[0189] This embodiment also discloses a person re-identification method, which utilizes the above-mentioned person re-identification learning network and includes the following steps:

[0190] Obtain a person re-identification dataset, preprocess the person re-identification dataset, and obtain pedestrian images;

[0191] Input the pedestrian image into the pedestrian re-identification learning network to extract global features, local features and attribute features;

[0192] The global features, local features and attribute features are input into the global identity classifier, local identity classifier and attribute classifier respectively to confirm the pedestrian identity, local identity and pedestrian attributes, and obtain the pedestrian re-identification results.

[0193] In this technical solution, a pedestrian re-identification network based on attribute soft sharing and local context (ASCLNet) is designed. It is a novel multi-level feature learning network. In addition, two modules are proposed to learn local features and attribute features: the context local module (CLM) uses the contextual information between the local and full body to reconstruct the characteristics of a person based on local features; the attribute soft sharing module (ASM) learns the intrinsic connection between attributes by sharing feature parameters between the attributes of a person. Through extensive experiments on widely used datasets, sufficient visual explanations are given. Experimental results show that ASCLNet achieves significant performance and surpasses many advanced methods.

[0194] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0195] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a person re-identification learning network, characterized in that: The following steps are involved: Obtain a person re-identification dataset, preprocess the person re-identification dataset, and obtain a training dataset; The training dataset is input into the attribute soft sharing and context local network to extract global features, local features and attribute features; The global features, local features and attribute features are input into the global identity classifier, local identity classifier and attribute classifier respectively to obtain the predicted probabilities of pedestrian identity, local identity and pedestrian attributes; Calculate the overall loss based on the predicted probabilities of pedestrian identity, local identity, and pedestrian attributes; The model parameters of the attribute soft sharing and context local network are iteratively updated according to the overall loss back propagation until the model converges to obtain the pedestrian re-identification learning network; Among them, the attribute soft sharing and context local network includes: a backbone network, a global feature branch, a local feature branch, and an attribute feature branch; the backbone network adopts the ResNet-50 model, and the classifier in the ResNet-50 model is removed; the global feature branch includes a global identity classifier, the local feature branch includes a context local module and a local identity classifier, and the attribute feature branch includes an attribute soft sharing module and an attribute classifier; the global feature branch is used to provide global-level features, namely global features; the local feature branch, through the designed context local module, uses the potential spatial relationship and context information between the local subject and the whole to obtain local features; the attribute feature branch, through the designed attribute soft sharing module, establishes the intrinsic connection between attributes and constructs attribute features; Extract attribute features, specifically: Input the feature map X into a fully convolutional network to extract the low-level attribute feature map A i , i∈(1,2,…,r), r represents the number of attribute groups; For each attribute feature map A i , except A i All feature maps except A j Perform averaging, then feature maps from other types for: Attribute feature map A i Perform two soft sharing operations, the formula is as follows: Where, Attribute feature graph representing the shared information in the first stage; Obtain attribute features through global average pooling Avg The number of channels is C a .

2. The method for constructing a person re-identification learning network according to claim 1, characterized in that: Obtaining the training data set includes the following steps: We select datasets with attribute labels, including Market-1501 and DukeMTM-reID, as person re-identification datasets. Each time, the same number of samples are randomly extracted from the pedestrian re-identification dataset, the image size of each sample is adjusted, and horizontal flipping and random erasing are performed with a preset probability to obtain batch samples as the training dataset.

3. The method for constructing a person re-identification learning network according to claim 1, characterized in that: Extract global features, specifically: In the global feature branch, generalized average pooling is introduced to aggregate global features; among them, the global feature F is obtained by generalized average pooling. g The calculation formula is: Where, Indicates F g The i-th channel of the feature to be aggregated in , X represents the feature map; q represents the aggregation parameter of generalized average pooling; when q tends to infinity, generalized average pooling degenerates to maximum pooling; when q = 1, generalized average pooling degenerates to average pooling.

4. The method for constructing a person re-identification learning network according to claim 1, characterized in that: Extract local features, specifically: Two branches are designed in the context local module, namely context local maximum pooling and context local average pooling; The spatial attention modules are introduced in the context local maximum pooling and the context local average pooling, which can be expressed by the following formula: S(X)=σ(Conv 3×3 ([Avg(X);Max(X)])) (2); Where S represents the spatial attention module; [A; B] represents the connection between A and B along the channel; X represents the input feature map; σ represents the sigmoid function; Conv 3×3 Represents a 3×3 convolutional layer; Avg and Max represent the average pooling and maximum pooling along the channel, respectively, which generate their own feature attention maps for the feature map X; Each pixel of the feature map X contains preliminary spatial context information after passing through the spatial attention module, so the feature map X containing spatial context information is obtained. m The formula is as follows: X m =Conv(X⊙S(X)) (3); Where Conv is a fully convolutional network consisting of a 1×1 convolutional layer, a batch normalization layer, and a rectified linear activation unit layer; ⊙ represents pixel-level multiplication; The feature map X containing spatial context information m Input to the maximum pooling layer / average pooling layer to aggregate features and obtain P m 、P a And the i-th row is used as the local feature of the context local maximum pooling / context local average pooling Will Connect along the channel to get p i , the specific formula is as follows: will p i Input into the fully connected layer, and use the output of the fully connected layer as the local feature of the current context local module 5. The method for constructing a person re-identification learning network according to claim 1, characterized in that: Calculate the overall loss, which includes the following steps: Using the cross entropy loss function l ce And the triple loss function l tri , supervise the global feature branch and the local feature branch; adopt the triple loss function l tri And the weighted binary cross entropy loss function l bce , supervised attribute feature branch; Cross entropy loss function l ce The formula is as follows: Where N represents the number of images in each batch; w i represents the predicted probability of identity i; y i Indicates the identity of the character; K indicates the total number of identities; Triple loss function l tri Contains anchor images Positive sample image and negative sample images The formula is as follows: Where N M and N K Represents the number of images and identities in each batch respectively; α represents the parameter that controls the distance between a set of positive and negative sample pairs; i and j represent the person identity index and image index respectively; Weighted binary cross entropy loss function l bce The formula is as follows: Where, σ represents the sigmoid function; represents the probability prediction of the attributes of character i and j; Labels representing attributes of images i and j; Introducing weight coefficient p j , the specific formula is as follows: Where q j Indicates the proportion of attribute j in the training data set; The overall loss of attribute soft sharing and context local network is the sum of all branch losses, which is calculated as follows: l t ＝l g +l l +l a (12)； l g ＝l tri +l ce (13)； Where, l g 、l l 、l a They represent the losses of the global feature branch, local feature branch, and attribute feature branch respectively; i and j represent the indexes of local features and attribute features respectively.

6. A pedestrian re-identification method, characterized in that: Using the person re-identification learning network according to any one of claims 1 to 5, comprising the following steps: Obtain a person re-identification dataset, preprocess the person re-identification dataset, and obtain pedestrian images; Input the pedestrian image into the pedestrian re-identification learning network to extract global features, local features and attribute features; The global features, local features and attribute features are input into the global identity classifier, local identity classifier and attribute classifier respectively to confirm the pedestrian identity, local identity and pedestrian attributes, and obtain the pedestrian re-identification results.

Images