A pedestrian re-identification method based on multi-granularity segmentation attention network

Abstract

The present application considers the problems of pedestrian misalignment and difficulty in capturing multi-granularity features, and proposes a pedestrian re-identification method based on a multi-granularity segmentation attention network. It contains the extraction and processing of coarse-grained and fine-grained features of the image, which improves the robustness of the model. Through multi-granularity segmentation attention, the relationship within the feature is extracted. The channel attention mechanism is added to extract the relationship between the features. Finally, GeMPooling is used for aggregation to produce a compact image representation, and BN layer is used to prevent gradient vanishing or explosion, prevent overfitting, and speed up convergence, etc. A new effective method for pedestrian re-identification is constructed, which provides a more efficient framework for pedestrian re-identification in practical applications. Then, combined with ID loss and hard sample mining triplet loss, the loss of the optimized model is iteratively optimized, so that the final loss of the model reaches a minimum value, so as to improve the accuracy and robustness of the features.

Description

[0001] This invention relates to the field of computer vision technology and is a pedestrian re-identification method based on a multi-granularity segmentation attention network. Background Technology

[0002] Person re-identification, also known as ReID, is a technique that uses computer vision to determine the presence of a specific pedestrian in an image or video sequence. Due to its significant application value in intelligent video surveillance and security systems, this task is of great research importance. In recent years, with the continuous advancement of deep learning, research in this area has made significant progress. However, the large differences in background, lighting, pose, and camera resolution between images captured by different cameras make learning robust discriminative features extremely challenging.

[0003] In recent years, with the active improvement of deep learning, significant progress has been made in person re-identification research. Despite this progress, the large intra-class variability caused by cluttered backgrounds, varying lighting, pose changes, and different resolutions still makes identifying people from a large number of candidate images and learning their discriminative features challenging. Attention mechanisms have been widely used in ReID systems to promote high-performance recognition and demonstrate powerful representational capabilities by discovering discriminative regions and mitigating misalignments. Attention mechanisms still play a significant role in person re-identification research. However, they cannot effectively capture multi-granular features. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a pedestrian re-identification method based on a multi-granularity segmentation attention network. Its purpose is to solve the problems of pedestrian misalignment and difficulty in capturing multi-granularity features, and to extract more robust and discriminative features with internal correlation.

[0005] To achieve the above objectives, this invention provides a person re-identification method based on a multi-granularity segmentation attention network, comprising the following steps:

[0006] S1. Construct the network framework, which consists of a multi-granularity segmentation attention module and a channel attention module;

[0007] S2. Input the given pedestrian image into the network to extract features, and the obtained features are processed by the multi-granularity segmentation attention module;

[0008] S3. The obtained features are processed through a channel attention mechanism;

[0009] S4. The obtained features are processed through GeM Pooling and then through a BN layer;

[0010] S5. Finally, calculate the ID loss and hard sample mining triplet loss based on the obtained features.

[0011] This invention provides a person re-identification method based on a multi-granularity segmentation attention network. Compared with existing technologies, it has the following advantages:

[0012] This invention addresses the challenges of pedestrian misalignment and the difficulty in capturing multi-granularity features by introducing multi-granularity segmentation attention to obtain the internal feature relationships at multiple granularities. A channel attention mechanism is then added to further clarify these relationships, effectively solving the problem of capturing multi-granularity features. GeM pooling is then used to pool the features in the final layer of the network, followed by a Batch Normalization (BN) layer. Finally, an iterative feedback process is implemented, combining ID loss and hard sample mining triplet loss, to minimize the model's final loss. This optimizes the model and improves the accuracy and robustness of pedestrian re-identification. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is the overall network structure diagram of the pedestrian re-identification based on a multi-granularity segmentation attention network in this invention patent.

[0015] Figure 2 This is a structural diagram of the multi-granularity segmentation attention module in this invention patent.

[0016] Figure 3 This is a structural diagram of the channel attention mechanism in this invention patent. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0018] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Pedestrian re-identification based on a multi-granularity segmentation attention network includes steps S1 to S5:

[0019] S1. Construct the network framework, which consists of a multi-granularity segmentation attention module and a channel attention module;

[0020] S2. Input the given pedestrian image into the network to extract features, and the obtained features are processed by the multi-granularity segmentation attention module;

[0021] S3. The obtained features are processed through a channel attention mechanism;

[0022] S4. The obtained features are processed through GeM Pooling and then through a BN layer;

[0023] S5. Finally, calculate the ID loss and hard sample mining triplet loss based on the obtained features.

[0024] The following is a detailed description of each step.

[0025] In step S1, the network architecture is constructed, such as... Figure 1 As shown, the network consists of two parts: a multi-granularity segmentation attention module and a channel attention module. The specific steps are as follows:

[0026] S101. Construct a framework consisting of a multi-granularity segmentation attention module and a channel attention mechanism.

[0027] S102. The extracted features are used for classification.

[0028] In step S2, the given pedestrian image is input into the backbone network to extract features, and then further captured through a multi-granularity segmentation attention module. The specific steps are as follows:

[0029] S201. The backbone network used in the feature extraction section is a five-stage ResNet-50. The stride of the fifth (final) convolution is changed from 2 to 1 to preserve more image details. ResNet-50 has appeared in many classic network architectures and has been widely accepted and proven. Therefore, ResNet-50 is used for feature extraction from pedestrian images.

[0030] S202. Given a person image, we first extract feature maps X∈R using a backbone network. C×H×W Where C represents the channel dimension, H represents the height, and W represents the width. Features undergo various stages, and then pass through a multi-granularity segmentation attention module, such as... Figure 2 As shown. This module is mainly designed to capture multi-granularity features and improve the internal correlation of channel-level features. We can capture multi-granularity features using the following multi-granularity segmentation attention module:

[0031] First, the input features are divided into two parts along the channel axis. The number of feature channels is defined as C, so the number of channels in each part is C / 2, as shown in the following formula:

[0032] Split1,Split2=S(X) (1)

[0033] Where S(·) represents the input feature being divided into two features along the channel. i This indicates the result of the first segmentation.

[0034] After feature segmentation, we use sub-channel attention to learn discriminative cues for each feature tensor based on the channel-separated features. We then pass the segmented features through sub-channel attention A... i This improves the internal correlation of channel-level features. Then, the features are merged to obtain an attention feature A with the same size as the input features, as shown in the formula:

[0035]

[0036] The asterisk (*) represents element-wise multiplication. The summation symbol is used to merge the attention maps obtained after segmentation. This merged attention map represents the salient regions in the feature map.

[0037] As the network continuously extracts features, the feature dimension becomes very large (2048 dimensions) in later stages. Even after one segmentation, the number of channels in each group is still very high. Therefore, we use a re-segmentation approach to achieve finer-grained feature extraction. Taking Split1, obtained after the first segmentation, as an example, its formula is as follows:

[0038]

[0039] This represents the result of Split1. After re-segmentation, the re-segmentation features are processed through sub-channel attention and then merged to obtain the re-segmentation attention, the formula of which is as follows:

[0040]

[0041] Using formulas (3) and (4) above, we can obtain the features of Split1 after initial segmentation and attention merging. These features are then used as the Split1 features for the first segmentation in formula (1). Similarly, Split2 obtains features in the same way as Split1. The first segmentation yields coarser-grained features, and the second segmentation yields finer-grained features. Through multi-granularity segmentation attention processing, multi-granularity features are captured, improving the internal correlation of channel-level features.

[0042] In step S3, the obtained features are processed through a channel attention mechanism.

[0043] S301, First, the framework diagram of this attention mechanism is as follows: Figure 3As shown, this attention is primarily focused on the relationships between features.

[0044] S302. We introduce a channel attention module after the multi-granularity segmentation attention at each stage of the backbone network. Here, we directly use the SE module. The input feature map is of size H×W×C. First, we perform global average pooling on it to obtain a feature map of size 1×1×C. Then, we pass it through a fully connected layer for dimensionality reduction, and the feature size becomes... (r is a scaling factor used to control the number of channels). Next, ReLU activation is performed. Finally, a fully connected layer and a sigmid layer are passed to obtain channel weights. These channel weights are applied to the original input features, resulting in features with different weights for each channel. The SE module is a channel attention module, which can be seen as adaptively learning a weight for the feature map on each channel. Given a feature map X, the channel attention A is defined as:

[0045] A = Sigmoid(W2ReLU(W1Pool) avg (X))) (5)

[0046] Where W1 and W2 represent two fully connected operations, Pool avg This represents the average pooling layer. Channel attention A is applied to the original feature map X through channel multiplication. Furthermore, this channel attention is also used in the attention part of our multi-granularity segmentation module. The segmented multi-granularity features are then used to replace the original feature map X to learn the segmented local channel attention.

[0047] In step S4, the input features are processed through GeM Pooling and BN layers.

[0048] S401. The features from stage 5 undergo a generalized mean pooling (GeM Pooling) to become a vector feature, which is then input into the hard sample mining triplet loss during the training phase. For multi-granularity instance retrieval, widely used max pooling or mean pooling cannot capture discriminative features specific to a particular domain. We employ a learnable pooling layer called generalized mean (GeM) pooling. This pooling layer takes the original feature map X as input and produces a vector f as the output of the pooling process, with the following formula:

[0049]

[0050] Here, Pk is a pooling hyperparameter that can be learned during backpropagation. As Pk approaches infinity, it approximates max pooling, and as Pk = 1, it approximates average pooling. The feature vector ultimately consists of a single value from each feature map, i.e., the generalized average activation, with a dimension equal to K. In many popular networks, this equals 256, 512, or 2048, making it a compact image representation. In the network proposed in this invention, this value is 2048.

[0051] S402. The obtained features are processed through a BN layer, i.e., batch normalization, to solve the problem of changes in the distribution of intermediate layer data during training, so as to prevent gradient vanishing or exploding, prevent overfitting, and speed up convergence.

[0052] In step 5, the obtained features are used to calculate the ID loss and the hard sample mining triplet loss, which are then fed back into the model for iterative loss optimization. The specific steps are as follows:

[0053] S501. The overall network loss consists of ID loss and hard sample mining triplet loss. We treat each ID as a distinct category, and the ID loss yields the ID prediction for the image, similar to classification loss. The ID loss is defined as:

[0054]

[0055] Where y and p i Let represent the actual ID label and the predicted score for the i-th class, respectively. N represents the number of classes, and q... i This indicates label smoothing. ε is the label smoothing parameter, which is set to 0.1.

[0056] Furthermore, to leverage the characteristics of person re-identification and facilitate finding the person most similar to the target image from the image database, a hard sample mining triplet loss is introduced during training. Unlike traditional triplet losses, this loss selects the least similar positive sample and the most similar negative sample to form a triplet, and then calculates the triplet loss. This approach allows the network to learn better representations. The formula is as follows:

[0057]

[0058] Where M represents each selected pedestrian, and K represents the randomly selected image for each pedestrian, meaning a batch contains M×K images. (·) + Let max(0,·) represent the max function. A, P, and N represent the anchor sample, positive sample, and negative sample, respectively. max d A,P Mind represents the least similar positive sample. A,N The most similar negative sample is represented by α. α represents the boundary of the similarity difference between the positive sample and the anchor sample, and its value is set to 0.3. The total loss L is:

[0059] L = L id +L hard (9)

[0060] S502. Because the training process of a network model is a process of continuously optimizing the loss, the current loss is fed back into the network model for continuous iterative optimization to reduce the loss and thus obtain more robust features.

[0061] This invention improves the accuracy and robustness of a model by refining feature extraction methods. Addressing the issues of pedestrian misalignment and the difficulty in capturing multi-granularity features, this paper extracts and processes image features at a higher granularity, thus enhancing the model's robustness. Multi-granularity segmentation attention is used to extract relationships within features. A channel attention mechanism is incorporated to extract relationships between features. Finally, GeM Pooling is applied to generate a compact image representation, and Batch Normalization (BN) layers are used to prevent gradient vanishing or exploding, overfitting, and faster convergence. A novel and effective method for pedestrian re-identification is constructed, providing a more efficient framework for practical applications.

[0062] This invention also proposes a person re-identification method based on a multi-granularity segmentation attention network, comprising:

[0063] A network framework is constructed, which consists of a multi-granularity segmentation attention module and a channel attention module.

[0064] The given pedestrian image is input into the network to extract features, and the obtained features are then processed by a multi-granularity segmentation attention module.

[0065] The obtained features are processed through a channel attention mechanism;

[0066] The obtained features are processed through GeM Pooling and then through a BN layer;

[0067] Finally, the obtained features are used to calculate the ID loss and the hard sample mining triplet loss.

[0068] The present invention also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0069] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A person re-identification method based on a multi-granularity segmentation attention network, characterized in that, The method is executed by a computer and includes the following steps: S1. Construct the network framework, which consists of a multi-granularity segmentation attention module and a channel attention module; S2. Input the given pedestrian image into the network to extract features. The obtained features are then processed by a multi-granularity segmentation attention module. The specific implementation process of S2 is as follows: S201. The backbone network used in the feature extraction part is ResNet-50 with five stages. The convolution stride of the 5th stage is changed from 2 to 1 to retain more image details. ResNet-50 has appeared in many classic network architectures and has been widely recognized and proven. Therefore, ResNet-50 is used to extract features from pedestrian images. S202. Given a person image, we first extract feature maps using a backbone network. Where C represents the channel dimension, H represents the height, and W represents the width; the features go through various stages, and then through a multi-granularity segmentation attention module. This module is mainly designed to capture multi-granularity features and improve the internal correlation of channel-level features. We can capture multi-granularity features through the following multi-granularity segmentation attention module: First, the input features are divided into two parts along the channel axis. The number of feature channels is defined as C, so the number of channels in each part is C / 2, as shown in the following formula: (1) in This indicates that the input features are divided into two features along the channel, representing the result of the first segmentation; After feature segmentation, we use sub-channel attention to learn discriminative cues for each feature tensor based on the channel-separated features. We then pass the segmented features through sub-channel attention. To improve the internal correlation of channel-level features, they are then merged to obtain attention feature A with the same size as the input features. The formula is: (2) Where * represents element-wise product, the attention maps obtained after segmentation are merged by summation, and the merged attention map represents the salient region in the feature map; As the network continues to extract features, the feature dimension becomes very large in later stages, reaching 2048 dimensions. Even after one segmentation, the number of channels in each group is still very high. Therefore, we employ a second segmentation approach to achieve more granular feature extraction, using the features obtained after the first segmentation. For example, its formula is as follows: (3) express The segmentation result is further segmented, and the segmented features are then processed through sub-channel attention to obtain the re-segmentation attention, the formula of which is as follows: (4) Using the above formulas (3) and (4), we can obtain The features that are first segmented and then merged through attention are used as the first segmentation feature in formula (1). Features; similar It is also like The same feature acquisition process is used. The first segmentation yields coarser-grained features, and the second segmentation yields finer-grained features. Through multi-granularity segmentation attention processing, multi-granularity features are captured, thereby improving the internal correlation of channel-level features. S3. The obtained features are processed through a channel attention mechanism; S4. The obtained features are processed through GeM Pooling and then through a BN layer; S5. Finally, calculate the ID loss and hard sample mining triplet loss based on the obtained features.

2. The person re-identification method based on a multi-granularity segmentation attention network as described in claim 1, characterized in that, The specific implementation process of S1 is as follows: S101. Construct a framework consisting of a multi-granularity segmentation attention module and a channel attention mechanism; S102. The extracted features are used for classification.

3. The person re-identification method based on a multi-granularity segmentation attention network as described in claim 1, characterized in that, The specific implementation process of S3 is as follows: S301, Channel attention primarily focuses on the relationships between features; S302. We introduce a channel attention module after the multi-granularity segmentation attention at each stage of the backbone network. Here, we directly use the SE module; the input size is... The feature map is first subjected to global average pooling to obtain... Size characteristics; Then, dimensionality reduction is performed using a fully connected layer. Its feature size becomes Where r is a scaling factor used to control the number of channels, then ReLU activation is performed, followed by a fully connected layer and a sigmoid layer to obtain channel weights. These channel weights are applied to the original input features, resulting in features with different weights for each channel. The SE module is a channel attention module, which can be seen as adaptively learning a weight for the feature map on each channel. Given a feature map X, the channel attention A is defined as: (5) and This represents two fully connected operations. The average pooling layer applies channel attention A to the original feature map X through channel multiplication. In addition, the attention part in our multi-granularity segmentation module also uses this channel attention, replacing the original feature map X with the segmented multi-granularity features to learn the local channel attention after segmentation.

4. The person re-identification method based on a multi-granularity segmentation attention network as described in claim 1, characterized in that, The specific implementation process of S4 is as follows: S401, the features of stage 5 undergo a generalized average pooling (GeM Pooling) to become a vector feature, which is input into the hard sample mining triplet loss during the training phase for multi-granularity instance retrieval. Since widely used max pooling or average pooling cannot capture discriminative features specific to a particular domain, we employ a learnable pooling layer called generalized average pooling. This pooling layer takes the original feature map X as input and produces a vector f as the output of the pooling process. Its formula is: (6) in It is a pooling hyperparameter that can be learned during backpropagation. When it approximates max pooling, when The time approximates average pooling, and the feature vector is ultimately composed of a single value from each feature map, i.e., generalized average activation, with a dimension equal to K. In many popular networks, this is equal to 256, 512, or 2048, making it a compact image representation. In the network proposed in this invention, this value is 2048. S402. The obtained features are processed through a BN layer, i.e., batch normalization, to solve the problem of changes in the distribution of intermediate layer data during training, so as to prevent gradient vanishing or exploding, prevent overfitting, and speed up convergence.

5. The person re-identification method based on a multi-granularity segmentation attention network as described in claim 1, characterized in that, The specific implementation process of S5 is as follows: S501. The overall network loss consists of ID loss and hard sample mining triplet loss. We treat each ID as a different category. ID loss can obtain the ID prediction of the image, which is similar to classification loss. ID loss is defined as: (7) Where y and p i Let q represent the actual ID label and the predicted score for the i-th class, respectively, where N represents the number of classes and q represents the predicted score for the i-th class. i Indicates a smooth label. The label smoothing parameter is set to 0.1; Furthermore, to leverage the characteristics of person re-identification and facilitate finding the person most similar to the target image from the image database, a hard sample mining triplet loss is introduced during training. Unlike the traditional triplet loss, this loss selects the least similar positive sample and the most similar negative sample to form a triplet, and calculates the triplet loss. This method allows the network to learn a better representation, and its formula is as follows: (8) Where M represents each selected pedestrian, and K represents the randomly selected image for each pedestrian, meaning a batch contains M×K images. express function A, P, and N represent anchor samples, positive samples, and negative samples, respectively. Represents the least similar positive sample. Indicates the most similar negative sample. Let L represent the boundary of the difference between the positive sample and the anchor sample, and set its value to 0.

3. The total loss L is: (9) S502. Because the training process of a network model is a process of continuously optimizing the loss, the current loss is fed back into the network model for continuous iterative optimization to reduce the loss and thus obtain more robust features.

Images