A fully supervised occluded person re-identification method based on deep learning

Abstract

The present invention discloses a fully supervised occluded pedestrian re-identification method based on deep learning. Different cameras are used to collect images, and pedestrian labels and corresponding camera labels in the images are annotated to obtain a training set. A multi-branch pedestrian re-identification network including a global branch, an attention branch and a channel branch is established, and the trained multi-branch pedestrian re-identification network is used to re-identify the image to be tested; the global branch is responsible for extracting holistic features, and the attention branch selects cluster centers for attention construction by limiting the correlation between channel clusters to avoid the network from mistakenly focusing on the occluded area; the channel branch dynamically segments the feature map to avoid errors caused by spatial segmentation. The method of the present invention integrates the prediction results of the three branches to maintain a certain robustness to occlusion. There is no need to additionally mark and capture the occlusion. Only an image-level label data set is required to achieve good pedestrian matching accuracy in occluded scenes.

Description

Technical Field

[0001] The present invention relates to a cross-camera pedestrian image retrieval method in the field of computer vision technology, and in particular to a pedestrian re-identification method based on an attention mechanism. Background Art

[0002] Person re-identification, also known as "pedestrian identification," involves taking a given pedestrian image and obtaining its remaining images across multiple cameras. This involves finding images of the same person captured by other cameras. Due to cross-camera constraints, the varying lighting and angles of the images, as well as the ever-changing backgrounds and poses of pedestrians, pose significant challenges to the recognition accuracy of convolutional neural networks (CNNs), attracting a large number of researchers to the study of person re-identification.

[0003] Due to the complexity of real-world scenarios, cameras may not be able to capture complete images of pedestrians. The lack of features makes it more difficult for neural networks to match the correct images than ideally. Therefore, the occlusion problem is also one of the difficulties in the current pedestrian re-identification task.

[0004] The earliest use of convolutional neural networks for occluded person re-identification was in Lingxiao He et al.'s paper, "Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-free Approach," published in the IEEE Conference on Computer Vision and Pattern Recognition. Existing methods are often extensions of Jiaxu Miao et al.'s paper, "Pose-Guided Feature Alignment for Occluded Person Re-Identification," also published in the IEEE International Conference on Computer Vision.

[0005] Most existing methods rely on additional posture estimation modules to locate pedestrian parts and avoid the impact of occlusion. However, the posture estimation module requires additional training and its performance cannot be guaranteed. How to effectively immunize against the impact of occlusion while avoiding the use of the posture estimation module is still a problem worth exploring. Summary of the Invention

[0006] In order to solve the problems existing in the re-identification of occluded pedestrians, the present invention provides an occluded pedestrian re-identification method based on deep learning. On the basis of the channel attention module, a high-order salient attention module is designed by limiting the interaction range of the feature map channels to increase the diversity of attention coverage. A feature multi-splitting module is designed to process features using parameters of different sparsity. The combination of the two can effectively achieve the goal of reducing occlusion confusion, so that the features extracted by the network under occlusion conditions only contain pedestrian information.

[0007] The present invention is a method for pedestrian re-identification in occlusion scenarios with only image-level category annotations.

[0008] The technical solutions of the present invention are as follows:

[0009] (1) Use different cameras to collect images of the occluded target scene for pedestrian re-identification, and mark the pedestrian labels and corresponding camera labels in the images to obtain the training set;

[0010] (2) Establish a multi-branch person re-identification network and train it using the training set in step (1);

[0011] (3) After the training is completed, the trained multi-branch person re-identification network is used to re-identify the image to be tested.

[0012] In step (1), when labeling, there is no need to mark whether there is occlusion or not. It is only necessary to mark the pedestrian label Y and the camera label c corresponding to the image.

[0013] In the specific implementation, the image is divided into multiple parts according to the completeness of the pedestrians, and the pedestrians at the back and away from the camera are labeled as pedestrians.

[0014] In step (2), the multi-branch person re-identification network mainly consists of an overall feature extraction network and three parallel branches connected after the feature extraction network, namely the global branch, the attention branch and the channel branch;

[0015] The feature extraction network is formed by modifying the OSNet model. The OSNet model is split at the output of the third bottleneck block (Bottleneck) and then divided into two parts: the front part and the back part. The front part serves as the feature extraction part shared by all branches, and the back part is copied into two to form three back parts in total. The input ends of the three back parts are connected in parallel to the output end of the front part. The three back parts do not share weights. The output ends of the three back parts are connected to the global branch, attention branch, and channel branch respectively.

[0016] The image is first input into the front part of the feature extraction network and then into the three back parts respectively to obtain the global feature map FG, attention feature map F, and channel feature map FC. The three feature maps are respectively input into the global branch, attention branch, and channel branch to obtain global features, attention features, and channel features.

[0017] The global branch primarily consists of multi-layer modules, each consisting of a global average pooling layer, a batch normalization layer, and a fully connected layer. In the global branch, feature encoding is performed as follows: the global feature map FG is processed through a global average pooling layer to obtain the pre-normalized feature fg, which is then processed through a normalization layer BN to obtain the normalized feature fg_bn, and finally through a fully connected layer to obtain the classification feature fg_c as the global feature. The global feature map FG is used for feature encoding and training.

[0018] The channel branch is mainly composed of a feature multi-slicing module, a splicing layer, a convolution layer, and a multi-layer module in sequence. The multi-layer modules are composed of a global average pooling layer, a batch normalization layer, and a fully connected layer connected in sequence; the channel feature map FC is first input into the feature multi-slicing module to obtain multiple split feature maps FC, and then the multiple split feature maps FC are spliced. After the splicing result is convolved, it is processed by the global average pooling layer, the batch normalization layer, and the fully connected layer in sequence to obtain the channel features; in the feature multi-slicing module, the channel feature map FC is divided into n feature sub-maps according to the multi-slicing method from the channel dimension, and then the multi-layer transfer convolution operation of the group is performed to obtain the split feature map FP corresponding to the n feature sub-maps merged into a unique group.

[0019] The feature multi-splitting module is as follows: for the channel feature map FC, two splits are performed along the channel dimension. The first split is to divide the entire feature map into h feature sub-maps equally along the channel dimension. The second split is to divide all j feature sub-maps obtained from the previous split into two equal halves along the channel dimension to obtain feature sub-maps, and a total of h+j=n feature sub-maps are obtained. The value of h is set according to the following formula:

[0020] The feature multi-slicing module includes a total of D layers of convolutional unit layers, and the first convolutional unit layer contains E l A number of parallel independent convolution operations;

[0021] S1. After segmentation, each feature subgraph is grouped separately, and one feature subgraph is added to one group;

[0022] S2. Sort the groups according to the channel dimension of the feature subgraph from small to large, and set the convolution processing parameter T(S) for each group according to the sorting. l ,l),l=D,…,2,1,S l =1, 2, ..., El , T(S l , l) indicates that each feature subgraph in the group is convolutional layer S l Convolution operation is performed:

[0023] D and E l The value of is calculated as follows:

[0024]

[0025] l=D,…,2,1

[0026]

[0027] in, Represents rounding down, l decreases from D to 1;

[0028] In S2, initially, all feature subgraphs in all groups are set to be processed by the convolution operation of the lth convolution unit layer. l Set the convolution processing parameters T(S l , D).

[0029] S3, firstly, the feature subgraphs in each group are subjected to the convolution operations in the D-th convolution unit layer of the feature multi-slicing module according to the settings of the above-mentioned convolution processing parameters;

[0030] S4. Sort the groups obtained after S3 in ascending order according to the channel dimension, and then merge them into a group of two adjacent groups. The last group with an odd number remaining during the merging is not merged, and the groups are assigned their own weights during the merging. Then merge; expressed as:

[0031]

[0032] S5. Return to step S2 and pass it to the next l-1th convolutional unit layer for processing. Repeat steps S2-S4 until all feature sub-graphs are in one group, and obtain a unique group T(1, 1). The group contains n feature graphs with the same channel dimension, which are used as the feature graph FC after segmentation.

[0033] The Sth convolutional unit layer of the lth layer l Weights of the convolution operation All are pre-set.

[0034] The attention branch is mainly composed of a channel attention module, a high-order saliency attention module, and two multi-layer modules. The multi-layer modules are composed of a global average pooling layer, a batch normalization layer, and a fully connected layer connected in sequence; the input end of the high-order saliency attention module is connected to the middle of the channel attention module, and the outputs of the channel attention module and the high-order saliency attention module are processed by their respective multi-layer modules and then output; the attention feature map F is input into the channel attention module, and the intermediate quantity in the channel attention module is input into the high-order saliency attention module. The first-order feature map FA and the second-order feature map FS are output by the channel attention module and the high-order saliency attention module respectively, and the first-order feature map FA and the second-order feature map FS are processed by the global average pooling layer, batch normalization layer, and fully connected layer in their respective multi-layer modules to obtain the attention features.

[0035] The channel attention module and high-order saliency attention module in the attention branch are as follows:

[0036] In the channel attention module, the attention feature map F is multiplied by its own transposed matrix to obtain the Gram matrix G. For each row in the Gram matrix G, the channel attention map GS is calculated according to the following formula. Then, the first-order feature map FA is obtained based on the channel attention map GS. The formula is expressed as:

[0037]

[0038]

[0039] Among them, G ij Represents the element in the i-th row and j-th column of the Gram matrix G, GS ij Represents the element in row i and column j of the attention graph GS, F i Represents the sub-feature map corresponding to the i-th channel of the attention feature map F, FA i Represents the sub-feature map corresponding to the i-th channel in the first-order feature map FA, exp() represents the exponential operation, j represents the ordinate of the corresponding feature map, C represents the number of channels of the feature map, and γ1 represents the proportional coefficient, which is a parameter learned by the network;

[0040] In the high-order salient attention module, according to the channel attention map GS obtained by the channel attention module, the vertical coordinate k corresponding to the maximum value pixel point in each row of the channel attention map GS is calculated. i , count the vertical coordinates k of all rows i And delete the elements with the same value to get the vertical coordinate k i The array k is constructed; according to the Gram matrix G, an all-zero matrix Z of the same size is established, and the all-zero matrix Z is assigned according to the array k according to the following formula:

[0041]

[0042] Among them, i represents the horizontal coordinate, j represents the vertical coordinate, and Z ij represents the element in row i and column j of the all-zero matrix Z;

[0043] The obtained all-zero matrix Z is dot-multiplied with the attention map GS to obtain the dot product map GZ, and then the dot product map GZ is used to obtain the high-order salient attention map M through the following formula. Then, the high-order salient attention map M is processed to obtain the second-order feature map FS. The formula is expressed as:

[0044]

[0045]

[0046] Among them, M ij Represents the element in the i-th row and j-th column of the high-order salient attention map M, γ2 is a hyperparameter for adjusting the proportion of high-order salient attention, FS i Represents the feature map corresponding to the i-th channel of the second-order feature map FS.

[0047] In the global branch, attention branch and channel branch, the cross entropy loss function and triplet loss function are used to generate losses. At the same time, the orthogonal singular value decomposition regularization operation is performed in the attention branch to generate regularized loss. The cross entropy loss and triplet loss of the three branches and the regularized loss of the attention branch are added as the total loss, and then the Adam optimization algorithm is used for optimization training.

[0048] In the global branch, the cross entropy loss function and triple loss function are established as follows:

[0049] Under the supervision of the pedestrian label Y, the cross entropy loss function is calculated based on the classification feature fg_c; at the same time, the triplet loss function is calculated based on the normalized feature fg obtained by the global average pooling layer.

[0050] Under the supervision of image-level labels Y, the cross entropy loss function is as follows:

[0051]

[0052] Where p(k) is the probability that the classification result fg_c belongs to pedestrian k, calculated by the Softmax operation, q(k) is the distribution of one-hot codes generated by the pedestrian ID annotations in the image, P is the total number of pedestrians, and for the label Y of the image X, q(k) = 1 if and only if k = y, and q(k) = 0 otherwise.

[0053] The triplet loss function is as follows:

[0054]

[0055] Where m is a manually set threshold, [·]+ represents max(0, ·), and D(·) represents the Euclidean distance. Represents the i-th pedestrian corresponding to the feature with attribute x. When x = a, it means that the feature comes from the anchor image. When x = p, it means that the feature comes from the positive sample image. When x = n, it means that the feature comes from the negative sample image. For each batch of images read from the dataset, it will contain P pedestrians, each pedestrian contains K images. The anchor image is randomly selected from this batch of images. After the anchor image is confirmed, the positive sample images can only be selected from images of the same pedestrian as the anchor image, and the negative sample images can only be selected from images of different pedestrians than the anchor image. The selected positive sample image must be the farthest from the anchor image among all possible positive sample images, and the selected negative sample image must be the closest to the anchor image among all possible negative sample images.

[0056] In the channel branch, the feature encoding generation method and loss function constraints consistent with the global branch are adopted for the channel feature map FP, that is, the features obtained by the global average pooling layer are used to calculate the triple loss function, and the features obtained by the fully connected layer are used to calculate the cross entropy loss function.

[0057] In the attention branch, the first-order feature map FA and the second-order feature map FS adopt the same feature encoding and loss function constraints as the global branch, that is, the features obtained by the global average pooling layer are used to calculate the triple loss function, and the features obtained by the fully connected layer are used to calculate the cross entropy loss function.

[0058] In (2), the training process adopts the Adam optimization algorithm and uses cross entropy loss and triplet loss for supervision.

[0059] The OSNet model is pre-imported into ImageNet for pre-training to obtain parameters.

[0060] In step (3), the image to be tested is re-identified as follows: for the image to be tested, the image to be tested is input into the trained multi-branch pedestrian re-identification network and multiple feature codes are output, the multiple feature codes are spliced ​​as the final pedestrian feature code, the pedestrian feature code and each picture in the known picture database are pre-input into the trained multi-branch pedestrian re-identification network and then compared with the pedestrian feature codes obtained by the same splicing, the similarity between the pedestrian feature code of the image to be tested and each picture in the picture database is obtained, and the pictures in the picture database are sorted from large to small according to the similarity and presented.

[0061] The method first acquires and annotates images to obtain a training set, then builds a multi-branch network consisting of a global branch, an attention branch, and a channel branch. The global branch extracts holistic features, while the attention branch selects cluster centers for attention construction by limiting the correlation between channel clusters, preventing the network from incorrectly focusing on occluded areas. The channel branch dynamically segments the feature map, avoiding errors caused by spatial segmentation. The constructed multi-branch network is then trained and processed on the test image. The prediction results of the three branches are combined to maintain a certain degree of robustness to occlusion.

[0062] The present invention has the following advantages:

[0063] 1. This paper uses a high-order attention mechanism to capture salient feature areas in the image and integrates the salient area features with the global features, thus achieving high robustness to occlusion.

[0064] 2. The present invention does not need to segment the pedestrian feature map from the spatial dimension to recognize detailed features, but instead uses targeted segmentation from the channel dimension, thereby avoiding recognition errors caused by changes in pedestrian posture and position.

[0065] 3. The training of the present invention does not require the assistance of an additional posture estimator, which reduces the data collection burden and model training burden brought by the separate training of additional modules.

[0066] The method of the present invention does not require additional annotation and capture of occlusions, and only requires an image-level label dataset to achieve good pedestrian matching accuracy in occlusion scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of the application process of the present invention.

[0068] Figure 2 Schematic diagram of some re-identification results of the present invention. DETAILED DESCRIPTION

[0069] The present invention will be further described below with reference to the accompanying drawings.

[0070] The embodiments of the present invention and their implementation process are:

[0071] S01: Obtain images in the scene where pedestrian re-identification is required, and manually annotate all images at the image level to form a training set.

[0072] In this example, the person re-identification dataset selected consists of images of people captured by multiple cameras placed at different locations within a certain area. All images are RGB images, and the number of cameras used is at least two. All images of a particular person are assigned the same label; different labels are required for different persons. All images captured by a given camera are assigned the same camera label. All images in the training set are uniformly resized to 384×128 using bilinear interpolation.

[0073] S02, such as Figure 1 As shown in the figure, a multi-branch person re-identification network based on an attention mechanism was constructed. OSNet was selected as the backbone network model, and the parameters pre-trained on ImageNet were imported. The last fully connected layer of the original OSNet was discarded. After setting up the OSNet backbone network model, a three-branch network model was constructed. Kaiming initialization was used for batch normalization layers and fully connected layers that did not perform classification. Fully connected layers that performed classification were initialized with a mean of 0 and a variance of 0.001.

[0074] S03, during training, the training set images are input into the multi-branch person re-identification network in batches. Each batch of data contains 8 pedestrians, each with 8 images, for a total of 64 images per batch. The input images are calculated by the backbone network model to obtain feature maps, which are respectively input into the subsequent global branch, attention branch, and channel branch. The channel dimensions of the input and output of the three branches are all 512, and the split fraction n of the channel branch is set to 3. After obtaining the feature vectors of each output, they are processed through the fully connected layer to obtain the corresponding prediction results. The established person re-identification model is trained using the Adam algorithm, and regularization loss, cross entropy loss function, and triplet loss function are used for constraints. One round of training for all training set images is considered as one round, and a total of 120 rounds of training are conducted. The learning rate is changed using the WarmUp strategy. From the first round to the end of the 10th round, it linearly increases from 0.000035 to 0.00035, changes to 0.000035 at the beginning of the 50th round, and changes to 0.0000035 at the beginning of the 90th round. Finally, until convergence, the parameters of the pedestrian re-identification model are saved.

[0075] S04: Use the trained person re-identification model to infer the unknown pedestrian image to be retrieved, and obtain the encoding for pedestrian feature extraction in this scene.

[0076] To verify the effectiveness of each module of the present invention, the performance of each module was evaluated on the publicly available Occluded-Duke dataset, a standard occlusion dataset derived from the DukeMTMC-reID dataset. The Occluded-Duke dataset features 9%, 100%, and 10% occlusion images in the training, search, and query sets, respectively. The training set contains 15,618 images of 702 people; the query set contains 17,661 images of 1,110 people, with no overlap between the 702 people in the query set and the 1,110 people in the training set; and the query set contains 2,210 images of the 519 people in the query set. All images in the search set are occluded, and an image is considered occluded if it contains multiple people or is obscured by trees or cars. Among all occlusion datasets, this dataset is larger and more comprehensive, making it the most suitable for evaluating the performance of models dealing with occlusion.

[0077] This paper mainly uses two commonly used evaluation indicators in pedestrian re-identification: Rank-1 and mAP to evaluate the re-identification accuracy. The performance verification results of each module are shown in Table 1:

[0078] Table 1

[0079]

[0080] Among them, ResNet50 and OSNet represent network structures with different backbone network models and only global branches. The attention branch represents the branch containing the channel attention module and the high-order salient attention module. SVDO represents the application of regularization loss to the high-order salient attention module. The channel branch represents the branch containing the feature multi-slicing module. As can be observed from the table, OSNet selected by the present invention is more suitable for solving the occlusion problem than ResNet50. After adding the attention branch to OSNet, the performance did not change significantly because the attention branch at this time did not learn the correct content. After selecting the regularization loss, the Rank-1 performance of the attention branch increased by 6%, and the mAP performance increased by 1.7%, which is a significant increase, fully demonstrating the effectiveness of the attention branch. After adding the channel branch to OSNet, the performance improvement is even more significant, indicating that the feature multi-slicing module of the channel branch is very effective for solving the occlusion problem. After combining all branches, the performance is further improved, thereby verifying the effectiveness of the multi-branch pedestrian re-identification network proposed by the present invention.

[0081] To verify the effectiveness of the proposed method, we compared it with other occluded person re-identification methods on the publicly available Occluded-Duke dataset:

[0082] Table 2

[0083]

[0084] The first part of Table 2 represents methods that rely on additional pose estimators or occlusion existence information, the second part represents methods without additional information intervention, and ACPNet in the third part refers to the method of the present invention. It can be observed from the table that the methods in the first part generally perform better than the methods in the second part due to the application of additional information, while the method of the present invention successfully surpasses the performance of the methods in the first part without the intervention of additional information. This shows that even without the intervention of additional information, the method of the present invention is still at a better level among a series of methods in terms of the two indicators of Rank-1 and mAP, which is the advantage of the method of the present invention.

[0085] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any changes and modifications made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

[0086] Figure 2 The Ranking Activation Map proposed in the paper "Towards rich feature discovery with class activation mapsaugmentation for person re-identification, CVPR 2019" is used for visualization. There are three rows in the figure, each row includes 9 pictures. The pictures in the first column represent the pictures used for re-identification, and the second to ninth columns represent the eight pictures most similar to the pictures in the first column matched by the multi-branch person re-identification network, with the similarity decreasing from left to right. Each picture used for re-identification has different degrees of occlusion, and the matched pictures are also partially occluded. It can be seen from the figure that regardless of whether the picture is occluded or not, the multi-branch person re-identification network can achieve completely correct matching, which further illustrates the effectiveness of the present invention.

Claims

1. A fully supervised occluded person re-identification method based on deep learning, characterized by: The method comprises the following steps: (1) Use different cameras to collect images of the occluded target scene for pedestrian re-identification, and mark the pedestrian labels and corresponding camera labels in the images to obtain the training set; (2) Establish a multi-branch person re-identification network consisting of a global branch, an attention branch, and a channel branch, and train it using the training set in step (1); The channel branch is composed of a feature multi-slicing module, a splicing layer, a convolution layer, and a multi-layer module in sequence. The multi-layer module is composed of a global average pooling layer, a batch normalization layer, and a fully connected layer connected in sequence. The channel feature map FC is first input into the feature multi-slicing module to obtain multiple split feature maps FP, and then the multiple split feature maps FP are spliced. After the splicing result is convolved, it is processed in sequence by the global average pooling layer, the batch normalization layer, and the fully connected layer to obtain the channel feature. In the feature multi-segmentation module, the channel feature map FC is divided into n feature sub-maps according to the multi-segmentation method from the channel dimension, and then the segmented feature map FP corresponding to the n feature sub-maps is obtained through the multi-layer transfer convolution operation of the group; The attention branch consists of a channel attention module, a high-order saliency attention module, and two multi-layer modules. The multi-layer modules are composed of a global average pooling layer, a batch normalization layer, and a fully connected layer connected in sequence; the input end of the high-order saliency attention module is connected to the middle of the channel attention module, and the outputs of the channel attention module and the high-order saliency attention module are respectively processed by their respective multi-layer modules and output; the attention feature map F is input into the channel attention module, and the intermediate quantity in the channel attention module is then input into the high-order saliency attention module. The first-order feature map FA and the second-order feature map FS are output by the channel attention module and the high-order saliency attention module respectively, and the first-order feature map FA and the second-order feature map FS are processed by the global average pooling layer, batch normalization layer, and fully connected layer in their respective multi-layer modules to obtain attention features; (3) Use the trained multi-branch person re-identification network to re-identify the test image.

2. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: In step (1), when labeling, it is only necessary to label the pedestrian label Y and the camera label c corresponding to the image.

3. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: In step (2), the multi-branch person re-identification network consists of an overall feature extraction network and three parallel branches connected after the feature extraction network, wherein the three branches are a global branch, an attention branch, and a channel branch; The feature extraction network is formed by modifying the OSNet model. The OSNet model is split at the output of the third bottleneck block and then divided into two parts: the front part and the back part. The back part is copied into two to form three back parts in total. The input ends of the three back parts are connected in parallel to the output end of the front part, and the output ends of the three back parts are connected to the global branch, attention branch, and channel branch respectively. The image is first input into the front part of the feature extraction network and then into the three back parts respectively to obtain the global feature map FG, attention feature map F, and channel feature map FC. The three feature maps are respectively input into the global branch, attention branch, and channel branch to obtain global features, attention features, and channel features.

4. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: The global branch is composed of multi-layer modules, each of which is composed of a global average pooling layer, a batch normalization layer and a fully connected layer connected in sequence; in the global branch, the following feature encoding is performed: the global feature map FG is processed by the global average pooling layer to obtain the pre-normalization feature fg, then the normalization feature fg_bn is obtained by the normalization layer BN, and finally the classification feature fg_c is obtained as the global feature by the fully connected layer.

5. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: The feature multi-splitting module is specifically as follows: for the channel feature map FC, two splits are performed along the channel dimension. The first split is to equally split the entire feature map into h feature sub-maps along the channel dimension. The second split is to equally split the j feature sub-maps among all the feature sub-maps obtained in the previous split again along the channel dimension to obtain feature sub-maps, and a total of h+j=n feature sub-maps are obtained. The feature multi-slicing module includes a total of D layers of convolutional unit layers, and the first convolutional unit layer contains E l The number of convolution operations; S1. After segmentation, each feature subgraph is grouped separately, and one feature subgraph is added to one group; S2. Sort the groups according to the channel dimension of the feature subgraph from small to large, and set the convolution processing parameter T(S) for each group according to the sorting. l ,l),l=D,…,2,1,S l =1,2,…,E l , T(S l ,l) indicates that each feature subgraph in the group is convolutional layer S l Convolution operation is performed: D and E l The value of is calculated as follows: l＝D,…,2,1 in, Represents rounding down, l decreases from D to 1; S3, firstly, the feature subgraphs in each group are subjected to the convolution operations in the D-th convolution unit layer of the feature multi-slicing module according to the settings of the above-mentioned convolution processing parameters; S4: Sort the groups obtained after S3 processing from small to large according to the channel dimension, and then merge each two adjacent groups into a group. When merging, the groups are assigned their own weights. Then merge; S5. Return to step S2 and pass it to the next l-1th convolutional unit layer for processing. Repeat steps S2-S4 until all feature sub-graphs are in one group, and obtain a unique group T(1,1). The group contains n feature graphs with the same channel dimension, which are used as the feature graph FP after segmentation.

6. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: The channel attention module and high-order saliency attention module in the attention branch are as follows: In the channel attention module, the attention feature map F is multiplied by its own transposed matrix to obtain the Gram matrix G. For each row in the Gram matrix G, the channel attention map GS is calculated according to the following formula. Then, the first-order feature map FA is obtained based on the channel attention map GS. The formula is expressed as: Among them, G ij Represents the element in the i-th row and j-th column of the Gram matrix G, GS ij Represents the element in row i and column j of the attention graph GS, F i Represents the sub-feature map corresponding to the i-th row of the attention feature map F, FA i represents the sub-feature map corresponding to the i-th channel in the first-order feature map FA, exp( ) represents the exponential operation, j represents the j-th column of the corresponding feature map, C represents the total number of columns of the feature map, and γ1 represents the scale factor, which is a parameter learned by the network; In the high-order salient attention module, according to the channel attention map GS obtained by the channel attention module, the vertical coordinate k corresponding to the maximum value pixel point in the i-th row of the channel attention map GS is calculated. i , count the vertical coordinates k of all rows i And delete the elements with the same value to form an array k; according to the Gram matrix G, create an all-zero matrix Z of the same size, and assign the all-zero matrix Z according to the array k according to the following formula: Among them, i represents the i-th row, j represents the j-th column, and Z ij represents the element in row i and column j of the all-zero matrix Z; The obtained all-zero matrix Z is dot-multiplied with the attention map GS to obtain the dot product map GZ, and then the dot product map GZ is used to obtain the high-order salient attention map M through the following formula. Then, the high-order salient attention map M is processed to obtain the second-order feature map FS. The formula is expressed as: Among them, M ij represents the element in the i-th row and j-th column of the high-order salient attention map M, γ2 is a hyperparameter for adjusting the proportion of high-order salient attention, GZ ij Represents the element in the i-th row and j-th column of the dot product graph GZ, F ij Represents the element in the uth row and jth column of the attention feature map F, FS i Represents the feature map of the corresponding part of the i-th row of the second-order feature map FS.

7. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: In the global branch, attention branch and channel branch, cross entropy loss function and triplet loss function are used to generate losses. At the same time, orthogonal singular value decomposition regularization operation is performed in the attention branch to generate regularized loss. The cross entropy loss and triplet loss of the three branches and the regularized loss of the attention branch are added as the total loss, and then the Adam optimization algorithm is used for optimization training.

8. The method for fully supervised person re-identification under occlusion based on deep learning according to claim 1, characterized in that: In the step (3), the image to be tested is re-identified as follows: for the image to be tested, the image to be tested is input into the trained multi-branch pedestrian re-identification network and multiple feature codes are output, the multiple feature codes are spliced ​​as pedestrian feature codes, the pedestrian feature codes are pre-input into the trained multi-branch pedestrian re-identification network and then compared with the pedestrian feature codes obtained by splicing each picture in a known picture database, the similarity between the pedestrian feature code of the image to be tested and each picture in the picture database is obtained, and the pictures in the picture database are sorted from large to small according to the similarity and presented.

Images