A pedestrian re-identification method, device, equipment and medium

Abstract

The application discloses a pedestrian re-identification method and device, equipment and medium, and relates to the technical field of pedestrian re-identification. The method comprises the following steps: obtaining a to-be-identified global feature map and an original global feature map based on a to-be-identified pedestrian image and a plurality of original pedestrian images; obtaining a weighting weight coefficient corresponding to the to-be-identified global feature map and the original global feature map respectively, obtaining a weighted to-be-identified global feature map and a weighted original global feature map based on the corresponding weighting weight coefficients; obtaining pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively and pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively, obtaining pedestrian identification features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features; and extracting a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification features to realize pedestrian re-identification. The application improves the accuracy of pedestrian re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pedestrian re-identification, and in particular to a pedestrian re-identification method, device, equipment and medium. BACKGROUND

[0002] Person Re-identification (ReID) is a technology that uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence. It is widely considered a sub-problem of image retrieval, that is, given a monitoring pedestrian image, the image of the pedestrian under cross-device is retrieved.

[0003] The existing supervised pedestrian re-identification technology is similar to the conventional supervised image recognition model feature extraction and feature comparison method, that is, a strong feature extraction model is trained, the original pedestrian image is subjected to feature extraction using the feature extraction model, an abstract pedestrian description is obtained by transformation, and then a distance metric function is learned to determine the similarity of the pedestrian features, thereby obtaining a preliminary ranking list, and finally the original ranking list is optimized according to the relevant ranking, and finally the final pedestrian recognition result is obtained; specifically, the existing method uses a conventional CNN (Convolutional Neural Network) image classification model, such as a ResNet (residual network) series model, as the model framework backbone, and after introducing a channel attention operation in the Bottleneck (bottleneck layer), different weights are assigned to different channels, thereby learning the correlation between the channels; in order to make the network focus on local features, the global features are divided into H (high) * W (wide) block local blocks, each local block is subjected to a channel attention operation, and then the obtained weights are subjected to mean value and expansion processing to obtain a two-dimensional matrix of H*W, and finally multiplied with the global features to obtain new features; wherein, Bottleneck is a basic structural unit in a neural network, which uses a 1*1 neural network for dimension reduction processing, and is commonly used in ResNet.

[0004] However, in the prior art, the global feature is divided into W*H block local features, which are processed by a channel attention module. The channel attention module itself is usually composed of a fully connected layer, so it consumes more performance than a normal convolution layer. Therefore, if more channel attention modules are used, the overall model performance will be a big problem. Secondly, all local regions are processed by channel attention, and the local regions are still not targeted. The salient target other than the human body may still be focused on, so it may not be able to better focus on the key areas of the human body. In addition, the average processing of each local weight according to the channel may weaken the channel attention mechanism to learn the relevance of different channels, thereby reducing the accuracy of pedestrian re-identification. In addition, relying only on pedestrian ID information for pedestrian retrieval may not effectively identify different pedestrian features to achieve the purpose of re-identification, which will also reduce the accuracy of pedestrian re-identification.

[0005] In summary, how to improve the accuracy of pedestrian re-identification is a problem to be solved at present. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a pedestrian re-identification method, device, equipment and medium, which can improve the accuracy of pedestrian re-identification. The specific scheme is as follows:

[0007] In a first aspect, the present application discloses a pedestrian re-identification method, comprising:

[0008] obtaining a to-be-identified pedestrian image and a plurality of original pedestrian images, and obtaining a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model;

[0009] obtaining weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and weighting head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map;

[0010] obtaining pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by a pedestrian attribute branch, and obtaining pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by a target main global branch in the pedestrian re-identification model which is pre-trained based on a multi-dimensional global auxiliary branch, and then obtaining pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features;

[0011] extract a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification feature, so as to realize pedestrian re-identification.

[0012] Optionally, the acquiring of the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively comprises:

[0013] The to-be-identified global feature map and the original global feature map are compressed based on a global pooling layer, and the compressed images are excited based on a fully connected layer and an activation layer to acquire the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively.

[0014] Optionally, the weighting of the head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain the weighted to-be-identified global feature map and the weighted original global feature map comprises:

[0015] The to-be-identified global feature map and the original global feature map are divided into three equal parts in the horizontal direction to obtain the head-shoulder region images, the body region images and the lower body region images corresponding to the to-be-identified global feature map and the original global feature map respectively.

[0016] The weight coefficients and the head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively are multiplied to obtain the weighted head-shoulder region images corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively.

[0017] The weighted head-shoulder region images are spliced with the corresponding body region images and lower body region images to obtain the weighted to-be-identified global feature map and the weighted original global feature map.

[0018] Optionally, before the acquiring of the to-be-identified global feature map corresponding to the to-be-identified pedestrian image and the original global feature map corresponding to the original pedestrian image by using the pedestrian re-identification model, the method further comprises:

[0019] A training set is constructed by using a plurality of reference pedestrian pictures in a target region taken by cameras at different orientations, and the pedestrian re-identification network based on the residual network is trained by using the training set to obtain the pedestrian re-identification model.

[0020] Optionally, before the acquiring of the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map by using the target main global branch of the pedestrian re-identification model which is pre-trained based on the multi-dimensional global auxiliary branch, the method further comprises:

[0021] acquire reference global features of different dimensions corresponding to the training set by using the multi-dimensional global auxiliary branch added to the pedestrian re-identification network, and acquire target global features of a target dimension corresponding to the training set by using the original main global branch of the pedestrian re-identification network;

[0022] train the target main global branch by using the reference global features and the target global features and loss backpropagation, so as to acquire the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using the target main global branch.

[0023] Optionally, the acquiring of the pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features comprises:

[0024] The to-be-identified global feature and the pedestrian attribute feature corresponding to the weighted to-be-identified global feature map and the weighted original global feature map are spliced respectively by a feature fusion dimension reduction convolutional layer, and a dimension reduction convolution operation is performed on the spliced features to acquire the pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively.

[0025] Optionally, the extracting of the target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian recognition features to realize pedestrian re-identification comprises:

[0026] calculating a target distance between the pedestrian recognition features corresponding to the to-be-identified image and the pedestrian recognition features corresponding to all the original images;

[0027] extracting the original image corresponding to the target distance satisfying a preset condition as the target pedestrian image corresponding to the to-be-identified pedestrian image to realize pedestrian re-identification.

[0028] In a second aspect, the present application discloses a pedestrian re-identification device, comprising:

[0029] a global feature map acquisition module, configured to acquire a to-be-identified pedestrian image and a plurality of original pedestrian images, and acquire a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model;

[0030] a weighted global feature map acquisition module, configured to acquire a weighting weight coefficient corresponding to different channels of the to-be-identified global feature map and the original global feature map respectively, and perform weighting on a head-shoulder region image corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficient to obtain a weighted to-be-identified global feature map and a weighted original global feature map;

[0031] a pedestrian recognition feature acquisition module, configured to acquire a pedestrian attribute feature corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively, and acquire a pedestrian global feature corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch pre-trained based on a multi-dimensional global auxiliary branch in the pedestrian re-identification model, and then acquire a pedestrian recognition feature corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global feature and the pedestrian attribute feature;

[0032] a pedestrian re-identification module, configured to extract a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian recognition feature, so as to realize pedestrian re-identification.

[0033] In a third aspect, the present application discloses an electronic device, comprising a processor and a memory; wherein the processor implements the pedestrian re-identification method disclosed above when executing the computer program stored in the memory.

[0034] In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the pedestrian re-identification method disclosed above.

[0035] It can be seen that the application obtains a to-be-identified pedestrian image and a plurality of original pedestrian images, and obtains a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model; obtains weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and weights head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map; obtains pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by a pedestrian attribute branch, and obtains pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch obtained in advance in the pedestrian re-identification model based on a multi-dimensional global auxiliary branch, and then obtains pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features; and extracts a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian recognition features, so as to realize pedestrian re-identification. As can be seen from the above, the application weights the head-shoulder region images which are different from human bodies, improves the discrimination degree of different human bodies, and improves the accuracy of re-identification. The application extracts pedestrian global features by using the target main global branch obtained in advance based on the multi-dimensional global auxiliary branch, so that the pedestrian global features are more accurate and effective, which is conducive to improving the discrimination degree of different human bodies. The application adds pedestrian attribute features based on the pedestrian global features, which is conducive to further improving the discrimination degree of different human bodies and improving the re-identification effect and accuracy of the model. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0037] Figure 1 A flow chart of a pedestrian re-identification method provided by the present application;

[0038] Figure 2 An output head structure schematic diagram of a model training process provided by the present application;

[0039] Figure 3 An output head structure schematic diagram of a model using process provided by the present application;

[0040] Figure 4A specific pedestrian re-identification method flowchart provided for the present application;

[0041] Figure 5 A general ResNet residual unit structure schematic diagram provided for the present application;

[0042] Figure 6 A Key Area weighted residual unit structure schematic diagram provided for the present application;

[0043] Figure 7 A pedestrian re-identification device structure schematic diagram provided for the present application;

[0044] Figure 8 An electronic equipment structure diagram provided for the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0046] At present, in the prior art, the global feature is divided into W*H block local features, which are processed by a channel attention module. The channel attention module itself is generally composed of a full connection layer, so it consumes more performance than a common convolution layer. Therefore, if more channel attention modules are used, the overall model performance will be a big problem. Secondly, all local regions are processed by channel attention, and the local regions are still not targeted. The salient target other than the human body may still be focused on, so that the key areas of the human body may not be well focused. In addition, the local weight of each local feature is processed by the channel mean, which may weaken the channel attention mechanism to learn the relevance of different channels, thereby reducing the accuracy of pedestrian re-identification.

[0047] In order to overcome the above problems, the present application provides a pedestrian re-identification scheme, which can improve the accuracy of pedestrian re-identification.

[0048] Referring to Figure 1 The embodiments of the present application disclose a pedestrian re-identification method, which comprises:

[0049] Step S11: acquiring a to-be-identified pedestrian image and a plurality of original pedestrian images, and acquiring a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model.

[0050] In the embodiment of the present application, before the obtaining of the to-be-identified global feature map corresponding to the to-be-identified pedestrian image and the original global feature map corresponding to the original pedestrian image by using the pedestrian re-identification model, the method further comprises: constructing a training set by using a plurality of reference pedestrian pictures in the target area taken by cameras at different orientations, and training a pedestrian re-identification network based on a residual network by using the training set to obtain the pedestrian re-identification model. It should be pointed out that, first, the pedestrian pictures taken by cameras at different angles at each point are used as the pedestrian material data set, wherein one pedestrian corresponds to one ID, and the same ID contains pictures at different angles and directions, that is, the same pedestrian pictures at different postures. A general posture classification model can be used to roughly classify the posture of the training set pictures, so as to more clearly know the quantity distribution of the different posture data of the human body in the training material, and data enhancement is performed on the posture with a serious shortage of quantity to increase the quantity of the posture with a serious shortage of quantity to obtain the final training set. This method can appropriately alleviate the problem of unbalanced training samples, so that the difference in the amount of training data of each posture will not be too large. It should be pointed out that the data enhancement includes but is not limited to illumination augmentation, random erasing, and random crop. It should be pointed out that the ID is represented by Arabic numerals, starting from 0.

[0051] In the embodiment of the present application, the to-be-identified pedestrian image can be a query target image to be searched; and the plurality of original pedestrian images can be gallery base images.

[0052] Step S12: obtaining weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and weighting head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain weighted to-be-identified global feature map and weighted original global feature map.

[0053] In the embodiment of the present application, the obtaining of the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively comprises: performing feature compression on the to-be-identified global feature map and the original global feature map based on a global pooling layer, and performing feature excitation on the compressed images based on a fully connected layer and an activation layer to obtain the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively.

[0054] In the embodiments of the present application, the head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map are weighted based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map. It can be understood that weighting the key region of the head-shoulder region can enhance the distinction between different pedestrians, thereby improving the accuracy of pedestrian re-identification.

[0055] Step S13: obtaining pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively through the pedestrian attribute branch, and obtaining pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch in the pedestrian re-identification model which is obtained by pre-training based on a multi-dimensional global auxiliary branch, and then obtaining pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features.

[0056] In the embodiments of the present application, the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map are obtained by using the main global branch with the adjusted main network in the pedestrian re-identification model, and the pedestrian attribute features are obtained through the newly added pedestrian attribute classification branch. It should be noted that the main global branch outputs 512-dimensional global features.

[0057] It should be noted that, before obtaining the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map by using the target main global branch obtained by pre-training based on the multi-dimensional global auxiliary branch in the pedestrian re-identification model, the present application further comprises: obtaining reference global features of different dimensions corresponding to the training set by using the newly added multi-dimensional global auxiliary branch of the pedestrian re-identification network, and obtaining target global features of a target dimension corresponding to the training set by using the main global branch of the pedestrian re-identification network; based on the reference global features and the target global features, and by using loss backpropagation, adjusting the main network corresponding to the global branch and the main global branch to obtain an adjusted main network, so as to obtain the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map by using the main global branch with the adjusted main network.

[0058] It should be noted that the present application firstly uses multi-dimensional global auxiliary branch to assist the training of the main global branch, and the specific method is to add a global output branch of different dimensions after the output of the last feature map, for example, the main global branch outputs a feature of 512 dimensions, and a global branch of the same structure with a dimension of 384 can be added; the purpose is that the same pedestrian can obtain different dimensional information representations, and different information representations often also have differences in the focus of attention to pedestrians, and the multi-dimensional output branch (including the main global branch and the multi-dimensional global auxiliary branch) adjusts the main network through loss back propagation during the training process, thereby improving the representation ability of the network to pedestrians; it should be noted that there are two aspects, on the one hand, the dimension of the multi-dimensional global auxiliary branch is smaller than that of the main global branch, so as to ensure that the global main branch is dominant; on the other hand, the multi-dimensional global auxiliary branch is removed during the formal use of the model, and only the main global branch is used to extract features.

[0059] In the embodiment of the present application, the pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image are obtained based on the pedestrian global feature and the pedestrian attribute feature, comprising: the pedestrian global feature and the pedestrian attribute feature corresponding to the weighted global feature map of the to-be-identified pedestrian image and the weighted global feature map of the original pedestrian image are spliced through a feature fusion dimension reduction convolutional layer, and a dimension reduction convolutional operation is performed on the spliced features to obtain the pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image. Specifically, in the process of training the pedestrian re-identification network to obtain the pedestrian re-identification model, first, the main weight and the pedestrian ID information output branch weight are fixed, and the pedestrian attribute branch is trained using the existing main network weight; wherein fixing the main weight and the pedestrian ID information output branch weight means fixing the parameters of each layer in the neural network; secondly, after the pedestrian attribute branch converges, a feature fusion dimension reduction convolutional layer is added to splice the pedestrian main ID information feature and the pedestrian attribute feature, and then a 1*1 dimension reduction convolution is performed as output for training; the training process still adopts the two-stage training strategy of fixing and opening, and finally the dimension reduction convolution output feature is used as the final pedestrian retrieval feature (i.e. pedestrian recognition feature) of the network to improve the pedestrian retrieval effect.

[0060] It should be noted that the fixed stage means that the parameters of the main network and the parameters of the pedestrian ID information branch network are fixed and unchanged during the training process, and only the parameters of the pedestrian attribute branch network are changed; the open stage means that the model obtained based on the above fixed stage is used as a pre-trained model, and after loading, all the fixed network parameters are opened, and all parameters can be changed during the training stage.

[0061] It should be noted that, as described above, in the process of training the pedestrian re-identification model, the present application introduces two improvements in the output Head part of the network structure based on the pedestrian re-identification task to improve the generalization of the network and the pedestrian retrieval effect, that is, the multi-dimensional global auxiliary branch and the pedestrian attribute feature fusion in the output Head part, on the one hand, the network backbone combines different dimensional attention points for joint learning; on the other hand, the pedestrian attribute features can be fused for joint retrieval, thereby improving the generalization ability and retrieval effect of the model. Specifically, the network output Head structure is as shown in Figure 2 The figure includes two multi-dimensional global auxiliary branches and one backbone global branch, and the two multi-dimensional global branches are 128-dimensional and 384-dimensional respectively. It should be noted that the steps of obtaining the pedestrian attribute features corresponding to the weighted global feature map to be identified and the weighted original global feature map through the pedestrian attribute branch, and obtaining the pedestrian global features corresponding to the weighted global feature map to be identified and the weighted original global feature map using the target backbone global branch pre-trained based on the multi-dimensional global auxiliary branch in the pedestrian re-identification model do not pass through the multi-dimensional global branch, and the multi-dimensional global branch is only used in the training process, Figure 3 The pedestrian attribute branch used in the re-identification process in the present application is 64-dimensional, and the target backbone global branch is 512-dimensional.

[0062] It should be noted that the process of training the pedestrian re-identification network can be generally understood as three parts, the first part is the data input part, which includes reading data, data enhancement, etc.; the second part is the neural network, that is, the entire network structure belongs to one part; the third part is the loss function, that is, the result output by the neural network is processed, wherein the specific processing process is as follows: the loss function is to calculate the difference between the network prediction value and the true value of the label in the model training process, and then the difference result is fed back to the network, so that the network parameters are adjusted to make the prediction result more and more accurate. It should be noted that the present application uses two loss functions, arcSoftmax Loss and Triplet Loss, in combination. And the loss function Triplet Loss is applied to the feature comparison loss in the form of BNNeck, and the loss function arcSoftmax Loss is applied to the prediction classification loss after passing through the BN (Batch Normalization) layer. Among them, due to the existence of margin in the loss function arcSoftmax Loss, and the use of loss function Hard Triplet Loss in the later training, these will expand the features of different ID pedestrians and improve the model feature recognition.

[0063] Step S14: extracting a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification feature, to realize pedestrian re-identification.

[0064] In the embodiments of the present application, the pedestrian re-identification is realized by extracting a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification feature, including: calculating a target distance between the pedestrian identification feature corresponding to the to-be-identified image and the pedestrian identification features corresponding to all the original images; and extracting the original image corresponding to the target distance satisfying a preset condition as the target pedestrian image corresponding to the to-be-identified pedestrian image, to realize pedestrian re-identification. Specifically, the feature of each query to-be-retrieved target image is calculated with all the features in the gallery database image to obtain distance results and the distance values are sorted from large to small; the gallery database image corresponding to the feature of each query to-be-retrieved target image before n is extracted as a retrieval result; and the value of n is determined according to actual conditions and is not specifically limited herein.

[0065] As can be seen, the application obtains a to-be-identified pedestrian image and a plurality of original pedestrian images, and obtains a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model; obtains weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and weights head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map; obtains pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a pedestrian attribute branch, and obtains pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch obtained by pre-training based on a multi-dimensional global auxiliary branch in the pedestrian re-identification model, and then obtains pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features; and extracts a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian recognition features, so as to realize pedestrian re-identification. As can be seen from the above, the application weights the head-shoulder region images which are different from human bodies, improves the discrimination degree of different human bodies, and improves the accuracy of re-identification. The application extracts pedestrian global features by using the target main global branch obtained by pre-training based on the multi-dimensional global auxiliary branch, so that the pedestrian global features are more accurate and effective, which is conducive to improving the discrimination degree of different human bodies. The application adds pedestrian attribute features based on the pedestrian global features, which is conducive to further improving the discrimination degree of different human bodies and improving the re-identification effect and accuracy of the model.

[0066] Referring to Figure 4 As shown in the drawings, the embodiment of the application discloses a specific pedestrian re-identification method, which comprises the following steps:

[0067] Step S21: obtaining a to-be-identified pedestrian image and a plurality of original pedestrian images, and obtaining a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model.

[0068] In the embodiment, the specific process of step S21 can refer to the corresponding content disclosed in the foregoing embodiments, which will not be described here again.

[0069] Step S22: obtaining weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and performing horizontal trisection on the to-be-identified global feature map and the original global feature map to obtain head-shoulder region images, body region images and lower body region images corresponding to the to-be-identified global feature map and the original global feature map respectively.

[0070] In the embodiment of the present application, the acquisition of the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map comprises: performing feature compression on the to-be-identified global feature map and the original global feature map based on a global pooling layer, and performing feature excitation on the compressed image based on a fully connected layer and an activation layer to obtain the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map.

[0071] It should be noted that the Key Area weighting mechanism is introduced in each residual unit in the last two scales of the ResNet network structure, and the specific operation is as shown in Figure 5 The Key Area weighting mechanism is introduced at the end of the residual branch. Specifically, a set of global feature maps is output after the last convolution layer and the BN layer in the residual branch, and the size of the feature maps is B*C*H*W; two branches are introduced, wherein the left branch compresses the feature maps through a global pooling layer, and then excites the feature maps through two fully connected layers and a sigmod layer to obtain a set of weighting weight coefficients about the correlation of global features in different channels, and the size of the weight coefficients is B*C*1*1. The size of the weight coefficients can reflect the strength of different channels to the expression of the global pedestrian feature to a certain extent. The B is the Batch quantity, that is, how many pictures are sent to the neural network at a time during each iteration of the neural network; the C is the number of channels, that is, the number of the neural network at a certain layer, for example, if a picture is input and passes through a certain layer of the neural network, and the layer has 512 channels, then the output of the picture has 512 feature maps, and the Batch pictures are the output of Batch*C feature maps; the H is the height; and the W is the width.

[0072] In the embodiment of the present application, the to-be-identified global feature map and the original global feature map are horizontally divided into three equal parts to obtain head-shoulder region images, body region images and lower body region images corresponding to the to-be-identified global feature map and the original global feature map, and the specific operation is as shown in Figure 5 As shown in the right branch, the above global feature maps are divided into three parts through a slice layer, and the input is horizontally divided into three equal parts according to the aspect ratio of the network input and the analysis of the human body data in the training set, so as to obtain three sets of local feature maps, which are the head-shoulder region, the body region and the lower body region, and the size of each set of local feature maps is B*C*H / 3*W.

[0073] It should be noted that, as shown in Figure 5As shown is a Key Area weighted residual unit proposed in the application, like Figure 6 As shown is a conventional Resnet network residual unit; it is pointed out that under the premise of making the above-mentioned modifications to the left branch and the right branch in the Key Area weighted residual unit, introducing a dropblock (a regularization method) layer in the residual branch convolutional layer to inactivate the feature map at a small probability can improve the generalization of the model.

[0074] Step S23: point-multiply the weight coefficients and the shoulder region image corresponding to the to-be-identified global feature map and the original global feature map respectively to obtain the weighted shoulder region image corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively.

[0075] In the embodiment of the application, the channel weight coefficient B*C*1*1 obtained by the global feature is point-multiplied with the shoulder region B*C*H / 3*W through a Scale layer to obtain the weighted human shoulder region, which has a size of B*C*H / 3*W, and the purpose is to continue to strengthen the human shoulder region in the case that some channels extract the global feature of the pedestrian well.

[0076] It is pointed out that in the existing supervised pedestrian re-identification model, in the use of channel attention mechanism, the vast majority of them use the conventional attention mechanism using global features, which cannot well focus on the learning of local key areas; and for the pedestrian re-identification task, an attention module is often introduced to make the CNN model pay more attention to the learning of important areas of the human body, but the conventional attention module may also cause the key learning of the occlusion outside the human body, and does not achieve the purpose of accurately extracting important features of the human body; secondly, all global features are divided into regions and processed through channel attention, which will cause a huge burden on performance and weaken the reinforcement learning of the local area by the channel attention, and the application can effectively make the model focus on the learning of the human shoulder area by using the channel weight of the global feature in the residual unit and then combining it with the weighted operation of the targeted human shoulder area.

[0077] Step S24: splice the weighted shoulder region image and the corresponding body region image and lower body region image to obtain the weighted to-be-identified global feature map and the weighted original global feature map.

[0078] In the embodiment of the application, the weighted shoulder region is spliced with the human body trunk region and lower body region which are not processed (not weighted) through a Concat (merge) layer to obtain the pedestrian global feature map of the weighted human shoulder region as output, which has a size of B*C*H*W.

[0079] Step S25: Obtain the pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively through the pedestrian attribute branch, and obtain the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch in the pedestrian re-identification model which is pre-trained based on a multi-dimensional global auxiliary branch, and then obtain the pedestrian identification features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features.

[0080] In this embodiment, the specific process of step S25 can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0081] Step S26: Extract a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification features, so as to realize pedestrian re-identification.

[0082] In this embodiment, the specific process of step S26 can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0083] It can be seen that the weight coefficient of the global feature is obtained in the application, which is combined into the head-shoulder region of the human body for attention learning (i.e., the head-shoulder region image with large difference from the human body is weighted), and finally spliced with other regions of the human body as the output, so as to improve the recognition degree of different pedestrian features of the model, improve the differentiation degree of different human bodies, and improve the accuracy of re-identification. In the application, the pedestrian attribute feature is added on the basis of the pedestrian global feature, which is conducive to further improving the differentiation degree of different human bodies and improving the re-identification effect and accuracy of the model. In addition, the multi-dimensional global auxiliary branch is added, which is conducive to making the network trunk combine different dimensions of attention points for joint learning, and is conducive to improving the generalization ability and retrieval effect of the model.

[0084] In the application, a Key Area weighting and feature fusion network based on pedestrian re-identification is designed. The network is based on the ResNet network structure for pruning, uses multi-dimensional global branch to assist the main global branch at the output convolution head, and introduces a pedestrian attribute branch to perform feature fusion with the main global branch for training. The improvement of this network structure can effectively improve the generalization and retrieval effect of the model. Most importantly, in the application, for the pedestrian re-identification task, a channel weight is generated according to the global feature in the residual unit of the network, and the image is sliced according to the characteristics of the training set of the pedestrian re-identification task. Finally, the channel weight is combined into the region above the head-shoulder of the human body for attention learning, further improving the differentiation degree of different IDs of the model.

[0085] Reference is made to Figure 7As shown, the embodiment of the present application discloses a pedestrian re-identification device, comprising:

[0086] The global feature map acquisition module 11 is configured to acquire a to-be-identified pedestrian image and a plurality of original pedestrian images, and acquire a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model.

[0087] The weighted global feature map acquisition module 12 is configured to acquire weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and weight head-shoulder region images corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map.

[0088] The pedestrian identification feature acquisition module 13 is configured to acquire pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively, and acquire pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map by using a target main global branch trained in advance based on a multi-dimensional global auxiliary branch in the pedestrian re-identification model, and then acquire pedestrian identification features corresponding to the to-be-identified pedestrian image and the original pedestrian image based on the pedestrian global features and the pedestrian attribute features.

[0089] The pedestrian re-identification module 14 is configured to extract a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification features, so as to realize pedestrian re-identification.

[0090] The working processes of the above modules can be referred to the corresponding content disclosed in the foregoing embodiments, and will not be described here.

[0091] It can be seen that the head-shoulder region image with large difference in human body is weighted in the present application, the discrimination degree of different human bodies is improved, and the accuracy of re-identification is improved. The target main global branch trained in advance based on the multi-dimensional global auxiliary branch is used to extract the pedestrian global features in the present application, so that the pedestrian global features are more accurate and effective, which is conducive to improving the discrimination degree of different human bodies. The pedestrian attribute features are added on the basis of the pedestrian global features in the present application, which is conducive to further improving the discrimination degree of different human bodies and improving the re-identification effect and accuracy of the model.

[0092] Further, the embodiment of the present application further provides an electronic device, Figure 8 The electronic device 20 structure diagram shown in the figure cannot be considered as any limitation on the use range of the present application.

[0093] Figure 8 A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, an input / output interface 24, a communication interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps of the pedestrian re-identification method disclosed in any of the foregoing embodiments.

[0094] In the embodiments, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 25 can be any communication protocol applicable to the technical solutions of the present application, which is not specifically limited herein; the input / output interface 24 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0095] In addition, the memory 22 as a carrier of resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc. The memory 22 can include a random access memory as a running memory and a non-volatile memory for storage of external memory, and the storage resources thereon include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0096] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222 on the source host, and the operating system 221 can be Windows, Unix, Linux, etc. In addition to the computer program capable of completing the pedestrian re-identification method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0097] In the embodiments, the input / output interface 24 can specifically include but is not limited to a USB interface, a hard disk reading interface, a serial interface, a voice input interface, a fingerprint input interface, etc.

[0098] Further, the embodiments of the present application further disclose a computer readable storage medium for storing a computer program; wherein the computer program is executed by the processor to implement the pedestrian re-identification method disclosed above.

[0099] The specific steps of the method can refer to the corresponding contents disclosed in the foregoing embodiments, which will not be repeated here.

[0100] The computer readable storage medium mentioned herein includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, magnetic or optical disk, or any other form of storage medium known in the technical field. The computer program is executed by the processor to implement the aforementioned pedestrian re-identification method. For the specific steps of the method, refer to the corresponding content disclosed in the foregoing embodiments, which will not be described here again.

[0101] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the pedestrian re-identification method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0102] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly show the interchangeability of hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0103] The steps of the algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0104] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be used to distinguish one entity or action from another entity or action, without necessarily requiring or implying that there is any such actual relationship or order between these entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0105] The above describes in detail a pedestrian re-identification method, device, equipment and medium provided by the present application. The principles and implementation manners of the present application are described by applying specific examples in this document. The above example description is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A pedestrian re-identification method, characterized in that, The method comprises the following steps: obtaining a to-be-identified pedestrian image and a plurality of original pedestrian images, and obtaining a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model; obtaining weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively, and weighting head-shoulder region feature maps corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map; obtaining pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a pedestrian attribute branch, and obtaining pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch in the pedestrian re-identification model which is trained in advance based on a multi-dimensional global auxiliary branch, and then obtaining pedestrian identification features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features; extracting a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian identification features to realize pedestrian re-identification; the step of weighting the head-shoulder region feature maps corresponding to the to-be-identified global feature map and the original global feature map respectively based on the corresponding weighting weight coefficients to obtain the weighted to-be-identified global feature map and the weighted original global feature map comprises the following steps: horizontally trisecting the to-be-identified global feature map and the original global feature map to obtain head-shoulder region feature maps, body region feature maps and lower body region feature maps corresponding to the to-be-identified global feature map and the original global feature map respectively; point-multiplying the weight coefficients and the head-shoulder region feature maps corresponding to the to-be-identified global feature map and the original global feature map respectively to obtain weighted head-shoulder region feature maps corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively; splicing the weighted head-shoulder region feature maps with the corresponding unweighted body region feature maps and lower body region feature maps to obtain the weighted to-be-identified global feature map and the weighted original global feature map. 2.The pedestrian re-identification method of claim 1, wherein, the step of obtaining the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively comprises the following steps: performing feature compression on the to-be-identified global feature map and the original global feature map based on a global pooling layer, and performing feature excitation on the compressed images based on a fully connected layer and an activation layer to obtain the weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map respectively. 3.The pedestrian re-identification method of claim 1, wherein, before the step of obtaining a to-be-identified pedestrian image and a plurality of original pedestrian images, and obtaining a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and an original global feature map corresponding to the original pedestrian image by using a pedestrian re-identification model, the method further comprises the following steps: The training set is constructed by using a plurality of reference pedestrian pictures in a target area captured by cameras at different orientations, and the pedestrian re-identification model is obtained by training a pedestrian re-identification network based on a residual network using the training set. 4.The pedestrian re-identification method of claim 3, wherein, Before the target main global branch pre-trained based on the multi-dimensional global auxiliary branch is used to obtain the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map, the method further includes: The multi-dimensional global auxiliary branch added to the pedestrian re-identification network is used to obtain reference global features of different dimensions corresponding to the training set, and the original main global branch of the pedestrian re-identification network is used to obtain target global features of a target dimension corresponding to the training set; The original main global branch is trained to obtain a target main global branch based on the reference global features and the target global features and using loss backpropagation, so that the target main global branch is used to obtain the pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map. 5.The pedestrian re-identification method of claim 1, wherein, The pedestrian re-identification features corresponding to the to-be-identified pedestrian image and the original pedestrian image are obtained based on the pedestrian global features and the pedestrian attribute features, including: The pedestrian global features and the pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map are spliced by a feature fusion dimension reduction convolution layer, and a dimension reduction convolution operation is performed on the spliced features to obtain the pedestrian re-identification features corresponding to the to-be-identified pedestrian image and the original pedestrian image. 6.The pedestrian re-identification method of any one of claims 1-5, wherein, The target pedestrian image corresponding to the to-be-identified pedestrian image is extracted from the original pedestrian image based on the pedestrian re-identification features, so as to realize pedestrian re-identification, including: A target distance between the pedestrian re-identification features corresponding to the to-be-identified pedestrian image and the pedestrian re-identification features corresponding to all the original pedestrian images is calculated; The original pedestrian image corresponding to the target distance satisfying a preset condition is extracted as the target pedestrian image corresponding to the to-be-identified pedestrian image, so as to realize pedestrian re-identification.

7. A pedestrian re-identification apparatus characterized by comprising: The method includes: A global feature map acquisition module is configured to acquire a to-be-identified pedestrian image and a plurality of original pedestrian images, and acquire a to-be-identified global feature map corresponding to the to-be-identified pedestrian image and original global feature maps corresponding to the original pedestrian images using a pedestrian re-identification model; A weighted global feature map acquisition module is configured to acquire weighting weight coefficients of different channels corresponding to the to-be-identified global feature map and the original global feature map, and weight head-shoulder region feature maps corresponding to the to-be-identified global feature map and the original global feature map based on the corresponding weighting weight coefficients to obtain a weighted to-be-identified global feature map and a weighted original global feature map; The pedestrian recognition feature acquisition module is configured to acquire pedestrian attribute features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively, and acquire pedestrian global features corresponding to the weighted to-be-identified global feature map and the weighted original global feature map respectively by using a target main global branch in the pedestrian re-identification model which is pre-trained based on a multi-dimensional global auxiliary branch, and then acquire pedestrian recognition features corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively based on the pedestrian global features and the pedestrian attribute features. The pedestrian re-identification module is configured to extract a target pedestrian image corresponding to the to-be-identified pedestrian image from the original pedestrian image based on the pedestrian recognition features, so as to realize pedestrian re-identification. The weighting of the head-shoulder region feature maps corresponding to the to-be-identified global feature map and the original global feature map based on the corresponding weighting weight coefficients to obtain the weighted to-be-identified global feature map and the weighted original global feature map includes: The to-be-identified global feature map and the original global feature map are horizontally divided into three equal parts to obtain head-shoulder region feature maps, body region feature maps and lower body region feature maps corresponding to the to-be-identified global feature map and the original global feature map respectively. The weight coefficients and the head-shoulder region feature maps corresponding to the to-be-identified global feature map and the original global feature map are point multiplied to obtain weighted head-shoulder region feature maps corresponding to the to-be-identified pedestrian image and the original pedestrian image respectively. The weighted head-shoulder region feature maps are spliced with the corresponding unweighted body region feature maps and lower body region feature maps to obtain the weighted to-be-identified global feature map and the weighted original global feature map.

8. An electronic device, comprising: A processor and a memory are included; wherein the processor implements the pedestrian re-identification method of any one of claims 1 to 6 when executing the computer program saved in the memory.

9. A computer-readable storage medium, characterized in that, A computer program is stored; wherein the computer program is executed by a processor to implement the pedestrian re-identification method of any one of claims 1 to 6.

Images