Congested pedestrian re-identification method based on unequal-width convolution blocks

Abstract

The invention discloses a crowded pedestrian re-identification method based on unequal-width convolution blocks, and the method employs a data enhancement method to obtain a visible light image based on a pedestrian image, enables a network to effectively solve the complex problem of pedestrians in a crowded scene, and improves the generalization capability. And then constructing a key character attention module, obtaining a key character attention graph, and jointly enhancing the pedestrian distinguishing capability of the model by using the key character attention module and key character attention loss. Based on a key figure attention module, a ResNet50 network and an unequal-width convolution block, a pedestrian re-identification model is constructed and trained by taking a pedestrian image as input and a pedestrian identification result as output, and the pedestrian re-identification model is jointly trained by combining a difficult sample triple loss function and a label smooth regularized cross entropy loss function. By using the model provided by the invention, the rank-1 accuracy rate of pedestrian re-identification in a crowded scene is improved to 90.5%.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer vision technology, and specifically relates to a crowded pedestrian re-identification method based on unequal-width convolution blocks. Background Art

[0002] With the rapid development of social economy, public safety has become increasingly important. Today, surveillance equipment has spread to all aspects of daily life, however, the application of surveillance information is still limited to a relatively basic level. In an emergency, manually reviewing surveillance footage to identify and retrieve suspects is a time-consuming and labor-intensive task, especially in complex shooting environments, where the accuracy of human eye resolution is relatively low. Therefore, the industry has begun to actively explore pedestrian re-identification (Person Re-identification, Re-ID) technology. [1] , which has become an important part of the field of intelligent security.

[0003] Person re-identification (ReID) is a technology that aims to identify specific pedestrians from a large number of pedestrian images captured by different cameras to meet various search needs. The person re-identification task involves matching images or videos of related individuals captured across multiple non-overlapping cameras. This task has become crucial due to its wide range of practical applications, such as criminal investigation, danger warning, unmanned supermarket management, lost person rescue and many other fields. Person re-identification has become a hot topic in the field of scientific research and has received widespread attention.

[0004] The existing person re-identification technology has made great progress under the continuous advancement of scientific researchers. However, despite the continuous progress of person re-identification technology, the difficulty of the task has been greatly reduced because the target person always appears alone in a scene in the task setting, and the environment around the person is very different from the person. This task is often only applicable to person recognition in scenes with a small number of people, which is out of touch with real application scenarios.

[0005] For example, the paper "DiP: Learning Discriminative Implicit Parts for Person Re-Identification" [2]In this paper, the authors proposed a method to learn discriminative implicit parts (Dips) that are decoupled from explicit body parts. Therefore, Dips can learn to extract any features that help distinguish identities, which goes beyond predefined body parts (such as accessories). In addition, the authors proposed a novel implicit position that provides a geometric interpretation for each Dip. Finally, additional Dip weights are introduced to handle invisible or occluded situations and further improve the feature representation of Dips. This model solves the problem of conventional pedestrian re-identification well and brings certain improvements, but it is proved in the following examples that it is not suitable for crowded pedestrian re-identification, so the effect is not obvious.

[0006] In actual application scenarios, users often need to identify crowded scenes such as shopping malls, intersections, or schools, where a large number of people overlap closely. This makes it impossible to extract a picture with only one person for identification, as in existing pedestrian re-identification. When multiple people appear in the picture of a person in a crowded scene, the existing model will be confused because it does not have the ability to eliminate interference from strong similarities and is unable to identify people in crowded scenes. This has also led to the continuous improvement of the model in scientific research, but in actual applications, it still performs poorly. Overly ideal task settings have caused a huge derailment between scientific research and application.

[0007] [1]Masson H,Bhuiyan A,Nguyen-Meidine LT,et al.A survey of pruningmethods for efficient person re-identification across domains[J].arXivpreprint arXiv:1907.02547,2019.

[0008] [2]Li D,Chen S,Zhong Y,et al.Dip:Learning discriminative implicitparts for person re-identification[J].arXiv preprint arXiv:2212.13906,2022. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention provides a method for re-identifying crowded pedestrians based on unequal-width convolutional blocks. In this method, first, the network proposes a key person attention module, and by inputting the input picture into the module, the key person attention module and the key person attention loss are used to jointly enhance the model's ability to distinguish pedestrians. Secondly, two layers of unequal-width convolutional blocks are introduced into the convolutional neural network to make full use of its powerful nonlinear expression ability and the ability to capture large-scale spatial features, so as to mine the difference information of different positions in the global range of the feature map. Thirdly, a random person method is introduced for data enhancement, so that the network can effectively handle the complex problems of pedestrians in crowded scenes and enhance its generalization ability. Finally, the accuracy of the model in identifying the identity of pedestrians in crowded scenes is improved.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions: a crowded pedestrian re-identification method based on unequal width convolution blocks, comprising the following steps:

[0011] S1. Collect pedestrian images, and apply a data enhancement method for random person images to preprocess the pedestrian images to obtain visible light images;

[0012] S2. Construct a key person attention module, extract the character importance feature map using the visible light image as input, and splice it based on the original features of the image to obtain the key person attention map;

[0013] S3, using the ResNet50 network and the unequal width convolutional block NuCB, extract the image features rich in key person information from the key person attention map;

[0014] S4, based on the key person attention module, ResNet50 network and unequal width convolution block NUCB, takes pedestrian images as input and pedestrian recognition results as output to build and train a pedestrian re-identification model. The key person constraint loss is used in the training, and the image features output by S3 are combined with the difficult sample triple loss function and the cross entropy loss function of label smoothing regularization to jointly train the pedestrian re-identification model;

[0015] S5. The collected pedestrian image is input into a pedestrian re-identification model to obtain a pedestrian recognition result.

[0016] Furthermore, the aforementioned step S1 includes the following sub-steps:

[0017] S1.1. Use deeplabv3_resnet101 to extract the person images of ImageNet for pedestrian images, and use the random person image method for the person images. The images are randomly selected for erasure with probability P;

[0018] S1.2. For the selected image, randomly select a rectangular area whose total area is less than or equal to the preset area of ​​the total image, and randomly select one of the people images from ImageNet and modify its area to the size of the rectangular area;

[0019] S1.3. Use the resized ImageNet person images to fill the rectangular area selected in the image.

[0020] Furthermore, the aforementioned crowded pedestrian re-identification method based on unequal width convolutional blocks modifies the resized ImageNet person pictures and fills them into the rectangular area selected by the picture, as follows:

[0021]

[0022] Where S is a randomly selected image area, a(i,j) represents the image pixel input value, another(i,j) represents the pixel value of the filled image, i0 is the starting point of the horizontal coordinate of area S, and j0 is the starting point of the vertical coordinate of area S.

[0023] Furthermore, the aforementioned step S2 includes the following sub-steps:

[0024] S2.1. The visible light image can be used to identify key figures and output the key figure importance features;

[0025] S2.2. The key person importance feature map and the mask image of the picture are input into the key person constraint loss together; S2.3. The key person importance feature map and the original features of the picture are concatenated and input into the convolutional network to obtain the key person attention map, and then the key person attention map is dot-multiplied with the input image.

[0026] Further, in the aforementioned step S2.1, the key character importance features are extracted and output by constructing a key attention module; the key attention module includes a first basic convolution block, a first maximum pooling layer, a second basic convolution block, a second maximum pooling layer, a third basic convolution block, a third maximum pooling layer, a fourth basic convolution block, a first deconvolution layer, a fifth basic convolution block, a second deconvolution layer, a sixth basic convolution block, a third deconvolution layer, and a convolution layer connected in sequence;

[0027] The first basic convolution block, the second basic convolution block, and the third basic convolution block are used to increase the dimension of the input features;

[0028] The first, second, and third maximum pooling layers are used for downsampling to reduce the input feature size;

[0029] The fourth basic convolution block, the fifth basic convolution block, and the sixth basic convolution block are used to reduce the dimension of the input features;

[0030] The first deconvolution layer, the second deconvolution layer, and the third deconvolution layer are used to enlarge the size of the input features and convert the low-resolution feature map into a high-resolution feature map;

[0031] Convolutional layers are used to reduce the high-resolution feature map dimension to 1.

[0032] Furthermore, in the aforementioned step S2.2, the key person constraint loss is as follows:

[0033]

[0034] In the formula, y ij It means that when the horizontal and vertical coordinates are i and j respectively, the category corresponding to the real category of the mask here, It represents the value predicted by the model when the horizontal and vertical coordinates are i and j respectively. N represents the total number of images in a batch, and H and W represent the total height and width of the images respectively.

[0035] When the mask image is of multiple categories, ij The constraints are as follows:

[0036]

[0037] Apply the idea of ​​labelsmooth to the mask image. Based on the fact that the mask image is a three-value matrix, the following constraints are imposed on the matrix values:

[0038]

[0039] Wherein, is the hyperparameter set according to the experiment, ∈ is the preset hyperparameter, and K is the preset hyperparameter.

[0040] Furthermore, the aforementioned step S3 includes the following sub-steps:

[0041] S3.1. Input the key person attention map into a network consisting of ResNet50 and unequal width convolution blocks, perform convolution operations, and output information-rich image features;

[0042] In the unequal width convolution block NUCB, the convolution operation is implemented by unequal width convolution kernels. The width of a part of the small convolution kernel is increased to (n+2)*(n+2). The output image features are as follows:

[0043]

[0044] Where Input[c,h+i-1,w+j-1] is the element in the input feature map, h and w represent the height and width of the convolution kernel respectively, Kernel[k,c,i,j] is the weight of the (i,j) position of the cth channel of the kth convolution kernel, and b[k] is the bias term of the kth output channel;

[0045] S3.2. Perform MAX-pooling on the features obtained in step S3.2 and output the pooled feature map.

[0046] Furthermore, in the aforementioned step S4, the difficult sample triplet loss function is to select K images for each pedestrian in a batch sample containing P different pedestrians, and then form them into a data set of size P×K. Then, for each training, each image in this data set is used as an anchor image, and the two images with the longest intra-class distance and the shortest inter-class distance are selected to form a difficult triplet, and the difficult sample triplet loss function is constructed as follows:

[0047]

[0048] Where A is the anchor sample, Pos is the positive sample, Neg is the negative sample, α is the threshold parameter, and d A.Pos is the distance from the anchor point to the positive sample, d A.Neg is the distance from the anchor point to the negative sample.

[0049] Furthermore, in the aforementioned step S4, the smoothing label is introduced into the cross entropy loss as follows:

[0050]

[0051] Where N is the number of pedestrian IDs, y is the real label of the pedestrian, and p i is the output predicted identity probability value, ε is the error rate,.

[0052] Furthermore, in the aforementioned step S4, the joint loss function L sum As follows:

[0053] L sum =L keyperson +L tri_hard +L lsr

[0054] Among them, λ is a hyperparameter.

[0055] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0056] This method proposes a crowded pedestrian re-identification method based on unequal-width convolutional blocks. In this method, first, the network proposes a key person attention module. By inputting the input picture into the module, the key person attention module and the key person attention loss are used to jointly enhance the model's ability to distinguish pedestrians. Secondly, two layers of unequal-width convolutional blocks are introduced into the convolutional neural network to make full use of its powerful nonlinear expression ability and the ability to capture large-scale spatial features to mine the difference information of different positions in the feature map within the global range. Third, the random person method is introduced for data enhancement so that the network can effectively handle the complex problems of pedestrians in crowded scenes and enhance its generalization ability. These operations are designed to give higher weights to important features, thereby suppressing the expression of useless features. Finally, the rank-1 accuracy of pedestrian re-identification in crowded scenes is improved to 90.5%. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 These are some pedestrian images in the M1501-Crowed dataset.

[0058] Figure 2 It is a schematic diagram of the network framework of posture enhancement based on unequal-width convolution blocks of the present invention.

[0059] Figure 3 This is the key figure attention module diagram of the present invention.

[0060] Figure 4 It is a flow chart of network training of the present invention.

[0061] Figure 5 It is a schematic diagram of the unequal width convolution kernel of the present invention. DETAILED DESCRIPTION

[0062] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0063] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0064] The present invention proposes a crowded pedestrian re-identification method based on unequal width convolutional blocks to solve the crowded pedestrian re-identification task. In the embodiment, the training data set is M501-Crowed. Some pictures of the data set are as follows: Figure 1This training process uses mini-batch training. In each batch, P pedestrians are randomly selected, and K images are randomly selected for each of these pedestrians. There are a total of P*K pedestrian images. Next, we will take P*K images as an example. Figure 4 , the present invention comprises the following steps:

[0065] S1. Collect P*K pedestrian images, and use the data enhancement method of random person images to preprocess the pedestrian images to obtain visible light images;

[0066] S2. Construct a key person attention module, extract the character importance feature map using P*K visible light images as input, and splice them based on the original features of the images to obtain the key person attention map;

[0067] S3, using the ResNet50 network and unequal width convolutional blocks NUCB, extract image features rich in key person information from the key person attention map;

[0068] S4, based on the key person attention module, ResNet50 network and unequal width convolution block NUCB, takes pedestrian images as input and pedestrian recognition results as output to build and train the pedestrian re-identification model. The key person constraint loss is used in the training, and the hard sample triple loss function and the cross entropy loss function of label smoothing regularization are combined to jointly train the pedestrian re-identification model;

[0069] S5. The collected pedestrian image is input into a pedestrian re-identification model to obtain a pedestrian recognition result.

[0070] The images in step S1 are all from the M-Crowded dataset for crowded pedestrian re-identification. Visible light images are composed of three primary colors: red, green, and blue, so the image data contains three channels, each corresponding to a primary color. In M1501-Crowded, various situations of crowding are fully considered. For the original Market1501 dataset, a part of the crowd images are randomly selected, and in each image, a fine-grained image of a person in other images is randomly selected and placed in the picture, and randomly reduced and placed behind the person, or randomly enlarged and placed in front of the person, so as to simulate the complex relationship in the crowded scene, and the corresponding relationship of the person is recorded with a mask.

[0071] Step S1 includes the following sub-steps:

[0072] S1.1. Use deeplabv3_resnet101 to extract the person images of ImageNet for pedestrian images, and use the random person image method for the person images. The images are randomly selected for erasure with probability P;

[0073] S1.2. For the selected image, randomly select a rectangular area whose total area is less than or equal to the preset area of ​​the total image, and randomly select one of the people images from ImageNet and modify its area to the size of the rectangular area;

[0074] S1.3. Use the resized ImageNet person images to fill the rectangular area selected in the image.

[0075] Data augmentation is a technique to improve the generalization ability of a model by performing various transformations on the training data so that the model can better adapt to different scenarios. This technique generates more training samples without changing the data labels, thereby helping the model reduce overfitting. Among them, Random Erasing is an effective data augmentation strategy that simulates noise and missing information in the image by randomly occluding part of the image. This method not only improves the robustness of the model, but also enables the model to focus more on global features rather than relying on local details, thereby enhancing the model's performance when dealing with partial occlusion or missing information.

[0076] In order to further improve the generalization ability of convolutional neural networks, the present invention proposes a new data augmentation method - Random Person. The core idea of ​​this method is to set a predefined erasure probability, but it is different from the traditional random erasure method, which usually fills the three dimensions of the image with zero values, random values ​​or average values ​​in random areas. This method is effective in conventional pedestrian re-identification tasks. However, in order to cope with more complex and crowded scenes, the present invention uses pre-extracted person images in the Imagenet dataset as filling data, making the training process closer to the complex situations in actual scenes.

[0077] Due to its high fit with the task, it can be applied in the model and effectively improve the recognition effect of the model.

[0078] The implementation process of the random character method is as follows:

[0079] There is an input image I, and there is a certain probability P that it is selected for erasure, and the remaining probability 1-P remains unchanged. Let the randomly selected image area be S, and the resized ImageNet character image is filled into the rectangular area selected by the image, as follows:

[0080]

[0081] Where S is a randomly selected image area, a(i,j) represents the image pixel input value, another(i,j) represents the pixel value of the filled image, i0 is the starting point of the horizontal coordinate of area S, and j0 is the starting point of the vertical coordinate of area S.

[0082] Preferably, step S2 includes the following sub-steps:

[0083] S2.1. The visible light image can be used to identify key figures and output the key figure importance features;

[0084] S2.2. The key person importance feature map and the mask image of the picture are input into the key person constraint loss together; S2.3. The key person importance feature map and the original features of the picture are concatenated and input into the convolutional network to obtain the key person attention map, and then the key person attention map is dot-multiplied with the input image.

[0085] In step S2.1, the key attention module is constructed to extract and output the key person importance features; its overall structure is as follows Figure 3 As shown. The key attention module includes a first basic convolution block, a first maximum pooling layer, a second basic convolution block, a second maximum pooling layer, a third basic convolution block, a third maximum pooling layer, a fourth basic convolution block, a first deconvolution layer, a fifth basic convolution block, a second deconvolution layer, a sixth basic convolution block, a third deconvolution layer, and a convolution layer connected sequentially;

[0086] The first basic convolution block, the second basic convolution block, and the third basic convolution block are used to increase the dimension of the input features; the first basic convolution block increases the feature dimension from 3 to 32, and the second basic convolution block and the third basic convolution block multiply the dimension by 2 to gradually increase the dimension.

[0087] The first maximum pooling layer, the second maximum pooling layer, and the third maximum pooling layer are used for downsampling with a step size of 2. The height and width of the input feature are divided by 2 to achieve the effect of downsampling, thereby achieving the purpose of extracting key data in the image.

[0088] The fourth basic convolution block, the fifth basic convolution block, and the sixth basic convolution block are used to reduce the dimension of the input features; the fourth basic convolution block and the sixth basic convolution block divide the dimension by 2, and the fifth basic convolution block divides the dimension by 4 to adapt to the situation where the dimension doubles after feature superposition, thereby gradually reducing the dimension.

[0089] The first, second, and third deconvolution layers multiply the height and width of the input feature by two to enlarge the feature size. The low-resolution feature map can be converted into a high-resolution feature map. The convolution layer is used to reduce the dimension of the high-resolution feature map to 1. The deconvolution operation is essentially the inverse process of convolution. In actual implementation, the deconvolution layer expands the spatial size of the input feature map by using a transposed convolution kernel. The size of the transposed convolution kernel is usually larger than the ordinary convolution kernel to ensure that a feature map larger than the input is generated during the convolution operation.

[0090] Finally, the feature is enlarged to the same size as the original image, but the dimension is reduced to 1, which is used as the key person importance feature map and input into the key person constraint loss.

[0091] In the dataset, the mask image has three types of relationships, namely key figures, environmental features and interfering figures, as shown in red, green and black in Figure (1). In order to force the model to focus on the features of the key figure area, a key figure loss function is proposed in the key figure constraint loss. Step S2.2 Key figure constraint loss is as follows:

[0092]

[0093] In the formula, y ij It means that when the horizontal and vertical coordinates are i and j respectively, the category corresponding to the real category of the mask here, It represents the value predicted by the model when the horizontal and vertical coordinates are i and j respectively. N represents the total number of images in a batch, and H and W represent the total height and width of the images respectively.

[0094] When the mask image is of multiple categories, ij The constraints are as follows:

[0095]

[0096] Apply the idea of ​​labelsmooth to the mask image. Based on the fact that the mask image is a three-value matrix, the following constraints are imposed on the matrix values:

[0097]

[0098] Wherein, ∈ is the preset hyperparameter, which is set according to the experiment, and K is the preset hyperparameter, which is set to 10.

[0099] As a preference, in step 2.3, in order to make full use of the feature map Importance feature map of input and output for key person recognition module Perform weighted fusion. Concatenate the character importance feature map and the original features of the image, input them into the convolutional network, obtain the key character attention map, and perform dot multiplication on it and the input image.

[0100] Preferably, step S3 includes the following sub-steps:

[0101] S3.1. Input the key person attention map into a network composed of ResNet50 and unequal width convolution blocks, perform convolution operation, and output information-rich image features.

[0102] The common Convolutional Neural Networks (CNNs) are a method for efficiently processing and analyzing data in deep learning models, especially performing well in image processing tasks. Among them, the convolutional layer, as a key component of CNN, plays a vital role.

[0103] The convolution layer performs a convolution operation by applying a series of convolution kernels to local regions of the input data to extract features from the input data. These features can be low-level features such as edges and textures, or higher-level features such as shapes or patterns.

[0104] refer to Figure 5 , the dimension of the input feature map is C0×H×W, where C represents the number of channels, H represents the height, and W represents the width. After the n*n convolution kernel is passed and (n+1) / 2 padding is added, then without considering the convolution bias, C1 convolution kernels with dimensions of C0*n*n are required. Figure 5 As shown in Figure 5, when calculating the kth output dimension, the convolution kernel with the kth feature C0*n*n is used to perform convolution calculation on the input data and use it as the output value of the kth dimension. The calculation formula is shown in formula (5).

[0105]

[0106] Among them, Input[c,h+i-1,w+j-1] is the element in the input feature map, Kernel[k,c,i,j] is the weight of the (i,j) position of the cth channel of the kth convolution kernel, and b[k] is the bias term of the kth output channel.

[0107] In the existing convolutional neural networks, most still use small convolution kernels. This is because small convolution kernels have fewer parameters, which helps to reduce the complexity and computational complexity of the model, especially in deep networks. At the same time, small convolution kernels have stronger nonlinear expression capabilities at each layer and can better capture the nonlinear characteristics of the data. However, the receptive field of small convolution kernels is small and may not be able to effectively capture a larger range of spatial features. At the same time, if large convolution kernels are fully adopted, the number of parameters of the model may be increased, resulting in a significant increase in computational costs. All existing convolution kernels still use n*n*C convolution kernels, which are shaped like n*n*C cuboids. In order to have the advantages of both small and large convolution kernels, the present invention proposes a concept of non-uniform convolution block (NuCB), in which all convolutions are implemented by non-uniform convolution kernels (NuCK). In the unequal width convolution kernel, the original n*n*C rectangular shape is abandoned, and the width of some dimensions of the small convolution kernel is increased to (n+2)*(n+2). For some adjacent features, because they are facing the convolution layer of convolution kernels of different sizes, they are randomly assigned different weights, that is, when facing a small convolution kernel, it is equivalent to a weight of 0, so it can play an effect similar to the Dropout operation.

[0108] like Figure 5 As shown, C0 represents the input dimension, C1 represents the output dimension, H and W represent the height and width of the feature, respectively, and h and w represent the height and width of the convolution kernel, respectively.

[0109] S3.2. Perform MAX-pooling on the features obtained in step S3.2 and output the pooled feature map F.

[0110] Based on the key person attention module, ResNet50 network and unequal width convolution block NUCB, a pedestrian re-identification model is constructed and trained with pedestrian images as input and pedestrian recognition results as output. The model structure is as follows: Figure 2 .

[0111] Preferably, the loss function in step 4 adopts a joint training method, uses the key person constraint loss, and combines the difficult sample triplet loss function and the label smoothing regularized cross entropy loss function to jointly train the pedestrian re-identification model.

[0112] The loss function of the difficult sample triplet is to select K images of each pedestrian in a batch sample containing P different pedestrians, and then form them into a dataset of size P×K. Then, for each training, each image in this dataset is used as an anchor image, and the two images with the longest intra-class distance and the shortest inter-class distance of the image are selected to form a difficult triplet, and the loss function is calculated. The expression is shown in formula (6):

[0113]

[0114] Where A is the anchor sample, Pos is the positive sample, Neg is the negative sample, α is the threshold parameter, and d A.Pos is the distance from the anchor point to the positive sample, d A.Neg is the distance from the anchor point to the negative sample.

[0115] The cross entropy loss function is the most commonly used loss function in the pedestrian re-identification task. It measures the difference between the model prediction result and the true label. In order to further improve the robustness of the model and avoid the overfitting phenomenon that may occur when directly using the cross entropy loss function, a label smoothing method is introduced, that is, a small number of wrong labels are assigned to the network. The smoothing label is introduced in the cross entropy loss as follows:

[0116]

[0117] Where N is the number of pedestrian IDs, y is the real label of the pedestrian, and p i is the output predicted identity probability value, ε is the error rate, which is taken as 0.1, and the final constructed joint loss function L sum As follows:

[0118] L sum =L keyperson +L tri_hard +L lsr

[0119] Among them, λ is a hyperparameter and is set separately in the experiment.

[0120] The actual measurement process of the present invention is as follows:

[0121] Step 1: Enter the query set and gallery set, and proceed to step 2;

[0122] Step 2: Using the model obtained during the training process, perform feature extraction on all pedestrian images in the query set and gallery set input in step 1, and proceed to step 3;

[0123] Step 3: Calculate the similarity between the query set features and the gallery set features, and proceed to step 4;

[0124] Step 4: According to the similarity, obtain the matching result corresponding to each pedestrian image in the query set and proceed to step 5;

[0125] Step 5: End.

[0126] Preferably, the query set in step 1 of the test process represents a set of pedestrian images to be queried, and the gallery set represents a set of pedestrian images to be matched by the query set.

[0127] Preferably, the similarity calculation method in step 3 of the test process is point product similarity.

[0128] Embodiment 1:

[0129] In this embodiment, mini-batch gradient descent is used to update the model parameters of the present invention, that is, a small batch of samples is randomly selected for parameter update each time the gradient descent occurs. The ResNet50 network is selected as the basic framework, wherein the parameters of ResNet50 are pre-trained on ImageNet.

[0130] This paper uses ResNet50 as the basic skeleton of the network, in which the stride of the last bottleneck block is set to 1. At the same time, the edge and label smoothing regularization rates of the triple loss are set to 0.3 and 0.1 respectively. The input size of the image is 224×224 pixels, and the learning rate uses a warm-up method: it increases uniformly from 4×10^-6 to 4×10^-4 in the first 10 epochs, and then the learning rate decays exponentially by 0.1 every 40 epochs. A total of 200 epochs are trained. The Adam optimizer is used for model parameter optimization, and L2 regularization is applied at the same time, with a weight decay factor of 5×10^-4. In the test phase, the distance calculation between features is measured using the cosine distance metric. This experiment uses a deep learning framework based on PyTorch 1.8 and is accelerated on an NVIDIA3090GPU.

[0131] This example uses the dataset M1501-Crowded to complete the person re-identification task and test the performance of the model to evaluate the method in this paper. The performance of other methods is reproduced and applied to M1501-Crowded. The comparison results are shown in Table 1 Performance comparison (%) of the method of the present invention and other methods on the M1501-Crowded dataset.

[0132] Table 1

[0133]

[0134]

[0135] The experimental comparison results based on the dataset M1501-Crowded are shown in Table 1. The rank-1 and mAP of the model proposed in this paper are 90.0 and 80.5 respectively on this dataset. Compared with the BPBreID method using the attention mechanism, it uses the topological annotation of the local parts of the human body, uses multiple local parts to predict the attention map of the body parts, and generates the local body features of the ReID target. However, due to its over-reliance on the local annotations generated in the model, it cannot avoid paying attention to the local features of other characters because it fails to effectively exclude the interference objects in the crowded scene. Compared with the CLIP-ReID method using the visual Transformer, this method fixes the text encoder and image encoder in the first training stage, optimizes a set of learnable text features in the second training stage to generate text features, and then uses the text features to optimize the image encoder. However, due to the complex relationship between multiple people in the dataset, the text information generated by the model may be confused, thereby paying attention to the wrong information, and its recognition effect is still slightly lower than that of the method proposed in this paper. The comparison results with the existing advanced models on the dataset verify the effectiveness of this method.

[0136] Embodiment 2:

[0137] This embodiment will introduce an applicable scenario of the present invention.

[0138] In many public places in the city where people flow frequently, such as subway stations, bus stations or large-scale event sites, when the public security department needs to find specific suspects, the technology of the present invention can be used to identify the locations where these people have appeared, and combined with image information, such as address and time, to analyze the trajectory and whereabouts of these people.

[0139] Firstly, pedestrian detection technology is used to extract images showing pedestrians from the original videos of urban surveillance cameras, and these images are named according to the camera identification and shooting time.

[0140] These photos were then input into the model of the present invention.

[0141] The model then identifies occurrences of that individual in other images and finds all images of that person.

[0142] Second, resolve the movement path by image name.

[0143] Ultimately, based on this movement path information, the target individual can be located or further analyzed.

[0144] The model has the ability to quickly identify and track people in crowded environments. With just one picture, the person can be quickly found even if he or she is hiding in a crowd.

[0145] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A crowded pedestrian re-identification method based on unequal width convolutional blocks, characterized in that: The following steps are involved: S1. Collect pedestrian images, and apply a data enhancement method for random person images to preprocess the pedestrian images to obtain visible light images; S2. Construct a key person attention module, extract the character importance feature map using the visible light image as input, and splice it based on the original features of the image to obtain the key person attention map; S3, using the ResNet50 network and the unequal width convolutional block NuCB, extract the image features rich in key person information from the key person attention map; S4, based on the key person attention module, ResNet50 network and unequal width convolution block NUCB, takes pedestrian images as input and pedestrian recognition results as output to build and train a pedestrian re-identification model. The key person constraint loss is used in the training, and the image features output by S3 are combined with the difficult sample triple loss function and the cross entropy loss function of label smoothing regularization to jointly train the pedestrian re-identification model; S5. The collected pedestrian image is input into a pedestrian re-identification model to obtain a pedestrian recognition result.

2. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 1, characterized in that: Step S1 includes the following sub-steps: S1.

1. Use deeplabv3_resnet101 to extract the person images of ImageNet for pedestrian images, and use the random person image method for the person images. The images are randomly selected for erasure with probability P; S1.

2. For the selected image, randomly select a rectangular area whose total area is less than or equal to the preset area of ​​the total image, and randomly select one of the people images from ImageNet and modify its area to the size of the rectangular area; S1.

3. Use the resized ImageNet person images to fill the rectangular area selected in the image.

3. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 2, characterized in that: The resized ImageNet character image is filled into the rectangular area selected in the image, as follows: Where S is a randomly selected image area, a(i,j) represents the image pixel input value, another(i,j) represents the pixel value of the filled image, i0 is the starting point of the horizontal coordinate of area S, and j0 is the starting point of the vertical coordinate of area S.

4. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 1, characterized in that: Step S2 includes the following sub-steps: S2.

1. The visible light image can be used to identify key figures and output the key figure importance features; S2.2, the key person importance feature map and the mask image of the image are input into the key person constraint loss together; S2.

3. Concatenate the key person importance feature map and the original features of the image, input them into the convolutional network, obtain the key person attention map, and perform dot multiplication on it and the input image.

5. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 4, characterized in that: In step S2.1, the key character importance features are extracted and output by constructing a key attention module; The key attention module includes a first basic convolution block, a first maximum pooling layer, a second basic convolution block, a second maximum pooling layer, a third basic convolution block, a third maximum pooling layer, a fourth basic convolution block, a first deconvolution layer, a fifth basic convolution block, a second deconvolution layer, a sixth basic convolution block, a third deconvolution layer, and a convolution layer connected sequentially; The first basic convolution block, the second basic convolution block, and the third basic convolution block are used to increase the dimension of the input features; The first, second, and third maximum pooling layers are used for downsampling to reduce the input feature size; The fourth basic convolution block, the fifth basic convolution block, and the sixth basic convolution block are used to reduce the dimension of the input features; The first deconvolution layer, the second deconvolution layer, and the third deconvolution layer are used to enlarge the size of the input features and convert the low-resolution feature map into a high-resolution feature map; Convolutional layers are used to reduce the high-resolution feature map dimension to 1.

6. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 4, characterized in that: In step S2.2, the key person constraint loss is as follows: In the formula, y ij It means that when the horizontal and vertical coordinates are i and j respectively, the category corresponding to the real category of the mask here, It represents the value predicted by the model when the horizontal and vertical coordinates are i and j respectively. N represents the total number of images in a batch, and H and W represent the total height and width of the images respectively. When the mask image is of multiple categories, ij The constraints are as follows: Apply the idea of ​​labelsmooth to the mask image. Based on the fact that the mask image is a three-value matrix, the following constraints are imposed on the matrix values: Wherein, is the hyperparameter set according to the experiment, ∈ is the preset hyperparameter, and K is the preset hyperparameter.

7. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 1, characterized in that: Step S3 includes the following sub-steps: S3.

1. Input the key person attention map into a network consisting of ResNet50 and unequal width convolution blocks, perform convolution operations, and output information-rich image features; In the unequal width convolution block NUCB, the convolution operation is implemented by unequal width convolution kernels. The width of a part of the small convolution kernel is increased to (n+2)*(n+2). The output image features are as follows: Where Input[c,h+i-1,w+j-1] is the element in the input feature map, h and w represent the height and width of the convolution kernel respectively, Kernel[k,c,i,j] is the weight of the (i,j) position of the cth channel of the kth convolution kernel, and b[k] is the bias term of the kth output channel; S3.

2. Perform MAX-pooling on the features obtained in step S3.2 and output the pooled feature map.

8. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 6, characterized in that: In step S4, the difficult sample triplet loss function is to select K images of each pedestrian in a batch sample containing P different pedestrians, and then form them into a data set of size P×K. Then, for each training, each image in this data set is used as an anchor image, and the two images with the longest intra-class distance and the shortest inter-class distance are selected to form a difficult triplet, and the difficult sample triplet loss function is constructed as follows: Where A is the anchor sample, Pos is the positive sample, Neg is the negative sample, α is the threshold parameter, and d A.Pos is the distance from the anchor point to the positive sample, d A.Neg is the distance from the anchor point to the negative sample.

9. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 8, characterized in that: In step S4, smooth labels are introduced into the cross entropy loss as follows: Where N is the number of pedestrian IDs, y is the real label of the pedestrian, and p i is the output predicted identity probability value, ε is the error rate,.

10. The method for crowded pedestrian re-identification based on unequal width convolutional blocks according to claim 9, characterized in that: In step S4, the joint loss function L sum As follows: L sum =L keyperson +L tri_hard +L lsr Among them, λ is a hyperparameter.

Images