Pedestrian re-identification method, device and storage medium based on pedestrian re-identification model

Abstract

The present application discloses a pedestrian re-identification method based on a pedestrian re-identification model, a computer device and a computer-readable storage medium. The method comprises: optimizing a preset deep model by randomly selecting pedestrian images to be trained, presetting feature extraction modules, presetting local correlation attention modules and presetting overall correlation attention modules to obtain optimized pedestrian features of the pedestrian images to be trained and optimized pedestrian features with overall correlation, generating a pedestrian re-identification model to obtain pedestrian recognition results of the pedestrian images to be tested, learning the dependencies between pedestrian images and the dependencies within images through the deep model, and integrating the dependencies between images and the dependencies within images into a network structure, learning the robust features of pedestrian images, overcoming the uncertainty problems caused by the variability of the environment, pedestrians, etc., and improving the accuracy of pedestrian re-identification search.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a pedestrian re-identification method based on a pedestrian re-identification model, a computer device, and a computer-readable storage medium. Background Art

[0002] In order to discover existing safety hazards, installing surveillance cameras for all-weather monitoring is a timely and effective means. However, in the face of a large amount of surveillance data, achieving all-weather detection requires a lot of manpower and financial resources, which is costly and difficult to achieve. The purpose of person re-identification is to use computer technology to quickly and efficiently retrieve target pedestrians in video surveillance. However, due to the diversity of surveillance cameras, the complexity of scenes, and the uncertainty of pedestrians, pedestrian re-identification faces huge challenges. Researchers strive to explore more robust pedestrian features to improve the retrieval accuracy of pedestrian re-identification.

[0003] In recent years, the application of deep learning has greatly improved the correct recognition rate of pedestrian re-identification. Since the same pedestrian images taken by different lenses have potential correlation, people generally construct sample pairs to constrain the spatial distribution of pedestrian features and explore the dependencies between images. Varior et al. constructed positive and negative sample pairs and used the twin network to constrain the feature distribution of sample pairs, so that the feature distribution of positive sample pairs is similar and the feature distribution of negative sample pairs is different; Hermans et al. designed a difficult ternary loss to reduce the gap between positive samples and increase the difference between negative samples; Si et al. proposed a compact ternary loss to calculate the feature center of each pedestrian category, and then reduce the feature distribution of the same pedestrian category. In addition, since there is a certain correlation between the local features of pedestrian images, researchers also study the dependencies within the image. Sun et al. divided the convolution activation map of the image into six equal parts, and then performed classification operations on each part to learn the local information of the pedestrian image; Ye et al. directly used the non-local module to obtain the connection between any two pixels in the pedestrian convolution activation map and calculated the long-distance dependencies. However, the above methods of constraining the relationship between features only limit the feature distribution between the same pedestrian images, do not calculate the dependency between the same pedestrian images, and obtain less feature information; the method of learning local features does not pay attention to the foreground information of pedestrians and is prone to losing important features. Summary of the invention

[0004] The main purpose of this application is to provide a pedestrian re-identification method, computer device and computer-readable storage medium based on a pedestrian re-identification model, aiming to solve the technical problems that the existing method of constraining the relationship between features only limits the feature distribution between the same pedestrian images, does not calculate the dependency relationship between the same pedestrian images, and can only obtain less feature information, while the method of learning local features does not pay attention to the foreground information of the pedestrian and is prone to losing important features.

[0005] In a first aspect, the present application provides a pedestrian re-identification method based on a pedestrian re-identification model, the method comprising the following steps:

[0006] Randomly selecting pedestrian images to be trained, and preprocessing the pedestrian images to be trained;

[0007] Input the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain pedestrian features of the pedestrian image to be trained, and optimize the pedestrian features of the pedestrian image to be trained using a preset soft label loss function, a preset ternary loss function, and a preset center loss function;

[0008] Inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain a convolution activation map with local correlation;

[0009] Inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and optimizing the pedestrian features with overall correlation using a preset cross entropy loss function;

[0010] Optimizing the preset deep model based on the optimized pedestrian features of the pedestrian image to be trained and the optimized pedestrian features with overall correlation to generate a pedestrian re-recognition model;

[0011] Inputting the pedestrian image to be tested into the trained pedestrian re-recognition model;

[0012] Pedestrian features of the pedestrian image to be tested are extracted according to the pedestrian re-identification model, and feature similarity is calculated to obtain a pedestrian recognition result of the pedestrian image to be tested.

[0013] Preferably, before inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain the convolution activation map with local correlation, the method further includes:

[0014] A local correlation attention module is constructed, and the constructed local correlation attention module is embedded into the constructed preset feature extraction module to obtain local correlation in the image convolution activation map.

[0015] Preferably, before inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain the pedestrian features with overall correlation, the method further includes:

[0016] Construct an overall correlation attention module, and implant the constructed overall correlation attention module into the constructed preset feature extraction module to obtain the overall correlation of the image convolution activation map.

[0017] Preferably, the step of inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain pedestrian features of the pedestrian image to be trained includes:

[0018] Inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain a convolution activation map of the pedestrian image to be trained;

[0019] Performing a global pooling operation on the convolution activation map of the pedestrian image to be trained to obtain a first pedestrian feature of the pedestrian image to be trained;

[0020] A global pooling operation is performed on the convolution activation map of the pedestrian image to be trained, and a normalization process is performed to obtain a second pedestrian feature of the pedestrian image to be trained.

[0021] Preferably, the optimizing the pedestrian features of the pedestrian image to be trained by using a preset soft label loss function, a preset ternary loss function and a preset center loss function comprises:

[0022] Input the first pedestrian feature into a preset classifier for category prediction, and calculate the loss value of the first pedestrian feature using a preset soft label loss;

[0023] Calculate a first loss value of the second pedestrian feature using a preset ternary loss function;

[0024] A second loss value of the second pedestrian feature is calculated using a preset center loss function.

[0025] Preferably, the step of inputting the convolution activation map in the preset feature extraction module into a preset local correlation attention module to obtain a convolution activation map with local correlation comprises:

[0026] Based on a first weight matrix of a preset local correlation attention module, obtaining a first dependency coefficient of the convolution activation map;

[0027] Based on a second weight matrix of a preset local correlation attention module, obtaining a second dependency coefficient of the convolution activation map;

[0028] Obtaining a dependency coefficient between any two pixels in the convolution activation map through the first dependency coefficient, the second dependency coefficient, the third weight matrix of the preset local correlation attention module, and the convolution activation map;

[0029] A convolution activation map with local correlation is obtained by using the dependency coefficient between any two pixels in the convolution activation map, the fourth weight matrix of the preset local correlation attention module and the convolution activation map.

[0030] Preferably, the step of inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and optimizing the pedestrian features with overall correlation using a preset cross entropy loss function comprises:

[0031] Inputting the convolution activation map with local correlation into a preset overall correlation attention module, performing a matrix operation on the convolution activation map with local correlation, and obtaining a first eigenvector corresponding to the convolution activation map with local correlation;

[0032] According to the first feature vector, obtain a label embedding vector corresponding to the pedestrian type;

[0033] Concatenate the first feature vector and the corresponding label embedding vector to generate a second feature vector;

[0034] Inputting the pedestrian features of the pedestrian image to be trained into a preset overall relevant attention module, and initializing the initial unit state of the preset overall relevant attention module by using the pedestrian features of the pedestrian image to be trained;

[0035] Obtaining a corresponding hidden state according to the second feature vector, the preset initial hidden state of the LSTM, and the initialized initial unit state;

[0036] Using the hidden state as the pedestrian feature with overall relevance of the pedestrian image to be trained, and inputting the hidden state into a classifier to calculate the predicted probability that the pedestrian feature with overall relevance of the pedestrian image to be trained belongs to each category;

[0037] The predicted probability that the pedestrian feature with overall relevance belongs to each category is calculated by a cross entropy loss function to obtain a loss value of the pedestrian feature with overall relevance.

[0038] Preferably, optimizing the pedestrian features of the pedestrian image to be trained includes a loss value of a first pedestrian feature, a loss value of a second pedestrian feature, and a second loss value of the second pedestrian feature; optimizing the pedestrian features with overall correlation includes a loss value of the pedestrian features with overall correlation; optimizing a preset deep model based on the optimized pedestrian features and the optimized pedestrian features with overall correlation to generate a pedestrian re-identification model, including:

[0039] The objective function in the preset deep model is optimized by the loss value of the first pedestrian feature, the loss value of the second pedestrian feature, the second loss value of the second pedestrian feature and the loss value of the pedestrian feature with overall correlation to generate a pedestrian re-identification model.

[0040] In a second aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the deep model-based pedestrian re-identification method as described above are implemented.

[0041] In a third aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the deep model-based pedestrian re-identification method as described above are implemented.

[0042] The present application provides a pedestrian re-identification method, a computer device and a computer-readable storage medium based on a pedestrian re-identification model. The method comprises the following steps: randomly selecting a pedestrian image to be trained, preprocessing the pedestrian image to be trained, inputting the preprocessed pedestrian image to be trained into a preset feature extraction module, obtaining the pedestrian features of the pedestrian image to be trained, and optimizing the pedestrian features of the pedestrian image to be trained using a preset soft label loss function, a preset ternary loss function and a preset center loss function; inputting the convolution activation map in the preset feature extraction module into a preset local correlation attention module to obtain a convolution activation map with local correlation; inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain a pedestrian with overall correlation. Human features, and use a preset cross entropy loss function to optimize the pedestrian features with overall correlation; based on the optimized pedestrian features of the pedestrian image to be trained and the optimized pedestrian features with overall correlation, optimize the preset deep model to generate a pedestrian re-identification model; input the pedestrian image to be tested into the trained pedestrian re-identification model; extract the pedestrian features of the pedestrian image to be tested according to the pedestrian re-identification model, and calculate the feature similarity to obtain the pedestrian recognition result of the pedestrian image to be tested, realize learning the dependency between pedestrian images and the dependency within the image through the deep model, and integrate the dependency between images and the dependency within the image into a network structure, learn the robust features of pedestrian images, overcome the uncertainty problem caused by the variability of the environment, pedestrians, etc., and improve the accuracy of pedestrian re-identification search. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of a pedestrian re-identification method based on a pedestrian re-identification model provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structural framework of a pedestrian re-identification model provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structural framework of the overall relevant attention module provided in the embodiment of the present application;

[0047] Figure 4 The present invention is a block diagram showing the structure of a computer device according to an embodiment of the present application.

[0048] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0051] The embodiment of the present application provides a pedestrian re-identification method based on a pedestrian re-identification model, a computer device, and a computer-readable storage medium. The pedestrian re-identification method based on a pedestrian re-identification model can be applied to a computer device, which can be an electronic device such as a laptop computer or a desktop computer.

[0052] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0053] Please refer to Figure 1 , Figure 1 A flowchart of a pedestrian re-identification method based on a pedestrian re-identification model is provided in an embodiment of the present application.

[0054] like Figure 1 As shown, the method includes steps S101 to S107.

[0055] Step S101: randomly select pedestrian images to be trained, and preprocess the pedestrian images to be trained.

[0056] For example, randomly obtain a small batch of pedestrian images, which contains P pedestrians, each pedestrian has K images, that is, each small batch contains PK pedestrian images. These pedestrian images are normalized, and the data enhancement methods of random cropping, random horizontal flipping and random erasing are performed. For example, each small batch contains 16 pedestrians, each pedestrian has 8 images, and a total of 128 pedestrian images. The small batch images are cropped to 384×128, and all pixel values ​​are subtracted from the corresponding mean and divided by the corresponding variance to perform normalization.

[0057] Step S102: input the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain the pedestrian features of the pedestrian image to be trained, and use a preset soft label loss function, a preset ternary loss function and a preset center loss function to optimize the pedestrian features of the pedestrian image to be trained.

[0058] For example, a public pre-trained residual model is used to construct a feature extraction module, such as Figure 2 The pre-trained residual model used is ResNet-50.

[0059] Randomly obtain small batches of pedestrian images, which contain P pedestrians and K images of each pedestrian, that is, each small batch contains PK pedestrian images. These pedestrian images are normalized and data enhancement methods such as random cropping, random horizontal flipping, and random erasing are performed. Perform a global pooling operation on the last convolution activation map of the feature extraction module to obtain the image feature f e , the image feature f e Input the classifier to make category prediction, and then use the soft label loss to calculate the image feature f e The loss value is expressed as Among them, p(r)∈[0,1] represents the image feature f e The predicted probability of belonging to category r, ε∈[0,1] represents the hyperparameter, and y is the image feature f e The true label of , R represents the total number of pedestrian labels in the training set. The loss value of a small batch of pedestrian images is expressed as: Among them, p represents the number of pedestrian categories in the input mini-batch, k represents the number of images of each pedestrian category in the input mini-batch, Represents the loss value of the ath image. For example, the hyperparameter ε can be set to 0.1, and when training in the pedestrian database Market1501, the total number of sample categories R is 751.

[0060] A global pooling operation is performed on the final convolution activation map of the feature extraction module, and normalization (BN) is performed to obtain the pedestrian image feature f v , using the ternary loss function to calculate the pedestrian image feature f v Constraints are performed, and the loss value is expressed as:

[0061]

[0062] Where m represents the threshold value, represents the feature vector of the a-th pedestrian image in the input mini-batch, f p represents the feature vector of all images in the mini-batch that have the same category label as the a-th pedestrian image, f n represents the feature vector of all images in the mini-batch that have different category labels from the a-th pedestrian image, and Indicates the distance between two vectors, maxD represents the maximum distance, minD represents the minimum distance, [h] + =max(0,h) means taking the maximum value between 0 and h. The threshold m can be set to 0.3.

[0063] Based on the obtained pedestrian image features f v , using the central loss function to calculate the feature vector f v Constrained, the loss value calculated by the center loss can be expressed as:

[0064]

[0065] Among them, y a represents the label of the a-th pedestrian image in a mini-batch, represents the number of pedestrians y in a small batch a The feature center of the category. For example, the hyperparameter ε can be set to 0.1, and when training in the pedestrian database Market1501, the total number of sample categories R is 751.

[0066] Step S103: input the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain a convolution activation map with local correlation.

[0067] Exemplarily, a local correlation attention module is constructed and embedded into Block2 and Block3 in the constructed feature extraction module to obtain local correlation in the image convolution activation map, such as Figure 2 shown.

[0068] For example, the obtained convolution activation map with local correlation is represented as:

[0069] Z=W Z y+x, where x is the input convolution activation map, W Z represents the fourth weight matrix, y represents the dependency coefficient between two pixels at any position in the convolution activation map, and the dependency coefficient between any two pixels can be expressed as: y = [d1(x) + d2(x)]g(x), where g(x) = W g x represents a linear embedding, where W g is the third weight matrix, d1(x) and d2(x) both represent the dependency coefficients between any two pixels, and the dependency coefficients d1(x) and d2(x) can be expressed as:

[0070]

[0071] in, in represents the first weight matrix and Represents the second weight matrix. The weight matrices are all obtained through 3×3 convolution kernel learning.

[0072] Step S104: input the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and use a preset cross entropy loss function to optimize the pedestrian features with overall correlation.

[0073] Exemplarily, an overall correlation attention module is constructed and implanted after Block2 and Block3 in the constructed feature extraction module to obtain the overall correlation of the image convolution activation map, such as Figure 2 shown.

[0074] Based on the obtained convolution activation map, matrix operations are performed on the convolution activation map to obtain the feature vector corresponding to the feature map That represents the kth image of the pth category in the mini-batch; the matrix operation on the convolution activation map is as follows Figure 3 As shown, based on the feature vector Pedestrian category, obtain the label embedding vector l corresponding to the pedestrian category r , the label embedding vector l r It can be expressed as: r =E·r, where E represents the label embedding matrix and r represents the feature vector The corresponding label. The size of the label embedding matrix E on the database Market1501 is 751×256.

[0075] Based on the feature vector and the corresponding label embedding vector l r , concatenate them as the feature vector Based on the obtained feature vector Will Input LSTM to obtain the hidden state h at the pth image moment p , shown as hidden state h p It can be expressed as:

[0076]

[0077] Among them, the initial hidden state h of LSTM p and the initial cell state c p The feature vector f in the feature extraction module v Initialize. , the obtained image hidden state h p As the image feature, it is input into the classifier to calculate the image feature h pThe predicted probability of belonging to the pth class can be expressed as:

[0078] p=W p ·h t +b p

[0079] Among them, W p represents the weight matrix of the classifier, b p Denotes the deviation. The loss value of the small batch image feature is calculated using the cross entropy loss, which can be expressed as:

[0080]

[0081] Among them, p(a) represents the predicted probability that the a-th image belongs to the true label.

[0082] Step S105 , optimizing the preset deep model based on the optimized pedestrian features of the pedestrian image to be trained and the optimized pedestrian features with overall correlation, to generate a pedestrian re-recognition model.

[0083] Exemplarily, the loss value of the first pedestrian feature, the loss value of the second pedestrian feature, the second loss value of the second pedestrian feature, and the loss value of the pedestrian feature with overall correlation are obtained to optimize the objective function in the preset deep model. For example, the final objective function of optimizing the deep model is expressed as:

[0084] L SDN =L T +αL' I +γL C +λL G

[0085] Here, α, γ, and λ represent weights. For example, α=0.5, γ=0.0005, and λ=2 may be set.

[0086] Step S106: input the pedestrian image to be tested into the trained pedestrian re-recognition model.

[0087] Step S107 : extracting pedestrian features of the pedestrian image to be tested according to the pedestrian re-identification model, and calculating feature similarity to obtain a pedestrian recognition result of the pedestrian image to be tested.

[0088] Exemplarily, the test pedestrian image is input into the trained feature extraction module, and the pedestrian feature f is extracted based on the computing system. v , calculate the feature similarity, perform pedestrian matching, and then obtain the pedestrian recognition result.

[0089] Finally, the performance of the pedestrian re-identification method and computing system based on spatial driving features can also be evaluated according to the search accuracy.

[0090] Using a large-scale online public person re-identification database as a test object, for example, when testing on the Market 1501 database, the correct search rate of the person re-identification of the present invention reaches 95.8%. The person re-identification method of the present invention effectively uses a computing system to explore the dependencies between images and the dependencies within images, and integrates these two relationships into a model framework, which greatly improves the correct search rate of the person re-identification, which shows the effectiveness of the method of the present invention.

[0091] In an embodiment of the present application, by randomly selecting a pedestrian image to be trained, preprocessing the pedestrian image to be trained, inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain the pedestrian features of the pedestrian image to be trained, and using a preset soft label loss function, a preset ternary loss function and a preset center loss function to optimize the pedestrian features of the pedestrian image to be trained; the convolution activation map in the preset feature extraction module is input into a preset local correlation attention module to obtain a convolution activation map with local correlation; the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained are input into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and a preset cross entropy loss function is used. Optimize the pedestrian features with overall correlation; optimize the preset deep model based on the optimized pedestrian features of the pedestrian image to be trained and the optimized pedestrian features with overall correlation to generate a pedestrian re-identification model; input the pedestrian image to be tested into the trained pedestrian re-identification model; extract the pedestrian features of the pedestrian image to be tested according to the pedestrian re-identification model, and calculate the feature similarity to obtain the pedestrian recognition result of the pedestrian image to be tested, so as to realize the learning of the dependency between pedestrian images and the dependency within the image through the deep model, and integrate the dependency between images and the dependency within the image into a network structure, learn the robust features of pedestrian images, overcome the uncertainty problems caused by the variability of the environment, pedestrians, etc., and improve the accuracy of pedestrian re-identification search.

[0092] See also Figure 4 , Figure 4 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a terminal.

[0093] like Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0094] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the methods.

[0095] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0096] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the methods.

[0097] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0098] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0099] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0100] Randomly selecting pedestrian images to be trained, and preprocessing the pedestrian images to be trained;

[0101] Input the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain pedestrian features of the pedestrian image to be trained, and optimize the pedestrian features of the pedestrian image to be trained using a preset soft label loss function, a preset ternary loss function, and a preset center loss function;

[0102] Inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain a convolution activation map with local correlation;

[0103] Inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and optimizing the pedestrian features with overall correlation using a preset cross entropy loss function;

[0104] Optimizing the preset deep model based on the optimized pedestrian features of the pedestrian image to be trained and the optimized pedestrian features with overall correlation to generate a pedestrian re-recognition model;

[0105] Inputting the pedestrian image to be tested into the trained pedestrian re-recognition model;

[0106] Pedestrian features of the pedestrian image to be tested are extracted according to the pedestrian re-identification model, and feature similarity is calculated to obtain a pedestrian recognition result of the pedestrian image to be tested.

[0107] In one embodiment, the processor implements inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain the convolution activation map with local correlation, and is used to implement:

[0108] A local correlation attention module is constructed, and the constructed local correlation attention module is embedded into the constructed preset feature extraction module to obtain local correlation in the image convolution activation map.

[0109] In one embodiment, the processor implements inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain the pedestrian features with overall correlation, and is used to implement:

[0110] Construct an overall correlation attention module, and implant the constructed overall correlation attention module into the constructed preset feature extraction module to obtain the overall correlation of the image convolution activation map.

[0111] In one embodiment, the processor implements inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain pedestrian features of the pedestrian image to be trained, and is used to implement:

[0112] Inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain a convolution activation map of the pedestrian image to be trained;

[0113] Performing a global pooling operation on the convolution activation map of the pedestrian image to be trained to obtain a first pedestrian feature of the pedestrian image to be trained;

[0114] A global pooling operation is performed on the convolution activation map of the pedestrian image to be trained, and a normalization process is performed to obtain a second pedestrian feature of the pedestrian image to be trained.

[0115] In one embodiment, when the processor optimizes the pedestrian features of the pedestrian image to be trained by using a preset soft label loss function, a preset ternary loss function, and a preset center loss function, it is used to implement:

[0116] Input the first pedestrian feature into a preset classifier for category prediction, and calculate the loss value of the first pedestrian feature using a preset soft label loss;

[0117] Calculate a first loss value of the second pedestrian feature using a preset ternary loss function;

[0118] A second loss value of the second pedestrian feature is calculated using a preset center loss function.

[0119] In one embodiment, the processor implements inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain the convolution activation map with local correlation, and is used to implement:

[0120] Based on a first weight matrix of a preset local correlation attention module, obtaining a first dependency coefficient of the convolution activation map;

[0121] Based on a second weight matrix of a preset local correlation attention module, obtaining a second dependency coefficient of the convolution activation map;

[0122] Obtaining a dependency coefficient between any two pixels in the convolution activation map through the first dependency coefficient, the second dependency coefficient, the third weight matrix of the preset local correlation attention module, and the convolution activation map;

[0123] A convolution activation map with local correlation is obtained by using the dependency coefficient between any two pixels in the convolution activation map, the fourth weight matrix of the preset local correlation attention module and the convolution activation map.

[0124] In one embodiment, the processor implements inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain the pedestrian features with overall correlation, and optimizing the pedestrian features with overall correlation using a preset cross entropy loss function, to implement:

[0125] Inputting the convolution activation map with local correlation into a preset overall correlation attention module, performing a matrix operation on the convolution activation map with local correlation, and obtaining a first eigenvector corresponding to the convolution activation map with local correlation;

[0126] According to the first feature vector, obtain a label embedding vector corresponding to the pedestrian type;

[0127] Concatenate the first feature vector and the corresponding label embedding vector to generate a second feature vector;

[0128] Inputting the pedestrian features of the pedestrian image to be trained into a preset overall relevant attention module, and initializing the initial unit state of the preset overall relevant attention module by using the pedestrian features of the pedestrian image to be trained;

[0129] Obtaining a corresponding hidden state according to the second feature vector, the preset initial hidden state of the LSTM, and the initialized initial unit state;

[0130] Using the hidden state as the pedestrian feature with overall relevance of the pedestrian image to be trained, and inputting the hidden state into a classifier to calculate the predicted probability that the pedestrian feature with overall relevance of the pedestrian image to be trained belongs to each category;

[0131] The predicted probability that the pedestrian feature with overall relevance belongs to each category is calculated by a cross entropy loss function to obtain a loss value of the pedestrian feature with overall relevance.

[0132] In one embodiment, the processor implements optimization of the pedestrian features of the pedestrian image to be trained, including the loss value of the first pedestrian feature, the loss value of the second pedestrian feature, and the second loss value of the second pedestrian feature; optimizing the pedestrian features with overall correlation, including the loss value of the pedestrian features with overall correlation; optimizing the preset deep model based on the optimized pedestrian features and the optimized pedestrian features with overall correlation, and generating a pedestrian re-identification model, for implementing:

[0133] The objective function in the preset deep model is optimized by the loss value of the first pedestrian feature, the loss value of the second pedestrian feature, the second loss value of the second pedestrian feature and the loss value of the pedestrian feature with overall correlation to generate a pedestrian re-identification model.

[0134] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the pedestrian re-identification method based on the pedestrian re-identification model of the present application.

[0135] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0136] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0137] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A pedestrian re-identification method based on a pedestrian re-identification model, characterized in that: include: Randomly select pedestrian images to be trained, and preprocess the pedestrian images to be trained, including: randomly obtaining small batches of pedestrian images, including P pedestrians, K images of each pedestrian, each of the small batches of pedestrian images including PK pedestrian images, normalizing the pedestrian images, and performing data enhancement methods of random cropping, random horizontal flipping and random erasing, wherein each small batch contains 16 pedestrians, 8 images of each pedestrian, and a total of 128 pedestrian images, cropping the small batch images to 384×128, subtracting the corresponding mean from all pixel values, and dividing by the corresponding variance to perform normalization operation; Inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain the pedestrian features of the pedestrian image to be trained, and optimizing the pedestrian features of the pedestrian image to be trained using a preset soft label loss function, a preset ternary loss function, and a preset center loss function, including: inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain the pedestrian features of the pedestrian image to be trained, and optimizing the pedestrian features of the pedestrian image to be trained using a preset soft label loss function, a preset ternary loss function, and a preset center loss function, including: performing a global pooling operation on the last convolution activation map of the preset feature extraction module to obtain an image feature f e , the image feature f e Input the classifier to make category prediction, and then use the soft label loss to calculate the image feature f e The loss value is expressed as Among them, p(r)∈[0,1] represents the image feature f e The predicted probability of belonging to category r, ε∈[0,1] represents the hyperparameter, and y is the image feature f e The true label of , R represents the total number of pedestrian labels in the training set, and the loss value of the small batch of pedestrian images is expressed as: Among them, p represents the number of pedestrian categories in the input mini-batch, k represents the number of images of each pedestrian category in the input mini-batch, It represents the loss value of the a-th image. The hyperparameter ε can be set to 0.

1. When training in the pedestrian database Market1501, the total number of sample categories R is 751; The convolution activation map in the preset feature extraction module is input into the preset local correlation attention module to obtain a convolution activation map with local correlation, including: performing a global pooling operation on the last convolution activation map of the preset feature extraction module, and performing normalization (BN) to obtain the pedestrian image feature f v , using the ternary loss function to v Constraints are imposed, and the loss value is expressed as: Where m represents the threshold value, represents the feature vector of the a-th pedestrian image in the input mini-batch, f p represents the feature vector of all images in the mini-batch that have the same category label as the a-th pedestrian image, f n represents the feature vector of all images in the mini-batch that have different category labels from the a-th pedestrian image, and Indicates the distance between two vectors, maxD represents the maximum distance, minD represents the minimum distance, [h] + =max(0,h) means taking the maximum value between 0 and h, and the threshold m can be set to 0.3; Based on the obtained pedestrian image features f v , using the central loss function to calculate the feature vector f v Constrained, the loss value calculated by the center loss can be expressed as: Among them, y a represents the label of the a-th pedestrian image in a mini-batch, represents the number of pedestrians y in a small batch a The feature center of the category, the hyperparameter ε can be set to 0.

1. When training in the pedestrian database Market1501, the total number of sample categories R is 751; Inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain a convolution activation map with local correlation, including: constructing a local correlation attention module and embedding it into Block 2 and Block 3 in the constructed feature extraction module to obtain local correlation in the image convolution activation map; The obtained convolution activation map with local correlation is expressed as: Z=W Z y+x, where x is the input convolution activation map, W Z represents the fourth weight matrix, y represents the dependency coefficient between two pixels at any position in the convolution activation map, and the dependency coefficient between any two pixels can be expressed as: y = [d1(x) + d2(x)]g(x), where g(x) = W g x represents a linear embedding, where W g is the third weight matrix, d1(x) and d2(x) both represent the dependency coefficients between any two pixels, and the dependency coefficients d1(x) and d2(x) can be expressed as: in, in represents the first weight matrix and represents the second weight matrix, and the weight matrices are all obtained through 3×3 convolution kernel learning; Inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and optimizing the pedestrian features with overall correlation using a preset cross entropy loss function; Optimizing the preset deep model based on the optimized pedestrian features of the pedestrian image to be trained and the optimized pedestrian features with overall correlation to generate a pedestrian re-recognition model; Inputting the pedestrian image to be tested into the trained pedestrian re-recognition model; Pedestrian features of the pedestrian image to be tested are extracted according to the pedestrian re-identification model, and feature similarity is calculated to obtain a pedestrian recognition result of the pedestrian image to be tested.

2. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 1, characterized in that: Before inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain the convolution activation map with local correlation, the method further includes: A local correlation attention module is constructed, and the constructed local correlation attention module is embedded into the constructed preset feature extraction module to obtain local correlation in the image convolution activation map.

3. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 1, characterized in that: Before inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain the pedestrian features with overall correlation, the method further includes: Construct an overall correlation attention module, and implant the constructed overall correlation attention module into the constructed preset feature extraction module to obtain the overall correlation of the image convolution activation map.

4. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 1, characterized in that: The step of inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain pedestrian features of the pedestrian image to be trained includes: Inputting the preprocessed pedestrian image to be trained into a preset feature extraction module to obtain a convolution activation map of the pedestrian image to be trained; Performing a global pooling operation on the convolution activation map of the pedestrian image to be trained to obtain a first pedestrian feature of the pedestrian image to be trained; A global pooling operation is performed on the convolution activation map of the pedestrian image to be trained, and a normalization process is performed to obtain a second pedestrian feature of the pedestrian image to be trained.

5. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 4, characterized in that: The method of optimizing the pedestrian features of the pedestrian image to be trained by using a preset soft label loss function, a preset ternary loss function, and a preset center loss function includes: Input the first pedestrian feature into a preset classifier for category prediction, and calculate the loss value of the first pedestrian feature using a preset soft label loss; Calculate a first loss value of the second pedestrian feature using a preset ternary loss function; A second loss value of the second pedestrian feature is calculated using a preset center loss function.

6. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 1, characterized in that: The step of inputting the convolution activation map in the preset feature extraction module into the preset local correlation attention module to obtain the convolution activation map with local correlation comprises: Based on a first weight matrix of a preset local correlation attention module, obtaining a first dependency coefficient of the convolution activation map; Based on a second weight matrix of a preset local correlation attention module, obtaining a second dependency coefficient of the convolution activation map; Obtaining a dependency coefficient between any two pixels in the convolution activation map through the first dependency coefficient, the second dependency coefficient, the third weight matrix of the preset local correlation attention module, and the convolution activation map; A convolution activation map with local correlation is obtained by using the dependency coefficient between any two pixels in the convolution activation map, the fourth weight matrix of the preset local correlation attention module and the convolution activation map.

7. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 1, characterized in that: The step of inputting the convolution activation map with local correlation and the pedestrian features of the pedestrian image to be trained into a preset overall correlation attention module to obtain pedestrian features with overall correlation, and optimizing the pedestrian features with overall correlation using a preset cross entropy loss function comprises: Inputting the convolution activation map with local correlation into a preset overall correlation attention module, performing a matrix operation on the convolution activation map with local correlation, and obtaining a first eigenvector corresponding to the convolution activation map with local correlation; According to the first feature vector, obtain a label embedding vector corresponding to the pedestrian type; Concatenate the first feature vector and the corresponding label embedding vector to generate a second feature vector; Inputting the pedestrian features of the pedestrian image to be trained into a preset overall relevant attention module, and initializing the initial unit state of the preset overall relevant attention module by using the pedestrian features of the pedestrian image to be trained; Obtaining a corresponding hidden state according to the second feature vector, the preset initial hidden state of the LSTM, and the initialized initial unit state; Using the hidden state as the pedestrian feature with overall relevance of the pedestrian image to be trained, and inputting the hidden state into a classifier to calculate the predicted probability that the pedestrian feature with overall relevance of the pedestrian image to be trained belongs to each category; The predicted probability that the pedestrian feature with overall relevance belongs to each category is calculated by a cross entropy loss function to obtain a loss value of the pedestrian feature with overall relevance.

8. The pedestrian re-identification method based on the pedestrian re-identification model according to claim 1, characterized in that: Optimizing the pedestrian features of the pedestrian image to be trained includes a loss value of the first pedestrian feature, a loss value of the second pedestrian feature, and a second loss value of the second pedestrian feature; optimizing the pedestrian features with overall correlation includes a loss value of the pedestrian features with overall correlation; Optimizing the preset deep model based on the optimized pedestrian features and the optimized pedestrian features with overall correlation to generate a pedestrian re-identification model, including: The objective function in the preset deep model is optimized by the loss value of the first pedestrian feature, the loss value of the second pedestrian feature, the second loss value of the second pedestrian feature and the loss value of the pedestrian feature with overall correlation to generate a pedestrian re-identification model.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the deep model-based pedestrian re-identification method as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the deep model-based pedestrian re-identification method as claimed in any one of claims 1 to 8 are implemented.

Images