Pedestrian re-identification method, apparatus and device, and storage medium

Abstract

The embodiment of the invention relates to the technical field of data processing, in particular to a pedestrian re-identification method, device and equipment and a storage medium, and aims to improve the performance of a pedestrian re-identification model. The method comprises the following steps: selecting pedestrian images of various categories to obtain a first image combination; pedestrian identity features of the pedestrian images in the first image set are extracted, and an identity feature relation graph is constructed; according to the relation graph, forming a second image set by the pedestrian images of each category and the corresponding images of adjacent categories; pedestrian change features of the pedestrian images in the second image set are extracted; forming a third image set by each sample image and the pedestrian images of different pedestrian categories; fusing the pedestrian identity features of the sample image with the pedestrian change features of the other images to obtain generation features; combining the generated features with the original features to obtain a corresponding training set; using the training set to train a pedestrian re-identification model; and obtaining a pedestrian recognition result through the pedestrian re-recognition model.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data processing, and more specifically, to a pedestrian re-identification method, apparatus, device, and storage medium. Background Art

[0002] Pedestrian re-identification is an identity recognition technology that uses computer vision methods to retrieve the identity of a specific pedestrian in an image or video library. In the prior art, a pedestrian re-identification model constructed by a neural network is used to identify images or videos.

[0003] In the prior art, since the training data set often exhibits a long-tail distribution, that is, the head categories have a large number of samples, while the number of samples in the tail categories is extremely small, it is difficult for the neural network to fully learn the features of the tail categories during the training process, resulting in poor recognition of the tail categories by the model and unsatisfactory recognition effects. Summary of the Invention

[0004] The embodiments of the present application provide a pedestrian re-identification method, apparatus, device, and storage medium, aiming to improve the performance of the pedestrian re-identification model.

[0005] In the first aspect of the embodiments of the present application, a pedestrian re-identification method is provided. The method includes: Select one pedestrian image as a representative sample from each pedestrian category image in the pedestrian image data set to obtain a first image set; Extract the pedestrian identity features of all pedestrian images in the first image set through the pedestrian re-identification model, and construct a corresponding identity feature relationship graph. A category attention mechanism is added to the pedestrian re-identification model; According to the identity feature relationship graph, form a second image set with the pedestrian images of each category and the corresponding adjacent category images; Extract the pedestrian change features of all pedestrian images in the second image set through the pedestrian re-identification model; According to the pedestrian change features, form a third image set with each sample image in the second image set and the pedestrian images with different pedestrian categories from the sample image; Fuse the pedestrian identity features of the sample images in the third image set with the pedestrian change features of the remaining images to obtain generated features; Merge the generated features and the original features in the second image set into the same training batch to obtain a corresponding training set; Train the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model; Perform a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian recognition result.

[0006] Optionally, the steps for constructing the person re-identification model include: Extract first person convolution features from the person image through the backbone network of the person re-identification model; Convert the first person convolution features into query features, key features, and value features; Calculate corresponding attention values according to the semantic correlations between different channels of the query features and the key features; Fuse the attention values with the value features through an attention weighting method to obtain second person convolution features; Optimize the second person convolution features through a residual learning method to obtain third person convolution features; Map the third person convolution features to a class attention mask by using a preset mapping function, so as to decompose the first person convolution features into the person identity features and the person variation features through the class attention mask.

[0007] Optionally, the steps for constructing the person re-identification model further include: Input the person identity features and the person variation features into a person classifier; Optimize the parameters according to the person identity features through the identity decomposition loss function in the person classifier; Optimize the parameters according to the person variation features through the variation decomposition loss function in the person classifier.

[0008] Optionally, the extracting the person identity features of all person images in the first image set through the person re-identification model includes: Input the person image into the person re-identification model; Extract the person features corresponding to the person image through the person re-identification model; Decompose the person identity features from the person features through the class attention module in the person re-identification model.

[0009] Optionally, the constructing the corresponding identity feature relationship graph includes: Determine the class distance matrix between all the person identity features; Determine the adjacent classes corresponding to each person class from the class distance matrix; Construct the identity feature relationship graph with each person class as a vertex and the relationship between classes as an edge.

[0010] Optionally, the forming the second image set with the person images of each class and the corresponding adjacent class images according to the identity feature relationship graph includes: For any pedestrian category, retrieve the adjacent categories connected to the pedestrian category according to the identity feature relationship graph to obtain a corresponding category set; Select a preset number of the sample images for each category in the category set to obtain the second image set.

[0011] Optionally, the step of forming a third image set by using each sample image in the second image set and pedestrian images with pedestrian categories different from the sample image according to the pedestrian change characteristics includes: Perform clustering analysis on the pedestrian change characteristics; For each sample image, select pedestrian images that belong to different clustering centers and different pedestrian categories from the sample image to form the third image set.

[0012] A second aspect of the embodiments of the present application provides a pedestrian re-identification device, and the device includes: A first image set acquisition module, configured to select a pedestrian image as a representative sample from the images of each pedestrian category in a pedestrian image data set to obtain a first image set; An identity relationship graph construction module, configured to extract pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model and construct a corresponding identity feature relationship graph, and a category attention mechanism is added to the pedestrian re-identification model; A second image set acquisition module, configured to form a second image set by using the pedestrian images of each category and the corresponding adjacent category images according to the identity feature relationship graph; A pedestrian change feature extraction module, configured to extract pedestrian change features of all pedestrian images in the second image set through the pedestrian re-identification model; A third image set acquisition module, configured to form a third image set by using each sample image in the second image set and pedestrian images with pedestrian categories different from the sample image according to the pedestrian change characteristics; A feature fusion module, configured to fuse the pedestrian identity features of the sample images in the third image set with the pedestrian change features of the remaining images to obtain generated features; A training set acquisition module, configured to merge the generated features and the original features in the second image set into the same training batch to obtain a corresponding training set; A model training module, configured to train the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model; A pedestrian re-identification module, configured to perform a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian identification result.

[0013] Optionally, the steps for constructing the person re-identification model include: Extract first person convolution features from the person image through the backbone network of the person re-identification model; Convert the first person convolution features into query features, key features, and value features; Calculate corresponding attention values according to the semantic correlation between different channels of the query features and the key features; Fuse the attention values with the value features through an attention weighting method to obtain second person convolution features; Optimize the second person convolution features through a residual learning method to obtain third person convolution features; Use a preset mapping function to map the third person convolution features into a class attention mask, and decompose the first person convolution features into the person identity features and the person variation features through the class attention mask.

[0014] Optionally, the steps for constructing the person re-identification model further include: Input the person identity features and the person variation features into a person classifier; Optimize the parameters according to the person identity features through the identity decomposition loss function in the person classifier; Optimize the parameters according to the person variation features through the variation decomposition loss function in the person classifier.

[0015] Optionally, the identity feature relationship graph construction module includes: An image input sub-module for inputting the person image into the person re-identification model; A person feature extraction sub-module for extracting person features corresponding to the person image through the person re-identification model; A person identity feature extraction sub-module for decomposing the person identity features from the person features through the class attention module in the person re-identification model.

[0016] Optionally, the identity relationship graph construction module further includes: A class distance matrix calculation sub-module for determining the class distance matrix between all the person identity features; An adjacent class determination sub-module for determining the adjacent classes corresponding to each person class from the class distance matrix; An identity feature relationship graph construction sub-module for constructing the identity feature relationship graph with each person class as a vertex and the relationship between classes as an edge.

[0017] Optionally, the second image set acquisition module includes: A category set acquisition sub-module, configured to retrieve adjacent categories connected to the human category according to the identity feature relationship graph for any human category, so as to obtain a corresponding category set; A second image set acquisition sub-module, configured to select a preset number of the sample images for each category in the category set to obtain the second image set.

[0018] Optionally, the third image set acquisition module includes: A clustering analysis sub-module, configured to perform clustering analysis on the pedestrian change features; A third image set acquisition sub-module, configured to, for each of the sample images, select pedestrian images that belong to different clustering centers and different human categories from the sample image to form the third image set.

[0019] A third aspect of the embodiments of the present application provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect of the present application is implemented.

[0020] A fourth aspect of the embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0021] Using the pedestrian re-identification method provided by the present application, select a pedestrian image from each human category image in the pedestrian image data set as a representative sample to obtain a first image set; extract the pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model, and construct a corresponding identity feature relationship graph, and a category attention mechanism is added to the pedestrian re-identification model; according to the identity feature relationship graph, form a second image set with the pedestrian images of each category and the corresponding adjacent category images; extract the pedestrian change features of all pedestrian images in the second image set through the pedestrian re-identification model; according to the pedestrian change features, form a third image set with each sample image in the second image set and pedestrian images with different human categories from the sample image; fuse the pedestrian identity features of the sample images in the third image set with the pedestrian change features of the remaining images to obtain generated features; merge the generated features and the original features in the second image set into the same training batch to obtain a corresponding training set; train the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model; perform a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian recognition result.

[0022] In this application, a person re-identification model with a category attention mechanism is used to effectively distinguish the category information of different pedestrians in a pedestrian image, and then accurately extract the pedestrian identity features and pedestrian change features from the pedestrian image. The re-identification model is used to evaluate the distance or similarity between pedestrian categories, construct a corresponding relationship graph for all pedestrian categories, and thus sample the training data to construct the smallest training batch. For the pedestrian images of each category, by obtaining the pedestrian change features of the pedestrian images of other categories and fusing them with the pedestrian identity features of this category, pedestrian images with different changes are generated, thereby enriching the training samples of the pedestrian images of this category, achieving a balanced distribution of the training set samples, effectively improving the training effect of the pedestrian re-identification model, and further improving the accuracy of pedestrian re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the description of the embodiments of the present application will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 is a schematic diagram of pedestrian feature decomposition based on category attention proposed in an embodiment of the present application; Figure 2 is a flowchart of a pedestrian re-identification method proposed in an embodiment of the present application; Figure 3 is a flowchart of pedestrian feature generation based on intra-class change transfer proposed in an embodiment of the present application; Figure 4 is a schematic diagram of a pedestrian re-identification device proposed in an embodiment of the present application; Figure 5 is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0026] In the pedestrian re-identification task, the dataset usually exhibits a long-tailed distribution, which is an important concept in statistics and probability theory used to describe a situation where the tail (i.e., the part far from the mean) of a set of data contains relatively more extreme values. It is a subtype of the long-tailed distribution, characterized by a longer tail in the distribution, meaning there are a large number of extreme values. The occurrence probability of these extreme values in the distribution is low, but their existence has a significant impact on the characteristics of the overall distribution. In the dataset, it means there are more samples in the head categories and fewer samples in the tail categories. This distribution makes it difficult for deep neural networks to fully learn the tail features during training, resulting in poor recognition performance for the tail categories. Traditional feature generation methods mostly use random sampling, which is prone to introducing random noise and further hindering the optimization of the model and the improvement of pedestrian retrieval performance. Traditional pedestrian re-identification methods usually assume that the data follows a uniform distribution, but this assumption seriously does not match the distribution characteristics of the real scenario, so a special solution needs to be designed for the long-tailed distribution. Existing methods can be classified into four categories: data resampling, cost-sensitive learning, margin learning, and data generation.

[0027] The first method is data resampling, which usually adopts an oversampling strategy (increasing the sampling frequency of tail-category samples) or an undersampling strategy (decreasing the sampling frequency of head-category samples) to achieve a more balanced data distribution. Oversampling may lead to overfitting of the model to the noisy information of the tail categories, while undersampling may lose the effective information of the head-category samples. The second method is cost-sensitive learning, which can multiply different weights to the loss values of different categories to control the importance of the categories. Cost-sensitive learning methods often cannot handle large-scale and real-world long-tailed distribution data and will lead to difficulties in optimizing the objective function. The third method is margin learning, which usually uses a large-margin Softmax loss function to assign larger margin values to the tail categories and smaller margin values to the head categories. Since different data have different long-tailed distributions, it is not easy to select a suitable and general margin value. The fourth method is data generation, which uses an additional generative model to synthesize pseudo-samples or features to alleviate the long-tailed distribution problem of the training data. It is difficult for the generative model to guarantee the visual quality of the generated images and the category information of the generated features, so these additional generated pseudo-data will introduce some noise to the model training.

[0028] In this application, while taking into account identity consistency and variation differences, based on a real-scene image set with accurate annotations, the negative impact of the long-tail distribution of training samples on the person re-identification model is deeply analyzed. An effective person feature decomposition method is constructed using a class attention mechanism, decomposing the person feature into a person identity feature and a person variation feature to achieve semantic understanding of the person feature. Aiming at the unbalanced distribution of training data, an intra-class variation transfer method is proposed to transfer the variations of adjacent classes to the tail classes to restore the feature distribution space of the tail classes, which helps to balance the sample distribution of the training set.

[0029] In this application, the construction steps of the re-identification model include: S11: Extract the first person convolution feature from the person image through the backbone network of the person re-identification model.

[0030] In this embodiment, the re-identification model is a model constructed based on a deep learning model, and its backbone network is a convolutional network, which can extract features from images. The first convolution feature is the convolution feature extracted by the backbone network from the person image.

[0031] In this embodiment, refer to Figure 1 , Figure 1 is a schematic diagram of person feature decomposition based on class attention proposed in an embodiment of this application. As Figure 1 shown, when constructing the re-identification model, first construct an attention mechanism module, which is based on a deep learning network to extract the first person convolution feature of the image from the input image. The attention mechanism is an algorithm in deep learning that simulates the allocation of human cognitive resources. By dynamically assigning different weights to different input information, it helps the model focus on key information and improve the ability to handle complex tasks.

[0032] Exemplarily, the person convolution feature extracted from the backbone network is , where represents the number of channels, represents the height, represents the width.

[0033] S12: Convert the first person convolution feature into a query feature, a key feature, and a value feature.

[0034] In this embodiment, inspired by the self-attention mechanism, the person convolution feature is converted into a query feature, a key feature, and a value feature. As Figure 1 shown, a person feature extractor is added to the re-identification model, and a class attention mechanism is added to the person feature extractor to decompose the person feature.

[0035] Exemplarily, different 1 × 1 convolutional layers are used to convert the person feature Convert to query feature , key feature and value feature .

[0036] S13: Calculate the corresponding attention value according to the semantic correlation between different channels of the query feature and the key feature.

[0037] In this embodiment, the query, key, and value achieve information screening and focusing through their synergistic effects. Their core relationship can be summarized as: the query actively initiates a demand, the key passively matches the demand, and the value provides specific content.

[0038] In this embodiment, calculate the corresponding attention value according to the semantic correlation between different channels of the query feature and the key feature.

[0039] S14: Fuse the attention value and the value feature through an attention weighting method to obtain a second pedestrian convolution feature.

[0040] In this embodiment, fuse the attention value and the value feature through an attention weighting method to obtain a second pedestrian convolution feature. The second pedestrian convolution feature is a pedestrian feature that incorporates the attention mechanism.

[0041] Exemplarily, the self-attention mechanism of the convolution feature channel is expressed as:

[0042] where softmax(∙) represents the Softmax function. Since the Softmax function is very sensitive to large or small inputs (easily mapped to 1 or 0), it is scaled by the factor for scaling.

[0043] S15: Optimize the second pedestrian convolution feature through a residual learning method to obtain a third pedestrian convolution feature.

[0044] In this embodiment, the third pedestrian convolution feature is a pedestrian convolution feature obtained by optimizing the second pedestrian convolution feature through the residual learning method. The residual learning method effectively solves the problems of feature degradation and gradient transmission in deep networks by introducing residual mapping and cross-layer connections. After combining with the attention mechanism, it further improves the feature screening and reconstruction capabilities.

[0045] In this embodiment, optimize the second pedestrian convolution feature through a residual learning method, and combine with the attention mechanism to further improve the feature screening and reconstruction capabilities, so as to be able to learn pedestrian features with stronger expressive power.

[0046] Exemplarily, the expression of residual learning is:

[0047] where represents the mapping function of the SE attention module.

[0048] S16: Map the third pedestrian convolutional feature to a class attention mask using a preset mapping function, so as to decompose the first pedestrian convolutional feature into the pedestrian identity feature and the pedestrian variation feature through the class attention mask.

[0049] In this embodiment, the pedestrian identity feature is a feature vector representing the identity of a pedestrian in an image, and the pedestrian variation feature is a feature vector representing the changes in the actions, postures, appearances, etc. of the pedestrian.

[0050] In this embodiment, after obtaining the third pedestrian convolutional feature, map the third pedestrian convolutional feature to a class attention mask using a preset mapping function, and this class attention mask can decompose the first pedestrian convolutional feature into the pedestrian identity feature and the pedestrian variation feature.

[0051] Exemplarily, as Figure 1 shown, use the sigmoid function to map to the class attention mask . Since the attention mask mainly captures the specific semantic combinations of the feature patterns of each sample, the attention mask can be used to decompose the pedestrian identity feature and the pedestrian variation feature from the original features.

[0052]

[0053]

[0054] where represents the all - one matrix, and ⊙ represents the Hadamard product.

[0055] In this embodiment, the steps of constructing the pedestrian re - identification model further include: S17: Input the pedestrian identity feature and the pedestrian variation feature into a pedestrian classifier.

[0056] In this embodiment, the pedestrian classifier is used to classify pedestrians.

[0057] In this embodiment, as Figure 1 shown, after decomposing the pedestrian identity feature and the pedestrian variation feature, input them into the pedestrian classifier.

[0058] S18: Optimize the parameters according to the pedestrian identity features through the identity decomposition loss function in the pedestrian classifier.

[0059] In this embodiment, the identity decomposition loss function is used to train the pedestrian classifier. Through the identity decomposition loss function in the pedestrian classifier, minimize it according to the pedestrian identity features to optimize the network parameters.

[0060] S19: Optimize the parameters according to the pedestrian variation features through the variation decomposition loss function in the pedestrian classifier.

[0061] In this embodiment, the pedestrian variation decomposition loss function is used to train the pedestrian classifier. Through the pedestrian variation decomposition loss function in the pedestrian classifier, minimize it according to the pedestrian variation features to optimize the network parameters.

[0062] Exemplarily, input into the pedestrian classifier. By minimizing the identity decomposition loss function, can be accurately classified, which helps to remove the variation information in. Input into the pedestrian classifier. By minimizing the variation decomposition loss function, the classification probability distribution of gradually approaches the uniform distribution, which helps to remove the identity information in.

[0063] In this embodiment, during the model training process, jointly use the identity decomposition loss function and the pedestrian variation decomposition loss function to constrain and supervise the decomposition process of pedestrian features, so as to realize the semantic decomposition of pedestrian identity and pedestrian variation.

[0064] In this embodiment, a class self-attention module is constructed through the above steps, and the supervised model learns the pedestrian identity features and pedestrian variation features, so that the re-identification model can effectively decompose the original pedestrian features into pedestrian identity features and pedestrian variation features, and then better perform the re-identification task, improve the accuracy of re-identification, and provide a basis for the processing of long-tail data.

[0065] Refer to Figure 2 , Figure 2 is the flowchart of the pedestrian re-identification method proposed in an embodiment of this application. As Figure 1 shown, it specifically includes the following steps: S21: Select one pedestrian image as a representative sample from each pedestrian category image in the pedestrian image data set to obtain the first image set.

[0066] In this embodiment, the pedestrian image data set is a set of images containing pedestrians. The pedestrian category refers to the classification of pedestrians through specific features. For example, pedestrians wearing red clothes are one category of pedestrians, and pedestrians wearing white clothes are another category of pedestrians.

[0067] In this embodiment, to solve the long-tail distribution problem in the training data, first, one pedestrian image is selected from the images of each pedestrian category in the pedestrian image data set as a representative sample to obtain a first image set, which is a data set with a relatively small scale.

[0068] S22: Extract the pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model, and construct a corresponding identity feature relationship graph. A category attention mechanism is added to the pedestrian re-identification model.

[0069] In this embodiment, refer to Figure 3 , Figure 3 is the flowchart of pedestrian feature generation based on intra-class variation transfer proposed in an embodiment of this application. As Figure 3 shown, after obtaining the first image set, extract the pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model, and construct a corresponding identity feature relationship graph according to the pedestrian identity features. Since a category attention mechanism is added to this pedestrian re-identification model, it can well separate the pedestrian identity features from the original features.

[0070] In this embodiment, the specific steps of extracting the pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model include: S22-1: Input the pedestrian image into the pedestrian re-identification model.

[0071] In this embodiment, first input the pedestrian images in the first image set into the pedestrian re-identification model.

[0072] S22-2: Extract the pedestrian features corresponding to the pedestrian image through the pedestrian re-identification model.

[0073] In this embodiment, the pedestrian image is convolved through the backbone network in the pedestrian re-identification model, and the corresponding pedestrian features are extracted through a pedestrian feature extractor.

[0074] S22-3: Decompose the pedestrian identity features from the pedestrian features through the category attention module in the pedestrian re-identification model.

[0075] In this embodiment, after obtaining the pedestrian features, the pedestrian features are decomposed through the category attention module in the pedestrian re-identification model to obtain the pedestrian identity features.

[0076] In this embodiment, constructing a corresponding identity feature relationship graph includes: S22-4: Determine the class distance matrix between all the pedestrian identity features.

[0077] In this embodiment, the class distance matrix reflects the distance between the pedestrian features of each class.

[0078] In this embodiment, when constructing the identity feature relationship graph, first determine the class distance matrix between all the pedestrian identity features.

[0079] Exemplarily, calculate the class distance matrix between all the selected identity features . For each class , the first P-1 nearest neighboring classes can be retrieved from the class distance matrix . The corresponding set of neighboring classes can be expressed as: (5) where is the th nearest neighboring class.

[0080] S22-5: Determine the neighboring classes corresponding to each pedestrian class from the class distance matrix.

[0081] In this embodiment, from the distance matrix, the neighboring classes corresponding to each pedestrian class can be retrieved.

[0082] Exemplarily, the neighboring class of a pedestrian wearing red clothes is a pedestrian wearing pink clothes.

[0083] S22-6: Using each of the pedestrian classes as vertices and the relationships between classes as edges, construct the identity feature relationship graph.

[0084] In this embodiment, as Figure 3 shown, using each pedestrian class as vertices and the relationships between classes as edges, construct the identity feature relationship graph.

[0085] Exemplarily, a relationship graph can be constructed, where represents vertices, and each pedestrian class is a vertex; represents edges, and the relationships between classes are edges. Specifically, the vertices and edges in the relationship graph are respectively represented as: (6) (7) S23: According to the identity feature relationship graph, form a second image set with the pedestrian images of each class and the corresponding neighboring class images.

[0086] In this embodiment, according to the identity feature relationship graph, adjacent categories of each category are retrieved, and then the pedestrian images of each category and the pedestrian images of the corresponding adjacent categories are combined to form a second image set. In the second image set, there are several sample images of the pedestrian images of one category and several sample images of the pedestrian images of multiple categories adjacent to this category. The specific steps include: S23-1: For any pedestrian category, retrieve the adjacent categories connected to the pedestrian category according to the identity feature relationship graph to obtain a corresponding category set.

[0087] In this embodiment, for any pedestrian category, retrieve the adjacent categories connected to the pedestrian category according to the identity feature relationship graph to obtain a corresponding category set.

[0088] S23-2: Select a preset number of the sample images for each category in the category set to obtain the second image set.

[0089] In this embodiment, select a preset number of sample images for each category in the category set, and the selected sample images form the second image set.

[0090] Exemplarily, for small-batch sampling, randomly select a pedestrian category as an anchor point, and retrieve all the connected categories in the relationship graph to obtain a category set by combining the anchor point category and its adjacent categories: (8) where the set capacity is For each category in the set randomly extract instance samples for each category to form a minimum training batch with a sample number of where represents the instance sample, that is, the sample image.

[0091] For example, if the category set includes category A, category B, and category C, at this time, 5 images are selected from each of category A, category B, and category C to form the second image set.

[0092] S24: Extract the pedestrian change features of all the pedestrian images in the second image set through the pedestrian re-identification model.

[0093] In this embodiment, as Figure 3 shown, first extract the pedestrian change features (intra-class change features) of all the pedestrian images in the second image set through the pedestrian re-identification model.

[0094] Exemplarily, the pedestrian change feature , where represents the total number of categories in the training set represents the dimension of the pedestrian feature.

[0095] S25: According to the pedestrian change feature, each sample image in the second image set and the pedestrian images with different pedestrian categories from the sample image are combined to form a third image set.

[0096] In this embodiment, after obtaining the pedestrian change feature, each sample image in the second image set and the pedestrian images with different categories from the sample image are combined to form a third image set according to the obtained pedestrian change feature. The specific steps include: S25-1: Perform clustering analysis on the pedestrian change feature.

[0097] In this embodiment, as Figure 3 shown, perform clustering analysis on the pedestrian change feature to classify the pedestrian change feature, and classify similar pedestrian change features into one category.

[0098] Exemplarily, the k-means clustering algorithm is used to perform clustering analysis on all pedestrian change features.

[0099] S25-2: For each of the sample images, select the pedestrian images that belong to different clustering centers and different pedestrian categories from the sample image to form the third image set.

[0100] In this embodiment, after performing clustering analysis on the pedestrian change feature, for each sample image, select the pedestrian images that belong to different clustering centers and different pedestrian categories from the sample image to form the third image set. In this way, the images with different categories and different pedestrian change features from the sample image are formed into a set to enrich the sample quantity of the pedestrian images of this category.

[0101] Exemplarily, the formed image set is represented as: (9) where represents the pedestrian identity label of the instance sample of represents the instance sample the clustering label of the change feature of.

[0102] S26: Fuse the pedestrian identity feature of the sample image in the third image set with the pedestrian change features of the remaining images to obtain a generated feature.

[0103] In this embodiment, as Figure 3As shown in the figure, after obtaining the third image set, the pedestrian identity features of the selected sample images in the third set of images are fused with the pedestrian change features of the remaining images to obtain corresponding generated features. The generated features contain the pedestrian identity features of the selected category and the pedestrian change features of the remaining categories, so as to enrich the sample quantity of the selected category.

[0104] S27: Merge the generated features and the original features in the second image set into the same training batch to obtain the corresponding training set.

[0105] In this embodiment, the generated features and the original features in the second image set are merged into the same training batch to obtain the corresponding training set. In this way, the sample quantity of the pedestrian images of each category is enriched.

[0106] In this embodiment, by combining the pedestrian features of the instance sample x and the change features of the selected sample z, pedestrian features that are the same as the identity of the instance sample and different from the change of the instance sample are generated. By migrating the different change information of other categories to the tail category in this way, it helps to improve the change diversity of the tail category. Considering the differences in the sample distributions of different categories, more sample features are generated for the tail category and fewer sample features are generated for the tail category, and then the original features and the generated features are merged into one training batch, which helps to achieve the balance of the sample distribution of the training set.

[0107] Exemplarily, the selected sample image is a pedestrian image of category A, and the images of the remaining categories are pedestrian images of category B and category C. The pedestrian change features of the pedestrian images of category B and category C are respectively fused with the pedestrian identity features of the pedestrian image of category A, and the generated image is a pedestrian image of category A, where the action of the pedestrian is different from the original image, thereby enriching the sample quantity of the pedestrian images of category A.

[0108] In this embodiment, from the perspective of feature visualization, the ideological motivation for generating pedestrian features is analyzed, revealing the transferability of the change features between pedestrian categories. The pedestrian category features are selected to form the smallest training batch (min-batch), and the pedestrian change features of different categories are selected to generate pedestrian features with change diversity.

[0109] S28: Train the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model.

[0110] In this embodiment, the person re-identification model is trained with a training set that has an increased number of sample images to obtain a trained person re-identification model. Since the number of samples is enriched, the pedestrian features of each type and each action adjust the model parameters during the training process, and the trained model can more effectively and accurately identify pedestrians of each type and each action.

[0111] S29: Perform a person re-identification task through the person re-identification model to obtain a person identification result.

[0112] In this embodiment, after the person re-identification model is trained, a person re-identification task is performed through the person re-identification model to obtain a person identification result.

[0113] Exemplarily, multiple images are input into the person re-identification model, and the person re-identification model inputs the identity information of each person in each image, realizing effective identification of the identities of pedestrians in the pedestrian images, and can confirm the identity of the same person in different scenarios.

[0114] In the above embodiment of the present application, a class attention mechanism is proposed, which decomposes the original pedestrian features into pedestrian identity features and pedestrian variation features, captures the correlation between these semantic patterns, and highlights the class information that is beneficial to distinguishing different pedestrians.

[0115] An identity decomposition loss function and a variation decomposition loss function are constructed, and the pedestrian identity features and pedestrian variation features are input into the pedestrian classifier to constrain and supervise the decomposition process of pedestrian features, and more accurately decompose the pedestrian identity and pedestrian variation semantics.

[0116] A relationship graph about pedestrian classes is constructed, which models the semantic similarity relationship between pedestrian classes. Before the start of each training round, the re-identification model is used to evaluate the distance or similarity between pedestrian classes, and a corresponding relationship graph is constructed for all pedestrian classes. The relationship between pedestrian classes can be used for training data sampling to construct the smallest training batch for convenient training.

[0117] And a clustering relationship graph of pedestrian variations is constructed, which combines the pedestrian features of instance samples and the pedestrian variation features of selected samples to generate pedestrian features that have the same identity as the strength samples and different variations from the strength samples. In this way, the different variation information of other classes is migrated to the tail class, thereby enhancing the diversity of the tail class variations.

[0118] By the above method, a training set with evenly distributed samples of each class is formed to train the re-identification model, effectively improving the rationality of the model parameters. The trained re-identification model is used to perform the person re-identification task to obtain a more accurate identification result.

[0119] Based on the same inventive concept, an embodiment of the present application provides a pedestrian re-identification device. Refer to Figure 4 , Figure 4 which is a schematic diagram of a pedestrian re-identification device 400 proposed in an embodiment of the present application. As Figure 4 shown, the device includes: A first image set acquisition module 401, configured to select a pedestrian image as a representative sample from each pedestrian category image in the pedestrian image data set to obtain a first image set; An identity relationship graph construction module 402, configured to extract pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model, and construct a corresponding identity feature relationship graph, and a category attention mechanism is added to the pedestrian re-identification model; A second image set acquisition module 403, configured to form a second image set according to the identity feature relationship graph, including pedestrian images of each category and corresponding adjacent category images; A pedestrian change feature extraction module 404, configured to extract pedestrian change features of all pedestrian images in the second image set through the pedestrian re-identification model; A third image set acquisition module 405, configured to form a third image set according to the pedestrian change features, including each sample image in the second image set and pedestrian images with different pedestrian categories from the sample image; A feature fusion module 406, configured to fuse the pedestrian identity features of the sample images in the third image set with the pedestrian change features of the remaining images to obtain generated features; A training set acquisition module 407, configured to merge the generated features and the original features in the second image set into the same training batch to obtain a corresponding training set; A model training module 408, configured to train the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model; A pedestrian re-identification module 409, configured to perform a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian identification result.

[0120] Optionally, the construction steps of the pedestrian re-identification model include: Extracting first pedestrian convolutional features from a pedestrian image through a backbone network of the pedestrian re-identification model; Converting the first pedestrian convolutional features into query features, key features, and value features; Calculating corresponding attention values according to the semantic correlation between different channels of the query features and the key features; Fusing the attention values with the value features through an attention weighting method to obtain second pedestrian convolutional features; Optimize the second pedestrian convolution feature in a residual learning manner to obtain a third pedestrian convolution feature; Use a preset mapping function to map the third pedestrian convolution feature to a class attention mask, so as to decompose the first pedestrian convolution feature into the pedestrian identity feature and the pedestrian variation feature through the class attention mask.

[0121] Optionally, the construction steps of the pedestrian re-identification model further include: Input the pedestrian identity feature and the pedestrian variation feature into a pedestrian classifier; Optimize the parameters according to the pedestrian identity feature through the identity decomposition loss function in the pedestrian classifier; Optimize the parameters according to the pedestrian variation feature through the variation decomposition loss function in the pedestrian classifier.

[0122] Optionally, the identity feature relationship graph construction module includes: An image input sub-module for inputting the pedestrian image into the pedestrian re-identification model; A pedestrian feature extraction sub-module for extracting the pedestrian feature corresponding to the pedestrian image through the pedestrian re-identification model; A pedestrian identity feature extraction sub-module for decomposing the pedestrian identity feature from the pedestrian feature through the class attention module in the pedestrian re-identification model.

[0123] Optionally, the identity relationship graph construction module further includes: A class distance matrix calculation sub-module for determining the class distance matrix between all the pedestrian identity features; An adjacent class determination sub-module for determining the adjacent classes corresponding to each pedestrian class from the class distance matrix; An identity feature relationship graph construction sub-module for constructing the identity feature relationship graph with each pedestrian class as a vertex and the relationship between classes as an edge.

[0124] Optionally, the second image set acquisition module includes: A class set acquisition sub-module for retrieving the adjacent classes connected to any pedestrian class according to the identity feature relationship graph to obtain the corresponding class set; A second image set acquisition sub-module for selecting a preset number of the sample images for each class in the class set to obtain the second image set.

[0125] Optionally, the third image set acquisition module includes: A clustering analysis sub-module for performing clustering analysis on the pedestrian change features; A third image set acquisition sub-module for, for each of the sample images, selecting pedestrian images that belong to different clustering centers and different pedestrian categories from the sample images to form the third image set.

[0126] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the program is executed by a processor, it implements the pedestrian re-identification method described in any of the above embodiments of the present application.

[0127] Based on the same inventive concept, another embodiment of the present application provides an electronic device Figure 5 is a schematic diagram of an electronic device 500 proposed in an embodiment of the present application, including a memory 502, a processor 501, and a computer program stored on the memory and executable on the processor. When the processor executes, it implements the pedestrian re-identification method described in any of the above embodiments of the present application.

[0128] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.

[0129] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same and similar parts among the embodiments, refer to each other.

[0130] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a device, or a computer program product. Therefore, the embodiments of the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1means for the functions specified in one or more boxes.

[0132] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0134] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0135] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or terminal device including the said element.

[0136] The above has introduced in detail the person re-identification method, device, equipment and storage medium provided by the present application. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A pedestrian re-identification method, characterized in that, The method includes: Selecting one pedestrian image as a representative sample from each pedestrian category image in the pedestrian image data set to obtain a first image set; Extracting the pedestrian identity features of all pedestrian images in the first image set through a pedestrian re-identification model, and constructing a corresponding identity feature relationship graph, where a category attention mechanism is added to the pedestrian re-identification model; According to the identity feature relationship graph, forming a second image set with the pedestrian images of each category and the corresponding adjacent category images; Extracting the pedestrian change features of all pedestrian images in the second image set through the pedestrian re-identification model; According to the pedestrian change features, forming a third image set with each sample image in the second image set and the pedestrian images with different pedestrian categories from the sample image; Fusing the pedestrian identity features of the sample images in the third image set with the pedestrian change features of the remaining images to obtain generated features; Merging the generated features and the original features in the second image set into the same training batch to obtain a corresponding training set; Training the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model; Performing a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian recognition result.

2. The pedestrian re-identification method according to claim 1, wherein The construction steps of the pedestrian re-identification model include: Extracting first pedestrian convolution features from a pedestrian image through the backbone network of the pedestrian re-identification model; Converting the first pedestrian convolution features into query features, key features, and value features; Calculating corresponding attention values according to the semantic correlations between different channels of the query features and the key features; Fusing the attention values with the value features through an attention weighting method to obtain second pedestrian convolution features; Optimizing the second pedestrian convolution features through a residual learning method to obtain third pedestrian convolution features; Using a preset mapping function to map the third pedestrian convolution features into a category attention mask, so as to decompose the first pedestrian convolution features into the pedestrian identity features and the pedestrian change features through the category attention mask.

3. The pedestrian re-identification method according to claim 2, wherein The construction steps of the pedestrian re-identification model further include: Inputting the pedestrian identity features and the pedestrian change features into a pedestrian classifier; Optimizing parameters according to the pedestrian identity features through the identity decomposition loss function in the pedestrian classifier; Optimizing parameters according to the pedestrian change features through the change decomposition loss function in the pedestrian classifier.

4. The pedestrian re-identification method according to claim 1, characterized in that, The extracting the pedestrian identity features of all pedestrian images in the first image set through the pedestrian re-identification model includes: Inputting the pedestrian image into the pedestrian re-identification model; Extracting the pedestrian features corresponding to the pedestrian image through the pedestrian re-identification model; Decomposing the pedestrian identity features from the pedestrian features through the category attention module in the pedestrian re-identification model.

5. The pedestrian re-identification method according to claim 1, wherein The constructing the corresponding identity feature relationship graph includes: Determining a category distance matrix between all the pedestrian identity features; Determining the adjacent categories corresponding to each pedestrian category from the category distance matrix; Construct the identity feature relationship graph with each of the pedestrian categories as vertices and the relationships between the categories as edges.

6. The pedestrian re-identification method according to claim 1, wherein According to the identity feature relationship graph, form a second image set by combining the pedestrian images of each category and the corresponding adjacent category images, including: For any pedestrian category, retrieve the adjacent categories connected to the pedestrian category according to the identity feature relationship graph to obtain a corresponding category set; Select a preset number of the sample images for each category in the category set to obtain the second image set.

7. The pedestrian re-identification method according to claim 1, wherein According to the pedestrian change characteristics, form a third image set by combining each sample image in the second image set and the pedestrian images with different pedestrian categories from the sample images, including: Perform clustering analysis on the pedestrian change characteristics; For each of the sample images, select the pedestrian images that belong to different clustering centers and different pedestrian categories from the sample image to form the third image set.

8. A pedestrian re-identification device, characterized in that, The device includes: A first image set acquisition module, configured to select a pedestrian image as a representative sample from the images of each pedestrian category in the pedestrian image data set to obtain a first image set; An identity relationship graph construction module, configured to extract the pedestrian identity features of all the pedestrian images in the first image set through a pedestrian re-identification model and construct a corresponding identity feature relationship graph, and a category attention mechanism is added to the pedestrian re-identification model; A second image set acquisition module, configured to form a second image set by combining the pedestrian images of each category and the corresponding adjacent category images according to the identity feature relationship graph; A pedestrian change feature extraction module, configured to extract the pedestrian change features of all the pedestrian images in the second image set through the pedestrian re-identification model; A third image set acquisition module, configured to form a third image set by combining each sample image in the second image set and the pedestrian images with different pedestrian categories from the sample image according to the pedestrian change characteristics; A feature fusion module, configured to fuse the pedestrian identity features of the sample images in the third image set with the pedestrian change features of the remaining images to obtain generated features; A training set acquisition module, configured to merge the generated features and the original features in the second image set into the same training batch to obtain a corresponding training set; A model training module, configured to train the pedestrian re-identification model through the training set to obtain the trained pedestrian re-identification model; A pedestrian re-identification module, configured to perform a pedestrian re-identification task through the pedestrian re-identification model to obtain a pedestrian identification result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images