Method and device for determining semantic map of body part, model training and person re-identification

Abstract

The present invention discloses a method for determining a semantic map of body parts, which includes: extracting global feature maps of each image from a plurality of images of the same pedestrian; clustering all pixels in all the global feature maps to obtain a plurality of categories related to body parts; for each pixel, generating a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs according to the category to which the pixel belongs, and respectively determining each global feature map with a category label at the pixel position as a semantic map of the body part mapped by the global feature map, thereby improving the acquisition efficiency of images with body part labels.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular, to a method and device for determining a semantic map of body parts, a method and device for training a person re-identification model, an unsupervised person re-identification method and device, a computer device, and a storage medium. Background Art

[0002] Person re-identification (person re-ID) technology has become increasingly popular in the field of contemporary computer vision because its research and application in fields such as intelligent security have important significance. The goal of this technology is to identify the same person who wants to be queried and located on different monitoring devices. In real scenarios, due to factors such as human body postures, changes in camera shooting angles, and lighting conditions, how to accurately identify and match pedestrians has become a very challenging problem. With the successful application of deep learning-related algorithms in the field of computer vision and the emergence of large-scale databases, many advanced methods have been continuously proposed to solve this problem.

[0003] Common person re-identification methods are generally implemented in a supervised manner, that is, first obtain images with body part labels as training samples, then perform supervised training on the network model using the training samples, and finally use the trained network model for person re-identification. Currently, images with body part labels are obtained by manually annotating the body parts in the images. During the manual annotation process, the annotator needs to find the body parts to be annotated one by one in the image and label the corresponding labels at the pixel level at the positions of the body parts to be annotated to obtain images with body part labels. Since the annotation is pixel-level and manual, the annotation workload is huge and prone to errors, resulting in low annotation efficiency and low acquisition efficiency of images with body part labels.

[0004] In summary, the acquisition efficiency of images with body part labels is currently low. Summary of the Invention

[0005] (1) Object of the Invention

[0006] The object of the present invention is to provide a method and device for determining a semantic map of body parts, a method and device for training a person re-identification model, an unsupervised person re-identification method and device, a computer device, and a storage medium to solve the problem of low acquisition efficiency of images with body part labels existing in the prior art.

[0007] (2) Technical Solution

[0008] To solve the above problems, a first aspect of the present invention provides a method for determining a semantic map of body parts, including:

[0009] Extract the global feature maps of each image among several images of the same pedestrian;

[0010] Cluster all pixels in all the global feature maps to obtain several categories related to body parts;

[0011] For each pixel, generate a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs according to the category to which the pixel belongs, and respectively determine each global feature map with the category label at the pixel position as the body part semantic map mapped by the global feature map.

[0012] Optionally, cluster all pixels to obtain several categories related to body parts, including:

[0013] Perform a first clustering on all pixels to obtain foreground pixels;

[0014] According to the similarity between pixels, perform a second clustering on the foreground pixels to obtain several categories related to body parts.

[0015] The second aspect of the present invention provides a training method for a pedestrian re-identification model, including:

[0016] Adopt the method for determining the body part semantic map to determine the body part semantic map of the sample image;

[0017] Use the determined body part semantic map of the sample image to train the pedestrian re-identification model.

[0018] Optionally, using the body part semantic map of the sample image to train the pedestrian re-identification model includes:

[0019] According to the body part semantic map and the global feature map of the sample image, obtain the body part feature maps of the sample image;

[0020] Use the obtained body part feature maps of the sample image to calculate the loss function value, and iteratively update the parameters of the pedestrian re-identification model based on the calculated loss function value until the iteration termination condition is reached.

[0021] The third aspect of the present invention provides an unsupervised pedestrian re-identification method, including:

[0022] Use the pedestrian re-identification model trained according to the training method to obtain the body part feature maps of the query image and the candidate images;

[0023] Use the obtained body part feature maps of the query image and the candidate images to calculate the distance between the query image and each candidate image;

[0024] Sort the candidate images according to the calculated distances between the query image and each candidate image, and output the sorting result as the re-identification result.

[0025] Optionally, calculating the distance between the query image and each candidate image includes:

[0026] Calculating the feature distances of each body part of the two images using the feature maps of each body part of the two images, calculating the global feature distance of the two images using the global feature maps of the two images, and calculating the foreground feature distance of the two images;

[0027] Calculate the distance between the two images according to whether each body part in the two images is visible and the calculated feature distances of each body part, global feature distance, and foreground feature distance.

[0028] The fourth aspect of the present invention provides a device for determining a body part semantic map, including:

[0029] An extraction module for extracting the global feature map of each image in several images of the same pedestrian;

[0030] A clustering module for clustering all pixels in all the global feature maps to obtain several categories related to body parts;

[0031] A determination module for, for each pixel, generating a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs according to the category to which the pixel belongs, and respectively determining each global feature map with a category label at the pixel position as the body part semantic map mapped by the global feature map.

[0032] [[ID=2,4]]The fifth aspect of the present invention provides a training device for a pedestrian re-identification model, including:

[0033] A processing module for determining the body part semantic map of the sample image by using the method for determining the body part semantic map described above;

[0034] A training module for training a pedestrian re-identification model by using the determined body part semantic map of the sample image.

[0035] The sixth aspect of the present invention provides an unsupervised pedestrian re-identification device, including:

[0036] An acquisition module for obtaining the feature maps of each body part of the query image and candidate images by using the pedestrian re-identification model trained according to the training method;

[0037] A calculation module for calculating the distance between the query image and each candidate image by using the feature maps of each body part of the obtained query image and candidate images;

[0038] An output module, configured to sort candidate images according to the calculated distances between the query image and each candidate image, and output the sorting result as the re-identification result.

[0039] The seventh aspect of the present invention provides a computer device, including:

[0040] A memory;

[0041] A processor; and

[0042] A computer program;

[0043] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method for determining the body part semantic map, or the method for training the pedestrian re-identification model, or the unsupervised-based pedestrian re-identification method.

[0044] The eighth aspect of the present invention provides a storage medium, which is a computer-readable storage medium and stores a computer program thereon;

[0045] The computer program is executed by a processor to implement the method for determining the body part semantic map, or the method for training the pedestrian re-identification model, or the unsupervised-based pedestrian re-identification method.

[0046] (III) Advantageous Effects

[0047] The present invention provides a method for determining a body part semantic map, including: extracting global feature maps of each image in several images of the same pedestrian; clustering all pixels in all the global feature maps to obtain several categories related to body parts; for each pixel, generating a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs, and respectively determining each global feature map with a category label at the pixel position as the body part semantic map mapped by the global feature map. The body part semantic map of the present invention is an image with a body part label at the pixel position. In the present invention, through the pixel clustering and category label generation steps, several body part semantic maps can be obtained, thereby improving the acquisition efficiency of images with body part labels. Description of the Drawings

[0048] Figure 1 is a schematic flowchart of a method for determining a body part semantic map provided by an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of the category label generation process provided by an embodiment of the present invention;

[0050] Figure 3It is a schematic flow chart of a method for training a pedestrian re-identification model provided by an embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of a confidence map provided by an embodiment of the present invention;

[0052] Figure 5 It is a schematic flow chart of a method for unsupervised pedestrian re-identification provided by an embodiment of the present invention;

[0053] Figure 6 It is a schematic structural diagram of a device for determining a body part semantic map provided by an embodiment of the present invention;

[0054] Figure 7 It is a schematic structural diagram of a device for training a pedestrian re-identification model provided by an embodiment of the present invention;

[0055] Figure 8 It is a schematic structural diagram of a device for unsupervised pedestrian re-identification provided by an embodiment of the present invention;

[0056] Figure 9 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0057] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with the detailed implementation manners and with reference to the accompanying drawings. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0058] Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] In the description of the present invention, it should be noted that the terms "first" and "second" are only used for the purpose of distinction and cannot be construed as indicating or implying relative importance.

[0060] An embodiment of the present invention provides a method for determining a body part semantic map. As Figure 1 shown, the method includes:

[0061] Step 101, extract the global feature map of each image from several images of the same pedestrian;

[0062] Step 102, cluster all pixels in all the global feature maps to obtain several categories related to body parts;

[0063] Step 103: For each pixel, according to the category to which the pixel belongs, generate a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs, and respectively determine each global feature map with a category label at the pixel position as the body part semantic map mapped by the global feature map.

[0064] Among them, the body part semantic map in the embodiment of the present invention is an image with a body part label at the pixel position.

[0065] In implementation, in the embodiment of the present invention, through the pixel clustering and category label generation steps, several body part semantic maps can be obtained, thereby improving the acquisition efficiency of the image with a body part label.

[0066] In implementation, in the embodiment of the present invention, through the global feature map extraction, pixel clustering and category label generation steps, the body part semantic map can be directly learned from a conventional image without the need to obtain it through manual annotation, that is, without introducing additional annotation workload, thereby greatly saving labor costs.

[0067] In implementation, in the embodiment of the present invention, since the category label of the pixel is obtained through clustering, compared with the manual annotation in the prior art, the edge of the body part can be more accurately distinguished, so the accuracy of the obtained image with a body part label can be improved.

[0068] Among them, in step 101, in each image of the same pedestrian, there is only one pedestrian and no others.

[0069] Among them, in step 101, the implementation manner of extracting the global feature map of the image is similar to the implementation manner in the prior art and will not be elaborated here.

[0070] Among them, in step 102, all the obtained categories are related to body parts, and different categories are related to different body parts; for example, clustering all pixels obtains 4 categories related to body parts, namely category 1, category 2, category 3, and category 4. Category 1 is related to the head, category 2 is related to the upper body, category 3 is related to the lower body, and category 4 is related to the feet.

[0071] Among them, in step 102, any clustering method that can obtain several categories related to body parts is applicable to the embodiment of the present invention. For example, according to different clustering times, it is divided into one time and multiple times; according to different clustering types, it is divided into k-means (K-means clustering algorithm), CLARANS (clustering algorithm based on random selection), and PCM (fuzzy clustering algorithm), etc.

[0072] Optionally, in step 102, clustering all pixels to obtain several categories related to body parts includes:

[0073] Step 1021: Perform the first clustering on all pixels to obtain foreground pixels;

[0074] Step 1022: According to the similarity between pixels, perform the second clustering on the foreground pixels to obtain several categories related to body parts.

[0075] In implementation, by using hierarchical clustering, first perform the first clustering to eliminate the interference of background pixels, and then perform the second clustering to obtain several categories related to body parts, which can improve the accuracy of the obtained categories.

[0076] Among them, in step 1021, the implementation manner of performing the first clustering on all pixels to obtain foreground pixels is similar to the implementation manner of clustering pixels to obtain foreground pixels in the prior art. For example, clustering pixels according to the activation value and / or confidence of the pixels to obtain foreground pixels will not be elaborated here.

[0077] Among them, in step 1021, when performing the first clustering on all pixels according to the activation value of the pixels, the activation value of the pixels can be the activation value after normalization processing or the activation value without normalization processing.

[0078] In implementation, performing the first clustering on all pixels according to the activation value of the pixels after normalization processing can unify the comparison criteria of all pixels, thereby improving the accuracy of the determined foreground pixels.

[0079] Optionally, in step 1021, define the activation value at each pixel position M g in the global feature map M g (x, y) as ||M g (x, y)||2, and perform normalization processing on the activation value of each pixel by using the following formula:

[0080]

[0081] Among them, a(x, y) is the activation value of the pixel after normalization processing, max (i,j) ||M g (x, y)||2 is the maximum value of the activation values of all pixels, and (i, j) are the coordinates of the pixels in M g .

[0082] Among them, since foreground pixels have higher activation values compared to background pixels, in step 1021, when performing the first clustering on all pixels according to the activation values of the pixels, the pixels with activation values less than the division threshold are determined as background pixels. The division threshold can be determined based on experience or experiments. For example, if the activation values of the pixels are normalized, in the case of a clear image, the division threshold can be set to 0.05; in the case of an unclear image, the division threshold can be set to 0.1; if the activation values of the pixels are not normalized, different division thresholds can be set for the pixels in different global feature maps, and the division threshold of the pixels in the global feature Figure 1 is set to 100, and the division threshold of the pixels in the global feature Figure 2 is set to 200.

[0083] Among them, in step 1022, the similarity between pixels refers to the magnitude of the feature distance between pixels.

[0084] Among them, the foreground pixels include several body parts. In step 1022, any clustering algorithm based on the similarity between pixels that can obtain several categories related to body parts is applicable to the embodiments of the present invention, such as algorithms like k-means, CLARANS, and PCM and their extended algorithms.

[0085] Among them, in step 1022, when performing the second clustering on the foreground pixels according to the similarity between pixels, the position of the pixel can be the position after normalization or the position without normalization.

[0086] In implementation, using the pixel positions after normalization to complete the second clustering can exclude interference factors, thereby improving the accuracy of the obtained categories related to body parts.

[0087] Optionally, in step 1022, when performing the second clustering, more attention is paid to the similarity and difference between pixels rather than the activation values of the pixels. Therefore, the following formula is used to normalize the position of each pixel:

[0088]

[0089] Among them, D(x,y) is the position of the pixel after normalization, and M g (x,y) is the position of the pixel in the global feature map M g in, and ||M g (x,y)||2 is the activation value on M g [[ID=,33]](x,y).

[0090] Among them, since the pixels at the occluded positions are classified as background pixels when there is occlusion, in step 1022, when a body part is occluded, the number of categories obtained may be less than the number of body parts.

[0091] Among them, in step 103, the global feature map to which a pixel belongs and its position in the global feature map can be determined according to the position identifier of the pixel.

[0092] Among them, in step 103, the mapping relationship between categories and category labels can be customized; for example, set the category labels as positive integer values, the category label for the category related to the head is 1, the category label for the category related to the upper body is 2, the category label for the category related to the lower body is 3, and the category label for the category related to the feet is 4.

[0093] Among them, in step 103, for each pixel, the process of generating the corresponding category label at the position of the pixel in the global feature map to which the pixel belongs according to the category to which the pixel belongs is similar to the process of restoring the pixel with the category label to the global feature map.

[0094] For ease of understanding, the following takes Figure 2 as an example to give an exemplary description of the process of generating the corresponding category label at the position of the pixel in the global feature map to which the pixel belongs in step 103 according to the category to which the pixel belongs.

[0095] As Figure 2 shown in A, all the pixels in the global feature maps M g1 and M g2 are clustered to obtain 3 categories related to body parts. Category 1 is related to the head, Category 2 is related to the upper body, and Category 3 is related to the lower body.

[0096] For the pixel M g1 (25, 10), according to Figure 2 A, it can be seen that M g1 (25, 10) belongs to Category 1. Then, as Figure 2 shown in B, at the position coordinate (25, 10) in the global feature map M g1 to which M g1 (25, 10) belongs, the corresponding category label 1 is generated. For the pixel M g2 (35, 20), according to Figure 2 A, it can be seen that M g2 (35, 20) belongs to Category 1. Then, as Figure 2 shown in C, at the position coordinate (35, 20) in the global feature map M g2 to which M g2 (35, 20) belongs, the corresponding category label 1 is generated.

[0097] Similarly, if Figure 2 As shown in B, in M g1 The global feature map M to which (30,40) belongs g1 The corresponding category label 2 is generated at the position coordinate (30,40) in M g1 The global feature map M to which (20,60) belongs g1 The corresponding category label 3 is generated at the position coordinate (20,60); Figure 2 As shown in C, in M g2 The global feature map M to which (30,40) belongs g2 The corresponding category label 2 is generated at the position coordinate (30,40) in M g2 The global feature map M to which (35,60) belongs g2 The corresponding category label 3 is generated at the position coordinate (35,60).

[0098] In the same way, generate Figure 2 The category labels of other pixels not marked in the figure can be used to obtain the global feature map M. g1 Mapped body part semantics Figure 1 And the global feature map M g2 Mapped body part semantics Figure 2 .

[0099] To enhance the distinction, pixels in the body part semantic map that do not have a body part category label are given a background label. The background label can be customized, for example, by setting the background label to 0.

[0100] Based on the same inventive concept, the embodiment of the present invention also provides a method for training a pedestrian re-identification model, such as Figure 3 As shown, the method includes:

[0101] Step 301: Determine a body part semantic map of a sample image using the method for determining a body part semantic map according to an embodiment of the present invention.

[0102] Step 302: Use the determined body part semantic map of the sample image to train a person re-identification model.

[0103] In practice, the embodiments of the present invention improve the efficiency of acquiring images with body part labels, thereby improving the training efficiency of the pedestrian re-identification model.

[0104] In practice, the training method of the person re-identification model provided by the embodiment of the present invention does not introduce additional labeling workload, thereby greatly saving labor costs.

[0105] In practice, the embodiments of the present invention improve the accuracy of the trained person re-identification model by improving the accuracy of the acquired images with body part labels.

[0106] Among them, in step 301, the implementation manner of determining the body part semantic map of the sample image can refer to the specific implementation manners of steps 101 to 103 in the embodiments of the present invention.

[0107] Among them, in step 302, there are various implementation manners of training the person re-identification model by using the determined body part semantic map of the sample image. Three preferred implementation manners will be introduced below.

[0108] 1. Train the person re-identification model in an unsupervised manner.

[0109] Optionally, in step 302, training the person re-identification model by using the body part semantic map of the sample image includes:

[0110] Step 3021: Obtain the feature maps of each body part of the sample image according to the body part semantic map and the global feature map of the sample image;

[0111] Step 3022: Calculate the loss function value by using the obtained feature maps of each body part of the sample image, and iteratively update the parameters of the person re-identification model based on the calculated loss function value until the iteration termination condition is reached.

[0112] In implementation, the embodiments of the present invention provide a new method for training a person re-identification model based on unsupervised learning.

[0113] In implementation, by continuously iteratively updating and optimizing the parameters of the person re-identification model through training, a more accurate global feature map can be extracted, so that a more accurate body part semantic map can be obtained through steps such as clustering, and further a more accurate feature map of each body part can be determined.

[0114] Among them, in step 3021, since the global feature map contains feature information and the body part semantic map contains pixel-level body part segmentation information, and the core of pixel-level body part feature alignment is to represent the feature of a body part with all the pixels of a certain body part on the global feature map, the feature maps of each body part can be determined according to the global feature map and the body part semantic map.

[0115] Among them, in step 3021, any implementation manner of determining the feature maps of each body part according to the global feature map and the body part semantic map is applicable to the embodiments of the present invention. For example, project the body part semantic map onto the global feature map to obtain the feature maps of each body part.

[0116] Optionally, in step 3021, determining the feature maps of each body part according to the global feature map and the body part semantic map includes:

[0117] Step 30211: Obtain the confidence maps of each body part according to the body part semantic map;

[0118] Step 30212: Determine the feature maps of each body part according to the global feature map and the obtained confidence maps of each body part.

[0119] In implementation, determining the feature maps of each body part according to the global feature map and the confidence maps of each body part has a relatively small solution complexity.

[0120] Among them, the confidence map of the embodiment of the present invention has body part semantic information. In the confidence map of a certain body part, it does not contain the class labels of other body parts except this body part. For example, Figure 4 For example, Figure 4 As shown in A, the body part semantic map includes the head represented by class label 1, the upper body represented by class label 2, and the lower body represented by class label 3; the confidence map of the head obtained according to the body part semantic map is as shown in Figure 4 B. In the confidence map of the head, it contains the class label 1 of the head, and does not contain the class label 2 of the upper body and the class label 3 of the lower body.

[0121] Among them, in step 30211, any implementation manner of obtaining the confidence maps of each body part according to the body part semantic map is applicable to the embodiment of the present invention. For example, by means of masking and / or cropping, etc.

[0122] Among them, in step 30212, any implementation manner of determining the feature maps of each body part according to the global feature map and the confidence maps of each body part is applicable to the embodiment of the present invention. For example, by means of projection or calculation formula, etc.

[0123] Optionally, in step 30212, assuming that there are a total of (K - 1) body parts and a global background, a total of K confidence maps (i.e., each body part and the global background) are required, denoted as P0, P1,..., P K-1 , and each confidence map P k is semantically associated with a certain body part. Then, the feature map of each body part k can be determined according to the following formula:

[0124] M k = P k * M g

[0125] Among them, M k is the feature map of body part k, M g is the global feature map, k ∈ {0,..., K - 1}; * represents the product between elements.

[0126] Among them, in step 3022, the loss function value can be calculated using the loss functions commonly used in the prior art for training the person re-identification model. Specifically, one loss function or multiple loss functions can be used to calculate the loss function value.

[0127] Optionally, in step 3022, the softmax cross-entropy loss function and the triplet loss function in the prior art are used to calculate the loss function value. For example, the loss function value L is calculated using the following formula:

[0128] L = L tri + L ID

[0129] Among them, L tri is the triplet loss function value, which is determined according to the following formula: and are the features of the positive sample pair of the same person, and are the features of the negative sample pair of different persons, α is the defined boundary, N is the total number of samples, represents the square of the L2 norm;

[0130] Among them, L ID is the softmax cross-entropy loss function value, which is determined according to the following formula: S is the total number of categories, y is the category label, θ is the parameter of the model classification layer, and f is the feature.

[0131] Among them, in step 3022, after obtaining the feature maps of each body part, the loss function value can be calculated only based on the features of each body part, or the loss function value can be calculated based on the fusion features of the features of each body part and other features (such as global features, foreground features). The foreground feature is the feature of the foreground pixel region.

[0132] Among them, in step 3022, the implementation manner of updating the parameters of the person re-identification model based on the loss function value is similar to the implementation manner of updating the parameters of the person re-identification model based on the loss function value in the prior art, and will not be elaborated here.

[0133] Among them, in step 3022, the iteration termination condition can be preset, for example, preset to the convergence of the loss function, reaching the set number of iterations, or the person re-identification model tending to be stable, etc.

[0134] Among them, determining the body part semantic map according to steps such as clustering (i.e., step 301), learning the feature representation of body parts using the body part semantic map (i.e., step 3021), and calculating the loss function value and updating the model parameters using the body part features (i.e., step 3022) is an iterative process. During the training process, through continuous iteration, the model parameters are continuously optimized, so as to ensure that the trained person re-identification model can obtain accurate feature maps of each body part.

[0135] Second, train the person re-identification model in the first supervised manner.

[0136] Optionally, in step 302, use the body part semantic map of the sample image to train the person re-identification model, including:

[0137] Calculate the feature maps of each body part of the sample image according to the body part semantic map of the sample image, and use the person re-identification model to predict the feature maps of each body part of the sample image;

[0138] Calculate the loss according to the error between the predicted feature maps of each body part and the calculated feature maps of each body part, and iteratively update the parameters of the person re-identification model based on the loss.

[0139] Third, train the person re-identification model in the second supervised manner.

[0140] Optionally, in step 302, use the body part semantic map of the sample image to train the person re-identification model, including:

[0141] Use the person re-identification model to predict the body part semantic map of the sample image;

[0142] Calculate the loss according to the error between the predicted body part semantic map and the body part semantic map determined by the method for determining the body part semantic map according to the embodiments of the present invention, and iteratively update the parameters of the person re-identification model based on the loss;

[0143] Use the trained person re-identification model to predict the body part semantic map of the sample image and extract the feature maps of each body part according to the predicted body part semantic map, and calculate the feature maps of each body part of the sample image according to the body part semantic map of the sample image;

[0144] Calculate the loss according to the error between the extracted feature maps of each body part and the calculated feature maps of each body part, and iteratively update the parameters of the person re-identification model based on the loss.

[0145] Based on the same inventive concept, the embodiments of the present invention also provide an unsupervised person re-identification method, as Figure 5 shown, the method includes:

[0146] Step 501: Obtain the feature maps of each body part of the query image and the candidate images by using the person re-identification model trained by the training method of the person re-identification model according to the embodiments of the present invention;

[0147] Step 502: Calculate the distance between the query image and each candidate image by using the feature maps of each body part of the obtained query image and candidate images;

[0148] Step 503: Sort the candidate images according to the calculated distance between the query image and each candidate image, and output the sorting result as the re-identification result.

[0149] In implementation, the embodiments of the present invention improve the acquisition efficiency of the images with body part labels, thereby improving the training efficiency of the person re-identification model, and further improving the efficiency of person re-identification.

[0150] In implementation, the person re-identification method provided by the embodiments of the present invention does not introduce additional annotation workload, thus greatly saving the labor cost.

[0151] In implementation, the embodiments of the present invention improve the accuracy of the obtained images with body part labels, thereby improving the accuracy of the trained person re-identification model, and further improving the accuracy of person re-identification.

[0152] In implementation, the embodiments of the present invention provide a new unsupervised-based person re-identification method.

[0153] Among them, in step 501, for the implementation manner of obtaining the feature maps of each body part of the query image and the candidate images by using the person re-identification model, reference can be made to the implementation manner of obtaining the feature maps of each body part of the sample image in the training method of the person re-identification model provided by the embodiments of the present invention, which will not be elaborated here.

[0154] Among them, in step 502, the distance between the two images can be calculated only by using the feature maps of each body part, or the distance between the two images can be calculated by using the combination of the feature maps of each body part and the global feature map and / or the foreground feature, which can be specifically determined flexibly according to requirements.

[0155] Optionally, in step 502, calculating the distance between the query image and each candidate image includes:

[0156] Step 5021: Calculate the feature distances of each body part of the two images by using the feature maps of each body part of the two images, calculate the global feature distance of the two images by using the global feature maps of the two images, and calculate the foreground feature distance of the two images;

[0157] Step 5022: Calculate the distance between the two images according to whether each body part in the two images is visible and the calculated feature distances of each body part, global feature distance, and foreground feature distance.

[0158] In implementation, by using the solution in Step 5022 of the embodiments of the present invention to calculate the distance between the two images, the accuracy of the calculated distance between the two images can be improved, thereby improving the accuracy of pedestrian re-identification.

[0159] Among them, the two images in the embodiments of the present invention refer to the query image and any candidate image.

[0160] Among them, in Step 5021, the distance between the feature maps of each body part of the two images can be directly calculated, and the calculation result can be determined as the obtained feature distances of each body part; for the convenience of calculation, the distance can be calculated after converting the feature maps of each body part into other representation forms (such as feature vectors). The implementation manner of the global feature distance is similar to that of the body part feature distance, and will not be elaborated here.

[0161] Among them, in Step 5021, the foreground feature distance refers to the distance of the foreground feature. Any method for determining the foreground feature is applicable to the embodiments of the present invention.

[0162] Among them, in Step 5021, the distance in the embodiments of the present invention can be any distance in the prior art, such as Euclidean distance, cosine distance, Minkowski distance, or Manhattan distance, etc.

[0163] Among them, in Step 5022, any implementation manner of calculating the distance between the two images according to whether each body part is visible, the feature distances of each body part, the global feature distance, and the foreground feature distance is applicable to the embodiments of the present invention. For example, according to whether a certain body part is visible, it is determined whether to use the feature distance of this body part to calculate the distance between the two images, and the average value, weighted sum value, standard deviation value, or variance value of the feature distances of the body parts, global feature distance, and foreground feature distance is determined as the distance between the two images.

[0164] Optionally, in Step 5022, assuming that there are a total of (K - 1) body parts, the distance between the two images is calculated according to the following formula:

[0165]

[0166] Among them, d is the calculated distance between the two images; is the visibility of the k-th body part in the query image, is the visibility of the k-th body part in the candidate image. When the k-th body part is visible, the visibility value is 1, and when the k-th body part is invisible, the visibility value is 0. If the k-th body part is visible in both the query image and the candidate image, then The value is 1, otherwise the value is 0; d k is the k-th body part feature distance between the query image and the candidate image, D() represents the distance metric, and are respectively the k-th body part features of the query image and the candidate image; d g is the global feature distance between the query image and the candidate image, and are respectively the global features of the query image and the candidate image; d f is the foreground feature distance between the query image and the candidate image, and are respectively the foreground features of the query image and the candidate image.

[0167] Among them, the implementation manner of the solution in step 503 is similar to that in the prior art and will not be elaborated here.

[0168] Based on the same inventive concept, an embodiment of the present invention provides a device for determining a body part semantic map, which is used to execute the method for determining a body part semantic map described above, as Figure 6 shown, the device includes:

[0169] An extraction module 610, configured to extract the global feature map of each image in several images of the same pedestrian;

[0170] A clustering module 620, configured to cluster all pixels in all the global feature maps to obtain several categories related to body parts;

[0171] A determination module 630, configured to generate a corresponding category label at the position of each pixel in the global feature map to which the pixel belongs according to the category to which the pixel belongs for each pixel, and respectively determine each global feature map with a category label at the pixel position as the body part semantic map mapped by the global feature map.

[0172] Among them, the specific shape and structure of the extraction module 610, the clustering module 620, and the determination module 630 in the embodiment of the present invention are not limited, and those skilled in the art can set them arbitrarily according to their implemented functions and effects, which will not be elaborated here; in addition, the specific implementation process and implementation effect of the operation steps implemented by the extraction module 610, the clustering module 620, and the determination module 630 in the embodiment of the present invention are the same as those of steps 101 to 103 in the embodiment of the present invention, and can be specifically referred to the above description content, which will not be elaborated here.

[0173] Based on the above embodiments, continue to refer to Figure 6 , in the embodiments of the present invention, there is no limitation on the specific implementation process of the clustering module 620 to obtain several categories related to body parts. Those skilled in the art can set it according to specific design requirements. Preferably, when the clustering module 620 obtains several categories related to body parts, the clustering module 620 is specifically configured to perform: perform a first clustering on all pixels to obtain foreground pixels; perform a second clustering on the foreground pixels according to the similarity between pixels to obtain several categories related to body parts.

[0174] Based on the same inventive concept, an embodiment of the present invention provides a training device for a person re-identification model, which is used to execute the above-mentioned method for training a person re-identification model, as Figure 7 shown, the device includes:

[0175] A processing module 710, configured to determine the body part semantic map of the sample image by using the method for determining the body part semantic map in the embodiments of the present invention;

[0176] A training module 720, configured to train the person re-identification model by using the determined body part semantic map of the sample image.

[0177] Among them, in the embodiments of the present invention, there is no limitation on the specific shape and structure of the processing module 710 and the training module 720. Those skilled in the art can set them arbitrarily according to their implemented functions. Details are not described herein again; in addition, the specific implementation process and implementation effect of the operation steps implemented by the processing module 710 and the training module 720 in the embodiments of the present invention are the same as those of steps 301 to 302 in the embodiments of the present invention. For details, reference can be made to the above description, and details are not described herein again.

[0178] Based on the above embodiments, continue to refer to Figure 7 , in the embodiments of the present invention, there is no limitation on the specific implementation process of the training module 720 to train the person re-identification model. Those skilled in the art can set it according to specific design requirements. Preferably, when the training module 720 trains the person re-identification model, the training module 720 is specifically configured to perform: obtain the body part feature maps of the sample image according to the body part semantic map and the global feature map of the sample image; calculate the loss function value by using the obtained body part feature maps of the sample image, and iteratively update the parameters of the person re-identification model based on the calculated loss function value until the iteration termination condition is reached.

[0179] Based on the same inventive concept, an embodiment of the present invention provides an unsupervised person re-identification device, which is used to execute the above-mentioned unsupervised person re-identification method, as Figure 8 shown, the device includes:

[0180] An acquisition module 810, configured to obtain feature maps of each body part of a query image and candidate images by using a person re-identification model trained by a training method of a person re-identification model according to an embodiment of the present invention;

[0181] A calculation module 820, configured to calculate the distance between the query image and each candidate image by using the feature maps of each body part of the obtained query image and candidate images;

[0182] An output module 830, configured to sort the candidate images according to the calculated distance between the query image and each candidate image, and output the sorting result as a re-identification result.

[0183] Wherein, in the embodiment of the present invention, the specific shape and structure of the acquisition module 810, the calculation module 820, and the output module 830 are not limited, and those skilled in the art can set them arbitrarily according to their implemented functions and effects, which will not be elaborated herein; in addition, in the embodiment of the present invention, the specific implementation process and implementation effect of the operation steps implemented by the acquisition module 810, the calculation module 820, and the output module 830 are the same as those of steps 501 to 503 in the embodiment of the present invention. For details, reference can be made to the above description, which will not be elaborated herein.

[0184] Based on the above embodiment, continue to refer to Figure 8 , in the embodiment of the present invention, the specific implementation process of the calculation module 820 for calculating the distance between the query image and each candidate image is not limited, and those skilled in the art can set it according to specific design requirements. Preferably, when the calculation module 820 calculates the distance between the query image and each candidate image, the calculation module 820 is specifically configured to execute: calculating the feature distances of each body part of the two images by using the feature maps of each body part of the two images, calculating the global feature distance of the two images by using the global feature maps of the two images, and calculating the foreground feature distance of the two images; calculating the distance between the two images according to whether each body part in the two images is visible and the calculated feature distances, global feature distance, and foreground feature distance of each body part.

[0185] Based on the same inventive concept, an embodiment of the present invention provides a computer device, as Figure 9 shown, the device includes:

[0186] A memory 910;

[0187] A processor 920; and

[0188] A computer program;

[0189] Among them, the computer program is stored in the memory 910 and is configured to be executed by the processor 920 to implement the above method for determining the semantic map of body parts, or the training method of the pedestrian re-identification model, or the unsupervised pedestrian re-identification method.

[0190] Based on the same inventive concept, an embodiment of the present invention provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored;

[0191] The computer program is executed by the processor to implement the above method for determining the semantic map of body parts, or the training method of the pedestrian re-identification model, or the unsupervised pedestrian re-identification method.

[0192] Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0193] It should be understood that the above specific embodiments of the present invention are only used for exemplary illustration or explanation of the principles of the present invention, and do not constitute a limitation on the present invention. Therefore, any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and scope of the present invention shall be included within the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

[0194] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0195] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1means for the functions specified in one or more boxes.

[0196] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one Figure 1 one or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0197] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one Figure 1 one or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

Claims

1. A method for determining a semantic map of a body part, characterized in that, Including: Extracting the global feature map of each image from several images of the same pedestrian; Clustering all pixels in all the global feature maps to obtain several categories related to body parts, including: performing a first clustering on all pixels according to the activation value and / or confidence of the pixels to obtain foreground pixels; performing a second clustering on the foreground pixels according to the similarity between pixels to obtain several categories related to body parts; When performing the first clustering according to the activation value, the global feature map M g The activation value at each pixel position M g (x, y) is defined as ||M g (x, y)||2, and the following formula is used to normalize the activation value of each pixel: where a(x, y) is the activation value of the normalized pixel, and max (i,j) ||Mg(x, y)||2 is the maximum value of the activation values of all pixels, and (i, j) is the coordinate of the pixel in M g ; Determining the pixels with activation values less than the division threshold as background pixels during the first clustering; When performing the second clustering, using the following formula to normalize the position of each pixel: where D(x, y) is the position of the pixel after normalization, M g (x, y) is the position of the pixel in the global feature map M g in, ||Mg(x, y)||2 is M g (x, y) on the activation value; When there is occlusion, the pixels at the occluded positions are classified as background pixels during the second clustering; For each pixel, generating a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs according to the category to which the pixel belongs, and respectively determining each global feature map with the category label at the pixel position as the body part semantic map mapped by the global feature map.

2. A training method for a pedestrian re-identification model, characterized in that, Including: Using the method described in claim 1 to determine the body part semantic map of the sample image; Training the person re-identification model by using the determined body part semantic map of the sample image.

3. The method according to claim 2, wherein Training the person re-identification model by using the body part semantic map of the sample image, including: Obtaining the body part feature maps of the sample image according to the body part semantic map and the global feature map of the sample image; Calculating the loss function value by using the obtained body part feature maps of the sample image, and iteratively updating the parameters of the person re-identification model based on the calculated loss function value until the iteration termination condition is reached.

4. An unsupervised person re-identification method, characterized in that, Including: Using the person re-identification model trained by the method described in claim 2 or 3 to obtain the body part feature maps of the query image and the candidate images; Calculating the distance between the query image and each candidate image by using the obtained body part feature maps of the query image and the candidate images; Sorting the candidate images according to the calculated distance between the query image and each candidate image, and outputting the sorting result as the re-identification result.

5. The method according to claim 4, characterized in that, Calculating the distance between the query image and each candidate image, including: Calculating the body part feature distances between the two images by using the body part feature maps of the two images, calculating the global feature distance between the two images by using the global feature maps of the two images, and calculating the foreground feature distance; Calculating the distance between the two images according to whether each body part in the two images is visible and the calculated body part feature distances, global feature distance and foreground feature distance.

6. A device for determining a semantic map of a body part, characterized in that, Including: An extraction module for extracting the global feature map of each image from several images of the same pedestrian; A clustering module for clustering all pixels in all the global feature maps to obtain several categories related to body parts, including: performing a first clustering on all pixels according to the activation value and / or confidence of the pixels to obtain foreground pixels; performing a second clustering on the foreground pixels according to the similarity between pixels to obtain several categories related to body parts; When performing the first clustering according to the activation value, the global feature map M g defines the activation value at each pixel position M g (x, y) in it as ||M g (x, y)||2, and the following formula is used to normalize the activation value of each pixel: Among them, a(x, y) is the activation value of the normalized pixel, and max (i,j) ||Mg(x, y)||2 is the maximum value of the activation values of all pixels, and (i, j) is the coordinate of the pixel in M g ; Pixels with activation values less than the division threshold are determined as background pixels when performing the first clustering; When performing the second clustering, the following formula is used to normalize the position of each pixel: Among them, D(x, y) is the position of the pixel after normalization processing, and M g (x, y) is the position of the pixel in the global feature map M g ; ||Mg(x, y)||2 is the activation value on M g (x, y); When there is occlusion, the pixels at the occluded positions are classified as background pixels when performing the second clustering; A determination module, for each of the pixels, according to the category to which the pixel belongs, generate a corresponding category label at the position of the pixel in the global feature map to which the pixel belongs, and respectively determine each global feature map with the category label at the pixel position as the body part semantic map mapped by the global feature map.

7. A training device for a pedestrian re-identification model, characterized in that, Comprising: A processing module, configured to use the method described in claim 1 to determine the body part semantic map of the sample image; A training module, configured to use the determined body part semantic map of the sample image to train the person re-identification model.

8. An unsupervised person re-identification device, characterized in that, Comprising: An acquisition module, configured to use the person re-identification model trained by the method described in claim 2 or 3 to acquire the body part feature maps of the query image and the candidate images; A calculation module, configured to use the acquired body part feature maps of the query image and the candidate images to calculate the distance between the query image and each of the candidate images; An output module, configured to sort the candidate images according to the calculated distance between the query image and each candidate image, and output the sorting result as the re-identification result.

9. A computer device, characterized in that, Comprising: A memory; A processor; And A computer program; Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method for determining the body part semantic map described in claim 1, or the method for training the person re-identification model described in claim 2 or 3, or the unsupervised person re-identification method described in claim 4 or 5.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium on which a computer program is stored; The computer program is executed by a processor to implement the method for determining the body part semantic map described in claim 1, or the method for training the person re-identification model described in claim 2 or 3, or the unsupervised person re-identification method described in claim 4 or 5.

Images