Pedestrian re-identification model training method and pedestrian re-identification method

Abstract

The application discloses a pedestrian re-identification model training method and a pedestrian re-identification method. The training method comprises the following steps: obtaining a plurality of first samples; performing size scaling on each first sample to obtain a second sample and a third sample; performing feature extraction on each first feature map based on a backbone module and a convolution module in a preset model to obtain a first feature map and a corresponding second feature map; generating a first weight of the second feature map based on an attention module of the preset model; performing weighted summation on the second feature map of the sample and the corresponding first weight to obtain a feature representation of the sample; obtaining a prediction result by using the feature representation based on a prediction module of the preset model; calculating a loss by using the prediction result and updating the preset model to obtain a pedestrian re-identification model. Thus, the pedestrian re-identification model can generate a first weight according to the size of an input image and the size of a convolution layer, and can further pay attention to the details or outline information of the input image, thereby ensuring the re-identification effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target tracking, and in particular to a pedestrian re-identification model training method and a pedestrian re-identification method. BACKGROUND

[0002] In most pedestrian re-identification methods, a target detection model is used to identify and mark the boundary box of the pedestrian in each image to be detected, so that after determining the region image formed by the boundary box in which the pedestrian is located, each region image is input into a trained pedestrian re-identification model to determine the same pedestrian in each region image.

[0003] However, when the target detection model outputs the region image, the boundary box may be incorrectly labeled, that is, the boundary box includes too much background or does not completely include the pedestrian. Thus, the image quality of the region image input into the pedestrian re-identification model is not the same, resulting in poor pedestrian re-identification effect and difficulty in correctly identifying the same pedestrian in each region image. SUMMARY

[0004] Therefore, the present application provides a pedestrian re-identification model training method and a pedestrian re-identification method to improve the current situation that the image quality of the input image of the pedestrian re-identification model is not the same, resulting in poor pedestrian re-identification effect and difficulty in correctly identifying the same pedestrian in each region image.

[0005] In a first aspect, an embodiment of the present application provides a pedestrian re-identification model training method, comprising:

[0006] obtaining a preset number of first image samples;

[0007] reducing the size of each first image sample to obtain a preset number of second image samples, and increasing the size of each first image sample to obtain a preset number of third image samples;

[0008] performing feature extraction on each image sample based on a backbone module in a preset model to obtain a first feature map of each image sample;

[0009] performing feature extraction on the first feature map based on a convolution module in the preset model to obtain a plurality of second feature maps corresponding to the first feature map, wherein the convolution module includes a plurality of convolution layers arranged in parallel, and each convolution layer has different convolution kernel sizes;

[0010] inputting the first feature map into an attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map;

[0011] perform weighted summation on the second feature maps of each image sample and the corresponding first weights to obtain a corresponding feature representation of each image sample;

[0012] perform prediction on the category of pedestrians in the corresponding image sample based on the prediction module of the preset model by using each feature representation to obtain a prediction result;

[0013] calculate a loss function value based on the prediction result, and update the preset model reversely by using the loss function value to obtain a pedestrian re-identification model, wherein the attention module in the pedestrian re-identification model is configured to generate a larger first weight when the corresponding first feature map of the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the corresponding first feature map of the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0014] Optionally, in an available manner provided by the embodiment of the present application, after the first feature map is input into the attention module of the preset model to generate the first weight of each second feature map corresponding to the first feature map, and before the loss function value is calculated based on the prediction result, and the preset model is updated reversely by using the loss function value to obtain the pedestrian re-identification model, the method further comprises:

[0015] input the first feature map into a preset attention supervision module to generate a second weight of each second feature map corresponding to the first feature map, wherein the attention supervision module is configured to generate a larger second weight when the corresponding first feature map of the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger second weight when the corresponding first feature map of the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger;

[0016] the calculation of the loss function value based on the prediction result, and the update of the preset model reversely by using the loss function value to obtain the pedestrian re-identification model, comprises:

[0017] calculating a loss function value based on the first weight, the second weight and the prediction result;

[0018] updating the preset model reversely based on the loss function value to make the first weight of the second feature map close to the second weight, and obtaining the pedestrian re-identification model.

[0019] Optionally, in an available manner provided by the embodiment of the present application, after the preset number of first image samples are obtained, the method further comprises:

[0020] obtaining a class of each pedestrian in all the first image samples;

[0021] calculating a loss function value based on the first weight, the second weight, and the prediction result, comprises:

[0022] calculating a weight difference loss based on the first weight and the second weight of each second feature map;

[0023] calculating a feature classification loss based on the feature representation corresponding to each image sample;

[0024] calculating a class prediction loss based on the class of each pedestrian in all the first image samples and the prediction result;

[0025] obtaining the loss function value based on the weight difference loss, the feature classification loss, and the class prediction loss.

[0026] Optionally, in a feasible manner provided by the embodiment of the present application, the size of each first image sample is reduced to obtain a preset number of second image samples, and the size of each first image sample is increased to obtain a preset number of third image samples, comprising:

[0027] reducing the size of each first image sample to obtain a preset number of second image samples;

[0028] calculating the pixel mean of each first image sample;

[0029] increasing the size of each first image sample to obtain a preset number of third image samples by using the pixel mean of each first image sample.

[0030] Optionally, in a feasible manner provided by the embodiment of the present application, the convolution module comprises a first convolution layer with a convolution kernel size of 1x1, a second convolution layer with a convolution kernel size of 3x3, and a third convolution layer with a convolution kernel size of 5x5.

[0031] In a second aspect, the embodiment of the present application provides a pedestrian re-identification method, comprising:

[0032] obtaining a plurality of images to be identified;

[0033] inputting the plurality of images to be identified into a pedestrian re-identification model to determine the class of each pedestrian in the plurality of images to be identified, wherein the pedestrian re-identification model is obtained by the training method of the pedestrian re-identification model disclosed in any one of the first aspect.

[0034] In a third aspect, the embodiment of the present application provides a training device of a pedestrian re-identification model, comprising:

[0035] a sample acquisition module, configured to acquire a preset number of first image samples;

[0036] a scaling module, configured to reduce the size of each of the first image samples to obtain a preset number of second image samples, and increase the size of each of the first image samples to obtain a preset number of third image samples;

[0037] a first extraction module, configured to perform feature extraction on each image sample based on a backbone module in a preset model to obtain a first feature map of each image sample;

[0038] a second extraction module, configured to perform feature extraction on the first feature map based on a convolution module in the preset model to obtain a plurality of second feature maps corresponding to the first feature map, wherein the convolution module comprises a plurality of convolution layers arranged in parallel, and the convolution kernel size of each convolution layer is different;

[0039] a first generation module, configured to input the first feature map into an attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map;

[0040] a summation module, configured to perform weighted summation on the second feature map of each image sample and the corresponding first weight to obtain a feature representation corresponding to each image sample;

[0041] a prediction module, configured to predict the category of a pedestrian in each image sample by using each feature representation based on a prediction module of the preset model to obtain a prediction result;

[0042] an update module, configured to calculate a loss function value based on the prediction result, and update the preset model in a reverse direction by using the loss function value to obtain a pedestrian re-identification model, wherein the attention module in the pedestrian re-identification model is configured to generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0043] In a fourth aspect, an embodiment of the present application provides a pedestrian re-identification device, comprising:

[0044] an image acquisition module, configured to acquire a plurality of to-be-identified images;

[0045] a re-identification module, configured to input the plurality of to-be-identified images into a pedestrian re-identification model to determine the category of each pedestrian in the plurality of to-be-identified images, wherein the pedestrian re-identification model is obtained by using the training method of the pedestrian re-identification model disclosed in the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, the memory storing a computer program, the computer program performing the training method of the pedestrian re-identification model disclosed in the first aspect or the pedestrian re-identification method disclosed in the second aspect when running on the processor.

[0047] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program performing the training method of the pedestrian re-identification model disclosed in the first aspect or the pedestrian re-identification method disclosed in the second aspect when running on the processor.

[0048] In the training method of the pedestrian re-identification model provided by the embodiment of the present application, the computer device first acquires a preset number of first image samples, that is, acquires batch-size image samples; then, performs size scaling on each first image sample to obtain a second image sample and a third image sample corresponding to each first image sample; subsequently, performs feature extraction on each image sample based on a backbone module in the preset model to obtain a first feature map of each image sample; then, performs feature extraction on each first feature map by using a plurality of convolution layers with different convolution kernel sizes arranged in parallel in the preset model to obtain a second feature map under different receptive fields of the first feature map, that is, a second feature map; then, based on an attention module of the preset model, generates a first weight of each second feature map corresponding to the first feature map according to the size of the first feature map and the convolution kernel size of each convolution layer; then, performs weighted summation on the second feature map of each image sample and the corresponding first weight to obtain a feature representation of each image sample; based on a prediction module of the preset model, predicts the class of the pedestrian in each image sample according to the feature representation of each image sample to complete the re-identification prediction; finally, reversely updates the preset model by using a loss value corresponding to the prediction result, so that in the pedestrian re-identification model trained, the attention module can generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0049] Based on this, the embodiment of the present application sets and trains the attention module, so that when the pedestrian re-identification model predicts the input image with a larger size, the weight of the detailed information of the input image is larger, and then more attention is paid to the detailed information of the input image, avoiding the interference of background noise; when predicting the input image with a smaller size, the weight of the summarized information of the input image is higher, and then the model pays more attention to the summarized information of the input image, ensuring the complete capture of the pedestrian features. Moreover, the embodiment of the present application generates the second image sample and the third image sample on the basis of the first image sample, so that the number of image samples is increased, thereby ensuring the sufficient training of the model. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope of protection of the present application. In each drawing, similar components are marked with similar reference numerals.

[0051] Figure 1 A flowchart of a training method of a first pedestrian re-identification model provided by the embodiment of the present application is shown;

[0052] Figure 2 A flowchart of a training method of a second pedestrian re-identification model provided by the embodiment of the present application is shown;

[0053] Figure 3 A flowchart of a pedestrian re-identification method provided by the embodiment of the present application is shown;

[0054] Figure 4 A structural diagram of a training device of a pedestrian re-identification model provided by the embodiment of the present application is shown;

[0055] Figure 5 A structural diagram of a pedestrian re-identification device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0057] The components of the embodiments of the present application generally described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the application, but simply represents selected embodiments of the application. Based upon the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without paying creative effort are within the scope of the present application.

[0058] Hereinafter, the terms "include", "have", and their conjugates, used in various embodiments of the present application, merely indicate the presence of the features, numbers, steps, operations, elements, components, or combinations thereof, and should not be construed as precluding the presence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof.

[0059] In addition, the terms "first", "second", "third", and the like are used only to distinguish descriptions, and should not be understood as indicating or implying relative importance.

[0060] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present application belong. The terms (such as terms defined in a generally used dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized or overly formal meaning, unless clearly defined in various embodiments of the present application.

[0061] Embodiment 1

[0062] Referring to Figure 1 , a flowchart of a training method of a first pedestrian re-identification model provided by an embodiment of the present application is shown, and the training method of the pedestrian re-identification model provided by the embodiment of the present application comprises:

[0063] S110, a preset number of first image samples are obtained.

[0064] It should be noted that the image samples involved in the embodiments of the present application refer to images containing persons.

[0065] It should also be noted that the acquisition method of the image sample is content that can be set according to actual conditions, such as in a feasible manner, the embodiment of the present application obtains the bounding box of each pedestrian in the image by inputting the image containing multiple pedestrians into the pedestrian target detection model; the image is cropped according to the bounding box of each pedestrian, and then a plurality of image samples are obtained.

[0066] Further, it can be understood that the computer device in the embodiment of the present application extracts a batch size of image samples from the training set.

[0067] It can also be understood that the batch size, i.e., the size of the preset number, is content that can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0068] S120, the size of each first image sample is reduced to obtain a preset number of second image samples, and the size of each first image sample is increased to obtain a preset number of third image samples.

[0069] That is, after obtaining the original image sample, i.e., obtaining the first image sample, the embodiment of the present application reduces the size of each first image sample to obtain each first image sample after size reduction, i.e., a preset number of second image samples; at the same time, each first image sample is enlarged in size to obtain each first image sample after enlargement, i.e., a preset number of third image samples.

[0070] It should be understood that because the pedestrian target detection model may be difficult to generate a suitable bounding box for each pedestrian in the input image due to various factors, it may occur that the generated bounding box includes too much background image, or it may occur that only part of the image of a complete pedestrian is included, such as an image containing only the torso of the pedestrian. Therefore, in the pedestrian image obtained by cropping based on the bounding box, the larger size pedestrian image may have too much background image, and the smaller size pedestrian image may only contain part of the complete pedestrian.

[0071] In this case, the pedestrian re-identification model will be disturbed by too much background noise in the input image and / or by the incomplete pedestrian in the pedestrian image, making it difficult to extract correct / complete pedestrian features, resulting in difficulty in achieving the expected re-identification effect.

[0072] In view of this situation, the embodiment of the present application generates second image samples by scaling the size of the first image sample to simulate the case where the input image only contains part of the complete pedestrian, and generates third image samples to simulate the case where the input image has too much background image; and trains according to the first image sample, the second image sample and the third image sample, so that the model can correctly complete the re-identification for different image samples.

[0073] S130, based on the backbone module in the preset model, performing feature extraction on each image sample to obtain a first feature map of each image sample.

[0074] It can be understood that the backbone module refers to the network structure used for feature extraction in the target detection model.

[0075] It can also be understood that the backbone module can be composed of any network model, such as ResNet16 in a feasible manner.

[0076] S140, performing feature extraction on the first feature map based on a convolution module in the preset model to obtain a plurality of second feature maps corresponding to the first feature map, wherein the convolution module comprises a plurality of convolution layers arranged in parallel, and the convolution kernel sizes of each convolution layer are different.

[0077] It is not difficult to understand that when the input image contains too much background information, the pedestrian re-identification model should pay attention to the details / local information in the input image to avoid the interference of the background information; and when the input image contains too little pedestrian information, the pedestrian re-identification model should pay attention to the global / summary information in the input image to include the complete capture of the pedestrian features.

[0078] Based on this, for different sizes of image samples, that is, for different sizes of the first image sample, the second image sample and the third image sample, the embodiment of the application uses a plurality of convolution layers arranged in parallel and having different convolution kernel sizes to perform feature extraction of different dimensions / sizes on each image sample to generate a plurality of second feature maps corresponding to the first feature map, that is, using convolution layers with different convolution kernel sizes to capture feature information of different region sizes in the image sample. Therefore, for the second image sample with smaller size, the embodiment of the application can capture the global / summary information in the second image sample, and for the third image sample with larger size, the embodiment of the application will capture the details / local information in the third image sample.

[0079] Optionally, in a feasible manner provided by the embodiment of the application, the convolution module comprises a first convolution layer with a convolution kernel size of 1x1, a second convolution layer with a convolution kernel size of 3x3 and a third convolution layer with a convolution kernel size of 5x5.

[0080] S150, inputting the first feature map to an attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map.

[0081] It should be understood that if the image sample includes too much background information, the smaller the convolution kernel size of the plurality of second feature maps corresponding to the image sample, that is, the smaller the feature region to be paid attention to in the feature information with different convolution kernel sizes corresponding to the image sample, the less the background information contained in the corresponding feature information, so as to avoid the interference of the background information, and pay more attention to the information output by the convolution layer with smaller convolution kernel size.

[0082] And if the image sample includes incomplete pedestrian information, the information output by the convolution layer with larger convolution kernel size should be paid attention to.

[0083] Based on this, the embodiment of the present application sets an attention module in the pedestrian re-identification model, so that the pedestrian re-identification model pays more attention to the local / detail of the first feature map when the size of the first feature map is larger, that is, the weight of the second feature map output by the convolution layer with a smaller convolution kernel size in the second feature map corresponding to the first feature map is larger; at the same time, when the size of the first feature map is smaller, the weight of the second feature map output by the convolution layer with a larger convolution kernel size in the second feature map corresponding to the first feature map is larger.

[0084] Therefore, in the subsequent process, when the computer device performs weighted summation on each second feature map corresponding to the first feature map to obtain the feature representation, the larger the size of the image sample is, the higher the proportion of local / detail information in the feature representation corresponding to the image sample is; and the smaller the size of the image sample is, the higher the proportion of global / summary information in the feature representation corresponding to the image sample is.

[0085] In addition, it should be noted that in the training process of the preset model, that is, the pedestrian re-identification model, the attention module will be updated gradually in the continuous training / iterative update of the parameters, so that at the end of the training, the first weight as described above can be generated according to the size of the first feature map and the convolution kernel size of each convolution layer.

[0086] It can be understood that how to train the parameters in the attention module is a content that can be set according to actual conditions, for example, in a feasible manner, the embodiment of the present application calculates the corresponding weight loss value by using a preset weight loss function, and then updates the parameters in the attention module according to the weight loss value, so that the attention module has the above-mentioned function.

[0087] S160, weighted summation is performed on the second feature map of each image sample and the corresponding first weight to obtain the feature representation corresponding to each image sample.

[0088] For example, it is assumed that a first feature map corresponds to three second feature maps f1, f2 and f3, the first weight of f1 is w1, the first weight of f2 is w2, and the first weight of f3 is w3, then the feature representation of the image sample is (f1xw1)+(f2xw2)+(f3xw3).

[0089] S170, using each of the feature representations, a prediction module based on the preset model predicts the classification of each pedestrian in the preset number of first image samples to obtain a prediction result.

[0090] It can be understood that the prediction result represents the recognition result of the same classification of pedestrians by the preset model.

[0091] It can also be understood that the pedestrian re-identification task is used to detect and identify pedestrians in multiple images to determine the same pedestrian in the multiple images. That is, if different first image samples each contain the same pedestrian, the classification of the pedestrian in each first image sample is the same.

[0092] S180, calculate a loss function value based on the prediction result, and update the preset model in reverse using the loss function value to obtain a pedestrian re-identification model, wherein the attention module in the pedestrian re-identification model is used to generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0093] That is, through multiple iterations of training, the attention module in the pedestrian re-identification model that has completed training can generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and can also generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0094] It can be understood that although only one round of the multiple rounds of iteration of the model is described in the embodiment of the present application, it is not difficult to understand that before the training is completed to obtain the pedestrian re-identification model, the steps of "extracting a batch size of image samples from a data set" to "updating the model in reverse using the loss function value corresponding to the batch size of image samples" will be continuously executed, that is, the step of "calculating a loss function value based on the prediction result, and updating the preset model in reverse using the loss function value" in S110 to S180 will be continuously executed until the preset training termination condition is met, the training is completed, and the pedestrian re-identification model is obtained.

[0095] It can also be understood that the training termination condition is a content that can be set according to actual conditions, such as in one feasible manner, the training termination condition is to stop after the training period reaches a preset number. In another feasible manner, the training termination condition is to stop when the loss function converges.

[0096] In the training method of the pedestrian re-identification model provided in the embodiment of the present application, the computer device first acquires a preset number of first image samples, that is, acquires batch-size image samples; then, performs size scaling on each first image sample to obtain a second image sample and a third image sample corresponding to each first image sample; subsequently, performs feature extraction on each image sample based on a backbone module in the preset model to obtain a first feature map of each image sample; thereafter, performs feature extraction on each first feature map by using a plurality of convolution layers with different convolution kernel sizes arranged in parallel in the preset model to obtain a second feature map, that is, a feature map under different receptive fields of the first feature map; then, based on an attention module of the preset model, generates a first weight of each second feature map corresponding to the first feature map according to the size of the first feature map and the convolution kernel size of each convolution layer; then, performs weighted summation on the second feature map of each image sample and the corresponding first weight to obtain a feature representation of each image sample; based on a prediction module of the preset model, predicts the class of the pedestrian in each image sample according to the feature representation of each image sample to complete the re-identification prediction; finally, reversely updates the preset model by using a loss value corresponding to the prediction result, so that in the pedestrian re-identification model trained, the attention module can generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0097] Based on this, the embodiment of the present application trains the attention module to make the pedestrian re-identification model give more weight to the detailed information of the input image when predicting the input image with a larger size, so as to pay more attention to the detailed information of the input image and avoid the interference of background noise; and when predicting the input image with a smaller size, the model pays more attention to the summarized information of the input image, so as to ensure the complete capture of the pedestrian features. Moreover, the embodiment of the present application generates the second image sample and the third image sample on the basis of the first image sample, so that the number of image samples is increased, thereby ensuring the sufficient training of the model.

[0098] Optionally, in one feasible manner provided in the embodiment of the present application, the method can be specifically referred to Figure 2 FIG. 2 shows a flowchart of a second training method of a pedestrian re-identification model provided in the embodiment of the present application, that is, in this feasible manner, the method comprises the following steps after S140 and before S180:

[0099] S190, input the first feature map into a preset attention supervision module to generate a second weight of each second feature map corresponding to the first feature map, wherein the attention supervision module is used to generate a larger second weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger second weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger;

[0100] The S180 comprises:

[0101] S181, calculate a loss function value based on the first weight, the second weight and the prediction result;

[0102] S182, update the preset model reversely based on the loss function value, so that the first weight and the second weight of the second feature map are similar, and obtain a pedestrian re-identification model.

[0103] That is, in this feasible manner, the embodiment of the present application will supervise the attention module by using the preset attention supervision module, so as to ensure that the attention module can generate appropriate first weights.

[0104] It can be understood that the setting mode of the attention supervision module is content that can be set according to actual conditions, for example, in a feasible manner provided by the embodiment of the present application, the attention supervision module is a mapping table, which contains a plurality of second weights, a plurality of size of first feature maps and a plurality of convolution kernel sizes, one second weight corresponds to one size of first feature map and one convolution kernel size. Thus, after the size of the first feature map is obtained by the computer device, the second weight of each second feature map corresponding to the first feature map can be determined efficiently and simply according to the attention supervision module.

[0105] And in another feasible manner provided by the embodiment of the present application, the setting mode of the attention supervision module is shown in the following example:

[0106] Suppose that the convolution module comprises three convolution layers with convolution kernel sizes increasing successively, the first image sample is I1, the second image sample I2 is I1 zoomed inwards by p s to obtain, the third image sample I3 is I1 zoomed outwards by p b to obtain.

[0107] The first feature maps of I1, I2 and I3 are input into the convolution module to obtain three second feature maps f1 1 , f1 2 and f1 3 of I1, three second feature maps f2 1 , f2 2 and f23 And the three second feature maps f3 of I3 1 f3 2 and f3 3 .

[0108] Each first feature map is input into the attention module to generate the second weights w1 of the three second feature maps of I1. 1 w1 2 and W1 3 The second weights w2 of the three second feature maps of I2 1 w2 2 and W2 3 And the second weights w3 of the three second feature maps of I3. 1 w3 2 and w3 3 .

[0109] The attention supervision module, based on the first weight output by the attention module, combines p s and p b The second weights of the three second feature maps of I1 are w1. 1 w1 2 and W1 3 The second weights of the three second feature maps of I2 are w2. 1 w2 2 and W2 3 (1+p s ), and the second weights of the three second feature maps of I3 are w3. 1 (1+p b w3 2 and w3 3 .

[0110] It should be noted that, among the variables mentioned above, the numerical subscript of the variable indicates the image sample corresponding to the variable, such as subscript 1 indicating that the variable corresponds to the first image sample, and subscript 2 indicating that the variable corresponds to the second image sample; while the numerical superscript of the variable indicates the serial number of the variable.

[0111] It should also be noted that, in this feasible approach, the only difference between the outputs of the attention module and the attention supervision module lies in the first weight w2 corresponding to index 3 of I2. 3 And the second weight w2 with index 3 3 (1+p s ), and the first weight w3 corresponding to I3 is number 1. 1 And the second weight w3 with index 1 1 (1+p b), the other first weight and the second weight are the same. Based on this, in the training process of the pedestrian re-identification model, the second weight is certainly not less than the first weight, so as to ensure that the first weight will be updated in the correct direction.

[0112] In addition, it can be understood that the attention supervision module in the embodiment of the application only operates in the model training process, and the attention supervision module will stop running in the inference process of the model.

[0113] In addition, it can be understood that the attention supervision module in the embodiment of the application only operates in the model training process, and the attention supervision module will stop running in the inference process of the model. Figure 2 Only used as an example, and not used to limit S190 needs to be immediately after S140, the embodiment of the application only requires S190 to be executed after S140, and before S181.

[0114] Optionally, in a feasible manner provided by the embodiment of the application, the convolution module, the attention module and the attention supervision module are copied multiple times and placed in any intermediate layer after the backbone module in the form of a group of three.

[0115] Optionally, in order to ensure the training effect of the pedestrian re-identification model, in a feasible manner provided by the embodiment of the application, after S110, further comprising:

[0116] Obtaining the class of the pedestrian in each first image sample;

[0117] Further, S181 comprises:

[0118] Based on the first weight and the second weight of each second feature map, calculating a weight difference loss;

[0119] Based on the feature representation corresponding to each image sample, calculating a feature classification loss;

[0120] Based on the class of each pedestrian in all first image samples and the prediction result, calculating a class prediction loss;

[0121] Based on the weight difference loss, the feature classification loss and the class prediction loss, obtaining a loss function value.

[0122] It can be understood that in this case, the embodiment of the application will calculate the corresponding class prediction loss according to the class of the pedestrian in the first image sample, that is, the real class, and according to the predicted class of each pedestrian in the first image sample in the prediction result; at the same time, the feature classification loss will also be calculated according to the feature representation corresponding to each image sample; and the weight difference loss will be calculated according to the difference between the first weight and the second weight of each second feature map.

[0123] It can also be understood that the formula / algorithm used to calculate each loss is set according to actual conditions, such as in a preferred manner, the weight difference loss is calculated by a squared difference loss function formula, the feature classification loss is calculated by a triplet loss function formula, and the category prediction loss is calculated by a cross-entropy loss function.

[0124] Optionally, in an embodiment of the present application, the S120 comprises:

[0125] The size of each first image sample is reduced to obtain a preset number of second image samples;

[0126] The pixel mean value of each first image sample is calculated;

[0127] The size of each first image sample is increased using the pixel mean value of each first image sample to obtain a preset number of third image samples.

[0128] That is, in this feasible manner, the embodiment of the present application completes the generation of the third image sample based on the mean filling method.

[0129] It can be understood that mean filling is only one feasible way, and in actual production environment, the generation of the third image sample can also be realized by other methods / calculations, such as in another feasible manner, the third image sample is realized based on the maximum value filling method.

[0130] Embodiment 2

[0131] The embodiment 2 of the present application further provides a pedestrian re-identification method, referring to Figure 3 , a flowchart of a pedestrian re-identification method provided by the embodiment of the present application is shown, the pedestrian re-identification method provided by the embodiment of the present application comprises:

[0132] S210, a plurality of images to be identified are obtained;

[0133] S220, the plurality of images to be identified are input into a pedestrian re-identification model to determine the category of each pedestrian in the plurality of images to be identified, wherein the pedestrian re-identification model is obtained by the training method of the corresponding pedestrian re-identification model of the embodiment 1.

[0134] It can be understood that the pedestrian re-identification method provided by the embodiment 2 of the present application is based on the training method of the pedestrian re-identification model provided by the embodiment 1, and the pedestrian re-identification is completed by using the trained pedestrian re-identification model.

[0135] It can also be understood that in the pedestrian re-identification method provided by the embodiment of the present application, the computer device generates corresponding first weights for input images of different sizes based on the attention module in the pedestrian re-identification model, and then can perform pedestrian re-identification according to the details or summary information of the input image, thereby avoiding the interference of background noise in the input image, and at the same time, when the pedestrian image is incomplete, all pedestrian feature information in the input image can be completely captured.

[0136] Embodiment 3

[0137] Corresponding to the training method of the pedestrian re-identification model provided by Embodiment 1 of the present application, Embodiment 3 of the present application also provides a training device of a pedestrian re-identification model, which is described with reference to Figure 4 , which shows a structural schematic diagram of the training device of the pedestrian re-identification model provided by the embodiment of the present application. The training device 300 of the pedestrian re-identification model provided by the embodiment of the present application comprises:

[0138] The sample acquisition module 310 is configured to acquire a preset number of first image samples.

[0139] The scaling module 320 is configured to reduce the size of each first image sample to obtain a preset number of second image samples, and increase the size of each first image sample to obtain a preset number of third image samples.

[0140] The first extraction module 330 is configured to perform feature extraction on each image sample based on a backbone module in a preset model to obtain a first feature map of each image sample.

[0141] The second extraction module 340 is configured to perform feature extraction on the first feature map based on a convolution module in the preset model to obtain a plurality of second feature maps corresponding to the first feature map, wherein the convolution module comprises a plurality of convolution layers arranged in parallel, and the convolution kernel size of each convolution layer is different.

[0142] The first generation module 350 is configured to input the first feature map into an attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map.

[0143] The summation module 360 is configured to perform weighted summation on the second feature map of each image sample and the corresponding first weight to obtain a feature representation corresponding to each image sample.

[0144] The prediction module 370 is configured to predict the category of a pedestrian in each image sample based on a prediction module of the preset model using each feature representation to obtain a prediction result.

[0145] The updating module 380 is configured to calculate a loss function value based on the prediction result, and update the preset model reversely based on the loss function value to obtain a pedestrian re-identification model, wherein the attention module in the pedestrian re-identification model is configured to generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0146] Optionally, in an available manner provided by the embodiment of the present application, the device further comprises:

[0147] The second generating module is configured to input the first feature map into a preset attention supervision module to generate a second weight of each second feature map corresponding to the first feature map, wherein the attention supervision module is configured to generate a larger second weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger second weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

[0148] The updating module comprises:

[0149] The function value calculating submodule is configured to calculate a loss function value based on the first weight, the second weight and the prediction result.

[0150] The reverse updating submodule is configured to update the preset model reversely based on the loss function value, so that the first weight and the second weight of the second feature map are similar, to obtain a pedestrian re-identification model.

[0151] Optionally, in an available manner provided by the embodiment of the present application, the sample obtaining module further comprises:

[0152] The category obtaining submodule is configured to obtain the category of each pedestrian in all the first image samples.

[0153] Further, the function value calculating submodule comprises:

[0154] The difference loss calculating unit is configured to calculate a weight difference loss based on the first weight and the second weight of each second feature map.

[0155] The classification loss calculating unit is configured to calculate a feature classification loss based on the feature representation corresponding to each image sample.

[0156] The prediction loss calculating unit is configured to calculate a category prediction loss based on the category of each pedestrian in all the first image samples and the prediction result.

[0157] a loss function value acquisition unit, configured to obtain a loss function value based on the weight difference loss, the feature classification loss and the category prediction loss.

[0158] Optionally, in an available manner provided by the embodiment of the present application, the scaling module comprises:

[0159] a reducing submodule, configured to reduce the size of each first image sample to obtain a preset number of second image samples;

[0160] a mean value calculation submodule, configured to calculate the pixel mean value of each first image sample;

[0161] an enlarging submodule, configured to increase the size of each first image sample by using the pixel mean value of each first image sample to obtain a preset number of third image samples.

[0162] Optionally, in an available manner provided by the embodiment of the present application, the first convolutional layer with a kernel size of 1x1, the second convolutional layer with a kernel size of 3x3 and the third convolutional layer with a kernel size of 5x5.

[0163] The training device 300 of the pedestrian re-identification model provided by the embodiment of the present application can implement each process of the training method of the pedestrian re-identification model corresponding to the embodiment 1, and can achieve the same technical effects. To avoid repetition, details are not repeated here.

[0164] Embodiment 4

[0165] Corresponding to the pedestrian re-identification method provided by the embodiment 1 of the present application, the embodiment 2 of the present application further provides a pedestrian re-identification device, which refers to Figure 5 , shows the structure schematic diagram of the pedestrian re-identification device provided by the embodiment of the present application, the pedestrian re-identification device 400 provided by the embodiment of the present application comprises:

[0166] an image acquisition module 410, configured to acquire a plurality of to-be-identified images;

[0167] a re-identification module 420, configured to input the plurality of to-be-identified images into a pedestrian re-identification model to determine the category of each pedestrian in the plurality of to-be-identified images, wherein the pedestrian re-identification model is obtained by the training method of the pedestrian re-identification model corresponding to the embodiment 1.

[0168] The pedestrian re-identification device 400 provided by the embodiment of the present application can implement each process of the pedestrian re-identification method corresponding to the embodiment 2, and can achieve the same technical effects. To avoid repetition, details are not repeated here.

[0169] The embodiment of the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the computer program performs the training method of the pedestrian re-identification model corresponding to the embodiment 1 or the pedestrian re-identification method corresponding to the embodiment 2 when running on the processor.

[0170] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program performs the training method of the pedestrian re-identification model corresponding to the embodiment 1 or the pedestrian re-identification method corresponding to the embodiment 2 when running on the processor.

[0171] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other manners. The apparatus embodiments described above are only schematic, for example, the flow charts and structural diagrams in the drawings show the possible implementation architectures, functions and operations of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flow charts or structural diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that, in alternative implementation manners, the functions noted in the blocks can also occur in different orders from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the structural diagram and / or flow chart, and the combination of blocks in the structural diagram and / or flow chart, can be implemented by a dedicated hardware-based system for implementing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0172] In addition, each functional module or unit in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0173] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0174] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for training a pedestrian re-identification model, characterized in that, The method comprises: obtaining a preset number of first image samples; reducing the size of each first image sample to obtain a preset number of second image samples, and increasing the size of each first image sample to obtain a preset number of third image samples; performing feature extraction on each image sample based on a backbone module in a preset model to obtain a first feature map of each image sample; performing feature extraction on the first feature map based on a convolution module in the preset model to obtain a plurality of second feature maps corresponding to the first feature map, wherein the convolution module comprises a plurality of convolution layers arranged in parallel, and the convolution kernel size of each convolution layer is different; inputting the first feature map into an attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map; performing weighted summation on the second feature map of each image sample and the corresponding first weight to obtain a feature representation corresponding to each image sample; using each feature representation to predict the category of a pedestrian in the corresponding image sample based on a prediction module of the preset model to obtain a prediction result; calculating a loss function value based on the prediction result, and updating the preset model in a reverse direction using the loss function value to obtain a pedestrian re-identification model, wherein the attention module in the pedestrian re-identification model is configured to generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

2. The method of claim 1, wherein, After the first feature map is input into the attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map, and before the loss function value is calculated based on the prediction result, and the preset model is updated in a reverse direction using the loss function value to obtain a pedestrian re-identification model, the method further comprises: inputting the first feature map into a preset attention supervision module to generate a second weight of each second feature map corresponding to the first feature map, wherein the attention supervision module is configured to generate a larger second weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger second weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger; calculating a loss function value based on the first weight, the second weight, and the prediction result; updating the preset model in a reverse direction based on the loss function value to make the first weight and the second weight of the second feature map similar, and obtain a pedestrian re-identification model. After the preset number of first image samples are obtained, the method further comprises:

3. The method of claim 2, wherein, obtaining the category of each pedestrian in all the first image samples; ​ The computing loss function value based on the first weight, the second weight and the prediction result comprises: computing a weight difference loss based on the first weight and the second weight of each second feature map; computing a feature classification loss based on the feature representation corresponding to each image sample; computing a category prediction loss based on the category of each pedestrian in all the first image samples and the prediction result; obtaining the loss function value based on the weight difference loss, the feature classification loss and the category prediction loss.

4. The method of claim 1, wherein, The reducing the size of each first image sample to obtain a preset number of second image samples and increasing the size of each first image sample to obtain a preset number of third image samples comprises: reducing the size of each first image sample to obtain a preset number of second image samples; computing the pixel mean value of each first image sample; increasing the size of each first image sample to obtain a preset number of third image samples by using the pixel mean value of each first image sample.

5. The method of claim 1, wherein, The convolution module comprises a first convolution layer with a convolution kernel size of 1x1, a second convolution layer with a convolution kernel size of 3x3 and a third convolution layer with a convolution kernel size of 5x5.

6. A pedestrian re-identification method, characterized in that, It comprises: obtaining a plurality of to-be-identified images; inputting the plurality of to-be-identified images into a pedestrian re-identification model to determine the category of each pedestrian in the plurality of to-be-identified images, wherein the pedestrian re-identification model is obtained by the training method of the pedestrian re-identification model according to any one of claims 1-5.

7. A device for training a pedestrian re-identification model, comprising: It comprises: a sample acquisition module for acquiring a preset number of first image samples; a scaling module for reducing the size of each first image sample to obtain a preset number of second image samples and increasing the size of each first image sample to obtain a preset number of third image samples; a first extraction module for performing feature extraction on each image sample based on a backbone module in a preset model to obtain a first feature map of each image sample; a second extraction module for performing feature extraction on the first feature map based on a convolution module in the preset model to obtain a plurality of second feature maps corresponding to the first feature map, wherein the convolution module comprises a plurality of convolution layers arranged in parallel, and each convolution layer has a different convolution kernel size; a first generation module for inputting the first feature map into an attention module of the preset model to generate a first weight of each second feature map corresponding to the first feature map; a summation module for weighted summation of the second feature map of each image sample and the corresponding first weight to obtain a feature representation corresponding to each image sample; a prediction module for predicting the category of pedestrians in the corresponding image sample by using each feature representation based on a prediction module of the preset model to obtain a prediction result; The updating module is configured to calculate a loss function value based on the prediction result, and update the preset model reversely by using the loss function value to obtain a pedestrian re-identification model, wherein the attention module in the pedestrian re-identification model is configured to generate a larger first weight when the first feature map corresponding to the second feature map is larger and the convolution kernel size of the convolution layer outputting the second feature map is smaller, and generate a larger first weight when the first feature map corresponding to the second feature map is smaller and the convolution kernel size of the convolution layer outputting the second feature map is larger.

8. A pedestrian re-identification apparatus, characterized by comprising: The method comprises: The image acquisition module is configured to acquire a plurality of to-be-identified images; The re-identification module is configured to input the plurality of to-be-identified images into a pedestrian re-identification model to determine the category of each pedestrian in the plurality of to-be-identified images, wherein the pedestrian re-identification model is obtained by using the training method of the pedestrian re-identification model according to any one of claims 1 to 5.

9. A computer device, comprising: The computer program is stored in the memory and executed on the processor to perform the training method of the pedestrian re-identification model according to any one of claims 1 to 5, or the pedestrian re-identification method according to claim 6.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and executed on the processor to perform the training method of the pedestrian re-identification model according to any one of claims 1 to 5, or the pedestrian re-identification method according to claim 6.

Images