Human body attribute and identity recognition model training method, recognition method and related device

Abstract

The invention discloses a human body attribute and identity recognition model training method, a human body attribute and identity recognition method and a related device. The training method comprises the following steps: constructing a human body attribute and identity recognition network comprising a backbone network and a classification layer; performing feature extraction on the pedestrian training image through a backbone network, and performing attribute classification on a first pedestrian feature extracted by the backbone network through a classification layer to obtain a first attribute classification result; calculating an identity loss value based on the identity ID tag of the pedestrian training image and the first pedestrian feature, and calculating an attribute loss value based on the attribute tag of the pedestrian training image and the first attribute classification result; calculating a total loss value through the identity loss value and the attribute loss value; and network parameters of the backbone network and the classification layer are updated through the total loss value until the human body attribute and identity recognition network converges, and a trained human body attribute and identity recognition model is obtained. According to the model training method, the number of model training is reduced, resource consumption is saved, and the requirement for hardware equipment can be lowered.

Description

Technical Field

[0001] The present application relates to the technical field of pedestrian recognition, and in particular to a human attribute and identity recognition model training method, a recognition method and related devices. Background Art

[0002] In the application scenarios of passenger flow statistics and attribute analysis, customer attribute classification is realized by training attribute classification models. When performing passenger flow statistics, it is easy to count multiple customers that appear multiple times, but in actual situations, multiple customers only need to be counted once, that is, it is necessary to deduplicate the counted customers. The existing technology usually uses an additionally trained identity recognition model to identify the identity of each customer, and deduplicates the passenger flow statistics task according to the identity of each customer. This method requires training two models separately, consumes a lot of computing resources, and in actual applications, the two models need to be embedded in hardware devices, which has high requirements for hardware devices and is not suitable for low-performance hardware devices. Summary of the invention

[0003] The present application provides a human attribute and identity recognition model training method, recognition method and related devices, which are used to provide a model training method that can simultaneously recognize human attributes and identities, reduce the number of model trainings, and thus reduce the requirements for hardware equipment.

[0004] In view of this, the first aspect of the present application provides a method for training a human attribute and identity recognition model, comprising:

[0005] Constructing a human attribute and identity recognition network, wherein the human attribute and identity recognition network includes a backbone network and a classification layer;

[0006] Extracting features of pedestrian training images through the backbone network, and classifying attributes of first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result;

[0007] Calculating an identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature, and calculating an attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result;

[0008] Calculate a total loss value by using the identity loss value and the attribute loss value;

[0009] The network parameters of the backbone network and the classification layer are updated by the total loss value until the human body attribute and identity recognition network converges to obtain a trained human body attribute and identity recognition model.

[0010] Optionally, the method further includes:

[0011] Extracting features from the pedestrian training image using a first teacher model to obtain second pedestrian features; the first teacher model is a trained human identity recognition model;

[0012] The calculating the identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature includes:

[0013] Calculate a first identity loss value according to the identity ID label of the pedestrian training image and the first pedestrian feature;

[0014] A second identity loss value is calculated using the first pedestrian feature and the second pedestrian feature.

[0015] Optionally, the method further includes:

[0016] The pedestrian training image is subjected to attribute recognition by a second teacher model to obtain a second attribute classification result; the second teacher model is a trained human attribute recognition model;

[0017] The calculating the attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result includes:

[0018] Calculating a first attribute loss value according to the attribute label of the pedestrian training image and the first attribute classification result;

[0019] The second attribute loss value is calculated by using the first attribute classification result and the second attribute classification result.

[0020] Optionally, calculating the total loss value by using the identity loss value and the attribute loss value includes:

[0021] The identity loss value and the attribute loss value are weightedly summed to obtain a total loss value.

[0022] Optionally, the backbone network is a trained feature extraction model.

[0023] The second aspect of the present application provides a method for identifying human attributes and identities, comprising:

[0024] Inputting the image of the pedestrian to be identified into the human attribute and identity recognition model for feature extraction and attribute classification, and obtaining pedestrian features and human attribute classification results; wherein the human attribute and identity recognition model is trained using the human attribute and identity recognition model training method described in any one of the first aspects;

[0025] The pedestrian features are similarly calculated with pedestrian feature vectors stored in a preset pedestrian database, and the identity ID corresponding to the pedestrian feature vector with the highest similarity is output. The preset pedestrian database stores the corresponding relationship between the pedestrian feature vectors and the identity ID.

[0026] The third aspect of the present application provides a human attribute and identity recognition model training device, comprising:

[0027] A construction unit, used to construct a human attribute and identity recognition network, wherein the human attribute and identity recognition network includes a backbone network and a classification layer;

[0028] A first feature extraction and attribute classification unit, configured to extract features from a pedestrian training image through the backbone network, and to classify attributes of the first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result;

[0029] A first calculation unit is used to calculate an identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature, and calculate an attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result;

[0030] A second calculation unit, configured to calculate a total loss value by using the identity loss value and the attribute loss value;

[0031] A parameter updating unit is used to update the network parameters of the backbone network and the classification layer through the total loss value until the human body attribute and identity recognition network converges to obtain a trained human body attribute and identity recognition model.

[0032] A fourth aspect of the present application provides a human attribute and identity recognition device, comprising:

[0033] A feature extraction and attribute classification unit, used for inputting the pedestrian image to be identified into the human attribute and identity recognition model to perform feature extraction and attribute classification, and obtain pedestrian features and human attribute classification results; wherein the human attribute and identity recognition model is trained by the human attribute and identity recognition model training method described in any one of the first aspects;

[0034] The identity recognition unit is used to calculate the similarity between the pedestrian features and the pedestrian feature vectors stored in the preset pedestrian database, and output the identity ID corresponding to the pedestrian feature vector with the highest similarity. The preset pedestrian database stores the corresponding relationship between the pedestrian feature vectors and the identity ID.

[0035] A fifth aspect of the present application provides an electronic device, the device comprising a processor and a memory;

[0036] The memory is used to store program code and transmit the program code to the processor;

[0037] The processor is used to execute any one of the human attribute and identity recognition model training methods described in the first aspect according to the instructions in the program code, or to execute the human attribute and identity recognition method described in the second aspect.

[0038] In a sixth aspect, the present application provides a computer-readable storage medium for storing program code. When the program code is executed by a processor, it implements any one of the human attribute and identity recognition model training methods described in the first aspect, or implements the human attribute and identity recognition method described in the second aspect.

[0039] It can be seen from the above technical solutions that this application has the following advantages:

[0040] The human attribute and identity recognition model training method provided in the present application simultaneously performs identity recognition task training and attribute classification task training on the human attribute and identity recognition network, so that the trained human attribute and identity recognition model can perform identity recognition and attribute classification. The method only needs to train one model, which reduces the number of models to be trained and saves computing resource consumption. In actual applications, only one model needs to be embedded in the hardware device, which reduces the requirements for the hardware device. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A flowchart of a method for training a human attribute and identity recognition model provided in an embodiment of the present application;

[0043] Figure 2 A diagram of the first training process of the human attribute and identity recognition model provided in an embodiment of the present application;

[0044] Figure 3 A second training process diagram of a human attribute and identity recognition model provided in an embodiment of the present application;

[0045] Figure 4 A third training process diagram of the human attribute and identity recognition model provided in an embodiment of the present application;

[0046] Figure 5 A fourth training process diagram of a human attribute and identity recognition model provided in an embodiment of the present application;

[0047] Figure 6A schematic diagram of a flow chart of a human attribute and identity recognition method provided in an embodiment of the present application;

[0048] Figure 7 A structural schematic diagram of a human attribute and identity recognition model training device provided in an embodiment of the present application;

[0049] Figure 8 A structural schematic diagram of a human attribute and identity recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] For easier understanding, please refer to Figure 1 and Figure 2 The first embodiment of a human attribute and identity recognition model training method provided by the present application includes:

[0052] Step 110: construct a human attribute and identity recognition network, which includes a backbone network and a classification layer.

[0053] The existing feature extraction network (such as MobileFaceNet) can be used as the backbone network, and the softmax layer, fully connected layer, etc. can be used as the classification layer. In order to improve the training speed and training effect of the human attribute and identity recognition network, the existing trained feature extraction model with the best network parameters can also be used as the backbone network. For example, the trained model obtained by training the existing convolutional neural network with the human head and shoulder dataset in the prior art can be used as the backbone network.

[0054] Step 120: extract features of the pedestrian training image through the backbone network, and classify the attributes of the first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result.

[0055] The acquired pedestrian training image is input into the backbone network for feature extraction, and the first pedestrian feature fs is output. The fs vector shape is [N, D], where N is the number of pedestrian training images and D is the pedestrian feature dimension; the first pedestrian feature extracted is attribute classified through the classification layer to obtain the first attribute classification result logit_s. The logit_s tensor shape is [N, C], where C is the number of categories of a certain attribute. Pedestrian training images can be obtained from existing public pedestrian datasets. The pedestrians in the pedestrian training images at least include the pedestrian's head and shoulders, and the pedestrian's attributes include gender, age, etc.

[0056] Step 130: Calculate an identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature, and calculate an attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result.

[0057] This application introduces the ArcFace Loss function to calculate the identity loss value for the first pedestrian feature output by the backbone network, and introduces the Focal Loss function to calculate the attribute loss value for the first attribute classification result output by the classification layer.

[0058] Step 140: Calculate the total loss value using the identity loss value and the attribute loss value.

[0059] The total loss value can be obtained by weighted summing the identity loss value and the attribute loss value.

[0060] Step 150: Update the network parameters of the backbone network and the classification layer through the total loss value until the human attribute and identity recognition network converges to obtain a trained human attribute and identity recognition model.

[0061] The network parameters of the backbone network and the classification layer are updated through the total loss value until the human attribute and identity recognition network converge (such as the number of training iterations reaches the maximum number of iterations, or the training error is lower than the preset error threshold, or the training accuracy is higher than the preset accuracy threshold, etc.), and a trained human attribute and identity recognition model is obtained. The trained human attribute and identity recognition model can output pedestrian features for identifying human identity IDs (the pedestrian features of the pedestrian to be identified can be similar to the pedestrian feature vectors stored in the preset pedestrian database, and the identity ID corresponding to the pedestrian feature vector with the highest similarity is the identity ID of the pedestrian to be identified), and can also classify human attributes, which can be used for the deduplication task of passenger flow statistics, which helps save computing resources.

[0062] The human attribute and identity recognition model training method provided in the present application simultaneously performs identity recognition task training and attribute classification task training on the human attribute and identity recognition network, so that the trained human attribute and identity recognition model can perform identity recognition and attribute classification. The method only needs to train one model, which reduces the number of models to be trained and saves computing resource consumption. In actual applications, only one model needs to be embedded in the hardware device, which reduces the requirements for the hardware device.

[0063] Please refer to Figure 3 , the present application provides a second embodiment of a method for training a human attribute and identity recognition model, comprising:

[0064] Step 210: construct a human attribute and identity recognition network, which includes a backbone network and a classification layer.

[0065] The existing feature extraction network (such as MobileFaceNet) can be used as the backbone network, and the softmax layer, fully connected layer, etc. can be used as the classification layer. In order to improve the training speed and training effect of the human attribute and identity recognition network, the existing trained feature extraction model with the optimal network parameters can also be used as the backbone network.

[0066] Step 220: extract features of the pedestrian training image through the backbone network, and classify the attributes of the first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result.

[0067] The acquired pedestrian training image is input into the backbone network for feature extraction, and the first pedestrian feature fs is output. The fs vector shape is [N, D], where N is the number of pedestrian training images and D is the pedestrian feature dimension. The first pedestrian feature extracted is attribute classified through the classification layer to obtain the first attribute classification result logit_s. The logit_s tensor shape is [N, C], where C is the number of categories of a certain attribute. Among them, the pedestrians in the pedestrian training image at least include the head and shoulders of the pedestrians, and the attributes of the pedestrians include gender, age, etc.

[0068] Step 230: extract features of the pedestrian training image using the first teacher model to obtain second pedestrian features.

[0069] This application can use a human identity recognition model trained in the prior art as the first teacher model, such as a recognition model or a feature extraction model trained by human identity ID data (such as pedestrian ID data or face ID data) as the first teacher model. The pedestrian training image is input into the first teacher model for feature extraction, and the second pedestrian feature ft is output, and the ft vector shape is [N, D]. It should be noted that when the first teacher model is used for feature extraction, the first teacher model does not have a classification layer.

[0070] Step 240: Calculate a first identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature; and calculate a second identity loss value based on the first pedestrian feature and the second pedestrian feature.

[0071] The identity loss in the embodiment of the present application includes a first identity loss value and a second identity loss value. For the first pedestrian feature output by the backbone network and the identity ID label of the pedestrian training image, the ArcFace Loss function is introduced to calculate the first identity loss value; for the first pedestrian feature and the second pedestrian feature, the L2 Loss function is introduced to calculate the second identity loss value.

[0072] Step 250: Calculate the attribute loss value according to the attribute label of the pedestrian training image and the first attribute classification result.

[0073] This application introduces a Focal Loss function to calculate the attribute loss value based on the attribute labels and the first attribute classification results of the pedestrian training images.

[0074] Step 260: Calculate the total loss value using the identity loss value and the attribute loss value.

[0075] The first identity loss value, the second identity loss value and the attribute loss value are weightedly summed to obtain the total loss value.

[0076] Step 270: Update the network parameters of the backbone network and the classification layer through the total loss value until the human attribute and identity recognition network converges to obtain a trained human attribute and identity recognition model.

[0077] The network parameters of the backbone network and the classification layer are updated through the total loss value until the human attribute and identity recognition network converge (such as the number of training iterations reaches the maximum number of iterations, or the training error is lower than the preset error threshold, or the training accuracy is higher than the preset accuracy threshold, etc.), and a trained human attribute and identity recognition model is obtained. The trained human attribute and identity recognition model can output pedestrian features for identifying human identity IDs (the pedestrian features of the pedestrian to be identified can be similar to the pedestrian feature vectors stored in the preset pedestrian database, and the identity ID corresponding to the pedestrian feature vector with the highest similarity is the identity ID of the pedestrian to be identified), and can also classify human attributes, which can be used for the deduplication task of passenger flow statistics, which helps save computing resources.

[0078] It should be noted that during the training process, the network parameters of the first teacher model are not updated, and the human attribute and identity recognition network composed of the backbone network and the classification layer updates the network parameters.

[0079] The human attribute and identity recognition model training method provided in the present application simultaneously performs identity recognition task training and attribute classification task training on the human attribute and identity recognition network, so that the trained human attribute and identity recognition model can perform identity recognition and attribute classification. The method only needs to train one model, which reduces the number of models to be trained and saves computing resource consumption. In practical applications, only one model needs to be embedded in the hardware device, which reduces the requirements for the hardware device. When performing identity recognition task training, the present application introduces a teacher model to guide training to improve the identity recognition training effect of the human attribute and identity recognition model.

[0080] Please refer to Figure 4 The present application provides a third embodiment of a method for training a human attribute and identity recognition model, comprising:

[0081] Step 310: construct a human attribute and identity recognition network, which includes a backbone network and a classification layer.

[0082] The existing feature extraction network (such as MobileFaceNet) can be used as the backbone network, and the softmax layer, fully connected layer, etc. can be used as the classification layer. In order to improve the training speed and training effect of the human attribute and identity recognition network, the existing trained feature extraction model with the optimal network parameters can also be used as the backbone network.

[0083] Step 320: extract features of the pedestrian training image through the backbone network, and classify the attributes of the first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result.

[0084] The acquired pedestrian training image is input into the backbone network for feature extraction, and the first pedestrian feature fs is output. The fs vector shape is [N, D], where N is the number of pedestrian training images and D is the pedestrian feature dimension. The first pedestrian feature extracted is attribute classified through the classification layer to obtain the first attribute classification result logit_s. The logit_s tensor shape is [N, C], where C is the number of categories of a certain attribute. Among them, the pedestrians in the pedestrian training image at least include the head and shoulders of the pedestrians, and the attributes of the pedestrians include gender, age, etc.

[0085] Step 330: Perform attribute recognition on the pedestrian training image through the second teacher model to obtain a second attribute classification result.

[0086] This application can use a human attribute recognition model trained in existing calculations as the second teacher model, such as a large recognition model trained by human attribute data (including pedestrian images and attribute labels) as the second teacher model. Attribute labels include age, gender, etc. Taking age as an example, 0 represents less than 15 years old, 1 represents greater than 16 years old and less than 30 years old, 2 represents greater than 31 years old and less than 45 years old, 3 represents greater than 46 years old and less than 60 years old, and 4 represents greater than 60 years old, which is an integer 1-dimensional tensor. The pedestrian training image is input into the second teacher model for attribute recognition, and the second attribute classification result logit_t is output, and the shape of the logit_t tensor is [N, C].

[0087] Step 340: Calculate the identity loss value according to the identity ID label of the pedestrian training image and the first pedestrian feature.

[0088] This application introduces the ArcFace Loss function to calculate the first identity loss value based on the first pedestrian feature output by the backbone network and the identity ID label of the pedestrian training image.

[0089] Step 350: Calculate a first attribute loss value according to the attribute label of the pedestrian training image and the first attribute classification result; and calculate a second attribute loss value according to the first attribute classification result and the second attribute classification result.

[0090] The attribute loss value in the embodiment of the present application includes a first attribute loss value and a second attribute loss value. For the attribute label of the pedestrian training image and the first attribute classification result, the Focal Loss function is introduced to calculate the first attribute loss value; for the first attribute classification result and the second attribute classification result, the DKD Loss function is introduced to calculate the second attribute loss value.

[0091] Step 360: Perform a weighted summation on the first identity loss value, the second identity loss value, the first attribute loss value, and the second attribute loss value to obtain a total loss value.

[0092] The identity loss value, the first attribute loss value, and the second attribute loss value are weightedly summed to obtain the total loss value.

[0093] Step 370: Update the network parameters of the backbone network and the classification layer through the total loss value until the human attribute and identity recognition network converges to obtain a trained human attribute and identity recognition model.

[0094] The network parameters of the backbone network and the classification layer are updated through the total loss value until the human attribute and identity recognition network converge (such as the number of training iterations reaches the maximum number of iterations, or the training error is lower than the preset error threshold, or the training accuracy is higher than the preset accuracy threshold, etc.), and a trained human attribute and identity recognition model is obtained. The trained human attribute and identity recognition model can output pedestrian features for identifying human identity IDs (the pedestrian features of the pedestrian to be identified can be similar to the pedestrian feature vectors stored in the preset pedestrian database, and the identity ID corresponding to the pedestrian feature vector with the highest similarity is the identity ID of the pedestrian to be identified), and can also classify human attributes, which can be used for the deduplication task of passenger flow statistics, which helps save computing resources.

[0095] It should be noted that during the training process, the network parameters of the second teacher model are not updated, and the human attribute and identity recognition network composed of the backbone network and the classification layer updates the network parameters.

[0096] The human attribute and identity recognition model training method provided in the present application simultaneously performs identity recognition task training and attribute classification task training on the human attribute and identity recognition network, so that the trained human attribute and identity recognition model can perform identity recognition and attribute classification. The method only needs to train one model, which reduces the number of models to be trained and saves computing resource consumption. In practical applications, only one model needs to be embedded in the hardware device, which reduces the requirements for the hardware device. When performing attribute classification task training, the present application introduces a teacher model to guide training to improve the attribute classification training effect of the human attribute and identity recognition model.

[0097] Please refer to Figure 5 The present application provides a fourth embodiment of a method for training a human attribute and identity recognition model, comprising:

[0098] Step 410: construct a human attribute and identity recognition network, which includes a backbone network and a classification layer.

[0099] The existing feature extraction network (such as MobileFaceNet) can be used as the backbone network, and the softmax layer, fully connected layer, etc. can be used as the classification layer. In order to improve the training speed and training effect of the human attribute and identity recognition network, the existing trained feature extraction model with the optimal network parameters can also be used as the backbone network.

[0100] Step 420: extract features of the pedestrian training image through the backbone network, and classify the attributes of the first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result.

[0101] The acquired pedestrian training image is input into the backbone network for feature extraction, and the first pedestrian feature fs is output. The fs vector shape is [N, D], where N is the number of pedestrian training images and D is the pedestrian feature dimension. The first pedestrian feature extracted is attribute classified through the classification layer to obtain the first attribute classification result logit_s. The logit_s tensor shape is [N, C], where C is the number of categories of a certain attribute. Among them, the pedestrians in the pedestrian training image at least include the head and shoulders of the pedestrians, and the attributes of the pedestrians include gender, age, etc.

[0102] Step 430: extract features of the pedestrian training image through the first teacher model to obtain second pedestrian features, and perform attribute recognition on the pedestrian training image through the second teacher model to obtain a second attribute classification result.

[0103] This application can use a human identity recognition model trained in the prior art as the first teacher model, such as a recognition model or a feature extraction model trained by human identity ID data (such as pedestrian ID data or face ID data) as the first teacher model. The pedestrian training image is input into the first teacher model for feature extraction, and the second pedestrian feature ft is output, and the ft vector shape is [N, D]. It should be noted that when the first teacher model is used for feature extraction, the first teacher model does not have a classification layer.

[0104] This application can use a human attribute recognition model trained in the prior art as the second teacher model, such as a large recognition model trained by human attribute data (including pedestrian images and attribute labels) as the second teacher model. Attribute labels include age, gender, etc. Taking age as an example, 0 represents less than 15 years old, 1 represents greater than 16 years old and less than 30 years old, 2 represents greater than 31 years old and less than 45 years old, 3 represents greater than 46 years old and less than 60 years old, and 4 represents greater than 60 years old, which is an integer 1-dimensional tensor. The pedestrian training image is input into the second teacher model for attribute recognition, and the second attribute classification result logit_t is output, and the shape of the logit_t tensor is [N, C].

[0105] Step 440: Calculate a first identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature; and calculate a second identity loss value based on the first pedestrian feature and the second pedestrian feature.

[0106] The identity loss in the embodiment of the present application includes a first identity loss value and a second identity loss value. For the first pedestrian feature output by the backbone network and the identity ID label of the pedestrian training image, the ArcFace Loss function is introduced to calculate the first identity loss value; for the first pedestrian feature and the second pedestrian feature, the L2 Loss function is introduced to calculate the second identity loss value.

[0107] Step 450: Calculate a first attribute loss value according to the attribute label of the pedestrian training image and the first attribute classification result; and calculate a second attribute loss value according to the first attribute classification result and the second attribute classification result.

[0108] The attribute loss value in the embodiment of the present application includes a first attribute loss value and a second attribute loss value. For the attribute label of the pedestrian training image and the first attribute classification result, the Focal Loss function is introduced to calculate the first attribute loss value; for the first attribute classification result and the second attribute classification result, the DKD Loss function is introduced to calculate the second attribute loss value.

[0109] Step 460: Perform weighted summation on the first identity loss value, the second identity loss value, the first attribute loss value, and the second attribute loss value to obtain a total loss value.

[0110] The weighted sum of the first identity loss value, the second identity loss value, the first attribute loss value, and the second attribute loss value is obtained to obtain the total loss value Loss, that is:

[0111] Loss = a*Loss1 + b*Loss2+c*Loss3+d*Loss4

[0112] In the formula, a, b, c and d are the weight constants of each loss function respectively; Loss1 is ArcFace Loss; Loss2 is Focal Loss; Loss3 is L2 Loss, such as nn.MSE loss; Loss4 is DKD Loss.

[0113] Step 470: Update the network parameters of the backbone network and the classification layer through the total loss value until the human attribute and identity recognition network converges to obtain a trained human attribute and identity recognition model.

[0114] The network parameters of the backbone network and the classification layer are updated through the total loss value until the human attribute and identity recognition network converge (such as the number of training iterations reaches the maximum number of iterations, or the training error is lower than the preset error threshold, or the training accuracy is higher than the preset accuracy threshold, etc.), and a trained human attribute and identity recognition model is obtained. The trained human attribute and identity recognition model can output pedestrian features for identifying human identity IDs (the pedestrian features of the pedestrian to be identified can be similar to the pedestrian feature vectors stored in the preset pedestrian database, and the identity ID corresponding to the pedestrian feature vector with the highest similarity is the identity ID of the pedestrian to be identified), and can also classify human attributes, which can be used for the deduplication task of passenger flow statistics, which helps save computing resources.

[0115] It should be noted that during the training process, the network parameters of the first teacher model and the second teacher model are not updated, and the network parameters of the human attribute and identity recognition network composed of the backbone network and the classification layer are updated.

[0116] The human attribute and identity recognition model training method provided in the present application simultaneously performs identity recognition task training and attribute classification task training on the human attribute and identity recognition network, so that the trained human attribute and identity recognition model can perform identity recognition and attribute classification. The method only needs to train one model, which reduces the number of models to be trained and saves computing resource consumption. In practical applications, only one model needs to be embedded in the hardware device, which reduces the requirements for the hardware device. When performing attribute classification tasks and identity recognition task training, the present application introduces a teacher model to guide training to improve the identity recognition and attribute classification training effect of the human attribute and identity recognition model.

[0117] Please refer to Figure 6 , the present application also provides a method for human attribute and identity recognition, including:

[0118] Step 510: Input the image of the pedestrian to be identified into the human attribute and identity recognition model to perform feature extraction and attribute classification, and obtain pedestrian feature and human attribute classification results.

[0119] The collected pedestrian image to be identified is input into the human attribute and identity recognition model, pedestrian features are extracted through the backbone network, the extracted pedestrian features are attribute classified through the classification layer, and the pedestrian features and human attribute classification results are output. The human attribute and identity recognition model is trained using the human attribute and identity recognition model training method in any of the above method embodiments.

[0120] It should be noted that the pedestrian images to be identified and the pedestrian training images in the embodiments of the present application are derived from legally authorized sources or with the consent of the parties concerned.

[0121] Step 520: perform similarity calculation between the pedestrian features and the pedestrian feature vectors stored in the preset pedestrian database, and output the identity ID corresponding to the pedestrian feature vector with the highest similarity.

[0122] After the pedestrian features are extracted, the pedestrian features are similarly calculated with the pedestrian feature vectors stored in the preset pedestrian database, and the similarities exceeding the preset similarity threshold are screened out, and then the pedestrian feature vector corresponding to the highest similarity is selected. The preset pedestrian database stores the correspondence between the pedestrian feature vector and the identity ID. According to the correspondence, the identity ID corresponding to the pedestrian feature vector with the highest similarity is extracted as the identity ID of the pedestrian in the pedestrian image to be identified. After the pedestrian's identity ID and attribute category are obtained, passenger flow statistics, deduplication processing in passenger flow statistics, and customer attribute analysis can be performed according to the pedestrian's identity ID and attribute category.

[0123] The human attribute and identity recognition model in this application can not only output pedestrian features for identifying human identity ID, but also classify human attributes, which is helpful for the deduplication task of passenger flow statistics and saves computing resources.

[0124] Please refer to Figure 7 , the present application also provides a human attribute and identity recognition model training device, comprising:

[0125] A construction unit, used for constructing a human attribute and identity recognition network, wherein the human attribute and identity recognition network includes a backbone network and a classification layer;

[0126] A first feature extraction and attribute classification unit is used to extract features from a pedestrian training image through a backbone network, and to classify attributes of the first pedestrian features extracted by the backbone network through a classification layer to obtain a first attribute classification result;

[0127] A first calculation unit is used to calculate an identity loss value based on an identity ID label of a pedestrian training image and a first pedestrian feature, and calculate an attribute loss value based on an attribute label of the pedestrian training image and a first attribute classification result;

[0128] A second calculation unit, used for calculating a total loss value through the identity loss value and the attribute loss value;

[0129] The parameter updating unit is used to update the network parameters of the backbone network and the classification layer through the total loss value until the human attribute and identity recognition network converges to obtain a trained human attribute and identity recognition model.

[0130] As a further improvement, the device further comprises:

[0131] A second feature extraction unit is used to extract features from the pedestrian training image through the first teacher model to obtain a second pedestrian feature; a human identity recognition model trained by the first teacher model;

[0132] The second attribute classification unit is used to perform attribute recognition on the pedestrian training image through the second teacher model to obtain a second attribute classification result; the second teacher model is a trained human attribute recognition model.

[0133] Please refer to Figure 8 The present application also provides a human attribute and identity recognition device, which is characterized by comprising:

[0134] A feature extraction and attribute classification unit, used for inputting the pedestrian image to be identified into the human attribute and identity recognition model to perform feature extraction and attribute classification, and obtain pedestrian features and human attribute classification results; wherein the human attribute and identity recognition model is trained by the human attribute and identity recognition model training method in any of the aforementioned method embodiments;

[0135] The identity recognition unit is used to calculate the similarity between the pedestrian features and the pedestrian feature vectors stored in the preset pedestrian database, and output the identity ID corresponding to the pedestrian feature vector with the highest similarity. The preset pedestrian database stores the corresponding relationship between the pedestrian feature vectors and the identity ID.

[0136] The present application also provides an electronic device, the device comprising a processor and a memory;

[0137] The memory is used to store the program code and transmit the program code to the processor;

[0138] The processor is used to execute the human attribute and identity recognition model training method in any of the aforementioned method embodiments according to the instructions in the program code, or to execute the human attribute and identity recognition method in the aforementioned method embodiments.

[0139] The present application also provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, it implements the human body attribute and identity recognition model training method in any of the aforementioned method embodiments, or implements the human body attribute and identity recognition method in the aforementioned method embodiments.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0142] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0143] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0147] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for training a human attribute and identity recognition model, characterized in that: include: Constructing a human attribute and identity recognition network, wherein the human attribute and identity recognition network includes a backbone network and a classification layer; Extracting features of pedestrian training images through the backbone network, and classifying attributes of first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result; Calculating an identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature, and calculating an attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result; Calculate a total loss value by using the identity loss value and the attribute loss value; The network parameters of the backbone network and the classification layer are updated by the total loss value until the human attribute and identity recognition network converges to obtain a trained human attribute and identity recognition model.

2. The human attribute and identity recognition model training method according to claim 1, characterized in that: The method further comprises: Extracting features from the pedestrian training image using a first teacher model to obtain second pedestrian features; the first teacher model is a trained human identity recognition model; The calculating the identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature includes: Calculate a first identity loss value according to the identity ID label of the pedestrian training image and the first pedestrian feature; A second identity loss value is calculated using the first pedestrian feature and the second pedestrian feature.

3. The human attribute and identity recognition model training method according to claim 1, characterized in that: The method further comprises: The pedestrian training image is subjected to attribute recognition by a second teacher model to obtain a second attribute classification result; the second teacher model is a trained human attribute recognition model; The calculating the attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result includes: Calculating a first attribute loss value according to the attribute label of the pedestrian training image and the first attribute classification result; The second attribute loss value is calculated by using the first attribute classification result and the second attribute classification result.

4. The human attribute and identity recognition model training method according to claim 1, characterized in that: The calculating the total loss value by using the identity loss value and the attribute loss value comprises: The identity loss value and the attribute loss value are weightedly summed to obtain a total loss value.

5. The human attribute and identity recognition model training method according to claim 1, characterized in that: The backbone network is a trained feature extraction model.

6. A method for identifying human attributes and identity, characterized in that: include: Inputting the image of the pedestrian to be identified into the human attribute and identity recognition model for feature extraction and attribute classification, and obtaining pedestrian features and human attribute classification results; wherein the human attribute and identity recognition model is trained by the human attribute and identity recognition model training method according to any one of claims 1 to 5; The pedestrian features are similarly calculated with pedestrian feature vectors stored in a preset pedestrian database, and the identity ID corresponding to the pedestrian feature vector with the highest similarity is output. The preset pedestrian database stores the corresponding relationship between the pedestrian feature vectors and the identity ID.

7. A human attribute and identity recognition model training device, characterized in that: include: A construction unit, used to construct a human attribute and identity recognition network, wherein the human attribute and identity recognition network includes a backbone network and a classification layer; A first feature extraction and attribute classification unit, configured to extract features from a pedestrian training image through the backbone network, and to classify attributes of the first pedestrian features extracted by the backbone network through the classification layer to obtain a first attribute classification result; A first calculation unit is used to calculate an identity loss value based on the identity ID label of the pedestrian training image and the first pedestrian feature, and calculate an attribute loss value based on the attribute label of the pedestrian training image and the first attribute classification result; A second calculation unit, configured to calculate a total loss value by using the identity loss value and the attribute loss value; A parameter updating unit is used to update the network parameters of the backbone network and the classification layer through the total loss value until the human body attribute and identity recognition network converges to obtain a trained human body attribute and identity recognition model.

8. A human attribute and identity recognition device, characterized in that: include: A feature extraction and attribute classification unit, used for inputting the image of a pedestrian to be identified into a human attribute and identity recognition model to perform feature extraction and attribute classification, and obtain pedestrian features and human attribute classification results; wherein the human attribute and identity recognition model is trained using the human attribute and identity recognition model training method according to any one of claims 1 to 5; The identity recognition unit is used to calculate the similarity between the pedestrian features and the pedestrian feature vectors stored in the preset pedestrian database, and output the identity ID corresponding to the pedestrian feature vector with the highest similarity. The preset pedestrian database stores the corresponding relationship between the pedestrian feature vectors and the identity ID.

9. An electronic device, characterized in that: The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the human attribute and identity recognition model training method described in any one of claims 1-5 according to the instructions in the program code, or to execute the human attribute and identity recognition method described in claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and when the program code is executed by the processor, it implements the human attribute and identity recognition model training method described in any one of claims 1-5, or implements the human attribute and identity recognition method described in claim 6.

Images