Human attribute recognition model training and human attribute recognition method and related device

Abstract

The present invention provides a human attribute recognition model training and human attribute recognition method and related devices, the method comprising: obtaining a training data set, determining a sub-training data set matching the human attribute from the training data set for each human attribute, and training a first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute; obtaining a confidence label corresponding to each human attribute based on the training data set and each sub-recognition model; training a second residual network based on the confidence label corresponding to each human attribute and all human attribute annotation information to obtain a human attribute recognition model. The present invention is based on a training mode combining dual signal supervision of artificial labels and soft labels, which reduces the adverse effects caused by inconsistent convergence speeds during training of different human attributes and improves the recognition accuracy of the human attribute recognition model.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a human attribute recognition model training and human attribute recognition method and related devices. Background Art

[0002] With the rapid development of artificial intelligence technology and the deployment of massive high-definition surveillance equipment, video-based real-time human attribute recognition technology has good application prospects in many fields such as security monitoring, smart retail and targeted advertising.

[0003] The current human attribute recognition technology mainly obtains different human attribute information by uniformly analyzing the human attributes of the extracted human body, including gender, age group, top color, sleeve length, whether to play with mobile phones and other human attributes. As the demand continues to increase, the number of human attributes that need to be recognized is also increasing. As the number of human attributes increases, a single model cannot achieve good recognition effects for all human attributes, and the model complexity is getting higher and higher, and the requirements for storage space and computing cost are constantly increasing.

[0004] Therefore, how to provide a human attribute recognition model that guarantees good recognition accuracy for all human attributes at a lower computational cost is a problem that needs to be solved. Summary of the invention

[0005] One of the purposes of the present invention is to provide a human attribute recognition model training and human attribute recognition method and related devices, so as to obtain a human attribute recognition model that can ensure that all human attributes have good recognition accuracy at a lower computational cost.

[0006] In a first aspect, the present invention provides a method for training a human attribute recognition model, the method comprising:

[0007] Acquire a training data set; the training data set includes human image samples and human attribute annotation information corresponding to the human image samples;

[0008] For each human attribute, determining a sub-training data set matching the human attribute from the training data set, and training the first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each of the human attributes;

[0009] Based on the training data set and each of the sub-recognition models, obtaining a confidence label corresponding to each human attribute;

[0010] The second residual network is trained according to the confidence label corresponding to each human attribute and all the human attribute annotation information to obtain a human attribute recognition model.

[0011] In a second aspect, the present invention provides a method for identifying human attributes, the method comprising:

[0012] Obtain an image to be recognized;

[0013] Input the image to be identified into a pre-trained human attribute recognition model for identification, and obtain the confidence corresponding to each human attribute; wherein the human attribute recognition model is obtained by the human attribute recognition model training method described in the first aspect;

[0014] For a binary human attribute, if the confidence of the human attribute value of the binary human attribute is greater than a preset threshold, output the human attribute value and the confidence of the human attribute value;

[0015] For multiple categories of human attributes, the maximum confidence and the human attribute value corresponding to the maximum confidence are output.

[0016] In a third aspect, the present invention provides a training device for a human attribute recognition model, comprising:

[0017] An acquisition module is used to acquire a training data set; the training data set includes human image samples and human attribute annotation information corresponding to the human image samples;

[0018] A training module, for determining, for each human attribute, a sub-training data set matching the human attribute from the training data set, and training the first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute;

[0019] A determination module, configured to obtain a confidence label corresponding to each of the human attributes based on the training data set and each of the sub-recognition models;

[0020] The training module is also used to train the second residual network according to the confidence label corresponding to each of the human attributes and all the human attribute annotation information to obtain a human attribute recognition model.

[0021] In a fourth aspect, the present invention provides a human attribute recognition device, comprising:

[0022] An acquisition module, used for acquiring an image to be recognized;

[0023] A recognition module, used for inputting the image to be recognized into a pre-trained human attribute recognition model for recognition, and obtaining the confidence corresponding to each human attribute; wherein the human attribute recognition model is obtained according to the human attribute recognition model training method described in the first aspect;

[0024] An output module, for outputting, for a binary human attribute, a first predefined attribute value corresponding to the binary human attribute and the confidence if the confidence of the human attribute value of the binary human attribute is greater than a preset threshold; and a second predefined attribute value corresponding to the binary human attribute and the confidence if the confidence is less than the preset threshold;

[0025] The output module is also used to output the maximum confidence and the predefined attribute value corresponding to the maximum confidence for multiple categories of human attributes.

[0026] In a fifth aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method described in the first aspect or the second aspect.

[0027] In a sixth aspect, the present invention provides a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect or the second aspect.

[0028] The human attribute recognition model training and human attribute recognition method and related device provided by the present invention include: obtaining a training data set, then determining a sub-training data set matching the human attribute from the training data set for each human attribute, and training a first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute; then obtaining a confidence label corresponding to each human attribute based on the training data set and each sub-recognition model; finally, training a second residual network based on the confidence label corresponding to each human attribute and all the human attribute annotation information to obtain a human attribute recognition model. The embodiment of the present invention has two training processes. The model obtained in the first training process can obtain the confidence of the predefined human attribute, and then use the obtained confidence as a soft label. The training mode based on the combination of dual supervision signals of artificial labels and soft labels can effectively transfer the good recognition performance of the sub-recognition model to the human attribute recognition model of multiple human attribute recognition, reduce the adverse effects caused by the inconsistent convergence speed when training different human attributes, and improve the recognition effect of the human attribute recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A schematic diagram of a scene for human attribute recognition provided by an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of an application scenario of the human attribute recognition model training method provided in an embodiment of the present application;

[0032] Figure 3 A flowchart of a human attribute recognition model training method provided in an embodiment of the present application;

[0033] Figure 4 A schematic flowchart of step S302 provided in an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of the classification of the sub-recognition model provided by an embodiment of the present invention;

[0035] Figure 6 A schematic flow chart of step S304 provided in an embodiment of the present invention;

[0036] Figure 7 A schematic diagram of a human attribute recognition method provided in an embodiment of the present application;

[0037] Figure 8 A functional module diagram of a human attribute recognition model training device provided in an embodiment of the present application;

[0038] Fig. 9 A functional module diagram of a human attribute recognition device provided in an embodiment of the present application;

[0039] Fig.10 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0043] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0044] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0045] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0046] In order to facilitate the understanding of the present invention, some technical terms involved in the embodiments of the present invention are first explained.

[0047] 1. Human attributes are visual features of the human body that can be perceived by both computers and humans, such as gender, age, top texture, top color, sleeve type, bottom texture, bottom color, bottom length, and bottom type.

[0048] 2. Predefined attribute values ​​are values ​​assigned to each human attribute. For example, the human attribute value of human attribute-gender is male and female, the human attribute value of human attribute-age is infant, child, youth, middle-aged, elderly, etc., the human attribute value of human attribute-top color is red, yellow, blue, black, mixed colors, and the human attribute value of human attribute-whether to wear a hat is yes, no, etc.

[0049] 3. A human attribute recognition model is a model that can simultaneously recognize the human attribute values ​​of multiple human attributes.

[0050] 4. Sub-recognition model, which is a model that can determine the confidence of various human attributes.

[0051] With the rapid development of artificial intelligence technology and the deployment of massive high-definition monitoring equipment, real-time human attribute recognition technology based on video has good application prospects in many fields such as security monitoring, smart retail and targeted advertising. For example, Figure 1A schematic diagram of a scene of human attribute recognition provided by an embodiment of the present invention. Human attribute recognition refers to the process of detecting a human body based on a human image and then determining what human attributes the human body has. Human attributes are mostly semantic features related to an individual. Common human attributes include gender, age, color and style of clothing, and whether there are accessories such as backpacks and hats.

[0052] As the scene requirements continue to increase, the number of human attributes that need to be recognized is also increasing. A single model cannot achieve good recognition results for all human attributes. For this reason, a human multi-body attribute recognition model already exists to meet the growing demand for human attribute recognition.

[0053] In order to improve the accuracy of multi-body attribute recognition, the relevant technology provides a method that combines multiple single-body attribute recognition models with a single multi-body attribute recognition model, wherein each single-body attribute recognition model only recognizes the body attributes of a single area that has been divided, such as head, hand, etc. This solution firstly needs to rely on more models, and the storage space requirement increases with the increase of body attributes. Secondly, due to the large changes in uncontrolled human posture, the requirements for the body part division algorithm are high, and there is a risk of incorrect body part division. This solution uses a single model to analyze the body attributes of the whole body, avoiding the risk of incorrect body division and having lower space storage requirements.

[0054] In order to solve the above problems, an embodiment of the present invention proposes a multi-model fusion method, which can fuse multiple single human attribute recognition models with good recognition into a single model, thereby ensuring that all human attributes have good recognition accuracy at a low computational cost.

[0055] Firstly, an embodiment of the present invention provides a method for training a human attribute recognition model, which ensures that the trained neural network model can accurately recognize a variety of human attributes.

[0056] The following is an introduction to the implementation of the human attribute recognition model training method provided in the embodiment of the present application:

[0057] The human attribute recognition model training method provided in the embodiment of the present application can be applied to devices with model training functions, such as terminal devices, servers, etc. The terminal devices can specifically be smart phones, computers, personal digital assistants (PDAs), tablet computers, etc. The server can specifically be an application server or a Web server. In actual application deployment, the server can be an independent server or a cluster server.

[0058] In actual applications, the terminal device and the server can train the neural network model separately or interactively. When the two interactively train the neural network model, the terminal device can obtain the training data set from the server, and then use the training data set to train the model to obtain a human attribute recognition model, or the server can obtain the training data set from the terminal, and then use the training data set to train the model to obtain a human attribute recognition model.

[0059] It should be understood that after the terminal device or server executes the training method provided in the embodiment of the present application and obtains a human attribute recognition model through training, the human attribute recognition model can be sent to other terminal devices to run the above-mentioned human attribute recognition model on these terminal devices to realize the corresponding functions; the human attribute recognition model can also be sent to other servers to run the above-mentioned human attribute recognition model on other servers to realize the corresponding functions through these servers.

[0060] To facilitate understanding of the technical solution provided in the embodiment of the present application, the training method provided in the embodiment of the present application is introduced below by taking the server training a human attribute recognition model as an example and combining it with actual application scenarios.

[0061] See also Figure 2 , Figure 2 A schematic diagram of an application scenario of a human attribute recognition model training method provided in an embodiment of the present application. The scenario includes a terminal device 101 and a server 102 for model training, and the terminal device 101 and the server 102 are connected via a network. Among them, the terminal device 101 can provide the server with human image samples and human attribute annotation information corresponding to the human image samples.

[0062] After the server 102 obtains the human image samples and the human attribute annotation information corresponding to the human image samples from the terminal device 101 through the network, the human image samples and the human attribute annotation information corresponding to the human image samples are combined into a training data set. Next, the server can perform a two-stage model training process. The purpose of the first stage model training process is to obtain a sub-recognition model with a single human attribute recognition function corresponding to each human attribute. The purpose of the second stage model training process is to obtain a human attribute recognition model with multiple human attribute recognition functions. The final human attribute recognition model can be applied to various scenarios.

[0063] During the first stage model training process, the server 102 extracts sub-training data sets matching each human attribute from the training data set to perform model training until the model meets the convergence condition, thereby obtaining a plurality of sub-recognition models.

[0064] In the embodiment of the present application, the obtained sub-recognition model is used to determine the confidence label of each human attribute, which provides training information for the subsequent training of the human attribute recognition model, thereby making the model training effect better.

[0065] During the second stage model training process, the server 102 uses the training data set and the obtained confidence labels of the predefined human attributes to perform model training until the model meets the convergence condition, thereby obtaining a human attribute recognition model.

[0066] After the server 102 generates the human attribute recognition model, it can further send the human attribute recognition model to the terminal device 101 so that the human attribute recognition model can be run on the terminal device 101 and the corresponding functions can be realized by using the human attribute recognition model.

[0067] It should be noted that the above Figure 2 The application scenario shown is only an example. In actual applications, the neural network model training method provided in the embodiment of the present application can also be applied to other application scenarios. No limitation is made to the application scenario of the neural network model training method herein.

[0068] See also Figure 3 , Figure 3 A flowchart of a human attribute recognition model training method provided in an embodiment of the present application. For ease of description, the following embodiment is described with a server as the execution subject. It should be understood that the execution subject of the human attribute recognition model training method is not limited to the server, but can also be applied to devices with model training functions such as terminal devices. Figure 3 As shown, the human body attribute recognition model training method includes the following steps:

[0069] S301, obtaining a training data set; the training data set includes human image samples and human attribute annotation information corresponding to the human image samples;

[0070] S302, for each human attribute, determining a sub-training data set matching the human attribute from the training data set, and training the first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute;

[0071] S303, obtaining a confidence label corresponding to each human attribute based on the training data set and each sub-recognition model;

[0072] S304: training the second residual network according to the confidence label corresponding to each human attribute and all human attribute annotation information to obtain a human attribute recognition model.

[0073] According to the human attribute recognition model training method provided in the embodiment of the present application, a training data set is obtained, and then for each human attribute, a sub-training data set matching the human attribute is determined from the training data set, and based on the sub-training data set, a first residual network is trained to obtain a sub-recognition model corresponding to each human attribute; then based on the training data set and each sub-recognition model, a confidence label corresponding to each human attribute is obtained; finally, according to the confidence label corresponding to each human attribute and all human attribute annotation information, a second residual network is trained to obtain a human attribute recognition model. There are two training processes in the embodiment of the present invention. The model obtained in the first training process can obtain the confidence of the predefined human attribute, and then the obtained confidence is used as a soft label. The training mode based on the combination of dual signal supervision of artificial labels and soft labels can effectively transfer the good recognition performance of the sub-recognition model to the human attribute recognition model of multiple human attribute recognition, reduce the adverse effects caused by the inconsistent convergence speed when training different human attributes, and improve the recognition effect of the human attribute recognition model.

[0074] The above steps S301 to S304 are described in detail below.

[0075] In step S301, a training data set is obtained.

[0076] In the embodiment of the present application, the training data set includes human image samples and human attribute annotation information corresponding to the human image samples.

[0077] In the embodiment of the present application, the human image sample can come from a real-time captured image collected by a terminal device, or from a historical human image or historical captured image pre-stored by a server. In the embodiment of the present invention, a human image sample usually corresponds to a human body, so that when performing human attribute recognition, the pedestrian in the human image sample can be fine-grainedly recognized.

[0078] Therefore, in an optional implementation manner, the implementation manner of the above step S301 may be as follows:

[0079] a1, obtain multiple captured images;

[0080] a2, perform human body detection on each captured image and obtain a human body detection frame corresponding to each detected human body;

[0081] a3. Based on the obtained human body detection frame, each captured image is cropped, and the image corresponding to the human body detection frame is used as a human body image sample. A training data set is formed based on all human body image samples and the label information of all human attributes corresponding to each human body image sample.

[0082] In the embodiment of the present application, the human attribute annotation information of the human image sample is manually annotated. The human attribute annotation information of each human image sample may include the real attribute values ​​corresponding to one or more human attributes. The human attributes may include binary human attributes and multi-class human attributes. The binary human attributes have only two predefined attribute values, and the multi-class human attributes may have more than two predefined attribute values. For example, the human attributes in the embodiment of the present invention include but are not limited to gender, age group, top color, and whether to wear a hat, etc. Among them, gender and whether to wear a hat are binary human attributes, and age group and top color are multi-class human attributes. For example, for a human image sample, its corresponding human attribute information is male gender, black top color, wearing a hat, and youth.

[0083] It should be noted that in the training data set in the embodiment of the present application, it is not necessary for all human attributes of all human image samples to be labeled. It is allowed that some human attributes are missing, but the number of samples contained in each human attribute must be sufficient. For example, each human attribute contains no less than 3000 artificial labels. In other words, assuming that the ultimate goal of the embodiment of the present application is to enable the human attribute recognition model to recognize 10 human attributes, but during the training process, the human attribute annotation information corresponding to each human image sample can be less than 10 types of information.

[0084] In an optional implementation, after obtaining the training data set, the human image samples may be preprocessed, such as normalizing and scaling the human image samples to a specified size, the preset size may be customized, such as 384*128, and the training data set may be data enhanced, and the data enhancement method may be, but is not limited to, random cropping, random scaling, random horizontal flipping, random color jitter (not used for training color-based human attributes), random blurring, etc., so as to improve the efficiency of model training. Therefore, after the above step S301, the following steps may also be included:

[0085] b1, determining whether the size of each human image sample is consistent with the preset size;

[0086] b2, if they are inconsistent, adjust the size of the human image sample to a preset size;

[0087] b3, perform data enhancement on each human image sample; wherein the data enhancement includes any one of the following and their combination: cropping; scaling; flipping; blurring.

[0088] In step S302, for each human attribute, a sub-training data set matching the human attribute is determined from the training data set, and based on the sub-training data set, the first residual network is trained to obtain a sub-recognition model corresponding to each human attribute.

[0089] In an embodiment of the present application, a sub-training data set matching each human attribute represents that the human attribute annotation information of each human image sample in the sub-training data set contains information of the human attribute. For example, for the human attribute - gender, human image samples with gender human attributes can be extracted from the training data set to form a sub-training data set.

[0090] In the embodiment of the present application, since the obtained sub-recognition model is not used for productization, the size of the first residual network is not limited. For example, the first residual network can be but is not limited to Resnet101. The embodiment of the present invention improves the structure of the first residual network, that is, the output of the last layer of the first residual network - the fully connected layer is changed to 1, that is, only one confidence of a human attribute is output. Therefore, the input of the first residual network is the sub-training data set, and the output is the confidence of the human attribute.

[0091] In the embodiment of the present application, the sub-recognition model is used to predict the confidence corresponding to a certain human attribute. It should be pointed out that the confidence can represent the confidence of the self-recognition model in accurately identifying the human attribute, and can also represent the confidence of a predefined attribute value corresponding to the human attribute. For example, taking the human attribute-gender as an example, when the human image sample corresponding to the male gender is a positive sample, and the human image sample corresponding to the female gender is a negative sample, that is, the label of the male gender is represented by 1, and the label of the female gender is represented by 0, then the final confidence can also be used as the confidence of the male gender, and the confidence of the female gender can be obtained by subtracting the confidence of the male gender from the number 1.

[0092] During the training process, for multi-classification human attributes, it is equivalent to splitting them into multiple binary categories for training, so as to obtain a multi-classification human attribute sub-recognition model. Continuing with the human attribute-age as an example, when training the youth recognition sub-model, it can be divided into the binary cases of youth and non-youth for training, that is, the samples corresponding to youth are taken as positive examples, and the samples corresponding to other age attribute values ​​are taken as negative examples, so that the sub-recognition model corresponding to the age group can be obtained. The confidence obtained by the sub-recognition model is the confidence corresponding to the age group (it can also be youth). Similarly, assuming that it is divided into the binary cases of children and non-children for training, then the corresponding confidence of children can be obtained. Therefore, for multi-classification human attributes, multiple sub-recognition models can be trained, and each sub-recognition model can obtain the confidence of each predefined attribute value. If the confidence corresponding to a certain predefined attribute value is used to train the human attribute model, then the confidence of the predefined attribute value can be finally identified. For a certain image to be identified, the predefined attribute value with the largest confidence can be output as the identified attribute value.

[0093] For example, continuing with the example of human attribute-age, assuming that we can obtain the corresponding sub-recognition models and confidences for children, young people, the elderly and others respectively, the human attribute recognition model trained using the confidence corresponding to children and the manually labeled information in the training set, and the human attribute recognition model trained using the confidence corresponding to young people and the manually labeled information in the training set. For the same image to be recognized, the confidences obtained are the confidences corresponding to children and young people respectively, then the attribute value with the largest confidence and the human attribute can be output.

[0094] Therefore, in a possible implementation manner, the above step S302 can be performed as follows: Figure 4 As shown, Figure 4 A schematic flow chart of step S302 provided in an embodiment of the present invention:

[0095] S302-1, according to each target human attribute, extracting target human image samples having the target human attribute in the human attribute annotation information from the training data set to form a sub-training data set. The target human attribute is any one of all human attributes;

[0096] In the embodiment of the present application, since the human attribute annotation information includes the human attribute type and the human attribute value, all target human image samples corresponding to each human attribute can be determined according to the human attribute type. For example, if the human attribute information corresponding to a human image sample is male, black top color, wearing a hat, and young, then this human image sample can be a human image sample with four common human attributes: gender, top color, whether wearing a hat, and age group.

[0097] For each human attribute, a sub-training data set matching this human attribute can be extracted from the training data set. Taking human attribute-gender as an example, the human attribute annotation information corresponding to each human image sample in the training data set can be queried to see whether there is a "gender" label. As long as the human image samples containing the "gender" label are extracted as sub-training data sets matching human attribute-gender. It can also be understood that when constructing a sub-training data set, both positive samples and negative samples are required. For example, when constructing a sub-training data set corresponding to human attribute-gender, it contains both human image samples corresponding to male gender and human image samples corresponding to female gender.

[0098] S302-2, for each sub-training data set, calculate the loss value of the first loss function according to the predicted attribute value of the target human attribute of the target human image sample and the actual attribute value of the target human attribute in the human attribute annotation information.

[0099] S302-3, back-propagating the loss value of the first loss function to each layer of the first residual network to iteratively update the model parameters until the preset conditions are met, and using the trained first residual network as a sub-recognition model.

[0100] In the embodiment of the present application, the first loss function may satisfy the following relationship:

[0101]

[0102] Where n represents the total number of target human image samples; y i 'Represents the predicted attribute value of the target human attribute corresponding to the i-th target human image sample; y i Represents the true attribute value of the target human attribute corresponding to the i-th target human image sample;

[0103] For example, assuming that we continue to take the human body attribute - top color as an example, the human body attribute - top color corresponding to each target human body image sample is white. During the model training process, the predicted attribute value of the human body attribute - top color corresponding to each target human body image sample may be white or other colors. Therefore, the loss value of the first loss function can be calculated based on the difference between white and the predicted color.

[0104] During the model training process, you can also pre-set some training parameters to ensure the training effect. For example, you can set the batch size to 128, the learning rate to 0.001 as the initial value, the learning rate to decay in a step-by-step manner at a rate of 0.1, and use 50%, 75%, etc. of the total number of iterations as decay nodes. The total number of iterations is set to 85 epochs.

[0105] S302-4, traverse all human attributes to obtain a sub-recognition model corresponding to each human attribute.

[0106] The sub-recognition model obtained by training in the embodiment of the present invention can be seen in Figure 5 As shown, Figure 5 A classification diagram of a sub-recognition model provided by an embodiment of the present invention, from Figure 5 It can be seen that the embodiment of the present application can obtain a sub-model capable of identifying each predefined human attribute value corresponding to each human attribute, thereby improving the accuracy and efficiency of subsequently obtaining the confidence level of each predefined human attribute.

[0107] In step S303, based on the training data set and each sub-recognition model, a confidence label corresponding to each human attribute is obtained.

[0108] In the embodiment of the present application, the sub-recognition model corresponding to each human body attribute can be determined, and then the confidence level of each human body attribute can be determined based on the obtained sub-recognition model.

[0109] For example, for binary human attributes, the confidence of the binary human attributes can be determined by the above sub-recognition model. The confidence can also represent the confidence P of the predefined attribute value of the positive sample corresponding to the binary attribute. Then the confidence of the predefined attribute value corresponding to the negative sample is 1-P.

[0110] In the embodiment of the present application, after obtaining the confidence label corresponding to each human attribute, the confidence can be used as a soft label to train the human attribute recognition model. The effect of using confidence as a soft label for training in the present application is as follows:

[0111] 1. The soft-label-based form can fix the good performance of the sub-recognition model for each human attribute. Different from the usual distillation learning method, which requires the teacher model to perform online forward calculations to predict samples, it will take up a lot of computing resources, especially as the number of iterations increases. The embodiment of the present invention only needs one forward calculation.

[0112] 2. Based on the soft label signal supervised training mode, the good performance of the sub-recognition model can be transferred to the multi-human attribute model, that is, the human attribute recognition model obtained by subsequent training in this application.

[0113] 3. Based on the soft label signal supervision training mode, it can reduce the adverse effects caused by the mismatch of convergence speed of different human attribute training processes due to the imbalance in the number of samples of different human attributes, and effectively avoid the overfitting of human attribute training with sufficient artificial labels and the underfitting of human attribute training with insufficient artificial labels.

[0114] 4. A training mode based on the combination of dual signal supervision of artificial labels and soft labels. Due to the supervision of artificial signals during training, the adverse effects of large prediction errors in soft labels on model training can be reduced, thus improving the effect of model training.

[0115] In step S304, the second residual network is trained according to the confidence label corresponding to each human attribute and all human attribute annotation information to obtain a human attribute recognition model.

[0116] In an embodiment of the present application, since the obtained human attribute recognition model is used for productization, the size of the second residual network should be limited. For example, the first residual network can be but is not limited to Resnet50 or Resnet18. It can be seen that the network structure of the first residual network is larger than the network structure of the second residual network.

[0117] The embodiment of the present invention improves the structure of the second residual network, that is, the output of the last layer of the second residual network, the fully connected layer, is changed to the number of all human attributes that need to be recognized.

[0118] Therefore, in an optional implementation, the implementation of the above step S304 can refer to Figure 6 , Figure 6 A schematic flow chart of step S304 provided for an embodiment of the present invention:

[0119] S304-1, determining a first difference between a predicted attribute value of each human attribute corresponding to each human image sample and a real attribute value of each human attribute in the human attribute annotation information.

[0120] S304-2, determining a second difference between the predicted attribute value of each human attribute corresponding to each human image sample and the confidence label of each human attribute.

[0121] S304-3, calculating a loss value of a second loss function according to the first difference, the second difference, and respective weights of the first difference and the second difference;

[0122] S304-4, back-propagating the total loss value to each layer of the second residual network to iteratively update the model parameters until the preset conditions are met, and using the trained second residual network as a human attribute recognition model.

[0123] In this embodiment of the present invention, the second loss function may satisfy the following relationship:

[0124]

[0125] Where n represents the total number of target human image samples; m represents the number of types of human attributes; y i,j Represents the true attribute value of the i-th human attribute corresponding to the j-th human image sample; y' i,j Representing the predicted attribute value of the i-th human attribute corresponding to the j-th human image sample; z i,j represents the confidence label of the i-th human attribute corresponding to the j-th human image sample; when the attribute value of the i-th human attribute exists in the human attribute annotation information of the j-th human image sample, then w i,j =1, otherwise it is w i,j =0; λ represents the weight of the second difference.

[0126] It can be seen that the second loss function is composed of the cross entropy loss function (the formula on the left side of the plus sign) and the mean square error loss function (the formula on the right side of the plus sign), where the cross entropy loss function can be used to determine the above-mentioned first difference, and the mean square error loss function can be used to determine the above-mentioned second difference.

[0127] During the model training process, the model training parameters may be the same as the model training parameters set in step S302, and will not be described again here.

[0128] Based on the human attribute recognition model obtained through the above training, the embodiment of the present invention also provides a human attribute recognition method. Figure 7 , Figure 7 The schematic flow chart of the human attribute recognition method provided by the embodiment of the present invention is as follows. For the convenience of description, the following embodiment is described with the terminal device as the execution subject. It should be understood that the execution subject of the human attribute recognition model training method is not limited to the terminal device. Figure 7 As shown, the human attribute recognition method includes the following steps:

[0129] S401, obtaining an image to be recognized.

[0130] In this embodiment, the implementation method of obtaining the image to be identified may be similar to the implementation method of obtaining the human body image sample described above, and will not be described in detail here.

[0131] In an optional implementation manner, after the image to be identified is acquired, the image to be identified may also be processed using a data preprocessing method as described above.

[0132] S402: Input the image to be recognized into a pre-trained human attribute recognition model for recognition, and obtain the confidence level corresponding to each human attribute.

[0133] The human attribute recognition model is obtained according to the human attribute recognition model training method provided by the embodiment of the present invention, and will not be described in detail here;

[0134] S403, for the binary human attributes, if the confidence of the binary human attributes is greater than a preset threshold, outputting a first predefined attribute value and the confidence corresponding to the binary human attributes;

[0135] S404: If the confidence of the binary human attribute is less than a preset threshold, output a second predefined attribute value and the confidence corresponding to the binary human attribute.

[0136] In an embodiment of the present invention, since the confidence corresponding to the binary human attribute is the confidence of one of the predefined attribute values, taking gender as an example, during the training process, if male is a positive sample and female is a negative sample, the confidence obtained by the obtained sub-recognition model can also be used as the confidence of male gender. If male is a negative sample and female is a positive sample, the confidence obtained by the obtained sub-recognition model can also be used as the confidence of female gender.

[0137] In an optional implementation, the first predefined attribute value and the second predefined attribute value can be set based on the labels corresponding to the binary attributes in the human body annotation information, and the attribute value with the label 1 is set as the first predefined attribute value, and the attribute value with the label 0 is set as the second predefined attribute value.

[0138] For example, assuming that the confidence obtained by the sub-recognition model is also the confidence corresponding to the gender female, the first predefined attribute value is female, and the second predefined attribute value is male, then assuming that the confidence obtained in step S402 is 0.9 and the preset threshold is 0.5, the final output attribute value is female and the confidence is 0.9; if the confidence is 0.4, the final output attribute value is male and the confidence is 0.4.

[0139] S405: For the multi-classified human attributes, output the maximum confidence and the human attribute value corresponding to the maximum confidence.

[0140] According to the human attribute recognition method provided by the embodiment of the present invention, by assigning the image to be recognized to a pre-trained human attribute recognition model, the human attribute value corresponding to each human attribute and the confidence of the human attribute value can be obtained, and then for the binary human attributes, the human attribute value corresponding to the confidence greater than the preset threshold is output, and for the multi-class human attributes, the human attribute value corresponding to the maximum confidence is output, so that the user can intuitively grasp the accuracy of the recognition result.

[0141] It can be understood that, based on the same inventive concept, the embodiment of the present invention can also obtain a vehicle human attribute recognition model through a similar training method. During the training process, the human image samples are replaced with vehicle images, and the human attributes and each predefined human attribute value are replaced with the human attributes and predefined human attribute values ​​corresponding to the vehicle. In this way, a vehicle human attribute recognition model can be obtained, and vehicle human attribute recognition can be performed in actual scenarios.

[0142] The human attribute recognition model training method provided in the embodiment of the present application can be executed in a hardware device or in the form of a software module. When the human attribute recognition model training method is implemented in the form of a software module, the embodiment of the present application also provides a human attribute recognition model training method device, see Figure 8 , Figure 8This is a functional module diagram of a human attribute recognition model training device provided in an embodiment of the present application. The human attribute recognition model training device 500 may include:

[0143] The first acquisition module 510 is used to acquire a training data set; the training data set includes human image samples and human attribute annotation information corresponding to the human image samples;

[0144] A training module 520 is used to determine, for each human attribute, a sub-training data set matching the human attribute from the training data set, and train the first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute;

[0145] A determination module 530, configured to obtain a confidence label corresponding to each human attribute based on the training data set and each sub-recognition model;

[0146] The training module 520 is further used to train the second residual network according to the confidence label corresponding to each human attribute and all human attribute annotation information to obtain a human attribute recognition model.

[0147] It is understandable that the first acquisition module 510, the training module 520 and the determination module 530 can be executed in coordination. Figure 3 Each step in the process is performed to achieve the corresponding technical effect.

[0148] In an optional embodiment, the training module 520 can also be used to perform Figure 4 and Figure 6 Each step in the process is performed to achieve the corresponding technical effect.

[0149] In an optional implementation, the first acquisition module 510 is specifically configured to execute the above steps a1 to a3 to achieve corresponding technical effects.

[0150] In an optional implementation, the human attribute recognition model training device 500 may further include a preprocessing module for executing the above steps b1 to b3 to achieve corresponding technical effects.

[0151] The human attribute recognition method provided in the embodiment of the present application can be executed in a hardware device or in the form of a software module. When the human attribute recognition method is implemented in the form of a software module, the embodiment of the present application also provides a human attribute recognition model training method device, see Fig. 9 , Fig. 9 This is a functional module diagram of a human attribute recognition model training device provided in an embodiment of the present application. The human attribute recognition device 600 may include:

[0152] The second acquisition module 610 is used to acquire the image to be recognized;

[0153] The recognition module 620 is used to input the image to be recognized into the pre-trained human attribute recognition model for recognition, and obtain the confidence corresponding to each human attribute; wherein the human attribute recognition model is obtained according to the human attribute recognition model training method provided in an embodiment of the present invention.

[0154] The output module 630 is used to output the first predefined attribute value and confidence corresponding to the binary human attribute if the confidence of the human attribute value of the binary human attribute is greater than a preset threshold; if the confidence is less than the preset threshold, the second predefined attribute value and confidence corresponding to the binary human attribute is output.

[0155] The output module 630 is further used to output the maximum confidence and the human attribute value corresponding to the maximum confidence for multiple categories of human attributes.

[0156] It is understandable that the second acquisition module 610, the identification module 620 and the output module 630 can be executed in coordination. Figure 7 Each step in the process is performed to achieve the corresponding technical effect.

[0157] The present invention also provides an electronic device, see Fig.10 , Fig.10 This is a structural block diagram of an electronic device provided by an embodiment of the present invention.

[0158] like Fig.10 As shown, the electronic device 700 includes a memory 701, a processor 702 and a communication interface 703, and the memory 701, the processor 702 and the communication interface 703 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0159] The memory 701 can be used to store software programs and modules, such as the instructions / modules of the human attribute recognition model training device 500 or the human attribute recognition device 600 provided in the embodiment of the present invention, which can be stored in the memory 701 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 700. The processor 702 executes the software programs and modules stored in the memory 701 to perform various functional applications and data processing. The communication interface 703 can be used to communicate signaling or data with other node devices.

[0160] Among them, the memory 701 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0161] The processor 702 may be an integrated circuit chip with signal processing capability. The processor 702 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0162] Understandably, Fig.10 The structure shown is for illustration only. The electronic device 700 may also include Fig.10 More or fewer components as shown, or with Fig.10 Different configurations are shown. Fig.10 The components shown may be implemented in hardware, software or a combination thereof.

[0163] The present application also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the human attribute recognition model training method or human attribute recognition method in any of the above embodiments is implemented. The computer readable storage medium can be, but is not limited to, a U disk, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic disk, or an optical disk, and other media that can store program codes.

[0164] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A human attribute recognition model training method, characterized in that: The method comprises: Acquire a training data set; the training data set includes human image samples and human attribute annotation information corresponding to the human image samples; For each human attribute, determining a sub-training data set matching the human attribute from the training data set, and training the first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each of the human attributes; Based on the training data set and each of the sub-recognition models, obtaining a confidence label corresponding to each of the human attributes; According to the confidence labels corresponding to each of the human attributes and all the human attribute annotation information, the second residual network is trained to obtain a human attribute recognition model; According to the confidence labels corresponding to each of the human attributes and all the human attribute annotation information, the second residual network is trained to obtain a human attribute recognition model, including: Determine a first difference between a predicted attribute value of each human attribute corresponding to each human image sample and a real attribute value corresponding to each human attribute in the human attribute annotation information; Determine a second difference between the predicted attribute value of each human attribute corresponding to each of the human image samples and the confidence label corresponding to the each human attribute; Calculating a loss value of a second loss function according to the first difference, the second difference, and respective weights of the first difference and the second difference; The loss value of the second loss function is back-propagated to each layer of the second residual network to iteratively update the model parameters until the preset conditions are met, and the trained second residual network is used as the human attribute recognition model.

2. The method according to claim 1, characterized in that For each human attribute, a sub-training data set matching the human attribute is determined from the training data set, and a first residual network is trained based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute, including: According to the target human attributes, target human image samples having the target human attributes in the human attribute annotation information are extracted from the training data set to form the sub-training data set; wherein the target human attribute is any one of all the human attributes; For each of the sub-training data sets, calculating a loss value of a first loss function according to a predicted attribute value of the target human attribute of the target human image sample and a true attribute value of the target human attribute in the human attribute annotation information; Back-propagating the loss value of the first loss function to each layer of the first residual network to iteratively update the model parameters until a preset condition is met, and using the trained first residual network as the sub-recognition model corresponding to the target human attribute; All the human attributes are traversed to obtain a sub-recognition model corresponding to each human attribute.

3. The method according to claim 1, characterized in that After obtaining the training data set, the method further includes: Determining whether the size of each of the human body image samples is consistent with a preset size; If they are inconsistent, adjusting the size of the human body image sample to the preset size; Data enhancement is performed on each of the human body image samples; wherein the data enhancement includes any one of the following and their combination: cropping; scaling; flipping; blurring.

4. The method according to claim 2, characterized in that: The first loss function satisfies the following relationship: Wherein, n represents the total number of target human image samples; y i 'Represents the predicted attribute value of the target human attribute corresponding to the i-th target human image sample; y i Represents the true attribute value of the target human attribute corresponding to the i-th target human image sample; wherein, 5. The method according to claim 1, characterized in that The second loss function satisfies the following relationship: Wherein, n represents the total number of human image samples; m represents the number of human attributes; y i,j ,y' i,j and z i,j Respectively represent the true attribute value, predicted attribute value and confidence label of the i-th human attribute corresponding to the j-th human image sample; when the attribute value of the i-th human attribute exists in the human attribute annotation information of the j-th human image sample, then w i,j =1, otherwise it is w i,j =0; λ represents the weight of the second difference; 6. The method according to claim 1, characterized in that Get the training dataset, including: Get multiple captured images; Perform human body detection on each captured image to obtain a human body detection frame corresponding to each detected human body; Based on the obtained human body detection frame, each of the captured images is cropped, and the image corresponding to the human body detection frame is used as the human body image sample, and the training data set is composed based on all the human body image samples and the label information of all human body attributes corresponding to each human body image sample.

7. A method for identifying human attributes, characterized in that: The method comprises: Obtain an image to be recognized; Inputting the image to be identified into a pre-trained human attribute recognition model for identification, and obtaining the confidence corresponding to each human attribute; wherein the human attribute recognition model is obtained according to the human attribute recognition model training method according to any one of claims 1 to 6; For a binary human attribute, if the confidence of the binary human attribute is greater than a preset threshold, then output a first predefined attribute value corresponding to the binary human attribute and the confidence; if the confidence is less than the preset threshold, then output a second predefined attribute value corresponding to the binary human attribute and the confidence; For multiple categories of human attributes, the maximum confidence and the predefined attribute value corresponding to the maximum confidence are output.

8. A human attribute recognition model training device, characterized in that: include: An acquisition module is used to obtain a training data set; The training data set includes human image samples and human attribute annotation information corresponding to the human image samples; A training module, for determining, for each human attribute, a sub-training data set matching the human attribute from the training data set, and training the first residual network based on the sub-training data set to obtain a sub-recognition model corresponding to each human attribute; A determination module, configured to obtain a confidence label corresponding to each of the human attributes based on the training data set and each of the sub-recognition models; The training module is further used to train the second residual network according to the confidence label corresponding to each of the human attributes and all the human attribute annotation information to obtain a human attribute recognition model; The training module is specifically used to: determine a first difference between the predicted attribute value of each human attribute corresponding to each human image sample and the real attribute value corresponding to each human attribute in the human attribute annotation information; Determine a second difference between the predicted attribute value of each human attribute corresponding to each of the human image samples and the confidence label corresponding to the each human attribute; Calculating a loss value of a second loss function according to the first difference, the second difference, and respective weights of the first difference and the second difference; The loss value of the second loss function is back-propagated to each layer of the second residual network to iteratively update the model parameters until the preset conditions are met, and the trained second residual network is used as the human attribute recognition model.

9. A human attribute recognition device, characterized in that: include: An acquisition module, used for acquiring an image to be recognized; A recognition module, used for inputting the image to be recognized into a pre-trained human attribute recognition model for recognition, and obtaining the confidence corresponding to each human attribute; wherein the human attribute recognition model is obtained according to the human attribute recognition model training method according to any one of claims 1 to 6; An output module, for outputting, for a binary human attribute, a first predefined attribute value and the confidence of the binary human attribute if the confidence of the binary human attribute is greater than a preset threshold; and a second predefined attribute value and the confidence corresponding to the binary human attribute if the confidence is less than the preset threshold; The output module is also used to output the maximum confidence and the predefined attribute value corresponding to the maximum confidence for multiple categories of human attributes.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method according to any one of claims 1 to 6 or the method according to claim 7.

11. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images