Facial attribute recognition method, device and computer-readable storage medium

Abstract

The embodiment of the present application provides a facial attribute recognition method, device and computer-readable storage medium, the method comprising: obtaining a facial image to be recognized; inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network comprises a global feature extraction model and at least two parallel attribute classification models; obtaining the output of the facial attribute recognition network, wherein the output comprises at least two facial attributes corresponding to the at least two parallel attribute classification models. It can be seen that in the present application, multiple facial attributes can be obtained at the same time through the pre-trained facial attribute recognition network, avoiding the use of multiple different networks to obtain multiple facial attributes separately, simplifying the operation, being able to obtain multiple facial attributes more quickly, and improving processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a facial attribute recognition method, device and computer-readable storage medium. Background Art

[0002] When identifying different attributes of an object contained in an image, such as the object's gender, whether it wears sunglasses, whether it wears a mask, whether its hair color is red, etc., different attribute recognition models are generally used for recognition, and each attribute recognition model can only recognize one attribute of the object.

[0003] When training an attribute recognition model for identifying one attribute, it is necessary to train the attribute recognition model for identifying each attribute, that is, the parameters of each layer in the model need to be adjusted, the amount of calculation is relatively large, and the efficiency of adjusting the model is relatively low. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a facial attribute recognition method, device and computer-readable storage medium to solve the problem of large amount of calculation caused by retraining the model when updating some attributes of the model or adding new attributes in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a facial attribute recognition method, the method comprising:

[0006] Obtaining a facial image to be recognized;

[0007] Inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models;

[0008] An output of the facial attribute recognition network is obtained, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models.

[0009] In one embodiment, the step of inputting the facial image into a pre-trained facial attribute recognition network comprises:

[0010] Inputting the facial image into the global feature extraction model to obtain global facial features;

[0011] The global facial features are respectively input into the at least two parallel attribute classification models to obtain corresponding at least two facial attributes.

[0012] In one embodiment, the training process of the facial attribute recognition network includes:

[0013] Simultaneously obtaining the global feature extraction model and the first attribute classification model of the at least two parallel attribute classification models through training;

[0014] On the basis of the trained global feature extraction model, other attribute classification models in the at least two parallel attribute classification models are obtained through training.

[0015] In one embodiment, the training process of the global feature extraction model includes:

[0016] Constructing a first attribute recognition model, wherein the first attribute recognition model includes a global feature extraction model and a first attribute classification model;

[0017] The first attribute recognition model acquires facial images in a training set, and the first attribute classification model infers predicted facial attributes of the facial images in the training set;

[0018] According to the actual facial attributes of the facial images in the training set and the predicted facial attributes obtained by inference, the model parameters of the first attribute recognition model are adjusted to obtain a trained first attribute recognition model;

[0019] The first attribute classification model is removed from the trained first attribute recognition model to obtain the global feature extraction model.

[0020] In one embodiment, the training process of other attribute classification models in the at least two parallel attribute classification models is performed in parallel.

[0021] In one embodiment, the first attribute classification model is any one of the following: age model, gender model, mask model, glasses model, hair color model, eye model, beard model.

[0022] In a second aspect, an embodiment of the present application provides a facial recognition device, comprising:

[0023] A facial image acquisition module, used to acquire a facial image to be recognized;

[0024] A facial image processing module, used for inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models;

[0025] A facial attribute recognition module is used to obtain the output of the facial attribute recognition network, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models.

[0026] In one embodiment, the facial image processing module is specifically used to:

[0027] Inputting the facial image into the global feature extraction model to obtain global facial features;

[0028] The global facial features are respectively input into the at least two parallel attribute classification models to obtain corresponding at least two facial attributes.

[0029] In a third aspect, an embodiment of the present application provides a facial recognition device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the facial recognition device is running, the processor executes the machine-readable instructions to perform the steps of the method of the first aspect described above.

[0030] In one embodiment, it also includes an imaging element, the imaging element is coupled to the processor, the imaging element is configured to obtain a facial image captured by an image acquisition device, and the processor is configured to execute the machine-readable instructions to perform the steps of any method described in the first aspect when executed;

[0031] The communication device is coupled to the target device, and is configured to send the facial attribute recognition network for recognizing facial attributes under a plurality of attribute categories to the target device when the facial attribute recognition network appears.

[0032] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a computer or a processor, the steps of the method described in the first aspect or any embodiment are implemented.

[0033] The facial attribute recognition method provided by the embodiment of the present application obtains a facial image to be recognized; inputs the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models; obtains the output of the facial attribute recognition network, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models, so that when multiple attributes are recognized by the facial attribute recognition network, the global feature extraction model is shared, thereby reducing the number of network parameters and the amount of calculation, and improving the recognition efficiency of the facial attribute recognition network.

[0034] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A first flow chart of a facial attribute recognition method provided by an embodiment of the present application is shown;

[0037] Figure 2 A schematic diagram of the structure of a first attribute classification model provided in an embodiment of the present application is shown;

[0038] Figure 3 A schematic diagram of a process for determining a global feature extraction model provided in an embodiment of the present application is shown;

[0039] Figure 4 A schematic diagram showing a facial attribute recognition method provided by an embodiment of the present application is shown;

[0040] Figure 5 A schematic structural diagram of a facial recognition device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0042] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0043] When identifying object attributes, different attributes are identified through different neural network models. As the number of identified attributes increases, the number of models gradually increases with the types of attributes. If you want to identify multiple attributes, you need to train multiple models, and the amount of data calculation for training the model is relatively large.

[0044] In the related art, when multiple attribute recognition models are used to perform multiple attribute recognitions respectively, considering that the graphics processing unit (GPU) of the device running the attribute recognition model is limited, each attribute recognition model is generally run in sequence in the device, which results in a relatively long time for attribute recognition.

[0045] When multiple attributes are identified through one model, the facial feature extraction operation is shared, reducing the number of model parameters and computation. However, the existing training method often trains all modules of the entire model together. In actual application and later maintenance and update, the adjustment of one attribute will affect all other attributes, that is, the parameters of each layer of each attribute branch and feature extraction module in the entire model need to be adjusted. This increases the computational cost, time cost and difficulty of model update, affecting the application and maintenance of this technology in real-world scenarios.

[0046] In the embodiments of the present application, the above-mentioned problems existing in the prior art are taken into consideration. The facial attribute recognition network of the present application obtains an input facial image; the facial attribute recognition network includes a feature extraction model and multiple parallel first attribute classification models; the feature extraction model extracts global facial features from the input facial image; each first attribute classification model outputs a facial attribute under a corresponding attribute category based on inference of the global facial features. In this way, when multiple attributes are recognized by the facial attribute recognition network, the feature extraction model is shared, which reduces the number of parameters and the amount of calculation of the model, thereby improving the recognition efficiency of the facial attribute recognition network.

[0047] In view of the above situation, the present application embodiment first provides a facial attribute recognition method, such as Figure 1 As shown, the method comprises the following steps:

[0048] S101, obtaining a facial image to be recognized;

[0049] S102, inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models;

[0050] S103, obtaining an output of the facial attribute recognition network, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models.

[0051] It can be seen that in the present application, multiple facial attributes can be obtained simultaneously through a pre-trained facial attribute recognition network, avoiding the use of multiple different networks to obtain multiple facial attributes separately, simplifying the operation, being able to obtain multiple facial attributes more quickly, and improving processing efficiency.

[0052] In S101, the facial image may be an image captured by an image acquisition device targeting a target area. The image acquisition device may be a camera, a camera, etc. The image acquisition device may be a device set in an access control system, such as an image acquisition device set at the entrance of a residential area, or an image acquisition device set when an employee enters a workplace; the image acquisition device may also be a device set indoors, such as a classroom, a room in a nursing home, an entrance to a shopping mall, a place where a public security department implements monitoring, etc.; the image acquisition device may also be a device set outdoors, such as a device set at the entrance of an outdoor concert, or a device set on a city road or traffic intersection, which may be determined according to the actual application scenario.

[0053] In S102, a facial image including global facial features of the object is acquired through an image acquisition device, and the acquired facial image is input into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models.

[0054] In an embodiment of the present invention, each facial image corresponds to one or more attribute categories. Among them, the attribute categories include gender category, glasses category, mask category, hair color category, eye category, beard category, etc. The actual attribute corresponding to each attribute category is the actual facial attribute. For example, when the attribute category is the gender category, the actual facial attribute is male or female. For another example, when the attribute category is the glasses category, the actual facial attribute is wearing glasses or not wearing glasses; for another example, when the attribute category is the mask category, the actual facial attribute includes wearing a mask or not wearing a mask; for another example, when the attribute category is the eye category, the actual facial attribute is big eyes or small eyes; for another example, when the attribute category is the beard category, the actual facial attribute is having a beard or not having a beard; for another example, when the attribute category is the hair color category, the actual facial attribute is red, yellow, flax or black. It should be noted here that when the attribute category is the hair color category, the number of actual facial attributes under the hair color category used in model training can be determined according to the actual scenario.

[0055] The number of attribute classification models is the same as the number of attribute categories. Different attribute classification models are used to identify facial attributes under different attribute categories. The facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models. As an optional embodiment, the facial image is input into a pre-trained facial attribute recognition network, including:

[0056] Inputting the facial image into the global feature extraction model to obtain global facial features;

[0057] The global facial features are respectively input into the at least two parallel attribute classification models to obtain corresponding at least two facial attributes.

[0058] In the embodiment of the present invention, the global feature extraction model and the attribute classification model can be a CNN model, a residual network, or a GAN trained based on sample data, etc. The model structure of the attribute classification model includes a convolutional neural network layer, a batch normalization processing layer, an activation function, a convolutional neural network model, a batch normalization processing layer, an activation function, an average pooling layer (batch normalization processing layer), and a fully connected layer. The layer connection method inside the attribute classification model can refer to Figure 2 .

[0059] The purpose of the present application is to obtain a facial attribute recognition network, which can recognize facial attributes of an object (such as a person) in a facial image under multiple attribute categories. During the training process of the entire facial attribute recognition network, if an untrained global feature extraction model is used, on the one hand, the recognition accuracy of the facial attribute recognition network finally obtained will be reduced. On the other hand, during the training process, if the model parameters of the global feature extraction model are adjusted each time, the amount of calculation in the training process will increase. Therefore, the training process of the facial attribute recognition network includes:

[0060] The global feature extraction model and the first attribute classification model among the at least two parallel attribute classification models are obtained simultaneously through training; based on the trained global feature extraction model, other attribute classification models among the at least two parallel attribute classification models are obtained through training.

[0061] In one implementation, a first attribute recognition model may be constructed, wherein the first attribute recognition model includes a global feature extraction model and a first attribute classification model. Subsequently, a trained first attribute recognition model may be obtained through training based on the training data set. In other words, a global feature extraction model and a first attribute classification model are obtained at the same time.

[0062] Specifically, the first attribute recognition model obtains the facial images in the training set, and the first attribute classification model infers the predicted facial attributes of the facial images in the training set; according to the actual facial attributes of the facial images in the training set and the inferred predicted facial attributes, the model parameters of the first attribute recognition model are adjusted to obtain the first attribute recognition model that has been trained. In the iterative process, the model parameters of the global feature extraction model and the model parameters of the first attribute classification model are adjusted simultaneously.

[0063] The facial images in the training set may be labeled with a plurality of different actual facial attributes. The convergence status may be determined by constructing a loss function, which may be constructed based on the difference between the actual facial attributes and the predicted facial attributes.

[0064] After the first attribute recognition model is trained, the global feature extraction model and the first attribute classification model can be obtained by segmentation. For example, the remaining part of the first attribute recognition model after removing the first attribute classification model is used as the global feature extraction model. The global feature extraction model can be a shared CNN model.

[0065] Afterwards, the global feature extraction model may be further used to train other attribute classification models in at least two parallel attribute classification models.

[0066] Taking the second attribute classification model as an example, a second attribute recognition model can be constructed, which includes the global feature extraction model and the second attribute classification model that have been trained above. Subsequently, the trained second attribute recognition model can be obtained through training based on the training data set. It should be noted that when training the second attribute classification model, only the parameters in the second attribute classification model are adjusted and updated during the iteration process, and the parameters in the global feature extraction model are not adjusted. That is, in the process of training the second attribute recognition model, the parameters in the global feature extraction model are fixed. In this way, fewer parameters are adjusted during training, so that the training can converge faster and the training speed is fast.

[0067] It is understandable that after completing the training of the second attribute recognition model, the second attribute classification model can be obtained by removing the global feature extraction model in the second attribute recognition model. It is also understandable that through a similar process, other attribute classification models in at least two parallel attribute classification models can be obtained. In addition, since the training processes of other attribute classification models except the first attribute classification model in at least two parallel attribute classification models are independent of each other and do not depend on each other, the training process can be performed in parallel, which can further shorten the training time and improve efficiency.

[0068] It should be noted that different training data sets may be used when training different attribute classification models, as long as the training data sets used have been annotated with corresponding facial attributes.

[0069] Furthermore, by splicing the global feature extraction model with at least two parallel attribute classification models, the facial attribute recognition network in the present application can be obtained.

[0070] The facial attribute recognition network in the present application is scalable. Assuming that the output of the pre-trained facial attribute recognition network in S102 includes N facial attributes, it is also possible to continue to train another attribute classification model corresponding to one or more facial attributes based on the pre-trained facial attribute recognition network, so that the number of facial attributes included in the output of the facial attribute recognition network after retraining is greater than N. It can be seen that the present application facilitates network expansion and realizes simultaneous recognition of a greater number of facial attributes.

[0071] In another implementation, when constructing each attribute classification model, a trained global feature extraction model is used. The training process of the global feature extraction model is described in detail below.

[0072] refer to Figure 3 , determine the feature extraction model according to the following steps:

[0073] S301, constructing a first attribute recognition model, wherein the first attribute recognition model includes a global feature extraction model and at least two parallel second attribute classification models, wherein the second attribute classification model is used to recognize facial attributes in each attribute category of multiple attribute categories;

[0074] S302, the first attribute recognition model acquires the facial image, and infers the predicted facial attributes under each second attribute category according to the facial image;

[0075] S303, adjusting the model parameters of the first attribute recognition model according to the actual facial attributes of the facial image and the predicted facial attributes obtained by inference, to obtain a trained first attribute recognition model;

[0076] S304, removing at least two parallel second attribute classification models from the trained first attribute recognition model to obtain the global feature extraction model.

[0077] The function of the global feature extraction model is to extract feature information from facial images to obtain global facial features. If only the global feature extraction model is trained, it is necessary to manually mark the feature vectors corresponding to each facial image, but it is impossible to manually mark the feature maps corresponding to each facial image. The model cannot be trained alone. Therefore, by training the first attribute recognition model including the global feature extraction model, that is, the global feature extraction model is spliced ​​with the second attribute classification model. The second attribute classification model is used to identify the attributes of objects in facial images, so as to achieve the purpose of training the global feature extraction model.

[0078] Considering that the global feature extraction model is a shared network of models for recognizing multiple attributes, the second attribute classification model spliced ​​with the global feature extraction model is used to identify facial attributes in each of the multiple attribute categories, in order to improve the accuracy of the obtained global feature extraction model. The global feature extraction model and the second attribute classification model can both be CNN models, residual networks, or GANs trained based on sample data.

[0079] In a specific implementation process, after the first attribute recognition model is constructed, a sample facial image may be selected from the acquired facial images. The sample facial image may be an image including an object having attributes under multiple attribute categories.

[0080] A plurality of sample facial images are input into the first attribute recognition model to obtain a second predicted facial attribute of each sample facial image under each attribute category, and the model parameters of the global feature extraction model in the first attribute recognition model are adjusted according to the principle of the shortest distance between each second predicted facial attribute and the corresponding actual facial attribute, so as to finally obtain a trained first attribute recognition model.

[0081] Considering that when only the model parameters of the global feature extraction model are adjusted without adjusting the model parameters of the second attribute classification model, the gap between the second predicted facial attributes predicted by the first attribute recognition model and the actual facial attributes will be relatively large. When the gap is relatively large, the distance determined by the second predicted facial attributes and the actual facial attributes will also be relatively large. In this case, multiple trainings are required to obtain the trained first attribute recognition model, and the amount of calculation is relatively large. In order to reduce the amount of calculation, the model parameters of the global feature extraction model and the model parameters of the second attribute classification model can be adjusted at the same time, that is, according to the principle of the shortest distance between each second predicted facial attribute and the corresponding actual facial attribute, the model parameters of the global feature extraction model and the second attribute classification model in the first attribute recognition model are adjusted respectively, and finally the trained first attribute recognition model is obtained.

[0082] After obtaining the trained first attribute recognition model, it is necessary to remove at least two parallel second attribute classification models to obtain a network that can share a global feature extraction network. When removing the second attribute classification model, the following steps may be included:

[0083] Acquire a model file of a first attribute recognition model that has completed training; the model file includes model structure description information and model parameter information;

[0084] From the model file of the trained first attribute recognition model, delete the model structure description information and model parameter information of at least two parallel second attribute classification models.

[0085] Here, the model file of the first attribute recognition model that has completed training includes model structure description information and model parameter information, wherein the model structure description information includes the model structure, execution order, input and output formats between layers, etc., and the model parameter information includes the model parameters included in each layer in the model, which is not exemplified here.

[0086] In the specific implementation process, after obtaining the model file of the first attribute recognition model that has completed training, the model structure description information and model parameter information of the second attribute classification model are deleted from the model file. For example, the model structure description information and model parameter information of the second attribute classification model are manually deleted. In this way, the model information of the global feature extraction model is finally obtained. The model information is spliced ​​with the model information of multiple attribute classification models (each attribute classification model is used to identify attributes under an attribute category), which is described in detail below.

[0087] Multiple iterative trainings are performed for each first attribute classification model. In a specific implementation process, one attribute category is arbitrarily selected from multiple attribute categories, and facial images with actual facial attributes under this attribute category are selected from a large number of facial images. Taking a first attribute classification model in a facial attribute recognition network as an example, for each iterative training, the model parameters of the feature extraction model in the facial attribute recognition network are controlled to remain unchanged, the extracted global facial features are input into the first attribute classification model, and the first predicted facial attributes of the global facial features under this attribute category are inferred. It is determined whether the training has converged based on the actual facial attributes of the input facial image and the first predicted facial attributes obtained by inference, as well as a predetermined convergence condition; if it is determined that it has not converged, the next iterative training is performed; if it is determined that it has converged, the training of the first attribute classification model is completed. That is, according to the principle of the shortest distance between the first predicted facial attribute and the actual facial attribute under this attribute category, if the distance between the first predicted facial attribute and the actual facial attribute is greater than the preset distance threshold, it is determined that there is no convergence. After it is determined that there is no convergence, the model parameters of the first attribute classification model are adjusted, that is, after adjusting the model parameters of the first attribute classification model in the facial attribute recognition network, the next iterative training is performed. If the distance between the first predicted facial attribute and the actual facial attribute is less than or equal to the preset distance threshold, it is determined that there is convergence. When convergence is determined, the training of the first attribute classification model is completed.

[0088] For example, taking the attribute category as the gender category, the facial attribute recognition network as the gender recognition model, and the first attribute classification model as the gender model as an example, the training process of the gender model is introduced, the model parameters of the global feature extraction model in the gender model are controlled to remain unchanged, the selected facial image is input into the global feature extraction model in the gender model, the global feature extraction model performs feature extraction to obtain the global facial features of the facial image, the obtained global facial features are input into the gender model, the gender model obtains the predicted gender attributes under the gender category based on the input global facial features, and judges whether convergence is achieved according to the convergence condition of the principle of shortest distance between the predicted gender attributes and the actual gender attributes. If convergence does not occur, the model parameters of the gender model used to classify gender in the gender model are adjusted to perform the next iterative training until convergence, and finally a gender model that has completed training is obtained.

[0089] As an optional embodiment, the first attribute classification model may also be any one of the following: age model, mask model, glasses model, hair color model, eye model, beard model.

[0090] It should be noted here that the above example is only used as an example for explanation. The training method of other attribute recognition models for identifying attributes under other attribute categories is similar to the above process. For example, for the training of a facial attribute recognition network containing multiple first attribute classification models, the global feature extraction module in the facial attribute recognition network extracts global facial features from the input facial image, and inputs the extracted global facial features into a first attribute classification model. After the training of the first attribute classification model is completed, the global facial features are input into the next first attribute classification model until the last first attribute classification model is trained. It is not described in detail here. As another optional embodiment, in order to improve the efficiency of training, on the basis of the global feature extraction model that has been trained, the training process of other attribute classification models in at least two parallel attribute classification models is carried out in parallel. For example, on the basis of the global feature extraction model that has been trained, the gender model, age model, mask model, glasses model, hair color model, eye model, and beard model can be trained in parallel, and finally the trained gender model, age model, mask model, glasses model, hair color model, eye model, and beard model are obtained. That is, the global feature extraction module in the facial attribute recognition network extracts global facial features from the input facial image, and inputs the extracted global facial features in parallel to the gender model, age model, mask model, glasses model, hair color model, eye model, and beard model, thereby training the gender model, age model, mask model, glasses model, hair color model, eye model, and beard model in parallel to obtain the corresponding completed training model.

[0091] It should be noted here that after a first attribute classification model is trained, the model parameters of the first attribute classification model are saved.

[0092] It should be noted that when the attribute category is the hair color category, the hair colors that can be recognized by the hair color attribute recognition model may include red, black, brown, white, etc., then the actual hair color attributes under the hair color category may include red, black, brown, white, etc. The number of hair colors that can be recognized by the facial attribute recognition network can be determined according to actual needs.

[0093] After obtaining the trained first attribute classification models, it is necessary to separate the trained first attribute classification models from the trained facial attribute recognition network so as to splice the global feature extraction model and each trained first attribute classification model to obtain a facial attribute recognition network for identifying attributes under multiple attribute categories. In this way, if a new attribute needs to be identified, a first attribute classification model that can classify the new attribute can be trained separately, and the trained first attribute classification model can be spliced ​​into the existing facial attribute recognition network. Compared with the prior art, which requires training of each first attribute recognition model, it is not necessary to train the global feature extraction model in the facial attribute recognition network, nor is it necessary to train other attribute classification models, which greatly reduces the amount of calculation and the training time. In addition, the facial attribute recognition network finally obtained has strong scalability, and attribute classification models for identifying different attributes can be spliced ​​according to needs without affecting the application performance of the existing model.

[0094] After obtaining the model information corresponding to each first attribute classification model, a facial attribute recognition network for recognizing facial attributes under multiple attribute categories may be determined according to the following steps:

[0095] After the training of all the first attribute classification models is completed, the model parameters of the global feature extraction model and the model parameters of all the first attribute classification models are written into the model file of the facial attribute recognition network. When writing the model parameters in the model file of the facial attribute recognition network, the model parameters of the global feature extraction model can be written first, and then the model parameters of each first attribute classification model can be written in sequence. When writing the model parameters of each first attribute classification model, each model parameter can be written in sequence regardless of the order.

[0096] Facial attributes include natural face attributes and / or face occlusion attributes.

[0097] Here, the natural attributes of the face can be attributes under the gender category (for example, female), attributes under the hair color category (for example, black hair), attributes under the eye category (for example, small eyes), and attributes under the beard category (for example, having a beard), etc. The face occlusion attributes can be attributes under the glasses category (for example, wearing glasses), attributes under the mask category (for example, wearing a mask), etc.

[0098] For example, refer to Figure 4After obtaining the face picture, the face image is input into the shared network, that is, the face image is input into the global feature extraction network to extract the global face features including the global attributes of the face in the face image, and the extracted global face features are respectively input into the hair color model, mask model, glasses model, and gender model in the facial attribute recognition network to obtain the attributes of the gender category corresponding to the face image as female, the attribute of the mask category as not wearing a mask, the attribute of the glasses category as wearing glasses, and the attribute of the hair color category as black.

[0099] Corresponding to the above-mentioned facial recognition method, an embodiment of the present application provides a facial recognition device, including:

[0100] A facial image acquisition module, used to acquire a facial image to be recognized;

[0101] A facial image processing module, used for inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models;

[0102] A facial attribute recognition module is used to obtain the output of the facial attribute recognition network, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models.

[0103] The facial image processing module is specifically used for:

[0104] Inputting the facial image into the global feature extraction model to obtain global facial features;

[0105] The global facial features are respectively input into the at least two parallel attribute classification models to obtain corresponding at least two facial attributes.

[0106] The training process of the facial attribute recognition network includes:

[0107] Simultaneously obtaining the global feature extraction model and the first attribute classification model of the at least two parallel attribute classification models through training;

[0108] On the basis of the trained global feature extraction model, other attribute classification models in the at least two parallel attribute classification models are obtained through training.

[0109] Among them, it also includes:

[0110] Global feature extraction model generation module, used for:

[0111] Constructing a first attribute recognition model; the first attribute recognition model includes a global feature extraction model and at least two parallel second attribute classification models, the second attribute classification model is used to identify facial attributes under each second attribute category in multiple second attribute categories;

[0112] The first attribute recognition model acquires the facial image and infers the predicted facial attributes under each second attribute category based on the facial image;

[0113] According to the actual facial attributes of the facial image and the predicted facial attributes obtained by inference, the model parameters of the first attribute recognition model are adjusted to obtain a trained first attribute recognition model;

[0114] At least two parallel second attribute classification models are removed from the trained first attribute recognition model to obtain the global feature extraction model.

[0115] The training process of other attribute classification models in the at least two parallel attribute classification models is performed in parallel.

[0116] The first attribute classification model is any one of the following: age model, gender model, mask model, glasses model, hair color model, eye model, beard model.

[0117] The present application also provides a facial recognition device 50, such as Figure 5 FIG. 1 is a schematic diagram of the structure of a facial recognition device 50 provided in an embodiment of the present application. The facial recognition device 50 is a computer device, including a processor 51, a memory 52, and a bus 53. The memory 52 stores machine-readable instructions executable by the processor 51. When the facial recognition device 50 is running, the processor 51 communicates with the memory 52 via the bus 53. When the machine-readable instructions are executed by the processor 51, the following processing is performed:

[0118] Obtaining a facial image to be recognized;

[0119] Inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models;

[0120] An output of the facial attribute recognition network is obtained, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models.

[0121] As those skilled in the art know, with the development of computer hardware, the specific implementation method and name of the bus may undergo various changes. The bus referred to here conceptually covers any information transmission line that can provide services to various components in an electronic device, including but not limited to FSB, HT, QPI, Infinity Fabric, etc.

[0122] In the embodiments of the present application, the processor may be a general-purpose processor, including a central processing unit (CPU), or a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural network processor (NPU), a tensor processing unit (TPU), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0123] In a possible implementation manner, the instructions executed by the processor 51, wherein the step of inputting the facial image into a pre-trained facial attribute recognition network, includes:

[0124] Inputting the facial image into the global feature extraction model to obtain global facial features;

[0125] The global facial features are respectively input into the at least two parallel attribute classification models to obtain corresponding at least two facial attributes.

[0126] In a possible implementation manner, in the instructions executed by the processor 51, the training process of the facial attribute recognition network includes:

[0127] Simultaneously obtaining the global feature extraction model and the first attribute classification model of the at least two parallel attribute classification models through training;

[0128] On the basis of the trained global feature extraction model, other attribute classification models in the at least two parallel attribute classification models are obtained through training.

[0129] In a possible implementation manner, in the instructions executed by the processor 51, the training process of the global feature extraction model includes:

[0130] Constructing a first attribute recognition model; the first attribute recognition model includes a global feature extraction model and at least two parallel second attribute classification models, the second attribute classification model is used to identify facial attributes under each second attribute category in multiple second attribute categories;

[0131] The first attribute recognition model acquires the facial image and infers the predicted facial attributes under each second attribute category based on the facial image;

[0132] According to the actual facial attributes of the facial image and the predicted facial attributes obtained by inference, the model parameters of the first attribute recognition model are adjusted to obtain a trained first attribute recognition model;

[0133] At least two parallel second attribute classification models are removed from the trained first attribute recognition model to obtain the global feature extraction model.

[0134] In a possible implementation manner, in the instructions executed by the processor 51, the training process of other attribute classification models in the at least two parallel attribute classification models is performed in parallel.

[0135] In a possible implementation, in the instructions executed by the processor 51, the first attribute classification model is any one of the following: age model, gender model, mask model, glasses model, hair color model, eye model, beard model.

[0136] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned facial attribute recognition method are executed.

[0137] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned facial attribute recognition method can be executed, thereby solving the problem of large amount of calculation when the model is trained in the prior art. The present application obtains a facial image to be recognized; inputs the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models; obtains the output of the facial attribute recognition network, wherein the output includes at least two facial attributes corresponding to the at least two parallel attribute classification models. In this way, when multiple attributes are recognized by the facial attribute recognition network, the feature extraction model is shared, the parameter amount and the calculation amount of the model are reduced, thereby improving the recognition efficiency of the facial attribute recognition network; further, when training the first attribute recognition model, the model parameters of the feature extraction model are controlled to remain unchanged, and only the model parameters of the first attribute classification model are adjusted. Compared with adjusting the model parameters of the feature extraction model and the attribute classification model at the same time, the calculation amount in the training process is reduced, and the training efficiency of the facial attribute recognition network is improved.

[0138] Also includes:

[0139] An imaging element, the imaging element is coupled to the processor, the imaging element is configured to acquire a facial image captured by an image acquisition device, and the processor is configured to execute the machine-readable instructions to perform the steps of the facial attribute recognition method as described above;

[0140] The communication device is coupled to the target device, and is configured to send the facial attribute recognition network for recognizing facial attributes under a plurality of attribute categories to the target device when the facial attribute recognition network appears.

[0141] Optionally, the electronic device involved in the present application may be a camera device (such as a camera, a camcorder, an edge computing box, etc.) used in environments such as homes, classrooms, and nursing homes; the target device may be an associated and bound mobile phone, tablet, or other device; the communication device may be a device based on Bluetooth technology, fourth-generation mobile communication technology (4th generation mobile networks, 4G), fifth-generation mobile communication technology (5th generation mobile networks, 5G), wireless local area network technology (Wi-Fi Alliance, Wi-Fi), and other technologies, and the communication device sends an alarm information about the status of the target object to the target device through a router or sends an alarm information about the status of the target object directly to the target device through a wireless wide area network (Wireless Wide Area Network, WWAN).

[0142] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0143] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0145] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0146] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A facial attribute recognition method, characterized in that: The method includes: Obtaining a facial image to be recognized; Inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models; Obtaining an output of the facial attribute recognition network, the output comprising at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models; The training process of the facial attribute recognition network includes: Simultaneously obtaining the global feature extraction model and the first attribute classification model of the at least two parallel attribute classification models through training; Based on the trained global feature extraction model, other attribute classification models in the at least two parallel attribute classification models are obtained through training; The training process of the global feature extraction model includes: Constructing a first attribute recognition model, wherein the first attribute recognition model includes a global feature extraction model and a first attribute classification model; The first attribute recognition model acquires facial images in a training set, and the first attribute classification model infers predicted facial attributes of the facial images in the training set; According to the actual facial attributes of the facial images in the training set and the predicted facial attributes obtained by inference, the model parameters of the first attribute recognition model are adjusted to obtain a trained first attribute recognition model; The first attribute classification model is removed from the trained first attribute recognition model to obtain the global feature extraction model.

2. The method according to claim 1, characterized in that The step of inputting the facial image into a pre-trained facial attribute recognition network comprises: Inputting the facial image into the global feature extraction model to obtain global facial features; The global facial features are respectively input into the at least two parallel attribute classification models to obtain corresponding at least two facial attributes.

3. The method according to claim 1, characterized in that The training process of other attribute classification models in the at least two parallel attribute classification models is performed in parallel.

4. The method according to claim 1, characterized in that The first attribute classification model is any one of the following: age model, gender model, mask model, glasses model, hair color model, eye model, beard model.

5. A facial recognition device, characterized in that: include: A facial image acquisition module, used to acquire a facial image to be recognized; A facial image processing module, used for inputting the facial image into a pre-trained facial attribute recognition network, wherein the facial attribute recognition network includes a global feature extraction model and at least two parallel attribute classification models; The training process of the facial attribute recognition network includes: Simultaneously obtaining the global feature extraction model and the first attribute classification model of the at least two parallel attribute classification models through training; Based on the trained global feature extraction model, other attribute classification models in the at least two parallel attribute classification models are obtained through training; A facial attribute recognition module, configured to obtain an output of the facial attribute recognition network, wherein the output includes at least two facial attributes corresponding one-to-one to the at least two parallel attribute classification models; The training process of the global feature extraction model includes: Constructing a first attribute recognition model, wherein the first attribute recognition model includes a global feature extraction model and a first attribute classification model; The first attribute recognition model acquires facial images in a training set, and the first attribute classification model infers predicted facial attributes of the facial images in the training set; According to the actual facial attributes of the facial images in the training set and the predicted facial attributes obtained by inference, the model parameters of the first attribute recognition model are adjusted to obtain a trained first attribute recognition model; The first attribute classification model is removed from the trained first attribute recognition model to obtain the global feature extraction model.

6. A facial recognition device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the facial recognition device is running, the processor executes the machine-readable instructions to perform the steps of any one of the methods according to claims 1 to 4.

7. The facial recognition device according to claim 6, characterized in that: Also includes: An imaging element, the imaging element being coupled to a processor, the imaging element being configured to acquire a facial image captured by an image acquisition device, the processor being configured to execute machine-readable instructions to perform the steps of any one of the methods of claims 1 to 4 when executed; The communication device is coupled to the target device, and is configured to send the facial attribute recognition network for recognizing facial attributes under a plurality of attribute categories to the target device when the facial attribute recognition network appears.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a computer or a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images