Face attribute prediction network generation method, face attribute prediction method and device

Abstract

The application is suitable for the technical field of image processing, and provides a face attribute prediction network generation method, a face attribute prediction method and device. The face attribute prediction network generation method comprises: obtaining a sample face image set; performing augmentation processing on each sample face image to obtain a first augmented image and a second augmented image; alternately training a first branch network and a second branch network according to a plurality of first augmented images and a plurality of second augmented images to obtain a trained first branch network; and determining the trained first branch network as a face attribute prediction network. Thus, the mutual supervision of the first branch network and the second branch network can enable the trained first branch network to extract the features of the attributes corresponding to the face in the face image, thereby avoiding attribute labeling on the sample face image and reducing the human resources and time cost of attribute labeling.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a face attribute prediction network generation method, a face attribute prediction method and device. BACKGROUND

[0002] A face image contains a lot of information. In actual applications, attribute prediction is needed for a face in a face image to obtain multiple attributes corresponding to the face. For example, attribute prediction is performed on a face image to obtain gender attributes, age attributes and the like corresponding to the face.

[0003] Therefore, it is crucial to design a multi-task attribute prediction method for obtaining multiple attributes corresponding to a face in a face image so as to be able to simultaneously obtain multiple attributes corresponding to the face.

[0004] In related technologies, a multi-task attribute prediction method is a supervised learning model based on a convolutional neural network.

[0005] However, when training the above model, attribute labels corresponding to a face need to be set for each sample face image in a sample face image set before training, and setting attribute labels for each sample face image consumes a large amount of time and labor cost, resulting in low training efficiency of the model. SUMMARY

[0006] The present application provides a face attribute prediction network generation method, a face attribute prediction method and device, which can avoid the consumption of a large amount of time and labor cost for setting attribute labels for each sample face image in a sample face image set when training a network, ensure the training efficiency of the network, and save time cost and human resources.

[0007] In a first aspect, the present application provides a face attribute prediction network generation method, comprising:

[0008] obtaining a sample face image set, the sample face image set comprising multiple sample face images;

[0009] augmenting each sample face image to obtain a first augmented image and a second augmented image, the first augmented image and the second augmented image maintaining the same attributes corresponding to a face in a corresponding sample face image;

[0010] alternately training a first branch network and a second branch network according to multiple first augmented images and multiple second augmented images to obtain a trained first branch network, so that the trained first branch network can extract features of attributes corresponding to a face in a face image through mutual supervision of the first branch network and the second branch network, the attributes corresponding to the face including at least age attributes and gender attributes.

[0011] determining the trained first branch network as a face attribute prediction network.

[0012] The first and second augmented images are obtained by augmenting each sample face image, and the first and second branch networks are alternately trained according to the plurality of first augmented images and the plurality of second augmented images, to obtain a trained first branch network. Thus, the trained first branch network can extract the features of the face attributes corresponding to the face in the face image by mutual supervision learning of the second branch network and the first branch network, so that the sample face image is not labeled with attributes, and the human resources and time cost of attribute labeling are reduced.

[0013] In a second aspect, the present application provides a face attribute prediction method, comprising:

[0014] obtaining a region corresponding to a face in a to-be-tested face image;

[0015] inputting the region corresponding to the face into a face attribute prediction network, and outputting predicted attributes corresponding to the face, the face attribute prediction network being used to predict the features of the attributes corresponding to the face in the face image, the attributes corresponding to the face including at least age attributes and gender attributes, and the face attribute prediction network being obtained according to the face attribute prediction network generation method.

[0016] The region corresponding to the face is inputted into the face attribute prediction network to output the predicted attributes corresponding to the face. Two augmented images obtained by randomly augmenting each sample are inputted into the first and second branch networks for alternately training, and the face attribute prediction network obtained has a higher prediction accuracy for the attributes corresponding to the face.

[0017] In a third aspect, the present application provides a face attribute prediction network generation device, which is used to execute the method in the first aspect or any possible implementation manner of the first aspect. Specifically, the device can include:

[0018] an acquisition module, configured to acquire a sample face image set, the sample face image set including a plurality of sample face images;

[0019] an augmentation module, configured to augment each sample face image to obtain a first augmented image and a second augmented image, the first and second augmented images maintaining the same attributes corresponding to the face in the corresponding sample face image;

[0020] a training module, configured to perform alternating training on the first branch network and the second branch network according to the first plurality of augmented images and the second plurality of augmented images, to obtain a trained first branch network, so that the trained first branch network can extract features of attributes corresponding to a face in a face image through mutual supervision of the first branch network and the second branch network, the attributes corresponding to the face at least including an age attribute and a gender attribute;

[0021] a determination module, configured to determine the trained first branch network as a face attribute prediction network.

[0022] In a fourth aspect, the present application provides a face attribute prediction apparatus, which is configured to execute the method in the second aspect or any possible implementation manner of the second aspect. Specifically, the apparatus can include:

[0023] an acquisition module, configured to acquire a region corresponding to a face in a face image to be tested;

[0024] a prediction module, configured to input the region corresponding to the face into a face attribute prediction network, and output predicted attributes corresponding to the face, the face attribute prediction network being configured to predict features of attributes corresponding to the face in the face image, the attributes corresponding to the face at least including an age attribute and a gender attribute, the face attribute prediction network being obtained according to the face attribute prediction network generation method as described above.

[0025] In a fifth aspect, the present application provides an electronic device, which includes a memory and a processor. The memory is configured to store instructions; and the processor is configured to execute the instructions stored in the memory, so that the device executes the face attribute prediction network generation method in the first aspect or any possible implementation manner of the first aspect, and / or the face attribute prediction method in the second aspect or any possible implementation manner of the second aspect.

[0026] In a sixth aspect, a computer readable storage medium is provided, which stores instructions, when the instructions are executed on a computer, causing the computer to execute the face attribute prediction network generation method in the first aspect or any possible implementation manner of the first aspect, and / or the face attribute prediction method in the second aspect or any possible implementation manner of the second aspect.

[0027] In a seventh aspect, a computer program product is provided, which includes instructions, when the instructions are executed on a device, causing the device to execute the face attribute prediction network generation method in the first aspect or any possible implementation manner of the first aspect, and / or the face attribute prediction method in the second aspect or any possible implementation manner of the second aspect.

[0028] It can be understood that the beneficial effects of the third aspect to the seventh aspect described above can be referred to the related description in the first aspect and / or the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0030] Figure 1 is a flowchart of a face attribute prediction network generation method provided by an embodiment of the present application;

[0031] Figure 2a is a flowchart of a face attribute prediction network generation method provided by an embodiment of the present application;

[0032] Figure 2b is a schematic diagram of a face attribute prediction network provided by an embodiment of the present application;

[0033] Figure 3a is a flowchart of a face attribute prediction method provided by an embodiment of the present application;

[0034] Figure 3b is a flowchart of a face attribute prediction network attribute prediction method provided by an embodiment of the present application;

[0035] Figure 4 is a structural schematic diagram of a face attribute prediction network generation device provided by an embodiment of the present application;

[0036] Figure 5 is a structural schematic diagram of a face attribute prediction device provided by an embodiment of the present application;

[0037] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, persons skilled in the art will understand that the present application can be practiced without these specific details. In other instances, well-known structures, devices, circuits and methods have not been described in detail in order to avoid obscuring the present application.

[0039] It should be understood that the term "includes" when used in the specification and the appended claims herein, specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0040] It should also be understood that the term "and / or" when used in the specification and the appended claims herein, means any one or more of the associated listed items and includes all possible combinations of the associated listed items.

[0041] As used in the specification and the appended claims herein, the term "if' can be interpreted as meaning "when" or "once" or "in response to a determination" or "in response to a detection" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted as meaning "once determined" or "in response to a determination" or "once detected [the described condition or event]" or "in response to a detection [the described condition or event]" depending on the context.

[0042] In addition, in the description of the specification and the appended claims herein, the terms "first", "second", "third", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0043] Reference in the specification to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearances of the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments", and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms "including", "containing", "having", and variations thereof mean "including but not limited to", unless otherwise specifically stated.

[0044] The present application provides a face attribute prediction network generation method, a face attribute prediction method and device, which can be applied to case investigation, missing person search and intelligent security, etc.

[0045] The face attribute prediction network generation method and the face attribute prediction method can be implemented by an electronic device.

[0046] The electronic device can execute the face attribute prediction network generation method through a network training system, and the electronic device can execute the face attribute prediction method through an identification system.

[0047] It should be understood that the electronic devices performing the face attribute prediction network generation method and the face attribute prediction method can be the same device or different devices.

[0048] The electronic device can be a smartphone, a tablet computer, a desktop computer, a notebook computer, a handheld device, a server, a vehicle-mounted device, etc. The embodiments of the present application do not make any limitation on the specific type of the electronic device.

[0049] Before introducing the face attribute prediction network generation method and the face attribute prediction method provided by the embodiments of the present application, the scenarios to which the face attribute prediction network generation method and the face attribute prediction method are applied are exemplarily described as follows:

[0050] Taking searching for a missing person as an example, the face attribute of the missing person A is output by inputting the region corresponding to the face of the missing person A into the face attribute recognition network trained by the face attribute prediction network generation method. After obtaining the predicted attribute of the missing person A, the image matching the missing person A is searched for in the missing person database according to the predicted attribute of the missing person A.

[0051] The predicted attribute of the missing person A includes but is not limited to gender attribute and age attribute, for example, can also include hair attribute, hairline attribute, single / double eyelid attribute, hat attribute, mask attribute, etc.

[0052] Based on the above scenario description, the face attribute prediction network generation method provided by the embodiments of the present application is described in detail below by taking an electronic device as an example in combination with the accompanying drawings and application scenarios.

[0053] Please refer to Figure 1 , Figure 1 The flowchart of the face attribute prediction network generation method provided by an embodiment of the present application is shown.

[0054] As shown in Figure 1 , the face attribute prediction network generation method provided by the present application can include:

[0055] S101, obtaining a sample face image set.

[0056] The sample face image set includes multiple frames of sample face images.

[0057] Optionally, the sample face image set includes at least one frame of sample face image.

[0058] The sample face image set can be selected from an existing public face image dataset, can be obtained by pre-shooting face images through a camera, or can be downloaded from the Internet.

[0059] In some embodiments, when taking a face image through a camera, a camera with high precision is needed to take the sample face from multiple angles in order to obtain a clear sample face image at any angle.

[0060] The camera for taking the face image can be a camera, a camera of a smart phone, a camera of a notebook computer, or a camera of a tablet computer.

[0061] S102, augment each sample face image to obtain a first augmented image and a second augmented image.

[0062] The first augmented image and the second augmented image maintain the same attributes corresponding to the face in the corresponding sample face image.

[0063] For example, the gender attribute of a certain sample face image is male, and the age attribute is eight years old. After the sample face image is augmented, the gender attribute of the first augmented image and the second augmented image is male, and the age attribute is eight years old.

[0064] Augmentation is a method of generating similar but different training samples by making a series of random changes to sample images to expand the size of the training data set.

[0065] The augmentation can be one or more selected from random horizontal flipping, random cropping, random translation, random rotation, random scaling, random color change, random noise addition, random grayscale transformation, random size transformation, random mirror transformation, and random color space transformation.

[0066] For example, a certain sample face image in the sample face image set is flipped and cropped to obtain a first augmented image after flipping and a second augmented image after cropping.

[0067] Again, for example, a certain sample face image in the sample face image set is sequentially subjected to random horizontal flipping, random cropping, and random translation twice to obtain a first augmented image and a second augmented image.

[0068] S103, alternately train the first branch network and the second branch network according to the plurality of first augmented images and the plurality of second augmented images to obtain a trained first branch network.

[0069] The electronic device alternately trains the first branch network and the second branch network so that the trained first branch network can extract the features of the attributes corresponding to the face in the face image through mutual supervision of the first branch network and the second branch network.

[0070] The attributes corresponding to the face include at least an age attribute and a gender attribute.

[0071] In some embodiments, the electronic device alternately trains the first branch network and the second branch network by performing the following steps:

[0072] inputting the plurality of first augmented images into the first branch network to output first features, and inputting the plurality of second augmented images into the second branch network to output second features;

[0073] inputting the plurality of second augmented images into the first branch network to output third features, and inputting the plurality of first augmented images into the second branch network to output fourth features;

[0074] adjusting network parameters of the first branch network according to the first features, the second features, the third features, and the fourth features to obtain the trained first branch network.

[0075] In some embodiments, the first branch network includes a first backbone network and an initial prediction network, the first backbone network is configured to extract features of a face in a face image, and the initial prediction network is configured to extract features of attributes corresponding to the face in the face image.

[0076] The initial prediction network includes a plurality of attribute prediction modules, and each attribute prediction module is configured to predict different attributes. The number of attribute prediction modules included in the initial prediction network is set according to the types of attributes to be predicted, and will not be described here.

[0077] For example, the initial prediction network includes two attribute prediction modules, which are respectively configured to predict a gender attribute and an age attribute corresponding to the face.

[0078] The second branch network includes a second backbone network, and the second backbone network is configured to extract features of the face in the face image.

[0079] It should be noted that the first backbone network and the second backbone network are both backbone networks. The first backbone network and the second backbone network have the same structure and share weights.

[0080] In some embodiments, the first backbone network and the second backbone network each include an encoder f and a projection multilayer perceptron (MLP).

[0081] The encoder f is a convolutional network, for example, a residual network (ResNet) or a depthwise separable convolutional network (MobileNet).

[0082] In some embodiments, the initial prediction network is a predictor h, which is specifically a multilayer perceptron (MLP).

[0083] The electronic device performs mutual supervision learning on the features of the face in the face image output by the first branch network and the features of the face in the face image output by the second branch network, so that the trained first branch network can extract the features of the attributes corresponding to the face in the face image through mutual supervision of the first branch network and the second branch network.

[0084] S104, determining the trained first branch network as a face attribute prediction network.

[0085] It can be understood that the face attribute prediction network is a modified twin network, except that in the network of step S104, the first branch network includes a first backbone network and an initial prediction network, and the second branch network includes a second backbone network.

[0086] The face attribute prediction network generation method provided in the present application generates a first augmented image and a second augmented image by performing augmentation processing on each frame of sample face image by an electronic device, and alternately trains a first branch network and a second branch network according to a plurality of first augmented images and a plurality of second augmented images, to obtain a trained first branch network. Therefore, with the aid of the second branch network, the trained first branch network can extract the features of the attributes corresponding to the face in the face image through mutual supervision learning of the second branch network and the first branch network, thereby avoiding attribute labeling on the sample face image and reducing the human resources and time cost of attribute labeling.

[0087] Based on the description of the above Figure 1 According to the description of the above embodiment, the electronic device adjusts the network parameters of the first branch network by means of a loss function when adjusting the network parameters of the first branch network according to the first feature, the second feature, the third feature and the fourth feature.

[0088] Next, the specific implementation process of the face attribute prediction network generation method of the present application will be described in detail. Figure 2a

[0089] The electronic device adjusts the network parameters of the first branch network by means of a loss function, wherein the loss function adopts a symmetric loss function calculated by a stochastic gradient descent method.

[0090] ​Stochastic Gradient Descent (SGD) refers to that in a machine learning algorithm, sometimes a loss function needs to be constructed for an original network, and then an optimization algorithm is used to optimize the loss function so as to find the optimal parameters to minimize the value of the loss function. In the optimization algorithm for solving machine learning parameters, the optimization algorithm based on gradient descent is mostly used.

[0091] Optionally, the symmetric loss function is a symmetric loss function in which gradient operation is stopped.

[0092] Please refer to Figure 2a , Figure 2a A flowchart of a face attribute prediction network generation method provided by an embodiment of the application is shown.

[0093] As Figure 2a shown, the face attribute prediction network generation method provided by the application can include:

[0094] S201, determining a symmetric loss function or an optimized symmetric loss function according to the first feature, the second feature, the third feature and the fourth feature.

[0095] In some embodiments, the electronic device calculates the symmetric loss function by performing the following steps:

[0096] The similarity between the first feature and the second feature is determined as a first similarity, and the similarity between the third feature and the fourth feature is determined as a second similarity.

[0097] The symmetric loss function or the optimized symmetric loss function is determined according to the first similarity and the second similarity.

[0098] The optimized symmetric loss function refers to a symmetric loss function in which gradient operation is stopped.

[0099] For the calculation of the similarity and the symmetric loss function, the following is shown:

[0100] The calculation formula of the feature vector of the first feature is:

[0101]

[0102] Where p1 represents the feature vector of the first feature, and x1 represents the first augmented image.

[0103] The calculation formula of the feature vector of the second feature is:

[0104]

[0105] wherein z2 represents a feature vector of the second feature, and x2 represents the second augmented image.

[0106] The formula for calculating the feature vector of the third feature is:

[0107]

[0108] wherein p2 represents a feature vector of the third feature.

[0109] The formula for calculating the feature vector of the fourth feature is:

[0110]

[0111] wherein z1 represents a feature vector of the fourth feature.

[0112] The formula for calculating the first similarity is:

[0113]

[0114] wherein D(p1, z2) represents the first similarity, i.e., the negative cosine similarity between the first feature and the second feature, p1 represents a feature vector of the first feature, z2 represents a feature vector of the second feature, ||p1||2 represents a modulus of the feature vector of the first feature, and ||z2||2 represents a modulus of the feature vector of the second feature.

[0115] The formula for calculating the second similarity is:

[0116]

[0117] wherein D(p2, z1) represents the second similarity, i.e., the negative cosine similarity between the third feature and the fourth feature, p2 represents a feature vector of the third feature, z1 represents a feature vector of the fourth feature, ||p2||2 represents a modulus of the feature vector of the third feature, and ||z1||2 represents a modulus of the feature vector of the fourth feature.

[0118] The formula for calculating the symmetric loss function is:

[0119]

[0120] wherein D(p1, z2) represents the first similarity, i.e., the similarity between the first feature and the second feature;

[0121] D(p2, z1) represents the second similarity, i.e., the similarity between the third feature and the fourth feature.

[0122] In some embodiments, a stop-grad operation is performed on the second branch network.

[0123] Then, the symmetric loss function formula when the gradient operation is stopped is as follows:

[0124]

[0125] where stopgrad() represents a stop gradient operation, (p1, stopgrad(z2)) represents that, when the symmetric loss is calculated, the first augmented image x1 is output as p1 through the first branch network, and the second augmented image x2 is output as z2 through the second branch network, which is equivalent to that the first augmented image x1 is processed through the initial prediction network in the first branch network, and the second augmented image x2 is not processed through the initial prediction network in the second branch network, and since the second branch network has the stop gradient (stop-grad), z2 does not participate in the gradient update of the network parameters;

[0126] Similarly, (p2, stopgrad(z1)) represents that, when the symmetric loss function is calculated, the second augmented image x2 is output as p2 through the first branch network, and the first augmented image x1 is output as z2 through the second branch network, which is equivalent to that the second augmented image x2 is processed through the initial prediction network in the first branch network, and the first augmented image x1 is not processed through the initial prediction network in the second branch network, and since the second branch network has the stop gradient (stop-grad), z1 does not participate in the gradient update of the network parameters;

[0127] In the first term (p1, stopgrad(z2)), the second backbone network does not receive the gradient from z2, but receives the gradient from p1;

[0128] In the second term (p2, stopgrad(z1)), the second backbone network does not receive the gradient from z1, but receives the gradient from p2.

[0129] where the network parameters represent a vector containing all parameters of the network, and the process of updating the network parameters is a process of gradually optimizing the network. The optimization goal of the network is to minimize the symmetric loss, and the symmetric loss is derived with respect to the network parameters to update the network parameters.

[0130] After the first branch network and the second branch are alternately trained for a plurality of times (for example, 100 times), they tend to converge, and the first branch network obtained at this time is the final trained first branch network.

[0131] It can be understood that the stop gradient operation can maximize the similarity between the vectors of the features output by the first branch network and the second branch network.

[0132] S202, adjusting the network parameters of the initial prediction network according to the symmetric loss function or the optimized symmetric loss function.

[0133] In some embodiments, the electronic device adjusts the network parameters of the initial prediction network according to the symmetric loss function.

[0134] In some other embodiments, the electronic device adjusts the network parameters of the initial prediction network according to the optimized symmetric loss function (the symmetric loss function when the gradient operation is stopped).

[0135] S203, determining a total loss function according to the symmetric loss function or the optimized symmetric loss function and the number of attribute prediction modules included in the initial prediction network.

[0136] The calculation formula of the total loss function is:

[0137]

[0138] Wherein, n represents the number of attribute prediction modules included in the initial prediction network, and i represents the i-th attribute prediction module.

[0139] S203, adjusting the network parameters of the first backbone network according to the total loss function.

[0140] S204, connecting the adjusted initial prediction network to the output end of the adjusted first backbone network to obtain the trained first branch network.

[0141] In some embodiments, the adjusted initial prediction network is connected to the output end of the adjusted first backbone network, and after the first backbone network outputs the features of the face in the face image, the features corresponding to the face are input into the adjusted prediction network, and the adjusted prediction network outputs the features of the attributes corresponding to the face in the face image.

[0142] In this application, the electronic device determines the symmetric loss function or the optimized symmetric loss function through the first feature, the second feature, the third feature and the fourth feature, adjusts the network parameters of the initial prediction network according to the symmetric loss function or the optimized symmetric loss function, and finally adjusts the network parameters of the first backbone network according to the total loss function. Therefore, the electronic device adjusts the initial prediction network through the optimized symmetric loss function (the symmetric loss function when the gradient operation is stopped), which can maximize the similarity of the features output by the first branch network and the second branch network, ensure that the network parameters of the initial prediction network and the first branch network are more optimal, and the features of the face in the output face image are more accurate.

[0143] Based on the above description, in one specific embodiment, as Figure 2bAs shown, the left side is the first branch network, and the right side is the second branch network, the first branch network includes an encoder f (a first main network) and a predictor h (an initial prediction network), and the second branch network includes an encoder f (a second main network), and the two encoders f share weights.

[0144] Based on the above, the electronic device performs a face attribute prediction network generation method, which can include the following steps:

[0145] Step 11, the electronic device sequentially performs random horizontal flipping, random cropping, and random translation processing twice on the face image x to obtain a face image x1 and a face image x2.

[0146] Step 12, the electronic device inputs the face image x1 into the encoder f of the first branch network to output the features corresponding to the face, and inputs the features corresponding to the face into the predictor h of the first branch network to output the features of the attributes corresponding to the face.

[0147] Step 13, the electronic device inputs the face image x2 into the encoder f of the second branch network to output the features corresponding to the face.

[0148] Step 14, the electronic device calculates the negative cosine similarity between the features of the attributes corresponding to the face output by the first branch network and the features corresponding to the face output by the second branch network.

[0149] Step 15, the electronic device calculates the symmetric loss function when the gradient operation is stopped according to the negative cosine similarity.

[0150] Step 16, the electronic device adjusts the network parameters of the predictor h according to the symmetric loss function when the gradient operation is stopped.

[0151] Step 17, the electronic device calculates the total loss function according to the symmetric loss function corresponding to the stopped gradient and the number of attribute prediction modules included in the predictor h.

[0152] Step 18, the electronic device adjusts the network parameters of the encoder f of the first branch network according to the total loss function;

[0153] Step 19, the electronic device determines the trained first branch network according to the adjusted predictor h and the adjusted encoder f.

[0154] Step 20, the electronic device determines the trained first branch network as the face attribute prediction network.

[0155] It can be understood that the execution order of steps 12 and 13, and steps 16 and 17 is not sequential, and they can be performed simultaneously.

[0156] Based on the above scenario description, the face attribute prediction method provided by the embodiments of the present application will be described in detail below with the electronic device as an example, in combination with the drawings and application scenarios.

[0157] Please refer to Figure 3a , Figure 3a The flowchart of the face attribute prediction method provided by an embodiment of the present application is shown.

[0158] As Figure 3a shown, the face attribute prediction method provided by the present application can include:

[0159] S301, obtaining a region corresponding to a face in a to-be-tested face image.

[0160] In some embodiments, the region corresponding to the face is obtained by performing face detection on the to-be-tested face image, obtaining a detection window, and then performing image interception on the image in the detection window.

[0161] The to-be-tested face image can be directly given by a user, or can be extracted from video / image data collected by an image collection device such as a monitoring camera or a camera.

[0162] In some embodiments, the image collection device is communicatively connected to the electronic device, and the image collection device sends the collected video / image to the electronic device, and the electronic device obtains the to-be-tested face image from the video / image.

[0163] It can be understood that the detection window refers to a wireframe that can extract the face in the to-be-tested face image.

[0164] The face detection algorithm can be used to perform face detection on the to-be-tested face.

[0165] The face detection algorithm can be stored in a storage device. The storage device can communicate with the electronic device, so that the electronic device can retrieve the face detection algorithm from the storage device. The storage method and specific type of the storage device are not limited by the present application.

[0166] In some embodiments, the YOLO (you only look once) algorithm is used to perform face detection on the to-be-tested face image. The YOLO algorithm is an object recognition and positioning algorithm based on a deep neural network, and its biggest feature is fast running speed.

[0167] For example, after the electronic device obtains the to-be-tested face image, the face detection algorithm is retrieved to perform face detection on the image captured by the monitoring camera, and the region corresponding to the face in the to-be-tested face image is obtained.

[0168] S302, inputting the region corresponding to the face into a face attribute prediction network, and outputting the predicted attribute corresponding to the face.

[0169] A face attribute prediction network is used to predict the features of the attributes corresponding to a face in a face image. The attributes corresponding to the face include at least age and gender attributes.

[0170] The face attribute prediction network includes a backbone network and a prediction network. The backbone network is used to extract the features of the face in the face image, and the prediction network is used to predict the features of the attributes corresponding to the face.

[0171] The prediction network comprises multiple attribute prediction modules, each used to predict different facial attributes. For example, the prediction network may include two attribute prediction modules: one for age and the other for gender.

[0172] The age attribute prediction module is used to predict the age attribute of a face, and the gender attribute prediction module is used to predict the gender attribute of a face.

[0173] Of course, the prediction network can also be configured with other attribute prediction modules, depending on the attribute categories of the faces to be predicted.

[0174] For example, the prediction network may also include a hair attribute prediction module, a hairline attribute prediction module, a single / double eyelid attribute prediction module, a hat attribute prediction module, a mask attribute prediction module, etc.

[0175] In some embodiments, the region corresponding to the face is input into the backbone network to output the features of the face; the features of the face are input into the prediction network to output the predicted attributes corresponding to the face.

[0176] The backbone network and prediction network, which are used for face attribute prediction, are pre-stored in a storage device that communicates with the electronic device.

[0177] It should be noted that the face attribute prediction network is the first branch network after training. The first branch network after training is used to train the first backbone network and the second branch network alternately through the first augmented image and the second augmented image. Through mutual supervision between the first branch network and the second branch network, the first branch network after training can extract the features of the attributes corresponding to the face in the face image. The first augmented image and the second augmented image are obtained by augmenting each frame of sample face image. Each frame of sample image comes from the sample face image set.

[0178] The first augmented image and the second augmented image retain the same attributes as the faces in the corresponding sample face images.

[0179] Augmentation processing can be one or more of the following: random horizontal flipping, random cropping, random translation, random rotation, random scaling, random color change, random noise addition, random grayscale transformation, random image size transformation, random mirror transformation, and random color space transformation.

[0180] For example, after acquiring a face image to be tested, the electronic device invokes a face detection algorithm to detect faces in the image and obtain the corresponding face region. The electronic device then inputs the face region into the backbone network, outputs the face features, and then inputs the face features into the prediction network, outputting the predicted attributes of the face.

[0181] In a missing persons search scenario, the region corresponding to the face of missing person A is input into a facial attribute recognition network, which outputs the gender and age attributes of missing person A. After obtaining the gender and age attributes of missing person A, an image matching missing person A is searched in the missing persons database based on these attributes.

[0182] like Figure 3b As shown, the backbone network, as a shared network, outputs facial features, while the prediction network, as a task-specific network, receives the facial features output by the shared network and outputs the predicted attributes corresponding to the facial features.

[0183] The face attribute prediction method provided in this application uses an electronic device to input the region corresponding to a face into a backbone network, outputting the face features. These face features are then input into a prediction network to output the predicted attributes corresponding to the face. In other words, by alternately training the first and second branch networks with two augmented images obtained from randomly augmenting each frame of samples, the resulting face attribute prediction network achieves higher accuracy in predicting the attributes corresponding to the face.

[0184] Corresponding to the above Figure 1 The present application also provides a face attribute prediction network generation apparatus, in addition to the face attribute prediction network generation method described in the illustrated embodiment.

[0185] Below, in conjunction with Figure 4 This application provides a detailed description of a face attribute prediction network generation apparatus according to an embodiment of the present application.

[0186] Please see Figure 4 , Figure 4 A schematic block diagram of a face attribute prediction network generation apparatus provided in an embodiment of this application is shown.

[0187] like Figure 4As shown in the figure, an embodiment of this application provides a face attribute prediction network generation device, including an acquisition module 401, an augmentation module 402, a training module 403, and a determination module 404.

[0188] The acquisition module 401 is used to acquire a sample face image set, which includes multiple frames of sample face images;

[0189] The augmentation module 402 is used to perform augmentation processing on each frame of sample face image to obtain a first augmented image and a second augmented image. The first augmented image and the second augmented image maintain the same attributes as the face in the corresponding sample face image.

[0190] Training module 403 is used to alternately train a first branch network and a second branch network based on multiple first augmented images and multiple second augmented images to obtain a trained first branch network, so that the trained first branch network can extract features of the attributes corresponding to the face in the face image through mutual supervision between the first branch network and the second branch network, wherein the attributes corresponding to the face include at least age attributes and gender attributes.

[0191] The determination module 404 is used to determine the trained first branch network as a face attribute prediction network.

[0192] In some embodiments, the training module 403 is specifically used for:

[0193] Multiple first augmented images are input into the first branch network to output a first feature, and multiple second augmented images are input into the second branch network to output a second feature;

[0194] Multiple second augmented images are input into the first branch network to output a third feature, and multiple first augmented images are input into the second branch network to output a fourth feature;

[0195] Based on the first feature, the second feature, the third feature, and the fourth feature, the network parameters of the first branch network are adjusted to obtain the trained first branch network.

[0196] In some embodiments, the first branch network includes a first backbone network and an initial prediction network, wherein the first backbone network is used to extract features of a face in a face image, and the initial prediction network is used to extract features of attributes corresponding to the face in the face image.

[0197] In some embodiments, the second branch network includes a second backbone network, which is used to extract facial features from a face image.

[0198] In some embodiments, the training module 403 is specifically used for:

[0199] Based on the first feature, the second feature, the third feature, and the fourth feature, determine the symmetric loss function or the optimized symmetric loss function;

[0200] Adjust the network parameters of the initial prediction network according to the symmetric loss function or the optimized symmetric loss function;

[0201] The total loss function is determined based on the symmetric loss function or the optimized symmetric loss function and the number of attribute prediction modules included in the initial prediction network;

[0202] Adjust the network parameters of the first backbone network according to the total loss function;

[0203] The adjusted initial prediction network is connected to the output of the adjusted first backbone network to obtain the trained first branch network.

[0204] In some embodiments, the training module 403 is specifically used for:

[0205] The similarity between the first feature and the second feature is determined as the first similarity.

[0206] The similarity between the third feature and the fourth feature is determined as the second similarity.

[0207] Based on the first similarity and the second similarity, the symmetric loss function or the optimized symmetric loss function is determined.

[0208] Corresponding to the above Figure 3a The present application also provides a face attribute prediction device 500, in addition to the face attribute prediction method described in the illustrated embodiment.

[0209] Below, in conjunction with Figure 5 This application provides a detailed description of a face attribute prediction device according to an embodiment.

[0210] Please see Figure 5 , Figure 5 A schematic block diagram of a face attribute prediction device provided in an embodiment of this application is shown.

[0211] like Figure 5 As shown in the figure, an embodiment of this application provides a face attribute prediction device, including an acquisition module 501 and a prediction module 502.

[0212] The acquisition module 501 is used to acquire the region corresponding to the face in the face image to be tested;

[0213] The prediction module 502 is used to input the region corresponding to the face into the face attribute prediction network and output the predicted attribute corresponding to the face. The face attribute prediction network is used to predict the features of the attribute corresponding to the face in the face image. The attribute corresponding to the face includes at least age attribute and gender attribute.

[0214] In some embodiments, the face attribute prediction network includes a backbone network and a prediction network, wherein the backbone network is used to extract features of a face in a face image, and the prediction network is used to predict features of attributes corresponding to the face.

[0215] In some embodiments, the prediction module is specifically used for:

[0216] The region corresponding to the face is input into the backbone network, and the features of the face are output.

[0217] The features of the face are input into the prediction network, and the predicted attributes corresponding to the face are output.

[0218] It should be understood that the face attribute prediction network generation device 400 and / or face attribute prediction device 500 of this application can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented using software. Figure 1 The face attribute prediction network generation method shown in Figure 3 and / or the face attribute prediction method shown in Figure 3, when implemented in software... Figure 1 When using the face attribute prediction network generation method shown in Figure 3 and / or the face attribute prediction method shown in Figure 3, the face attribute prediction network generation device 400 and / or the face attribute prediction device 500 and their respective modules can also be software modules.

[0219] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 6As shown, device 600 includes a processor 601, a memory 602, a communication interface 603, and a bus 604. The processor 601, memory 602, and communication interface 603 communicate via the bus 604, or via other means such as wireless transmission. The memory 602 stores instructions, and the processor 601 executes the instructions stored in the memory 602. The memory 602 stores program code 6021, and the processor 601 can call and execute the program code 6021 stored in the memory 602. Figure 1 The face attribute prediction network generation method shown in Figure 3 and / or the face attribute prediction method shown in Figure 3.

[0220] It should be understood that in this application, processor 601 can be a CPU, or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0221] The memory 602 may include read-only memory and random access memory, and provides instructions and data to the processor 601. The memory 602 may also include non-volatile random access memory. The memory 602 may be volatile memory or non-volatile memory, or may include both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0222] In addition to the data bus, the 604 bus may also include a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general designated all buses as Bus 604.

[0223] It should be understood that the device 600 according to this application may correspond to the device 400 and / or device 500 in this application, and may correspond to the figures in this application. Figure 1 and / or Figure 3a The device in the method shown, when device 600 corresponds to Figure 1 and / or Figure 3a When using the device in the method shown, the above and other operations and / or functions of each module in device 600 are respectively implemented to achieve Figure 1 and / or Figure 3a The operational steps of the methods performed by the device are omitted here for the sake of brevity.

[0224] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the various method embodiments described above.

[0225] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the steps described in the various method embodiments above.

[0226] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0227] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0228] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0229] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0230] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0231] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0232] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this application according to actual needs.

[0233] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating a face attribute prediction network, characterized in that, The method comprises: obtaining a sample face image set, the sample face image set comprising a plurality of sample face images; augmenting each sample face image to obtain a first augmented image and a second augmented image, the first augmented image and the second augmented image maintaining the same attributes of a face in the corresponding sample face image; alternately training a first branch network and a second branch network according to a plurality of first augmented images and a plurality of second augmented images to obtain a trained first branch network, so that the trained first branch network can extract features of attributes of a face in a face image through mutual supervision of the first branch network and the second branch network, the attributes of the face including at least age and gender; wherein the alternately training the first branch network and the second branch network according to the plurality of first augmented images and the plurality of second augmented images to obtain the trained first branch network comprises: inputting the plurality of first augmented images into the first branch network to output first features, and inputting the plurality of second augmented images into the second branch network to output second features; inputting the plurality of second augmented images into the first branch network to output third features, and inputting the plurality of first augmented images into the second branch network to output fourth features; and adjusting network parameters of the first branch network according to the first features, the second features, the third features, and the fourth features to obtain the trained first branch network; determining the trained first branch network as a face attribute prediction network.

2. The method of claim 1, wherein: the first branch network comprises a first backbone network and an initial prediction network, the first backbone network being configured to extract features of a face in a face image, and the initial prediction network being configured to extract features of attributes of the face in the face image; the second branch network comprises a second backbone network, the second backbone network being configured to extract features of a face in a face image.

3. The method of claim 2, wherein, the adjusting the network parameters of the first branch network according to the first features, the second features, the third features, and the fourth features to obtain the trained first branch network comprises: determining a symmetric loss function or an optimized symmetric loss function according to the first features, the second features, the third features, and the fourth features; adjusting network parameters of the initial prediction network according to the symmetric loss function or the optimized symmetric loss function; determining a total loss function according to the symmetric loss function or the optimized symmetric loss function and a number of attribute prediction modules included in the initial prediction network; adjusting network parameters of the first backbone network according to the total loss function; connecting the adjusted initial prediction network to an output end of the adjusted first backbone network to obtain the trained first branch network.

4. The method of claim 3, wherein, The determining the symmetric loss function or the optimized symmetric loss function according to the first feature, the second feature, the third feature and the fourth feature comprises: determining a similarity between the first feature and the second feature as a first similarity; determining a similarity between the third feature and the fourth feature as a second similarity; determining the symmetric loss function or the optimized symmetric loss function according to the first similarity and the second similarity.

5. A method for human face attribute prediction, characterized in that, comprises: obtaining a region corresponding to a face in a to-be-tested face image; inputting the region corresponding to the face into a face attribute prediction network to output a predicted attribute corresponding to the face, the face attribute prediction network being used for predicting a feature of an attribute corresponding to a face in a face image, the attribute corresponding to the face including at least an age attribute and a gender attribute, and the face attribute prediction network being obtained according to the face attribute prediction network generation method in any one of claims 1 to 4.

6. The method of claim 5, wherein, The face attribute prediction network comprises a backbone network and a prediction network, the backbone network being used for extracting a feature of a face in a face image, and the prediction network being used for predicting a feature of an attribute corresponding to the face; the inputting the region corresponding to the face into the face attribute prediction network to output the predicted attribute corresponding to the face comprises: inputting the region corresponding to the face into the backbone network to output the feature of the face; inputting the feature of the face into the prediction network to output the predicted attribute corresponding to the face.

7. A face attribute prediction network generation apparatus, characterized by comprising: comprises: an obtaining module configured to obtain a sample face image set, the sample face image set comprising a plurality of sample face images; an augmentation module configured to perform augmentation processing on each sample face image to obtain a first augmented image and a second augmented image, the first augmented image and the second augmented image maintaining the same attribute as an attribute corresponding to a face in a corresponding sample face image; The training module is configured to alternately train the first branch network and the second branch network according to the plurality of first augmented images and the plurality of second augmented images, to obtain a trained first branch network, so that the trained first branch network can extract features of attributes corresponding to a face in a face image through mutual supervision of the first branch network and the second branch network, and the attributes corresponding to the face at least include an age attribute and a gender attribute. The determination module is configured to determine the trained first branch network as a face attribute prediction network.

8. An apparatus for face attribute prediction, comprising: The method comprises: The acquisition module is configured to acquire a region corresponding to a face in a to-be-tested face image. The prediction module is configured to input the region corresponding to the face into a face attribute prediction network, and output predicted attributes corresponding to the face, the face attribute prediction network being configured to predict features of attributes corresponding to the face in the face image, and the attributes corresponding to the face at least including an age attribute and a gender attribute, and the face attribute prediction network being obtained according to the face attribute prediction network generation method in any one of claims 1 to 4.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method in any one of claims 1 to 4, and / or the method in claim 5 or 6.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method in any one of claims 1 to 4, and / or the method in claim 5 or 6.

Images