A face attribute recognition method and device, electronic equipment and medium

Abstract

The present disclosure provides a face attribute recognition method and device, electronic equipment and medium, wherein the method comprises: obtaining a face image of a target object, and extracting an image feature of the face image, the image feature being a shared feature of a first attribute branch and a second attribute branch; performing regression processing on the image feature through the first attribute branch to generate attribute information of a first face attribute of the target object, and performing classification processing on the image feature through the second attribute branch to generate attribute information of a second face attribute of the target object. The technical solution provided by the present disclosure can realize lightweight deployment of the model on the basis of ensuring the face attribute recognition effect.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and specifically to a method, apparatus, electronic device, and medium for recognizing facial attributes. Background Technology

[0002] In some application scenarios, it is necessary to recognize facial attributes. For example, some scenarios require detecting whether a face is wearing a mask or the degree of makeup. Recognizing different facial attributes may correspond to different recognition tasks. For example, recognition tasks can be divided into regression tasks and classification tasks.

[0003] Currently, different recognition models can be built to achieve different recognition tasks. For example, regression models and classification models can be built separately to handle their respective recognition tasks.

[0004] However, considering the performance of actual application devices, deploying multiple recognition models on these devices often consumes a significant amount of resources, failing to meet the requirements for lightweight design. Furthermore, different recognition models often have different data processing needs, making the fusion process of these models challenging. Summary of the Invention

[0005] In view of this, one or more embodiments of this disclosure provide a method, apparatus, electronic device and medium for facial attribute recognition, which achieves lightweight deployment of the model while ensuring the facial attribute recognition effect.

[0006] This disclosure provides a method for recognizing facial attributes. The method includes: acquiring a facial image of a target object and extracting image features from the facial image, wherein the image features are shared features of a first attribute branch and a second attribute branch; performing regression processing on the image features through the first attribute branch to generate attribute information of a first facial attribute of the target object; and performing classification processing on the image features through the second attribute branch to generate attribute information of a second facial attribute of the target object. In both the regression and classification processes, the image features are downsampled and expanded in both processes, and the number of expanded first channels in the regression processing is higher than the number of expanded second channels in the classification processing.

[0007] This disclosure also provides a facial attribute recognition device, the device comprising: a feature extraction unit, configured to acquire a facial image of a target object and extract image features from the facial image, the image features being shared features of a first attribute branch and a second attribute branch; and a branch processing unit, configured to perform regression processing on the image features through the first attribute branch to generate attribute information of a first facial attribute of the target object, and to perform classification processing on the image features through the second attribute branch to generate attribute information of a second facial attribute of the target object; wherein, during the regression processing and classification processing, the branch processing unit performs downsampling and channel expansion processing on the image features, and the number of expanded first channels in the regression processing is higher than the number of expanded second channels in the classification processing.

[0008] This disclosure also provides an electronic device including a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the above-described facial attribute recognition method.

[0009] This disclosure also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described facial attribute recognition method.

[0010] This disclosure provides a technical solution through one or more embodiments, which treats regression and classification tasks as two attribute branches of a model, thereby enabling simultaneous regression and classification processing within the same model. The image features corresponding to the facial image of the target object can be shared features between the two attribute branches, avoiding redundant feature extraction processes. Subsequent processing of the extracted image features involves downsampling and channel augmentation. Since facial attribute recognition is often more difficult in regression tasks than in classification tasks, the number of augmented first channels in regression processing can be higher than the number of augmented second channels in classification processing, ensuring higher accuracy in facial attribute recognition.

[0011] As can be seen, this disclosure processes shared image features through two different attribute branches in the same model, and incorporates the recognition difficulty into the processing, and designs the attribute branches differently, thereby ensuring the facial attribute recognition effect while also achieving lightweight deployment of the model. Attached Figure Description

[0012] The features and advantages of the embodiments of this disclosure will be more clearly understood by referring to the accompanying drawings, which are illustrative and should not be construed as limiting the present disclosure in any way. In the drawings:

[0013] Figure 1 A schematic diagram of the structure of a facial attribute recognition model in one embodiment of this disclosure is shown;

[0014] Figure 2 A schematic diagram illustrating the steps of a facial attribute recognition method according to one embodiment of this disclosure is shown.

[0015] Figure 3 A schematic diagram of the processing procedure for attribute branches in one embodiment of this disclosure is shown;

[0016] Figure 4 A schematic diagram illustrating the training process steps of a facial attribute recognition model in one embodiment of this disclosure is shown.

[0017] Figure 5 A schematic diagram of the functional modules of a facial attribute recognition device according to one embodiment of the present disclosure is shown;

[0018] Figure 6 A schematic diagram of the structure of an electronic device according to one embodiment of the present disclosure is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0020] This disclosure provides one embodiment of a facial attribute recognition model; please refer to [link to relevant documentation]. Figure 1 The recognition model can include a shared feature extraction network and two attribute branches. Each attribute branch can include a refinement module and a multilayer perceptron (MLP). The refinement modules in different attribute branches can perform differentiated processing on the image features output by the feature extraction network. The results are then passed through the MLP to output the final facial attribute recognition result.

[0021] Please see Figure 2 This disclosure provides a method for recognizing facial attributes, which may include the following steps.

[0022] S1: Obtain the facial image of the target object and extract the image features of the facial image, which are used as shared features of the first attribute branch and the second attribute branch.

[0023] S3: Regress the image features through the first attribute branch to generate attribute information of the first facial attribute of the target object, and classify the image features through the second attribute branch to generate attribute information of the second facial attribute of the target object; wherein, in both regression and classification processes, the image features are downsampled and channel augmented, and the number of augmented first channels in regression processing is higher than the number of augmented second channels in classification processing.

[0024] In this embodiment, the image features of the facial image can be obtained through... Figure 1 The feature extraction network shown extracts the features. In practical applications, the feature extraction network can be flexibly configured according to actual needs. For example, VGGNet or ResNet can be used.

[0025] Generally, image features extracted by feature extraction networks are often downsampled to some extent. For example, if the resolution of the original facial image is H*W, where H represents the height of the facial image and W represents the width of the facial image, then the resolution of the extracted image features can be (H / 32)*(W / 32).

[0026] In practical applications, the facial image of the target object can correspond to the scenario in which the recognition model is applicable. For example, if the recognition model is mainly used to recognize facial images in a vehicle environment, then the facial image of the target object can be an image taken in a vehicle environment. Or, if the recognition model performs well in both vehicle and typical outdoor environments, then the facial image of the target object can be an image taken in either a vehicle or outdoor environment.

[0027] In this embodiment, the image features extracted by the feature extraction network can be used as shared features for the subsequent first attribute branch and second attribute branch. The first attribute branch can be used to perform a regression task, and the second attribute branch can be used to perform a classification task.

[0028] Specifically, please combine Figure 1 and Figure 3 In order to reduce the amount of data to be processed, the first and second attribute branches can downsample the image features when processing shared image features, thus obtaining downsampled features with lower feature dimensions. In a specific application example, this downsampling process can be implemented using a depthwise convolutional layer with a stride of 2. For example, this depthwise convolutional layer can be a 3*3 convolutional kernel (conv3*3).

[0029] In related regression or classification processes, downsampled features can be directly pooled to obtain pooled features without changing the number of channels. However, each channel in a pooled feature typically corresponds to only a one-dimensional value. Directly compressing two-dimensional downsampled features into one-dimensional values ​​results in the loss of significant feature information. Therefore, to ensure the accuracy of facial attribute recognition, this embodiment does not directly perform pooling after obtaining the downsampled features. Instead, it first performs channel expansion on the downsampled features. The feature dimension in each channel can remain unchanged, but since the number of channels is expanded, the final feature dimension also increases. Specifically, to maintain the feature dimension in each channel, a 1*1 convolutional kernel (conv1*1) can be used to process the downsampled features. The number of expanded channels can be determined by setting the number of filters.

[0030] In this embodiment, after downsampling and channel expansion processing, image features with expanded channel count can be obtained. These expanded channel count image features are then reshaped into one-dimensional features, which can be input into a multilayer perceptron for processing, and the multilayer perceptron outputs facial attribute information.

[0031] Specifically, suppose the dimension of the image feature after the number of channels is expanded is (H / 64, W / 64, 2C), where H / 64 represents the height of the downsampled image feature, W / 64 represents the width of the downsampled image feature, 2C represents doubling the number of channels, and C is the number of channels before the expansion. Reshaping this three-dimensional image feature yields a one-dimensional feature of length H / 64*W / 64*2C, which retains the feature information from the original three-dimensional image feature.

[0032] Taking the image features in any channel as an example, suppose the image features in the channel can be represented by the following 3*3 matrix:

[0033]

[0034] After reshaping the image features in this channel, we can obtain the following one-dimensional feature with a length of 9:

[0035] (A,B,C,D,E,F,G,H,J)

[0036] Subsequently, similar one-dimensional features can be obtained for the image features in each channel. By concatenating the one-dimensional features of each channel in the channel order, a one-dimensional feature with a length of H / 64*W / 64*2C can be obtained.

[0037] As described above, in one embodiment of this disclosure, after obtaining the downsampled image features, pooling is not performed directly on the downsampled image features. Instead, channel expansion is performed first, and then the expanded channel image features are reconstructed into one-dimensional features. The advantage of this approach is that it avoids compressing the image features within a single channel into a single value, preserving the image features within that channel. Simultaneously, it expands the number of channels, increasing the richness of the features. Furthermore, when reconstructing into one-dimensional features, feature information is not lost, ensuring the accuracy of facial attribute recognition.

[0038] In one implementation, considering that facial attribute recognition is often more difficult in regression tasks than in classification tasks, the number of expanded first channels in regression processing can be higher than the number of expanded second channels in classification processing. For example, if the number of channels in the second attribute branch can be expanded to 2C, then the number of channels in the first attribute branch can be expanded to 4C. This ensures the accuracy of facial attribute recognition.

[0039] Please see Figure 1 and Figure 3 In one implementation, when performing regression processing on image features through the first attribute branch to generate attribute information for the first facial attribute, the image features can be downsampled according to a specified convolution stride, and the number of channels of the downsampled image features can be increased. The specified convolution stride can be 2, and downsampling can be implemented using a conv3*3 depthwise convolutional layer. The process of increasing the number of channels can be implemented using a conv1*1 convolutional kernel, and by setting the number of filtering components, the number of channels of the downsampled image features can be increased to 4C.

[0040] In this embodiment, the image features with expanded channel count can be reshaped into one-dimensional features. After the one-dimensional features are input into the multilayer perceptron, the distribution probability of the one-dimensional features over multiple discrete intervals of the first facial attribute representation can be calculated.

[0041] Specifically, although the first attribute branch performs a regression task, in this embodiment, the regression task can be transformed into a multi-class classification task containing multiple discrete intervals. For example, the first attribute branch needs to identify the degree of makeup in a facial image, which can have a preset numerical range. For example, if the numerical range is an integer from 1 to 100, then the regression task can actually be transformed into a multi-class classification task containing 100 discrete intervals. Each discrete interval can correspond to a specified integer from 1 to 100, and the specified integers corresponding to any two discrete intervals are unique. In this way, these 100 discrete intervals can serve as the discrete intervals representing the first facial attribute (degree of makeup).

[0042] In this embodiment, the reshaped one-dimensional feature is processed by a multilayer perceptron to obtain the probability distribution of the one-dimensional feature across various discrete intervals. Specifically, the multilayer perceptron may include fully connected layers and activation layers. After processing the one-dimensional feature, the fully connected layer yields logits (corrected values) of length N, where N represents the total number of discrete intervals. Subsequently, these logits of length N can be processed by the sigmoid or softmax function in the activation layer to obtain the final probability distribution. This probability distribution can be a vector of length N, where each element corresponds to the predicted probability value for each discrete interval.

[0043] After obtaining the aforementioned probability distribution, attribute information for the first facial attribute of the target object can be generated based on the probability distribution and the regression values ​​represented by the discrete intervals. Specifically, for any discrete interval, the predicted probability value of the probability distribution within that discrete interval is identified, and the attribute weight value of the discrete interval is determined based on the predicted probability value and the discrete values ​​represented by the discrete interval. For example, assuming the predicted probability value of the i-th discrete interval is pi, and the discrete value represented by the i-th discrete interval is i, then the attribute weight value of that discrete interval could be i*pi.

[0044] After obtaining the attribute weight values ​​for each discrete interval, the attribute information of the first facial attribute of the target object can be generated based on these values. Specifically, the attribute weight values ​​for each discrete interval can be added together, and the result can be used as the attribute information of the first facial attribute.

[0045] Please see Figure 1 and Figure 3 In one embodiment, when classifying image features through the second attribute branch to generate attribute information for the second facial attributes, the image features can also be downsampled according to a specified convolution stride, and the number of channels in the downsampled image features can be increased. Similarly, the image features with increased channels can be reshaped into one-dimensional features. This process is similar to the processing in the aforementioned embodiments and will not be described in detail again. However, when increasing the number of channels, different numbers of filtering components can be set to obtain different numbers of channels after expansion.

[0046] In this embodiment, after obtaining the reshaped one-dimensional features, a multilayer perceptron can be used to calculate the classification result of the one-dimensional features for the second facial attributes. The multilayer perceptron can also include fully connected layers and activation layers. After processing the one-dimensional features, the fully connected layers produce logits (corrected values) of length M, where M represents the number of pre-defined attribute categories. Subsequently, these M-length logits can be processed by the sigmoid or softmax function in the activation layers to obtain the classification result of the one-dimensional features for the second facial attributes. This classification result can be a probability vector, where each probability value corresponds one-to-one with an attribute category. Thus, based on the attribute categories of the second facial attributes represented by the classification result, attribute information of the target object's second facial attributes can be generated. For example, the highest probability value can be determined from the classification result, and the attribute category corresponding to this highest probability value can be used as the attribute category of the second facial attributes represented by the classification result. This attribute category can then be used to generate attribute information for the second facial attributes. For example, in a scenario where a face is being detected as to whether it is wearing a mask, the second facial attribute can be the mask itself, and the corresponding attribute category can be "wearing" or "not wearing." In some scenarios, it may also include attribute categories such as "unrecognizable." Thus, the process of detecting whether a face is wearing a mask can be viewed as a binary classification process (assuming the attribute categories are only "wearing" and "not wearing"). If the final classification result indicates that the probability value of the "wearing" attribute category is the highest, then attribute information representing the "wearing" attribute category can be output. This attribute information could be, for example, the value "1," while the attribute information representing the "not wearing" attribute category could be the value "0." Of course, the above explanation of attribute categories and attribute information is merely an example in a specific scenario. Those skilled in the art can flexibly interpret the specific meaning of attribute categories and attribute information according to the actual needs of the scenario, and all such interpretations should be included within the scope of this disclosure.

[0047] As described above, while the regression task in the first attribute branch can ultimately be transformed into a multi-class classification task, it differs somewhat from the classification task in the second attribute branch. For example, determining the attribute information of the first facial attribute involves a weighted sum of the probability distribution and the regression value; however, determining the attribute information of the second facial attribute only requires finding the attribute category with the highest probability value. The reasons for this difference will be explained during the subsequent training process.

[0048] In one embodiment, the step of extracting image features from the facial image can be implemented by a feature extraction network shared by the first attribute branch and the second attribute branch. This feature extraction network, as well as the first and second attribute branches, can be pre-trained, thereby enabling accurate facial attribute information to be obtained in the above embodiment. Please refer to [link to relevant documentation]. Figure 4 The training process includes the following steps.

[0049] S21: Obtain a set of facial image samples, wherein the facial image samples in the set of facial image samples have first standard information of the first facial attribute and second standard information of the second facial attribute that have been labeled.

[0050] S23: Extract sample features of the facial image sample through the feature extraction network.

[0051] S25: Perform regression processing on the sample features through the first attribute branch to generate first prediction information of the first facial attribute, and perform classification processing on the sample features through the second attribute branch to generate second prediction information of the second facial attribute.

[0052] S27: Based on the first prediction information and the first standard information, generate first error information of the first facial attribute, and based on the second prediction information and the second standard information, generate second error information of the second facial attribute.

[0053] S29: Generate multi-task error information based on the first error information and the second error information, and use the multi-task error information to correct the feature extraction network, the first attribute branch and the second attribute branch.

[0054] The first standard information and the second standard information can respectively represent the actual attribute information of the first facial attribute and the second facial attribute. After the model is trained, the attribute information output by the first attribute branch and the second attribute branch should correspond to the first standard information and the second standard information.

[0055] During training, the processing of sample features by the first and second attribute branches is similar to the processing of image features by the first and second attribute branches in the aforementioned embodiments. Specifically, when generating the first prediction information of the first facial attribute, the sample features can be downsampled according to a specified convolution stride, and the number of channels of the downsampled sample features can be increased. Then, the sample features with increased channel count can be reshaped into one-dimensional features, and the predicted distribution probability of the one-dimensional features on multiple discrete intervals represented by the first facial attribute can be calculated. Subsequently, the first prediction information of the first facial attribute can be generated based on the predicted distribution probability and the regression values ​​represented by the discrete intervals. Specifically, for any discrete interval, the predicted probability value of the predicted distribution probability in the discrete interval can be identified, and the attribute weight value of the discrete interval can be determined based on the predicted probability value and the discrete values ​​represented by the discrete interval. Then, the first prediction information of the first facial attribute is generated according to the attribute weight values ​​of each discrete interval.

[0056] The specific implementation methods of each of the above steps can be found in the corresponding descriptions in the foregoing implementation methods, and will not be repeated here.

[0057] In this embodiment, during the training of the regression and classification tasks, error information can be determined based on different rules. For the regression task, as described in the preceding steps, it can be converted into a multi-classification task with multiple discrete intervals. Unlike traditional classification tasks, regression tasks typically correspond to a numerical interval, which could be an interval representing the degree of makeup or the complexity of facial expressions. The first standard information in the regression task corresponds to a target value within this numerical interval, and a standard distribution probability related to the first standard information can be constructed based on this target value. This standard distribution probability can characterize the distribution pattern of the attribute information of the first facial attribute. In practical applications, this distribution pattern can be represented by a Gaussian distribution. For example, assuming the first facial attribute represents the degree of makeup, the numerical interval corresponding to this degree of makeup can be an integer from 1 to 100. In a set of facial image samples, if the degree of makeup of a target object is pre-labeled as 60, then the attribute information of the first facial attribute predicted by the first attribute branch should conform to a Gaussian distribution with an expected value of 60. Therefore, when generating the first error information of the first facial attribute, a standard distribution probability related to the first standard information can be constructed first. Specifically, the expected value of the standard attribute representing the first standard information can be used to construct a Gaussian distribution with a preset constant as the variance. The preset constant can be flexibly set according to the specific type of the first facial attribute, and this disclosure does not impose any limitations on it. Subsequently, by normalizing the constructed Gaussian distribution, the standard distribution probability related to the first standard information can be obtained.

[0058] As described in the aforementioned implementation, the first prediction information output by the first attribute branch can be the predicted distribution probability of the first facial attribute across multiple discrete intervals. Theoretically, the predicted distribution probability represented by this first prediction information should be the same as the standard distribution probability described above. However, since the first attribute branch has not been fully corrected during training, an error will occur between the predicted distribution probability and the standard distribution probability. This error can be used for back-correction of the model. Specifically, distribution error information can be generated based on the standard distribution probability and the predicted distribution probability. This distribution error information can be calculated using KL divergence (relative entropy), where the two parameters of KL divergence can be the standard distribution probability and the predicted distribution probability described above.

[0059] The calculated distribution error information can be used as part of the error information of the first attribute branch when performing the regression task. In addition, there is also attribute value error information between the predicted attribute value represented by the first prediction information and the standard attribute value represented by the first standard information. This attribute value error information can be represented by MAE (Mean Absolute Error) and / or MSE (Mean-Square Error).

[0060] In this embodiment, after obtaining the distribution error information and the attribute value error information, the first error information of the first facial attribute can be generated by weighted summation based on the distribution error information and the attribute value error information.

[0061] For the second attribute branch, since it performs the classification task, the second error information between the classification result represented by the second predicted information and the classification result represented by the second standard information can be calculated using the cross-entropy loss function. After obtaining the error information of each of the two attribute branches, the weights of the first and second error information can be determined respectively. Based on the determined weights, the first and second error information are weighted and summed to obtain multi-task error information used to correct the feature extraction network, the first attribute branch, and the second attribute branch. The weights of different error information can be flexibly selected according to actual needs, and this disclosure does not impose any limitations on this.

[0062] In this embodiment, after correcting the feature extraction network, the first attribute branch, and the second attribute branch using a large number of facial image samples, the corrected feature extraction network, the first attribute branch, and the second attribute branch can be used to extract features from the input facial image to be identified, generate attribute information of the first facial attribute, and generate attribute information of the second facial attribute, respectively.

[0063] In one implementation, the facial attribute recognition model described above typically needs to be applicable to specific scenarios while also being compatible with general scenarios. For example, the facial attribute recognition model needs to be applicable to in-vehicle scenarios while also being compatible with typical outdoor scenarios. Therefore, to better recognize facial images in different scenarios, facial image samples from different scenarios can be collected and mixed according to a specified ratio to form a facial image sample set. The recognition model trained in this way has better versatility and improves the efficiency of facial attribute recognition.

[0064] This disclosure provides a technical solution through one or more embodiments, which treats regression and classification tasks as two attribute branches of a model, thereby enabling simultaneous regression and classification processing within the same model. The image features corresponding to the facial image of the target object can be shared features between the two attribute branches, avoiding redundant feature extraction processes. Subsequent processing of the extracted image features involves downsampling and channel augmentation. Since facial attribute recognition is often more difficult in regression tasks than in classification tasks, the number of augmented first channels in regression processing can be higher than the number of augmented second channels in classification processing, ensuring higher accuracy in facial attribute recognition.

[0065] As can be seen, this disclosure processes shared image features through two different attribute branches in the same model, and incorporates the recognition difficulty into the processing, and designs the attribute branches differently, thereby ensuring the facial attribute recognition effect while also achieving lightweight deployment of the model.

[0066] Please see Figure 5 One embodiment of this disclosure provides a facial attribute recognition device, the device comprising:

[0067] The feature extraction unit 100 is used to acquire a facial image of a target object and extract image features from the facial image, wherein the image features are shared features of the first attribute branch and the second attribute branch.

[0068] The branch processing unit 200 is used to perform regression processing on the image features through the first attribute branch to generate attribute information of the first facial attribute of the target object, and to perform classification processing on the image features through the second attribute branch to generate attribute information of the second facial attribute of the target object.

[0069] In the process of performing regression processing and classification processing, the branch processing unit performs downsampling and channel expansion on the image features. In the regression processing, the number of the first channel after expansion is higher than the number of the second channel after expansion in the classification processing.

[0070] The specific processing logic of each functional module can be found in the description of the aforementioned method implementation method, and will not be repeated here.

[0071] The various units described in the above embodiments can be implemented by a computer chip or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0072] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0073] Please see Figure 6 One embodiment of this disclosure also provides an electronic device, which includes a memory and a processor. The memory is used to store a computer program, which, when executed by the processor, implements the above-described facial attribute recognition method.

[0074] One embodiment of this disclosure also provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the above-described facial attribute recognition method.

[0075] The processor can be a central processing unit (CPU). It can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0076] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods in the above-described embodiments.

[0077] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0078] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0079] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments of apparatus, devices, and storage media are basically similar to method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0080] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

[0081] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for recognizing facial attributes, characterized in that, The method includes: Acquire a facial image of the target object and extract image features from the facial image, which are used as shared features of the first attribute branch and the second attribute branch; The image features are regressed using the first attribute branch to generate attribute information of the first facial attribute of the target object, and the image features are classified using the second attribute branch to generate attribute information of the second facial attribute of the target object. In both regression and classification processes, the image features were downsampled and expanded, and the number of the first expanded channel in regression processing was higher than the number of the second expanded channel in classification processing. The step of extracting image features from the facial image is implemented by a feature extraction network shared by the first attribute branch and the second attribute branch; The feature extraction network, the first attribute branch, and the second attribute branch are trained in the following manner: Obtain a set of facial image samples, wherein the facial image samples in the set of facial image samples have first standard information of a first facial attribute and second standard information of a second facial attribute that have been labeled; The feature extraction network extracts sample features from the facial image samples; The sample features are regressed using the first attribute branch to generate first prediction information for the first facial attribute, and the sample features are classified using the second attribute branch to generate second prediction information for the second facial attribute. Based on the first prediction information and the first standard information, first error information of the first facial attribute is generated, and based on the second prediction information and the second standard information, second error information of the second facial attribute is generated. Multi-task error information is generated based on the first error information and the second error information, and the feature extraction network, the first attribute branch, and the second attribute branch are corrected using the multi-task error information.

2. The method according to claim 1, characterized in that, The attribute information of the first facial attribute of the target object is generated by performing regression processing on the image features through the first attribute branch, including: The image features are downsampled according to the specified convolution stride, and the number of channels of the downsampled image features is increased. The image features with expanded channel count are reshaped into one-dimensional features, and the distribution probability of the one-dimensional features over multiple discrete intervals represented by the first facial attribute is calculated. Based on the probability distribution and the regression values ​​represented by the discrete interval, attribute information of the first facial attribute of the target object is generated.

3. The method according to claim 2, characterized in that, The attribute information used to generate the first facial attribute of the target object includes: For any discrete interval, identify the predicted probability value of the distribution probability in the discrete interval, and determine the attribute weight value of the discrete interval based on the predicted probability value and the discrete value represented by the discrete interval. The attribute information of the first facial attribute of the target object is generated based on the attribute weight values ​​of each discrete interval.

4. The method according to claim 1, characterized in that, The attribute information of the second facial attribute of the target object is generated by classifying the image features through the second attribute branch, including: The image features are downsampled according to the specified convolution stride, and the number of channels of the downsampled image features is increased. The image features with increased channel count are reshaped into one-dimensional features, and the classification result of the one-dimensional features for the second facial attribute is calculated. Based on the attribute category of the second facial attribute represented by the classification result, the attribute information of the second facial attribute of the target object is generated.

5. The method according to claim 1, characterized in that, The first prediction information for generating the first facial attribute includes: The sample features are downsampled according to the specified convolution stride, and the number of channels of the downsampled sample features is increased. The sample features with expanded channel count are reshaped into one-dimensional features, and the predicted distribution probability of the one-dimensional features on multiple discrete intervals represented by the first facial attribute is calculated. Based on the predicted distribution probability and the regression values ​​represented by the discrete interval, the first predicted information of the first facial attribute is generated.

6. The method according to claim 5, characterized in that, The first prediction information for generating the first facial attribute includes: For any discrete interval, identify the predicted probability value of the predicted distribution probability in the discrete interval, and determine the attribute weight value of the discrete interval based on the predicted probability value and the discrete value represented by the discrete interval. First prediction information for the first facial attribute is generated based on the attribute weight values ​​of each of the discrete intervals.

7. The method according to claim 5, characterized in that, The first error information for generating the first facial attribute includes: Construct a standard distribution probability related to the first standard information, and generate distribution error information based on the standard distribution probability and the predicted distribution probability; Based on the predicted attribute value represented by the first prediction information and the standard attribute value represented by the first standard information, attribute value error information is generated. Based on the distribution error information and the attribute value error information, the first error information of the first facial attribute is generated.

8. The method according to claim 1, characterized in that, Generating multi-task error information based on the first error information and the second error information includes: The weights of the first error information and the second error information are determined respectively, and the first error information and the second error information are weighted and summed based on the determined weights to obtain the multi-task error information.

9. The method according to claim 1, characterized in that, The collection of facial image samples includes: Collect facial image samples from different scenarios and mix them according to a specified ratio to form a facial image sample set.

10. A facial attribute recognition device, characterized in that, The device includes: The feature extraction unit is used to acquire a facial image of the target object and extract the image features of the facial image, wherein the image features are shared features of the first attribute branch and the second attribute branch. The branch processing unit is configured to perform regression processing on the image features through the first attribute branch to generate attribute information of the first facial attribute of the target object, and to perform classification processing on the image features through the second attribute branch to generate attribute information of the second facial attribute of the target object. In the process of performing regression processing and classification processing, the branch processing unit performs downsampling and channel expansion on the image features, and the number of the first channel after expansion in regression processing is higher than the number of the second channel after expansion in classification processing. The step of extracting image features from the facial image is implemented by a feature extraction network shared by the first attribute branch and the second attribute branch; The feature extraction network, the first attribute branch, and the second attribute branch are trained in the following manner: Obtain a set of facial image samples, wherein the facial image samples in the set of facial image samples have first standard information of a first facial attribute and second standard information of a second facial attribute that have been labeled; The feature extraction network extracts sample features from the facial image samples; The sample features are regressed using the first attribute branch to generate first prediction information for the first facial attribute, and the sample features are classified using the second attribute branch to generate second prediction information for the second facial attribute. Based on the first prediction information and the first standard information, first error information of the first facial attribute is generated, and based on the second prediction information and the second standard information, second error information of the second facial attribute is generated. Multi-task error information is generated based on the first error information and the second error information, and the feature extraction network, the first attribute branch, and the second attribute branch are corrected using the multi-task error information.

11. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being used to store a computer program that, when executed by the processor, implements the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 9.

Images