Facial attribute recognition method and device, electronic equipment and medium

Abstract

The invention provides a facial attribute recognition method and device, electronic equipment and a medium, and the method comprises the steps: obtaining a facial image of a target object, extracting the image features of the facial image, and enabling the image features to serve as the shared features of a first attribute branch and a second attribute branch; and performing regression processing on the image features through the first attribute branch to generate attribute information of a first facial attribute of the target object, and performing classification processing on the image features through the second attribute branch to generate attribute information of a second facial attribute of the target object. According to the technical scheme provided by the invention, lightweight deployment of the model can be realized on the basis of ensuring the face attribute recognition effect.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a facial attribute recognition method, device, electronic device and medium. Background Art

[0002] In some application scenarios, it is necessary to identify the facial attributes of a person's face. For example, in some scenarios, it is necessary to detect whether the face is wearing a mask, or to detect the degree of makeup on the face. Recognizing different facial attributes may correspond to different recognition tasks. For example, the recognition task can be divided into regression tasks and classification tasks.

[0003] At present, different recognition models can be established when implementing different recognition tasks. For example, a regression model and a classification model can be established separately to handle their respective recognition tasks.

[0004] However, considering the performance of actual application devices, if multiple recognition models are deployed in the application device, it will often consume a lot of resources of the application device and fail to meet the lightweight requirements. Different recognition models often have different requirements for data processing, which also makes the fusion process of recognition models difficult. Summary of the invention

[0005] In view of this, one or more embodiments of the present disclosure provide a facial attribute recognition method, device, electronic device and medium, which achieve lightweight deployment of the model while ensuring the facial attribute recognition effect.

[0006] On the one hand, the present disclosure provides a method for recognizing facial attributes, the method comprising: acquiring a facial image of a target object, and extracting image features of the facial image, the image features being used as shared features of a first attribute branch and a second attribute branch; performing regression processing on the image features through the first attribute branch to generate attribute information of a first facial attribute of the target object, and performing classification processing on the image features through the second attribute branch to generate attribute information of a second facial attribute of the target object; wherein, in the process of regression processing and classification processing, the image features are downsampled and channel expanded, and the number of the expanded first channels in the regression processing is higher than the number of the expanded second channels in the classification processing.

[0007] On the other hand, the present disclosure also provides a facial attribute recognition device, the device comprising: a feature extraction unit, used to obtain a facial image of a target object and extract image features of the facial image, the image features serving as shared features of a first attribute branch and a second attribute branch; a branch processing unit, used to perform regression processing on the image features through the first attribute branch to generate attribute information of a first facial attribute of the target object, and to perform classification processing on the image features through the second attribute branch to generate attribute information of a second facial attribute of the target object; wherein the branch processing unit downsamples and expands channels on the image features during the regression and classification processes, and the number of expanded first channels in the regression process is higher than the number of expanded second channels in the classification process.

[0008] On the other hand, the present disclosure further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned facial attribute recognition method is implemented.

[0009] On the other hand, the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned facial attribute recognition method is implemented.

[0010] The technical solution provided by one or more embodiments of the present disclosure can use the regression task and the classification task as two attribute branches of the model, so that regression processing and classification processing can be implemented simultaneously in the same model. Among them, the image features corresponding to the facial image of the target object can be used as shared features of the two attribute branches, thereby avoiding redundant feature extraction processes. When the extracted image features are subsequently processed separately, they can all be processed by downsampling and channel expansion. Among them, since the recognition difficulty of facial attributes in the regression task is often higher than the recognition difficulty of facial attributes in the classification task, the number of the expanded first channels in the regression processing can be higher than the number of the expanded second channels in the classification processing, thereby ensuring the recognition accuracy of facial attributes.

[0011] It can be seen that the present invention processes shared image features through two different attribute branches in the same model, and takes recognition difficulty into consideration during the processing to differentiate the attribute branches, thereby achieving lightweight deployment of the model while ensuring the facial attribute recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The features and advantages of the embodiments of the present disclosure may be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0013] Figure 1 A schematic diagram showing the structure of a facial attribute recognition model in one embodiment of the present disclosure is shown;

[0014] Figure 2 A schematic diagram showing the steps of a method for recognizing facial attributes in one embodiment of the present disclosure is shown;

[0015] Figure 3 A schematic diagram of a processing process of an attribute branch in one embodiment of the present disclosure is shown;

[0016] Figure 4 A schematic diagram showing the steps of a training process for a facial attribute recognition model in one embodiment of the present disclosure is shown;

[0017] Figure 5 A schematic diagram showing the functional modules of a facial attribute recognition device in one embodiment of the present disclosure is shown;

[0018] Figure 6 A schematic structural diagram of an electronic device in one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0020] An embodiment of the present disclosure provides a recognition model for facial attributes. Figure 1 The recognition model may include a shared feature extraction network and two attribute branches. Each attribute branch may include a refinement module (refine) and a multi-layer perceptron (MLP). The refinement modules in different attribute branches may perform differential processing on the image features output by the feature extraction network. The processed results may be passed through the multi-layer perceptron to output the final recognition result of the facial attributes.

[0021] See also Figure 2 An embodiment of the present disclosure provides a method for recognizing facial attributes, which may include the following steps.

[0022] S1: Acquire a facial image of a target object, and extract image features of the facial image, wherein the image features are used as shared features of a first attribute branch and a second attribute branch.

[0023] S3: regressing the image features through the first attribute branch to generate attribute information of the first facial attributes of the target object, and classifying the image features through the second attribute branch to generate attribute information of the second facial attributes of the target object; wherein, in the regression processing and the classification processing, the image features are downsampled and channel expanded, and the number of the expanded first channels in the regression processing is higher than the number of the expanded second channels in the classification processing.

[0024] In this embodiment, the image features of the facial image can be Figure 1 The feature extraction network shown in the figure is extracted. In practical applications, the feature extraction network can be flexibly set according to actual needs. For example, a network such as VGGNet or ResNet can be used to implement it.

[0025] Generally speaking, the image features extracted by the feature extraction network are often downsampled to a certain extent. For example, the resolution of the original facial image is H*W, where H can represent the height of the facial image and W can represent the width of the facial image. Then the resolution of the extracted image features can be (H / 32)*(W / 32).

[0026] In practical applications, the facial image of the target object may correspond to the scene to which the recognition model is applicable. For example, if the recognition model is mainly used to recognize facial images in a vehicle environment, then the facial image of the target object may be an image taken in a vehicle environment. For another example, if the recognition model has good recognition effects in both a vehicle environment and a common outdoor environment, then the facial image of the target object may be an image taken in a vehicle environment or an outdoor environment.

[0027] In this embodiment, the image features extracted by the feature extraction network can be used as shared features of the subsequent first attribute branch and the second attribute branch. The first attribute branch can be used to implement the regression task, and the second attribute branch can be used to implement the classification task.

[0028] Specifically, please combine Figure 1 and Figure 3 , when the first attribute branch and the second attribute branch process the shared image features, in order to reduce the amount of data to be processed, the image features may be downsampled first, thereby obtaining downsampled features with lower feature dimensions. In a specific application example, the downsampling process may be implemented by a depth-wise convolution layer with a convolution stride of 2. For example, the depth-wise convolution layer may be a 3*3 convolution kernel (conv3*3).

[0029] In the relevant regression processing or classification processing process, for the down-sampled features, pooling processing can be directly performed to obtain pooled features without changing the number of channels. However, each channel in the pooled features usually corresponds to only a one-dimensional value. Directly compressing the two-dimensional down-sampled features into a one-dimensional value will lose more feature information. In view of this, in order to ensure the accuracy of facial attribute recognition, in this embodiment, after obtaining the down-sampled features, pooling processing is not directly performed, but the down-sampled features are first processed by channel expansion, and the feature dimension in each channel can remain unchanged. Since the number of channels is expanded, the feature dimension finally obtained will also increase. Specifically, in order to keep the feature dimension in the channel unchanged, a 1*1 convolution kernel (conv1*1) can be used to process the down-sampled features, and the number of channels after expansion can be determined by setting the number of filter components (filters).

[0030] In this embodiment, after downsampling and channel expansion processing, image features with expanded channel numbers can be obtained. After the image features with expanded channel numbers are reshaped into one-dimensional features, they can be input into a multi-layer perceptron for processing, and the attribute information of facial attributes can be output through the multi-layer perceptron.

[0031] Specifically, assuming that the dimension of the image feature with expanded channels is (H / 64, W / 64, 2C), where H / 64 represents the height of the downsampled image feature, W / 64 represents the width of the downsampled image feature, 2C represents the number of channels expanded to twice the original number, and C represents the number of channels before the channel expansion. Then, by reshaping the three-dimensional image feature, a one-dimensional feature with a length of H / 64*W / 64*2C can be obtained, and the feature information in the original three-dimensional image feature is not lost in the one-dimensional feature.

[0032] Taking the image features in any channel as an example, assuming that the image features in the channel can be represented by the following 3*3 matrix:

[0033]

[0034] Then after reshaping the image features in this channel, the following one-dimensional features with a length of 9 can be obtained:

[0035] (A,B,C,D,E,F,G,H,J)

[0036] Subsequently, similar one-dimensional features can be obtained for the image features in each channel. By concatenating the one-dimensional features of each channel in the order of the channels, a one-dimensional feature with a length of H / 64*W / 64*2C can be obtained.

[0037] From the above description, it can be seen that in one embodiment of the present disclosure, after obtaining the downsampled image features, the downsampled image features are not directly pooled, but first expanded through channels, and then the image features with the expanded number of channels are reshaped into one-dimensional features. The advantage of this processing is that the image features in a channel will not be compressed into a numerical value, but the image features in the channel are retained, and the number of channels can be expanded at the same time, the richness of the features is improved, and the feature information will not be lost when it is subsequently reshaped into a one-dimensional feature, thereby ensuring the recognition accuracy of facial attributes.

[0038] In one embodiment, considering that the difficulty of recognizing facial attributes in a regression task is often higher than that in a classification task, the number of expanded first channels in the regression process can be higher than the number of expanded second channels in the classification process. For example, the number of channels in the second attribute branch can be expanded to 2C, and the number of channels in the first attribute branch can be expanded to 4C. This can ensure the recognition accuracy of facial attributes.

[0039] See also Figure 1 and Figure 3 In one embodiment, when the image feature is regressed through the first attribute branch to generate the attribute information of the first facial attribute, the image feature can be downsampled according to the specified convolution step size, and the number of channels of the downsampled image feature can be expanded. The specified convolution step size can be 2, and the downsampling can be achieved through a conv3*3 deep convolution layer. The process of expanding the number of channels can be achieved through a conv1*1 convolution kernel, and the number of channels of the downsampled image feature can be expanded to 4C by setting the number of filter components.

[0040] In this embodiment, the image feature with expanded channels can be reshaped into a one-dimensional feature. After the one-dimensional feature is input into a multi-layer perceptron, the distribution probability of the one-dimensional feature in multiple discrete intervals represented by the first facial attribute can be calculated.

[0041] Specifically, although the first attribute branch implements a regression task, in this embodiment, the regression task can be converted into a multi-classification task containing multiple discrete intervals. For example, the first attribute branch needs to identify the degree of makeup in the facial image, and the degree of makeup can have a preset numerical interval. For example, if the numerical interval is an integer from 1 to 100, then the regression task can actually be converted into a multi-classification task containing 100 discrete intervals. Among them, each discrete interval can correspond to a specified integer from 1 to 100, and the specified integers corresponding to any two discrete intervals are not repeated. In this way, these 100 discrete intervals can be used as discrete intervals to represent the first facial attribute (degree of makeup).

[0042] In this embodiment, after the reshaped one-dimensional feature is processed by a multi-layer perceptron, the distribution probability of the one-dimensional feature in each discrete interval can be obtained. Specifically, the multi-layer perceptron may include a fully connected layer and an activation layer, wherein the fully connected layer processes the one-dimensional feature to obtain logits (corrected value) of length N, where N can represent the total number of discrete intervals, and subsequently, the logits of length N can be processed by the sigmoid function or the softmax function in the activation layer to obtain the final distribution probability. The distribution probability can be a vector of length N, and each element in the vector can correspond to the predicted probability value of each discrete interval.

[0043] After obtaining the above-mentioned distribution probability, the attribute information of the first facial attribute of the target object can be generated based on the distribution probability and the regression value represented by the discrete interval. Specifically, for any discrete interval, the predicted probability value of the distribution probability in the discrete interval is identified, and the attribute weight value of the discrete interval is determined based on the predicted probability value and the discrete value represented by the discrete interval. For example, assuming that the predicted probability value of the i-th discrete interval is pi, and the discrete value represented by the i-th discrete interval is i, then the attribute weight value of the discrete interval can be i*pi.

[0044] After obtaining the attribute weight values ​​of each discrete interval, the attribute information of the first facial attribute of the target object can be generated according to the attribute weight values ​​of each discrete interval. Specifically, the attribute weight values ​​of each discrete interval can be added together, and the result of the addition is used as the attribute information of the first facial attribute.

[0045] See also Figure 1 and Figure 3 In one embodiment, when the image features are classified and processed by the second attribute branch to generate the attribute information of the second facial attribute, the image features can also be downsampled according to the specified convolution step size, and the number of channels of the downsampled image features can be expanded, and the image features with the expanded number of channels can also be reshaped into one-dimensional features. This part of the process is similar to the processing process in the aforementioned embodiment, and will not be repeated here, except that when the number of channels is expanded, different numbers of channels after expansion can be obtained by setting different numbers of filter components.

[0046] In this embodiment, after obtaining the reshaped one-dimensional feature, the classification result of the one-dimensional feature for the second facial attribute can be calculated by a multi-layer perceptron. Among them, the multi-layer perceptron can also include a fully connected layer and an activation layer. After the fully connected layer processes the one-dimensional feature, a logits (corrected value) of length M can be obtained, where M can represent the number of pre-set attribute categories. Subsequently, the logits of length M can be processed by a sigmoid function or a softmax function in the activation layer to obtain the classification result of the one-dimensional feature for the second facial attribute. The classification result can be a probability vector, and each probability value in the probability vector can correspond to an attribute category one by one. In this way, based on the attribute category of the second facial attribute represented by the classification result, the attribute information of the second facial attribute of the target object can be generated. For example, the maximum probability value can be determined from the classification result, and then the attribute category corresponding to the maximum probability value can be used as the attribute category of the second facial attribute represented by the classification result, and the attribute category can be used to generate the attribute information of the second facial attribute. For example, in the scenario of detecting whether the face is wearing a mask, the second facial attribute may be a mask, and the attribute categories corresponding to the second facial attribute may be "wearing" and "not wearing". Of course, some scenarios may also include attribute categories such as "unrecognizable". In this way, the process of detecting whether the face is wearing a mask can be regarded as a binary classification process (assuming that the attribute categories are only "wearing" and "not wearing"). If the final classification result represents that the probability value of the attribute category "wearing" is the highest, then the attribute information representing the attribute category "wearing" can be output. The attribute information can be, for example, a value of "1", and the attribute information representing the attribute category "not wearing" can be a value of "0". Of course, the above explanations of attribute categories and attribute information are only examples in specific scenarios. Those skilled in the art can flexibly interpret the specific meanings of attribute categories and attribute information according to actual scenario requirements, and these explanations should be included in the protection scope of this disclosure.

[0047] From the above description, it can be seen that although the regression task in the first attribute branch can also be converted into a multi-classification task in the end, it is still different from the classification task in the second attribute branch. For example, when determining the attribute information of the first facial attribute, the weighted sum of the distribution probability and the regression value is used; when determining the attribute information of the second facial attribute, it is only necessary to find the attribute category with the largest probability value. The reason for this difference will be explained in the subsequent training process.

[0048] In one embodiment, the step of extracting the image features of the facial image can be implemented by a feature extraction network shared by the first attribute branch and the second attribute branch. The feature extraction network, the first attribute branch and the second attribute branch can be pre-trained, so that accurate attribute information of facial attributes can be obtained in the above embodiment. Figure 4 ,The training process includes the following steps.

[0049] S21: Acquire a facial image sample set, wherein the facial image samples in the facial image sample set have first standard information of a first facial attribute and second standard information of a second facial attribute that have been labeled.

[0050] S23: Extracting sample features of the facial image sample through the feature extraction network.

[0051] S25: performing regression processing on the sample features through the first attribute branch to generate first prediction information of the first facial attribute, and performing classification processing on the sample features through the second attribute branch to generate second prediction information of the second facial attribute.

[0052] S27: Generate first error information of the first facial attribute based on the first prediction information and the first standard information, and generate second error information of the second facial attribute based on the second prediction information and the second standard information.

[0053] S29: Generate multi-task error information according to the first error information and the second error information, and use the multi-task error information to correct the feature extraction network, the first attribute branch, and the second attribute branch.

[0054] Among them, the first standard information and the second standard information can respectively represent the actual attribute information of the first facial attribute and the second facial attribute. After the model is trained, the attribute information output by the first attribute branch and the second attribute branch should correspond to the first standard information and the second standard information.

[0055] During the training process, the processing process of the sample features by the first attribute branch and the second attribute branch is similar to the processing process of the image features by the first attribute branch and the second attribute branch in the aforementioned embodiment. Specifically, when generating the first prediction information of the first facial attribute, the sample features can be downsampled according to the specified convolution step size, and the number of channels of the downsampled sample features can be expanded. Then, the sample features with the expanded number of channels can be reshaped into a one-dimensional feature, and the predicted distribution probability of the one-dimensional feature on multiple discrete intervals represented by the first facial attribute can be calculated. Subsequently, the first prediction information of the first facial attribute can be generated based on the predicted distribution probability and the regression value represented by the discrete interval. Among them, for any discrete interval, the predicted probability value of the predicted distribution probability in the discrete interval can be identified, and the attribute weight value of the discrete interval can be determined based on the predicted probability value and the discrete value represented by the discrete interval, and then the first prediction information of the first facial attribute is generated according to the attribute weight value of each discrete interval.

[0056] For the specific implementation of each of the above steps, please refer to the corresponding description in the previous implementation manner, which will not be repeated here.

[0057] In this embodiment, during the training process of the regression task and the classification task, respective error information can be determined based on different rules. Among them, for the regression task, the regression task can be converted into a multi-classification task of multiple discrete intervals according to the description in the above steps. Different from the traditional classification task, the regression task usually corresponds to a numerical interval, which can be, for example, an interval of makeup degree or an interval of expression complexity. The first standard information in the regression task will correspond to a target value in the numerical interval, and a standard distribution probability related to the first standard information can be constructed based on the target value. The standard distribution probability can characterize the distribution law of the attribute information of the first facial attribute. In practical applications, the distribution law can be represented by a Gaussian distribution. For example, assuming that the first facial attribute characterizes the makeup degree, the numerical interval corresponding to the makeup degree can be an integer from 1 to 100. In the facial image sample set, the makeup degree of a target object is pre-marked as 60, then the attribute information of the first facial attribute predicted by the first attribute branch should conform to the Gaussian distribution with 60 as the expectation. In view of this, when generating the first error information of the first facial attribute, the standard distribution probability related to the first standard information can be constructed first. Specifically, the standard attribute value represented by the first standard information can be taken as an expectation, and a preset constant can be taken as a variance to construct a Gaussian distribution. The preset constant can be flexibly set according to the specific type of the first facial attribute, and the present disclosure does not limit this. Subsequently, by normalizing the constructed Gaussian distribution, the standard distribution probability related to the first standard information can be obtained.

[0058] According to the description of the aforementioned implementation, the first prediction information output by the first attribute branch can be the predicted distribution probability of the first facial attribute in multiple discrete intervals. Theoretically, the predicted distribution probability represented by the first prediction information should be the same as the above-mentioned standard distribution probability, but because the first attribute branch has not been fully corrected during the training process, it will cause an error between the predicted distribution probability and the standard distribution probability, and this error can be used to reversely correct the model. Specifically, distribution error information can be generated based on the standard distribution probability and the predicted distribution probability. The distribution error information can be calculated by KL divergence (relative entropy), and the two parameters of KL divergence can be the above-mentioned standard distribution probability and predicted distribution probability.

[0059] The calculated distribution error information can be used as part of the error information of the first attribute branch when performing the regression task. In addition, there will be attribute value error information between the predicted attribute value represented by the first prediction information and the standard attribute value represented by the first standard information. This attribute value error information can be represented by MAE (Mean Absolute Error) and / or MSE (Mean-Square Error).

[0060] In this implementation, after the distribution error information and the attribute value error information are obtained, the first error information of the first facial attribute can be generated according to the distribution error information and the attribute value error information by weighted summation.

[0061] For the second attribute branch, due to the classification task completed by the second attribute branch, the second error information between the classification result represented by the second prediction information and the classification result represented by the second standard information can be calculated by the cross-entropy loss function (Cross-Entropy Loss). After obtaining the error information of each of the two attribute branches, the weights of the first error information and the second error information can be determined respectively, and the first error information and the second error information are weighted and summed based on the determined weights to obtain multi-task error information for correcting the feature extraction network, the first attribute branch, and the second attribute branch. Among them, the weights of different error information can be flexibly selected according to actual needs, and the present disclosure does not limit this.

[0062] In this embodiment, after the feature extraction network, the first attribute branch and the second attribute branch are calibrated through a large number of facial image samples, the calibrated feature extraction network, the first attribute branch and the second attribute branch can be used to perform feature extraction, generate attribute information of the first facial attribute and generate attribute information of the second facial attribute on the input facial image to be identified.

[0063] In one embodiment, the above-mentioned facial attribute recognition model usually has the requirement to be applicable to specific scenarios and compatible with general scenarios. For example, the facial attribute recognition model needs to be applicable to vehicle-mounted scenarios and compatible with conventional outdoor scenarios. Then, in order to better recognize facial images in different scenarios, when selecting facial image samples, facial image samples in different scenarios can be collected, and the facial image samples in different scenarios can be mixed according to a specified ratio to form a facial image sample set. The recognition model trained in this way can have good versatility and improve the recognition efficiency of facial attributes.

[0064] The technical solution provided by one or more embodiments of the present disclosure can use the regression task and the classification task as two attribute branches of the model, so that regression processing and classification processing can be implemented simultaneously in the same model. Among them, the image features corresponding to the facial image of the target object can be used as shared features of the two attribute branches, thereby avoiding redundant feature extraction processes. When the extracted image features are subsequently processed separately, they can all be processed by downsampling and channel expansion. Among them, since the recognition difficulty of facial attributes in the regression task is often higher than the recognition difficulty of facial attributes in the classification task, the number of the expanded first channels in the regression processing can be higher than the number of the expanded second channels in the classification processing, thereby ensuring the recognition accuracy of facial attributes.

[0065] It can be seen that the present invention processes shared image features through two different attribute branches in the same model, and takes recognition difficulty into consideration during the processing to differentiate the attribute branches, thereby achieving lightweight deployment of the model while ensuring the facial attribute recognition effect.

[0066] See also Figure 5 In one embodiment of the present disclosure, a facial attribute recognition device is provided, the device comprising:

[0067] The feature extraction unit 100 is used to obtain a facial image of a target object and extract image features of the facial image, wherein the image features are used as shared features of the first attribute branch and the second attribute branch;

[0068] The branch processing unit 200 is used to perform regression processing on the image feature through the first attribute branch to generate attribute information of a first facial attribute of the target object, and to perform classification processing on the image feature through the second attribute branch to generate attribute information of a second facial attribute of the target object;

[0069] Wherein, the branch processing unit downsamples and expands channels of the image features during the regression and classification processes, and the number of expanded first channels in the regression process is higher than the number of expanded second channels in the classification process.

[0070] Among them, the specific processing logic of each functional module can be found in the description of the aforementioned method implementation method, which will not be repeated here.

[0071] Each unit described in the above embodiments may be implemented by a computer chip or a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0072] For the convenience of description, the above devices are described in terms of functions and are described separately in various units. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0073] See also Figure 6 An embodiment of the present disclosure further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned facial attribute recognition method is implemented.

[0074] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned facial attribute recognition method is implemented.

[0075] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0076] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0077] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0078] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0079] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0080] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0081] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for recognizing facial attributes, characterized in that: The method comprises: Acquire a facial image of a target object, and extract image features of the facial image, wherein the image features are used as shared features of the first attribute branch and the second attribute branch; Performing regression processing on the image feature through the first attribute branch to generate attribute information of a first facial attribute of the target object, and performing classification processing on the image feature through the second attribute branch to generate attribute information of a second facial attribute of the target object; In the process of regression processing and classification processing, the image features are downsampled and channel expanded, and the number of the expanded first channels in the regression processing is higher than the number of the expanded second channels in the classification processing.

2. The method according to claim 1, characterized in that Performing regression processing on the image feature through the first attribute branch to generate attribute information of the first facial attribute of the target object includes: Down-sampling the image features according to the specified convolution step size, and expanding the number of channels of the down-sampled image features; Reshape the image features with expanded channel numbers into one-dimensional features, and calculate the distribution probability of the one-dimensional features in multiple discrete intervals represented by the first facial attribute; Based on the distribution probability and the regression value represented by the discrete interval, attribute information of the first facial attribute of the target object is generated.

3. The method according to claim 2, characterized in that Generating attribute information of the first facial attribute of the target object includes: For any discrete interval, identifying a predicted probability value of the distribution probability in the discrete interval, and determining an attribute weight value of the discrete interval based on the predicted probability value and a discrete numerical value represented by the discrete interval; Attribute information of a first facial attribute of the target object is generated according to the attribute weight values ​​of each discrete interval.

4. The method according to claim 1, characterized in that: Classifying the image feature through the second attribute branch to generate attribute information of the second facial attribute of the target object includes: Down-sampling the image features according to the specified convolution step size, and expanding the number of channels of the down-sampled image features; Reshape the image features with expanded channels into one-dimensional features, and calculate the classification result of the one-dimensional features for the second facial attribute; Based on the attribute category of the second facial attribute represented by the classification result, attribute information of the second facial attribute of the target object is generated.

5. The method according to any one of claims 1 to 4, characterized in that: The step of extracting image features of the facial image is implemented by a feature extraction network shared by the first attribute branch and the second attribute branch; The feature extraction network, the first attribute branch, and the second attribute branch are trained in the following manner: Acquire a facial image sample set, wherein the facial image samples in the facial image sample set have first standard information of a first facial attribute and second standard information of a second facial attribute that have been annotated; Extracting sample features of the facial image sample through the feature extraction network; Performing regression processing on the sample features through the first attribute branch to generate first prediction information of the first facial attribute, and performing classification processing on the sample features through the second attribute branch to generate second prediction information of the second facial attribute; generating first error information of the first facial attribute based on the first prediction information and the first standard information, and generating second error information of the second facial attribute based on the second prediction information and the second standard information; Multi-task error information is generated according to the first error information and the second error information, and the multi-task error information is used to correct the feature extraction network, the first attribute branch, and the second attribute branch.

6. The method according to claim 5, characterized in that Generating first prediction information of the first facial attribute includes: Down-sampling the sample features according to the specified convolution step size, and expanding the number of channels of the down-sampled sample features; Reshape the sample features with the expanded number of channels into one-dimensional features, and calculate the predicted distribution probability of the one-dimensional features in multiple discrete intervals represented by the first facial attribute; Based on the predicted distribution probability and the regression value represented by the discrete interval, first prediction information of the first facial attribute is generated.

7. The method according to claim 6, characterized in that Generating first prediction information of the first facial attribute includes: For any discrete interval, identifying a predicted probability value of the predicted distribution probability in the discrete interval, and determining an attribute weight value of the discrete interval based on the predicted probability value and a discrete numerical value represented by the discrete interval; First prediction information of the first facial attribute is generated according to the attribute weight value of each discrete interval.

8. The method according to claim 6, characterized in that Generating first error information of the first facial attribute includes: Constructing a standard distribution probability related to the first standard information, and generating distribution error information based on the standard distribution probability and the predicted distribution probability; generating attribute value error information according to the predicted attribute value represented by the first prediction information and the standard attribute value represented by the first standard information; First error information of the first facial attribute is generated based on the distribution error information and the attribute value error information.

9. The method according to claim 5, characterized in that Generating multi-task error information according to the first error information and the second error information includes: The weights of the first error information and the second error information are determined respectively, and the first error information and the second error information are weighted summed based on the determined weights to obtain the multi-task error information.

10. The method according to claim 5, characterized in that Obtaining a facial image sample set includes: Facial image samples under different scenes are collected, and the facial image samples under different scenes are mixed according to a specified ratio to form a facial image sample set.

11. A facial attribute recognition device, characterized in that: The device comprises: A feature extraction unit, used to obtain a facial image of a target object and extract image features of the facial image, wherein the image features are used as shared features of the first attribute branch and the second attribute branch; a branch processing unit, configured to perform regression processing on the image feature through the first attribute branch to generate attribute information of a first facial attribute of the target object, and to perform classification processing on the image feature through the second attribute branch to generate attribute information of a second facial attribute of the target object; Wherein, the branch processing unit downsamples and expands channels of the image features during the regression processing and the classification processing, and the number of the expanded first channels in the regression processing is higher than the number of the expanded second channels in the classification processing.

12. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images