Age and gender estimation

Abstract

A method of age and gender estimation, comprising: receiving an input image; detecting a face image within the input image; estimating a head pose based on a set of face image intensities of the face image, wherein the head pose is represented as yaw, pitch, and roll; determining whether the yaw, pitch, and roll of the head pose are less than a predetermined threshold; if the yaw, pitch, and roll of the head pose are less than the predetermined threshold, aligning the face image; and predicting an age and gender of the aligned face image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to human attribute recognition, and more specifically to human age and gender recognition. BACKGROUND

[0002] Currently, age and gender estimation has proven to be difficult, in part based on insufficient datasets with age and gender labels, expression variations, pose variations, and illumination variations, and often uses a large amount of computational resources to perform adequately.

[0003] Predicting human age and gender, two of the intrinsic facial attributes without constraints, can increase in importance in real-world applications including human-computer interaction and autonomous driving performed with more constrained computational resources. SUMMARY

[0004] An example method of age and gender estimation includes receiving an input image, detecting a facial image within the input image, estimating a head pose based on a set of facial image intensities of the facial image, wherein the head pose is represented as yaw, pitch, and roll, determining whether the yaw, pitch, and roll of the head pose are less than a predetermined threshold, aligning the facial image if the yaw, pitch, and roll of the head pose are less than the predetermined threshold, and predicting an age and a gender of the aligned facial image. BRIEF DESCRIPTION OF DRAWINGS

[0005] In the drawings:

[0006] Figure 1 is a first example system diagram according to one embodiment of the present disclosure;

[0007] Figure 2 is a second example system diagram according to one embodiment of the present disclosure;

[0008] Figure 3 is an example flow of age and gender estimation according to one embodiment of the present disclosure;

[0009] Figure 4 is an example of an age and gender estimation network according to one embodiment of the present disclosure;

[0010] Figure 5 is an example of age and gender estimation using various convolutional neural network models and various dropout deletions according to one embodiment of the present disclosure;

[0011] Figure 6 is a second example of an age and gender estimation network according to one embodiment of the present disclosure;

[0012] Figure 7 is an example of age and gender estimation using various convolutional neural network models according to one embodiment of the present disclosure; and

[0013] Figure 8 is an example method of age and gender estimation according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] The embodiments listed below are merely to illustrate applications of the apparatus and methods and do not limit the scope. Modifications of equivalents of the apparatus and methods should be classified as within the scope of the claims.

[0015] Throughout the following description and in the claims, certain terminology is used to refer to particular system components. As will be understood by those skilled in the art, different companies can refer to components and / or methods by different names. The present document is not intended to distinguish between components and / or methods that have different names but function identically.

[0016] In the following discussion and in the claims, the terms "including" and "comprising" are used in an open-ended fashion, and thus can be construed to mean "including, but not limited to...." Additionally, the term "coupled" or "coupling" is intended to mean either an indirect or direct connection. Thus, if devices are coupled, that connection can be through some other device or interface.

[0017] Figure 1 An example hybrid computing system 100 that can be used to implement a neural network associated with operations of one or more portions or steps of a process is depicted. In this example, the processors associated with the hybrid system include a Field Programmable Gate Array (FPGA) 122, a Graphics Processing Unit (GPU) 120, and a Central Processing Unit (CPU) 118.

[0018] CPU 118, GPU 120, and FPGA 122 have the ability to provide neural networks. The CPU is a general purpose processor that can perform many different functions, its general purpose results in the ability to perform multiple different tasks, however, its processing of multiple data streams is limited and it is limited in its functionality relative to neural networks. The GPU is a graphics processor that has many small processing cores that are able to process parallel tasks in sequence. The FPGA is a field programmable device that has the ability to be reconfigured and perform any functionality that can be programmed into a CPU or GPU in the manner of a hardwired circuit. Because the programming of the FPGA is in the form of a circuit, it is many times faster than a CPU and much faster than a GPU.

[0019] There are other types of processors that a system can include, such as Accelerated Processing Units (APUs) that include CPUs with on-chip GPU elements, and Digital Signal Processors (DSPs) that are designed to perform high speed numerical data processing. Application Specific Integrated Circuits (ASICs) can also perform the hardwired functionality of the FPGA; however, the lead time to design and produce an ASIC is on the order of one year in quarters, as opposed to the fast turnaround implementation available in programming a FPGA.

[0020] The graphics processor unit 120, central processing unit 118, and field programmable gate array 122 are connected and connected to the memory interface and controller 112. The FPGA is connected to the memory interface through a programmable logic circuit to memory interconnect 180. This additional device is used due to the fact that the FPGA operates at very large bandwidths and to minimize the circuitry used from the FPGA to perform memory tasks. The memory interface and controller 112 is additionally connected to the persistent storage disk 110, system memory 114, and read only memory (ROM) 116.

[0021] Figure 1 The system of FIG. 1 can be used to program and train the FPGA. The GPU runs well with unstructured data and the GPU can be used to train, once the data has been trained, a deterministic inference model can be found, and the CPU can program the FPGA with the model data determined by the GPU.

[0022] The memory interface and controller is connected to a central interconnect 124 that is additionally connected to the GPU 120, CPU 118, and FPGA 122. The central interconnect 124 is additionally connected to an input and output interface 128 and a network interface 126.

[0023] Figure 2 A second example hybrid computing system 200 that can be used to implement a neural network associated with operations of one or more portions or steps of a process is depicted. In this example, the processors associated with the hybrid system include a field programmable gate array (FPGA) 210 and a central processing unit (CPU) 220.

[0024] The FPGA is electrically connected to an FPGA controller 212 that interfaces with a direct memory access (DMA) 218. The DMA is connected to an input buffer 214 and an output buffer 216 that are coupled to the FPGA to buffer data into and out of the FPGA, respectively. The DMA 218 includes two first input first output (FIFO) buffers, one for the host CPU and the other for the FPGA, that enable data to be written to and read from the appropriate buffer.

[0025] On the CPU side of the DMA is a host switch 228 that transfers data and commands to the DMA. The DMA is also connected to an SDRAM controller 224 that enables data to go to and from the FPGA to the CPU 220, which is also connected to an external SDRAM 226 and the CPU 220. The host switch 228 is connected to a peripheral interface 280. A flash controller 222 controls the persistent memory and is connected to the CPU 220.

[0026] The present disclosure describes a computationally efficient framework that can estimate age and gender to predict age and gender from red-blue-green (RGB) images and infrared (IR) images or videos. A head pose estimation module controls the head pose of the subject prior to performing age and gender estimation. A large dataset combination is used when training the model to avoid missing age groups, and the accuracy of the proposed model can be improved to enable different network architectures to trade off preference between higher accuracy and faster computation.

[0027] Age and gender classification can play an increasingly large role in computer vision, especially since the rise of autonomous driving. Given the vast variation in unconstrained images, manually designed features used by traditional methods are not satisfactory. Convolutional neural networks have alleviated this problem to some extent due to their powerful feature representation. To enable high accuracy with Convolutional Neural Networks (CNNs), the general trend is for more complex and computationally heavy systems, increasing the computational cost. One possible challenge in deploying large models in robotics, self-driving cars, and embedded vision applications is the limited computational capacity of embedded platforms, which hinders their use in computationally heavy and deep CNNs. Current models pursuing lightweight computation use come at the cost of accuracy, and one factor to pursue is to improve performance while limiting computational cost.

[0028] To train based on large-scale datasets and effectively trade off between computation and accuracy, an age and gender estimation framework based on a lightweight deep learning neural network is presented, which is based on four hybrid large public datasets as training data with wide range of age labels, gender annotations.

[0029] Structural adjustments have been made to current age and gender CNNs, including dropout deletion, changing the backbone, loss function, and age distribution representation.

[0030] A head pose estimation system controls the processing of age and gender estimation based on the subject's head pose. Due to the limitations of available datasets and inherent challenges caused by occlusions in large head poses, age and gender estimation is limited to near-frontal head poses. If the yaw, pitch, or roll of the head pose is greater than their threshold, age and gender estimation will not be processed.

[0031] In Figure 3The process of the proposed age and gender estimation solution is shown in 300. In the pre-processing regarding face detection 310, five-point facial landmarks are used to detect the subject's face. From the detected face, the subject's head pose can be estimated 312 using the predicted intrinsic Euler angles (i.e., yaw, pitch, and roll), based on which the system decides whether the age and gender estimation module should be processed. If the Euler angles (i.e., yaw, pitch, and roll) are within the threshold, face alignment 314 is performed. The system uses the five-point landmarks to first perform face alignment to establish correspondence between different faces. The aligned face images are input into a convolutional neural network 316 to complete the feature extraction task that presents age and gender prediction 318. In the CNN setup, we improved the original plain CNN and replaced it with ResNet50 and EfficientNetB4.

[0032] To train the CNN model, four public face datasets are fused for the data pre-processing stage to achieve data balancing and improve diversity.

[0033] Figure 4 An example of training and testing 400 is depicted, and the neural variance of the feature map can be caused by the dropout unit and batch normalization (BN) placed before the fully connected layer. The mismatch of variance can cause numerical instability.

[0034] Figure 5 It is indicated 500 that the model can achieve higher accuracy without using dropout. In subsequent examples, the dropout and batch normalization before the fully connected layer are removed partially due to variance shift. In one example, the dropout is moved after batch normalization, and it can be indicated that the removal of dropout results in higher performance of the original model.

[0035] To determine more accurate classification for large age and gender datasets, in other example implementations, the original plain backbone is replaced with ResNet-50-C4 and EfficientNetB4. Both ResNet-50-C4 and EfficientNetB4 indicate significantly improved performance. Such network architectures can be used when the embedded platform has more computational resources. Within the age and gender model, the original two-point distribution is replaced with a fully connected layer inserted between the feature layer and the regression layer. This example modification to the fully connected layer allows for age estimation with a Gaussian distribution and achieves the best results.

[0036] Figure 6Two example models 600 are depicted, in one model a face image 610 is input into a ResNet50 backbone 612, a feature map 614 is found, and a two-point distribution 616 is used for age and gender modeling in a fully connected layer, which is then sent to a regression layer 618.

[0037] Figure 6 A second example model inputs a face image 620 into an EfficientNetB4 backbone 622, determines a feature map 624, a Gaussian distribution 626 is used for age and gender modeling, which is then sent to a regression layer 628.

[0038] Figure 7 A comparison table 700 between a computationally light backbone model and two heavier backbone models is depicted.

[0039] Figure 8 An example method of age and gender estimation is depicted, the example method comprising receiving 810 an input image, detecting 812 a face image within the input image, and estimating 814 a head pose based on a set of face image intensities of the face image, wherein the head pose is represented as a yaw, a pitch, and a roll. The method further comprises determining 816 whether the yaw, the pitch, and the roll of the head pose are less than a predetermined threshold, aligning 818 the face image if the yaw, the pitch, and the roll of the head pose are less than the predetermined threshold, and predicting 820 an age and a gender of the aligned face image.

[0040] The method can further include retransforming and / or reorganizing a set of inverse transform coefficients of an inverse block process. The entropy decoding can be processed by at least one of a Huffman decoding and an arithmetic decoding. The dequantization factor applied to the dequantized can be based on a coefficient threshold and / or based on a set of coefficient frequencies. The inverse block process can perform reassembly of the dequantized dataset into a set of regular blocks. The inverse transform can include a luminance, a blue-green difference, a red-green difference, and a first green to second green difference. The decompressed image dataset can include at least three color channels.

[0041] Those skilled in the art will appreciate that the various illustrative blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or combinations of both. To illustrate the interchangeability of hardware and software, various illustrative blocks, modules, elements, components, methods, and algorithms have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application. Various components and blocks can be arranged differently (e.g., arranged in a different order, or partitioned in a different way) all without affecting the overall

[0042] It should be understood that the particular order or hierarchy of steps in the processes disclosed is an example. Based upon design preferences, it should be understood that specific order or hierarchy of steps in the processes can be rearranged, some can be performed simultaneously, the

[0043] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. The previous description provides various examples of the subject technology, and the subject technology is not limited to these examples. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather "one or more." Unless specifically stated otherwise, the term "some" refers to one or more. Singular participles including other grammatical numbers can convey the equivalent of the plural, and vice versa. Headings and subheadings, if any, are used for convenience only and do not limit the application. The recitation "configured to," "operable to," and "programmed to" do not mean any specific tangible or intangible modification to the subject, but rather mean the subject is merely capable of implementing, performing, or using the functions recited in any art-recognized manner. For example, a processor configured to monitor and control operations or components can also mean a processor programmed to monitor and control operations or a processor operable to monitor and control operations. Likewise, a processor configured to execute code can be interpreted as a processor programmed to execute code or a processor operable to execute code.

[0044] Phrases such as “aspect” do not mean that the aspect is essential to the subject technology or that the aspect applies to configurations of the subject technology. Disclosures in connection with an aspect can apply to configurations, or one or more configurations. An aspect can provide one or more examples. Phrases such as “aspect” can refer to one or more aspects, and vice versa. Phrases such as “embodiment” do not mean that the embodiment is essential to the subject technology or that the embodiment applies to configurations of the subject technology. Disclosures in connection with an embodiment can apply to embodiments, or one or more embodiments. An embodiment can provide one or more examples. Phrases such as “embodiment” can refer to one or more embodiments, and vice versa. Phrases such as “configuration” do not mean that the configuration is essential to the subject technology or that the configuration applies to configurations of the subject technology. Disclosures in connection with a configuration can apply to configurations, or one or more configurations. A configuration can provide one or more examples. Phrases such as “configuration” can refer to one or more configurations, and vice versa.

[0045] The use of the word “example” herein means “serving as an example or illustration.” Any aspect or design described herein as “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0046] The structure and function of elements of the various aspects described throughout this disclosure that are known or will be known to those of ordinary skill in the art are not explicitly described herein, but are intended to be included in the description, and are specifically contemplated to be included in the claims. Moreover, nothing disclosed herein is intended to be disclosed to the public unless explicitly recited in the claims. Moreover, with respect to the terms “comprising,” “having,” and the like, such terms are meant to encompass the items listed after such terms as well as other items. For example, a composition “comprising” an ingredient is meant to include having only the ingredient, as well as having the ingredient combined with other ingredients.

[0047] References to “one implementation,” “an implementation,” “some implementations,” “various implementations,” and the like indicate that the particular element or characteristic is included in at least one implementation of the present invention. These phrases do not necessarily refer to the same implementation, although they can. It is possible that one or more implementations can include elements or characteristics not specifically mentioned in connection with one or more other implementations. To the extent that a particular element or characteristic is expressly included in any claim, or is equivalent to an element or characteristic expressly included in an claim, it is hereby incorporated by reference into each of the claims.

[0048] It is understood that the present disclosure teaches only one example of an illustrative implementation and that many variations of the invention can be readily devised by those skilled in the art following the teachings herein, and that the scope of the invention is to be determined by the appended claims.

Claims

1. A method of age and gender estimation, characterized by, Comprising: receiving an input image; detecting a face image within the input image; estimating a head pose based on a set of face image intensities of the face image, wherein the head pose is represented as yaw, pitch, and roll; determining whether the yaw, pitch, and roll of the head pose are less than a predetermined threshold; aligning the face image if the yaw, pitch, and roll of the head pose are less than the predetermined threshold; and predicting an age and a gender of the aligned face image; the prediction of the age and the gender is performed via a convolutional neural network, the convolutional neural network comprising: a batch normalization, a fully connected layer, and a second batch normalization, dropout preceding the fully connected layer removed due to variance shift.

2. The method of age and gender estimation according to claim 1, characterized in that, the alignment of the face image is performed using five point facial landmarks.

3. The method of age and gender estimation according to claim 1, characterized in that, the predetermined threshold limits the head pose to near frontal head poses.

4. The method of age and gender estimation according to claim 1, characterized in that, Further comprising: extracting features of the aligned face image.

5. The method of age and gender estimation according to claim 1, characterized in that, the convolutional neural network used is one of ResNet-50-C4 and EfficientNetB4.

6. The method of age and gender estimation according to claim 1, characterized in that, the convolutional neural network comprises: a feature map of the aligned face image, a two point distribution within a fully connected layer, and a regression layer.

7. The method of age and gender estimation according to claim 1, characterized in that, the convolutional neural network comprises: a feature map of the aligned face image, a Gaussian distribution within a fully connected layer, and a regression layer.

8. The method of age and gender estimation according to claim 1, characterized in that, the age estimation is set as a Gaussian distribution.

9. The method of age and gender estimation of claim 1, wherein, the detection of the face image is performed using five point facial landmarks.

Images