A method and system for face age estimation with gender and ethnicity information fusion

Abstract

The present application belongs to the field of pattern recognition, and particularly relates to a face age estimation method and system fusing gender and race information, comprising: establishing a face age prediction model fusing gender and race information, including feature mapping and stacking a RepPsconv module into a convolutional neural network, the feature mapping including a main module and an auxiliary module, and the specific steps are: obtaining a face image set with labels of age, gender, race and the like for preprocessing; inputting the preprocessed face image into the convolutional neural network to extract features and fuse them; connecting the fused features to the main module and the auxiliary module, and optimizing the result of the main module by the auxiliary module; solving the loss of the output result of the main module and the label information of the face image, and iteratively training until convergence; inputting a face image to be measured into the trained model, and outputting an age estimation result, and the present application proposes an efficient RepPSconv module, fuses gender and race information, and improves the precision of face age estimation.

Description

Technical Field

[0001] The present invention belongs to the field of pattern recognition, and in particular relates to a method and system for estimating human face age by integrating gender and race information. Background Art

[0002] The human face contains a wealth of valuable information related to individual characteristics, which plays a key role in face-to-face interactions. In particular, biological details such as wrinkles, freckles, age spots, hair color, face shape, facial hair, and skin texture can be used to estimate a person's age. Age estimation based on the face plays a crucial role in social life.

[0003] Age estimation for faces is performed using two approaches. Traditional face age estimation methods, based on handcrafted features, require strong prior knowledge for manual design and cannot accurately predict a person's age. Deep learning methods, based on convolutional neural networks, are used as learning-based feature representation methods. Deep learning and convolutional neural networks learn feature representations directly from raw image pixels and project the face image into another feature space. These deep learning methods, based on convolutional neural networks, can automatically extract facial features clearly and robustly from face images. They possess strong learning capabilities and good robustness, while providing an end-to-end solution and achieving good age estimation accuracy.

[0004] Although the face age estimation method based on convolutional neural networks can effectively solve the task of age group classification, there are still huge challenges in the accuracy of its age value. Differences in gender and race also have an impact on face age estimation. Summary of the Invention

[0005] To address the above issues, the present invention provides a facial age estimation method that integrates gender and race information. A facial age prediction model that integrates gender and race information is established. The model includes a feature map and a convolutional neural network stacked using a RepPsconv module. The feature map includes a main module and an auxiliary module. The facial age estimation method that integrates gender and race information includes the following steps:

[0006] Obtain a collection of face images with labels such as face age, gender, and race, and preprocess the face image collection;

[0007] Input the preprocessed face image into the convolutional neural network for feature extraction;

[0008] The features extracted from the convolutional neural network are integrated at multiple scales at a fine-grained level;

[0009] The fused features are connected to the main module and the auxiliary module using a fully connected layer, and the auxiliary module optimizes the results of the main module;

[0010] Compare the output of the main module with the face image label information to calculate the loss, and iterate the training until the loss stops decreasing.

[0011] The face image to be tested is input into the trained face age prediction model that integrates gender and race information, and the age estimation result is output.

[0012] Furthermore, the RepPsconv module includes a 3×3PSconv branch, a 1×1conv branch, and an identity branch. The RepPsconv modules are stacked into a convolutional neural network. The overall network structure includes:

[0013] The first layer is the information fusion layer. This is a hidden layer added before the face image is input into the convolutional neural network. It has two neurons preset to represent the gender and race of the face. The two neurons representing gender and race are added and fused with the pixel values ​​of the face image.

[0014] The PSconv branch in the second-layer RepPSconv module is replaced with a normal convolution operation and the Identity branch is removed;

[0015] The third layer RepPSconv module has 64 input and 64 output channels, followed by a maximum pooling layer.

[0016] The input and output channels of the fourth layer RepPSconv module are 64 and 128 respectively;

[0017] The fifth layer RepPSconv module has 128 input and 128 output channels, followed by a maximum pooling layer.

[0018] The input and output channels of the sixth layer RepPSconv module are 128 and 256 respectively;

[0019] The input and output channels of the seventh layer RepPSconv module are 256 and 256 respectively;

[0020] The eighth layer RepPSconv module has input and output channels of 256 and 256 respectively, followed by a maximum pooling layer;

[0021] The input and output channels of the ninth layer RepPSconv module are 256 and 512 respectively;

[0022] The input and output channels of the tenth layer RepPSconv module are 512 and 512 respectively;

[0023] The input and output channels of the eleventh layer RepPSconv module are 512 and 512 respectively, followed by a maximum pooling layer;

[0024] The number of input and output channels of the twelfth layer RepPSconv module are 512 and 512 respectively;

[0025] The input and output channels of the thirteenth layer RepPSconv module are 512 and 512 respectively;

[0026] The input and output channels of the fourteenth RepPSconv module are 512 and 512 respectively, followed by a maximum pooling layer.

[0027] Furthermore, the acquired gender and race information is independent and discrete, making it an auxiliary module conditional branch to assist the main module in estimating facial age.

[0028] Furthermore, the main module includes multiple sub-networks, and the output of each sub-network will be multiplied by the different probability values ​​of each auxiliary attribute conditional branch of the auxiliary module, and the probability values ​​of each conditional branch of the auxiliary module are continuously updated during the training iteration process. Among the sub-networks of the main module, the sub-network that meets the actual conditions of the face image is multiplied by the high probability value, and the remaining sub-networks are multiplied by the small probability values ​​obtained by the auxiliary module.

[0029] Furthermore, the output results of the main modules are compared with the face image label information to calculate the loss, and the loss function is expressed as:

[0030]

[0031] Where N is the mini-batch of training samples, α and β are two hyperparameters, F is the number of sub-networks in the main module, and E is the number of auxiliary attributes in the auxiliary module, such as gender and race. It can also be extended to multiple auxiliary attributes. is the loss function of each sub-network of the main module, L i,age is the loss function of the absolute error between the predicted face age and the true value, is the deviation loss function of each sub-network of the main module, is the loss function used by the auxiliary module to supervise the auxiliary attribute estimation task.

[0032] Furthermore, the loss function of each sub-network of the main module is expressed as:

[0033]

[0034]

[0035] in, The i-th face image under the f-th sub-network of the main module can correctly estimate the real age y i The probability of Indicates that the i-th face image in the f-th sub-network of the main module belongs to the real age y i The probability of It refers to the probability that the i-th face image in the f-th subnetwork in the main module belongs to age category k, where K is the total number of age categories.

[0036] Furthermore, the auxiliary module uses the cross entropy loss function to supervise the estimation of auxiliary attributes such as gender and race of face images. The loss function is expressed as:

[0037]

[0038]

[0039] in, It is to estimate the probability of the auxiliary attribute being the true category e under the f-th sub-network of the main module, represents the probability that the auxiliary attribute of the i-th face image belongs to the true category e under the f-th sub-network of the main module, It refers to the probability that the auxiliary attribute of the i-th face image under the f-th sub-network in the main module belongs to category j.

[0040] Furthermore, the deviation loss function of each sub-network of the main module considers the correlation of adjacent age categories while estimating the true age category of the face image. The deviation loss function of each sub-network of the main module is expressed as:

[0041]

[0042] in, is the face image x i The estimated probability of belonging to age category k. In this loss function, the bias term (ky i ) 2 As a penalty factor, by penalizing those age groups that are far away from the real age category, the deviation loss will guide the network to output a more reasonable probability distribution, k is the age category, y i is the true age value of the face image, and K is the total number of age categories.

[0043] Furthermore, a loss function based on the absolute error between the predicted face age and the true value is introduced to make the predicted age closer to the true age. The loss function of the absolute error is expressed as:

[0044]

[0045] in, is the predicted age, yi is the true age value of the face image.

[0046] Furthermore, predicting age The calculation formula is:

[0047]

[0048] in, represents the age value predicted in the f-th sub-network of the main module, Indicates the probability value that the age predicted in the f-th sub-network of the main module is the true age value, Indicates that the i-th face image multiplied by the f-th sub-network of the main module belongs to the corresponding auxiliary attribute The probability value of .

[0049] Furthermore, the calculation formula for predicting age in the fth sub-network of the main module is:

[0050]

[0051] in, is the probability that the face image belongs to age category k under the f-th sub-network of the main module.

[0052] A facial age estimation system integrating gender and race information includes: an image acquisition module, a data preprocessing module, an information fusion layer, a convolutional neural network training module, and an output module. The convolutional neural network training module includes a RepPSconv module and a feature mapping module. The feature mapping module includes a main module and an auxiliary module.

[0053] The image acquisition module is used to obtain a collection of face images with labels such as face age, gender, and race, as well as the face image to be tested;

[0054] The data preprocessing module is used to preprocess the data obtained by the image acquisition module, including face detection, face alignment and face cropping;

[0055] The information fusion layer is used to add and fuse the gender and race information with the pixel values ​​of the face;

[0056] The RepPSconv module is used to extract feature maps from images and use the PSconv operator to fuse information between feature maps at multiple scales at a fine-grained level.

[0057] Feature mapping module, which is used to connect the extracted feature maps to the main module and auxiliary modules using a fully connected layer;

[0058] The main module is used to estimate the age of the face image to be tested;

[0059] Auxiliary module, used to optimize the results of the main module;

[0060] The output module is used to output the age estimation result of the face image to be tested and call the camera to predict the face age in real time.

[0061] Beneficial effects of the present invention:

[0062] 1. The present invention selects independent and discrete age-related attributes, gender and race, as auxiliary attributes, and integrates gender and race information into the estimation of facial age, thereby improving the accuracy of age estimation through faces.

[0063] 2. This paper proposes an efficient RepPSconv module. Based on the PSconv operator, this module extracts multi-scale information and performs feature fusion at a fine-grained level. It uses residual connections to redirect gradients, preventing gradient vanishing and gradient explosion problems in deep convolutional neural network training. The RepPSconv module forms a new convolutional neural network in a stacked form, characterized by fast training and convergence.

[0064] 3. The present invention proposes a main module and an auxiliary module. The output of each sub-network in the main module is multiplied by the different probability values ​​of each conditional branch of the auxiliary module. During the training process, the probability values ​​of each conditional branch of the auxiliary module will be updated in the iterative process. Finally, the sub-network that is consistent with the actual conditions of the face image will be multiplied by a high probability value, while the other conditional sub-networks will be multiplied by a small probability value. The estimated face age under actual conditions has the largest weight in the predicted final age, which makes the sub-network under actual conditions tend to learn related mapping models. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of a facial age estimation method that integrates gender and race information according to the present invention;

[0066] Figure 2 Schematic diagram of the structure of the information fusion layer of the present invention;

[0067] Figure 3 Schematic diagram of the structure of the RepPSconv module of the present invention;

[0068] Figure 4 This is a schematic diagram of the backbone network structure proposed by the present invention with an information fusion layer added and stacked by multiple RepPSconv modules;

[0069] Figure 5 Schematic diagram of the general framework of the feature mapping stage of the present invention;

[0070] Figure 6A structural diagram of the feature mapping stage of the present invention;

[0071] Figure 7 A framework diagram of the overall network of the present invention;

[0072] Figure 8 A flow chart of the entire method of the present invention;

[0073] Figure 9 A prototype system operation flow chart of the present invention;

[0074] Figure 10 This is a rendering of the age prediction of a face image, one of the functions of a prototype system of the present invention;

[0075] Figure 11 This is a diagram showing the effect of real-time prediction of facial age, one of the functions of a prototype system of the present invention. DETAILED DESCRIPTION

[0076] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0077] A facial age estimation method that integrates gender and race information, such as Figure 1 As shown, the following steps are included:

[0078] Obtain a collection of face images with labels such as age, gender, and race, and preprocess the face image collection;

[0079] Input the preprocessed face image into the convolutional neural network for feature extraction;

[0080] The features extracted from the convolutional neural network are integrated at multiple scales at a fine-grained level;

[0081] The fused features are connected to the main module and the auxiliary module using a fully connected layer, and the auxiliary module optimizes the results of the main module;

[0082] The output of the main module is compared with the face image label information to solve the loss, and the training is iterated until convergence;

[0083] The face image to be tested is input into the trained face age prediction model that integrates gender and race information, and the age estimation result is output.

[0084] In one embodiment, the overall network structure for estimating facial age by integrating gender and race information includes: inputting a sample image into the information fusion layer, integrating gender and race information into the sample image and feeding the sample image into the backbone network for training, performing feature mapping on the features extracted by the backbone network, iteratively training the backbone network, and finally saving the trained facial age prediction model integrating gender and race information, such as Figure 7 As shown;

[0085] While the backbone network is being trained, data augmentation is performed on the sample images. The data augmentation includes: random aspect ratio cropping; random horizontal flipping and random rotation at an angle of 5°.

[0086] In one embodiment, the method for estimating facial age by integrating gender and race information is specifically performed as follows:

[0087] The MORPH2 dataset is one of the most popular age estimation datasets. It's a cross-temporal dataset, containing images of the same person at different ages. The dataset is available in both commercial and academic versions. The academic version includes 55,134 images of 13,000 individuals, collected between 2003 and 2007. The individuals range in age from 16 to 77, with an average age of 33. In addition to age, the MORPH2 dataset also records other information about individuals, such as gender, race, and whether they wear glasses.

[0088] In the embodiment of the present invention, an academic version of a face image collection is used, and only three labels, namely age, gender and race, are used for training. The collection is split according to the agreement of 80% training set and 20% test set.

[0089] The MORPH2 dataset is preprocessed, including: using the MTCNN algorithm to align the original face images in the dataset, and cropping the aligned images to a fixed size.

[0090] Specifically, the MTCNN algorithm is a widely used algorithm in the field of face detection. It takes into account both accuracy and speed. It aligns the detected face image based on the principle that both eyes are on the same horizontal line, and uniformly scales the aligned face image to a size of 256×256 to facilitate subsequent training in the convolutional neural network.

[0091] The preprocessed training face image samples are input into the convolutional neural network to extract features. The face image samples are used as training samples and first enter the backbone network composed of a stack of RepPSconv modules with an information fusion layer added.

[0092] The information fusion layer is a hidden layer added before the face image is input into the backbone network. It has two neurons preset to represent the gender and race information of the face respectively. These two neurons are added and fused with the pixel values ​​of the face to obtain the new pixel value. Among them, the gender neuron weight is preset to 0.5 for male and 1 for female, and the race neuron weight is preset to 0.5 for white and 1 for black. These preset weights will be adaptively learned during the iterative training of the backbone network, as shown in the schematic diagram. Figure 2 shown.

[0093] like Figure 3 As shown, the RepPsconv module includes a 3×3PSconv branch, a 1×1conv branch, and an identity branch.

[0094] An information fusion layer is added to the convolutional neural network composed of stacked RepPSconv modules. The overall network structure is as follows: Figure 4 As shown, including:

[0095] The first layer is the information fusion layer. This is a hidden layer added before the face image is input into the convolutional neural network. It has two neurons preset, representing the gender and race of the face respectively. These two neurons are added and fused with the pixel values ​​of the face.

[0096] The PSconv branch in the second-layer RepPSconv module is replaced with a normal convolution operation and the Identity branch is removed. This is because the PSconv operator needs to group the feature maps and the number of channels in each group is 4, while the number of channels in the feature map after the information fusion layer is still 3. Therefore, the PSconv branch needs to be replaced with a normal convolution to increase the number of channels to 64.

[0097] The second layer has 3 input channels and 64 output channels;

[0098] The third layer RepPSconv module has 64 input and 64 output channels, followed by a maximum pooling layer.

[0099] The input and output channels of the fourth layer RepPSconv module are 64 and 128 respectively;

[0100] The fifth layer RepPSconv module has 128 input and 128 output channels, followed by a maximum pooling layer.

[0101] The input and output channels of the sixth layer RepPSconv module are 128 and 256 respectively;

[0102] The input and output channels of the RepPSconv module from the seventh to the eighth layer are 256 and 256 respectively, followed by a maximum pooling layer;

[0103] The input and output channels of the ninth layer RepPSconv module are 256 and 512 respectively;

[0104] The input and output channels of the tenth to fourteenth layers of the RepPSconv module are 512 and 512 respectively, and the eleventh and fourteenth layers are followed by a maximum pooling layer.

[0105] All modules use a 3×3 convolution kernel with a stride of 1.

[0106] During network training, common data augmentation methods are used to expand the training data, including: (a) random cropping of preprocessed images to 224×224×3 pixel regions, (b) random horizontal flipping, and (c) random rotation of images within the range of [-5°, 5°].

[0107] In one embodiment, if Figure 5 As shown in the figure, the extracted features are connected to the main module and the auxiliary module with a fully connected layer, and the auxiliary module optimizes the results of the main module. The main module includes multiple sub-networks, each of which is a possible situation of the image. Its output will be multiplied by the different probability values ​​of each conditional branch of the auxiliary module, and the probability values ​​of each conditional branch of the auxiliary module are continuously updated during the training iteration process. Among the sub-networks of the main module, the sub-network that meets the actual conditions of the face image is multiplied by the high probability value, and the remaining sub-networks are multiplied by the small probability values ​​obtained by the auxiliary module.

[0108] like Figure 6 As shown, an image sample is input into the main module. There are four possible image samples: a white male, a black male, a white female, and a black female. The four possibilities are regarded as four sub-networks. Each sub-network is multiplied by the different probability values ​​of the gender condition branch and the different probability values ​​of the race condition branch of the auxiliary module. During the training process, the probability values ​​of the gender condition branch and the race condition branch will be continuously updated. Finally, the sub-network with the same actual conditions as the image sample will be multiplied by a high probability value, and the remaining sub-networks will be multiplied by the small probability values ​​obtained by the auxiliary module.

[0109] Whether estimating based on the main attribute face or the auxiliary attributes (gender, race), it is regarded as a multi-classification task, and the softmax cross entropy loss function is used uniformly to supervise the individual attribute estimation task. For example, given a sample face image x i , the loss function used to supervise each sub-network of the main module is:

[0110]

[0111]

[0112] in, The i-th face image under the f-th sub-network of the main module can correctly estimate the real age y i The probability of Indicates that the i-th face image in the f-th sub-network of the main module belongs to the real age y i The probability of It refers to the probability that the i-th face image in the f-th subnetwork in the main module belongs to age category k, where K is the total number of age categories.

[0113] On the other hand, age estimation should not be simply regarded as a classification problem, but as a sequential classification problem, that is, the various age classes are not independent, but interrelated. For example, for the same person, a face image at the age of 54 should be closer to a face image at the age of 52 than to a face image at the age of 20. In other words, the similarity between two face images from the same person should be inversely proportional to the age span of the two face images. If age estimation is regarded as a classification problem, the expected probability distribution of age categories should also follow this principle. However, the cross entropy loss function used for age classification cannot meet this expectation. The function only attempts to maximize the probability of estimating the ground truth age category, while ignoring the correlation between adjacent age categories. Taking this defect into account, the use of deviation loss to assist. The deviation loss function of each sub-network of the main module is expressed as:

[0114]

[0115] in, is the face image x i The estimated probability of belonging to age category k, the bias term (ky i ) 2 As a penalty factor, by penalizing age groups that are far away from the true age category, the deviation loss will guide the network to output a more reasonable probability distribution, and K is the total number of age categories.

[0116] Preferably, the auxiliary module uses a cross entropy loss function to supervise the estimation of auxiliary attributes such as gender and race of face images. The loss function is expressed as:

[0117]

[0118]

[0119] in, is the probability of estimating the auxiliary attribute to be the true category e in the f-th subnetwork of the main module, represents the probability that the auxiliary attribute of the i-th face image belongs to the true category e under the f-th sub-network of the main module, It refers to the probability that the auxiliary attribute of the i-th face image in the f-th sub-network in the main module belongs to category j. Since the calculation starts from 0 in the formula, the total number of categories J is reduced by one. In this embodiment, since gender and race are each divided into two categories in the Morph2 dataset, J is set to 1.

[0120] A loss function based on the absolute error between the predicted face age and the true value is introduced to make the predicted age closer to the true age. The loss function of the absolute error is expressed as:

[0121]

[0122] in, is the predicted age, y i is the true age value of the face image.

[0123] Finally, a total loss function composed of the above individual losses is used to supervise the training of the entire network.

[0124]

[0125] Where N is the mini-batch of training samples, α and β are two hyperparameters, F is the number of sub-networks in the main module, and E is the number of auxiliary attributes in the auxiliary module. In this method, N = 64, α = 0.1, β = 0.004, F = 4, and E = 2 can be set.

[0126] In this invention, if there is no auxiliary module to optimize the main module, age estimation is just a common multi-classification problem under the single-task learning strategy, which is equivalent to Figure 6 A branch of the main module in . Compared with single-task learning, the multi-task learning strategy can improve the performance of face age estimation. Compared with traditional single-task learning, Figure 6 The subnetworks used for face age estimation in

[15] tend to provide more accurate predictions, especially the subnetwork under condition 1. In addition, the method can also predict auxiliary attributes such as gender and race with high accuracy.

[0127] Figure 5 This is a general framework for the feature mapping stage of the embodiment of the present invention, which can be extended to multiple auxiliary attributes. Figure 6 This is just an example of an embodiment of the present invention. Since the Morph2 dataset only provides the main attribute age label and the auxiliary attributes gender and race labels, if the acquired dataset includes or also has labels such as height, weight, BMI index, etc., the number of branches of the sub-network f in the main module can be adjusted according to the number of auxiliary attributes, and the number of branches of the corresponding auxiliary module should also be adjusted.

[0128] The present invention also provides a facial age estimation system that integrates gender and race information. The system includes: an image acquisition module, a data preprocessing module, an information fusion layer, a convolutional neural network training module, and an output module. The convolutional neural network training module includes a RepPSconv module and a feature mapping module. The feature mapping module includes a main module and an auxiliary module.

[0129] The image acquisition module is used to obtain a collection of face images with labels such as face age, gender, and race, as well as the face image to be tested;

[0130] The data preprocessing module is used to preprocess the data obtained by the image acquisition module, including face detection, face alignment and face cropping;

[0131] The information fusion layer is used to add and fuse the gender and race information with the pixel values ​​of the face;

[0132] The RepPSconv module is used to extract feature maps from images and use the PSconv operator to fuse information between feature maps at multiple scales at a fine-grained level.

[0133] Feature mapping module, which is used to connect the extracted feature maps to the main module and auxiliary modules using a fully connected layer;

[0134] The main module is used to estimate the age of the face image to be tested;

[0135] Auxiliary module, used to optimize the results of the main module;

[0136] The output module is used to output the age estimation result of the face image to be tested and call the camera to predict the face age in real time.

[0137] In another embodiment, a prototype system for facial age estimation that integrates gender and race information is designed based on the output module. The system includes two major functions: 1) predicting age from facial images; and 2) using a camera to predict facial age in real time.

[0138] In some embodiments, the present invention can use SGD optimizer for training adjustment, and can also use learning rate decay strategy. After multiple rounds of training, the convolutional neural network tends to be stable and the iterative training ends. The flowchart of the whole method is as follows: Figure 8 As shown:

[0139] After obtaining a dataset of facial images with age, gender, and race labels, preprocess the facial images;

[0140] Construct a convolutional neural network model with an information fusion layer and stacked by multiple RepPSconv modules;

[0141] Train the convolutional neural network using the dataset and perform multiple iterations;

[0142] Compare the output of the convolutional neural network with the face image label information to calculate the loss until the loss stabilizes;

[0143] At this point, the training is finished and the trained neural network model is saved.

[0144] Face age estimation process, such as Figure 9 As shown, select the function option. If you use pictures to predict age, input the original image, determine whether the file format is a picture, if so, check whether the face in the picture can be seen, otherwise re-input the original image, detect the face, load the face age prediction model that integrates gender and race information, and output the prediction result; Figure 10 As shown in the figure, after inputting the original face image on the left, the original image is preprocessed with face detection, face alignment, random cropping, color perturbation, random flipping, and scaling to a fixed size to make the input more diverse, thereby enhancing the generalization ability of the model; the processed original image is input into the face age estimation network that integrates gender and race information for feature extraction and outputs the face age estimation result. As can be seen from the image on the right, the final output includes the position of the face in the image and the corresponding face age estimation result.

[0145] If the camera is used to predict age, it is determined whether the camera is successfully called. If so, the face age prediction model integrating gender and race information is loaded to perform face age prediction and output the age prediction result in real time. Figure 11 As shown, the camera is called to obtain data and perform age prediction.

[0146] The face age estimation method provided by the present invention can be applied to the following scenarios:

[0147] 1. Improve surveillance intelligence and retrieval capabilities. Today, cameras are ubiquitous in our lives, with numerous surveillance cameras installed in every public place. The amount of video information captured is enormous, and relying solely on manpower and simple algorithms to retrieve this information would be extremely time-consuming. Age estimation from facial images can address this problem and be applied in many scenarios, such as monitoring minors in special locations, vending machine monitoring for sales research, and organizations using SkyEye to search and identify specific groups of people. Supermarkets can use facial age estimation to count the number of customers of different age groups in real time, thereby developing tailored promotional and marketing strategies.

[0148] 2. Access control for sensitive resources. Creating a healthy internet environment is crucial for the development of minors. If a facial age estimation system can identify minors as internet users, many web pages and videos inappropriate for minors can be blocked, thereby protecting the physical and mental development of minors. Furthermore, this system can be combined with a game addiction prevention system to better safeguard the healthy development of minors.

[0149] 3. Promote personalized development of human-computer interaction. Computers, mobile phones, and various smart terminals are now widely available to users of all ages. However, users of different age groups have different requirements for human-computer interaction systems. For example, due to decreased vision and hearing, the elderly may prefer larger fonts and louder system volume in the user interface. Meanwhile, young users may prefer a more lively and vivid user interface. If various smart terminals such as computers and mobile phones can identify the user's age, it will be possible to design corresponding human-computer interaction experiences for users of different age groups and provide personalized services for users.

[0150] 4. Improve the accuracy of face recognition systems. Age features play a very important role in face recognition feature systems, especially for faces with a wide range of age variations. Combining age estimation methods can further improve the accuracy of face recognition systems.

[0151] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A facial age estimation method integrating gender and race information, characterized in that: A facial age prediction model that integrates gender and race information is established. The model includes feature maps and a convolutional neural network stacked using RepPsconv modules. The feature maps include a main module and an auxiliary module. The facial age estimation method that integrates gender and race information includes the following steps: Obtain a collection of face images with age, gender, and race labels, and preprocess the face image collection; Input the preprocessed face image into the convolutional neural network for feature extraction; The features extracted from the convolutional neural network are integrated at multiple scales at a fine-grained level; The fused features are connected to the main module and the auxiliary module using a fully connected layer, and the auxiliary module optimizes the results of the main module; Compare the output of the main module with the face image label information to calculate the loss, and iterate the training until the loss stops decreasing. Input the face image to be tested into the trained face age prediction model that integrates gender and race information, and output the age estimation result; The RepPsconv module includes a 3×3PSconv branch, a 1×1conv branch, and an identity branch. The RepPsconv modules are stacked into a convolutional neural network. The overall neural network structure includes: The first layer is the information fusion layer. This is a hidden layer added before the face image is input into the convolutional neural network. It has two neurons preset to represent the gender and race of the face. The two neurons representing gender and race are added and fused with the pixel values ​​of the face image. The PSconv branch in the second-layer RepPSconv module is replaced by a normal convolution operation and there is no Identity branch; The third layer RepPSconv module has 64 input and 64 output channels, followed by a maximum pooling layer. The input and output channels of the fourth layer RepPSconv module are 64 and 128 respectively; The fifth layer RepPSconv module has 128 input and 128 output channels, followed by a maximum pooling layer. The input and output channels of the sixth layer RepPSconv module are 128 and 256 respectively; The input and output channels of the seventh layer RepPSconv module are 256 and 256 respectively; The eighth layer RepPSconv module has input and output channels of 256 and 256 respectively, followed by a maximum pooling layer; The input and output channels of the ninth layer RepPSconv module are 256 and 512 respectively; The input and output channels of the tenth layer RepPSconv module are 512 and 512 respectively; The input and output channels of the eleventh layer RepPSconv module are 512 and 512 respectively, followed by a maximum pooling layer; The number of input and output channels of the twelfth layer RepPSconv module are 512 and 512 respectively; The input and output channels of the thirteenth layer RepPSconv module are 512 and 512 respectively; The input and output channels of the fourteenth RepPSconv module are 512 and 512 respectively, followed by a maximum pooling layer.

2. The method for estimating facial age by integrating gender and race information according to claim 1, wherein: The main module includes multiple sub-networks. The output of each sub-network will be multiplied by the different probability values ​​of each auxiliary attribute conditional branch of the auxiliary module, and the probability values ​​of each conditional branch of the auxiliary module are continuously updated during the iterative training process. Among the sub-networks of the main module, the sub-network that meets the actual conditions of the face image is multiplied by the high probability value, and the remaining sub-networks are multiplied by the small probability values ​​obtained by the auxiliary module.

3. The method for estimating facial age by integrating gender and race information according to claim 1, wherein: The output of the main module is compared with the face image label information to calculate the loss. The loss function is expressed as: Where N is the mini-batch of training samples, α and β are two hyperparameters, F is the number of sub-networks in the main module, and E is the number of auxiliary attributes in the auxiliary module. is the loss function of each sub-network of the main module, is the loss function used by the auxiliary module to supervise the auxiliary attribute estimation task, L i,age is the loss function of the absolute error between the predicted face age and the true value, It is the deviation loss function of each sub-network of the main module.

4. The method for estimating facial age by integrating gender and race information according to claim 3, wherein: The loss function of each sub-network of the main module is expressed as: in, The i-th face image under the f-th sub-network of the main module can correctly estimate the real age y i The probability of Indicates that the i-th face image in the f-th sub-network of the main module belongs to the real age y i The probability of It refers to the probability that the i-th face image in the f-th subnetwork in the main module belongs to age category k, where K is the total number of age categories.

5. The method for estimating facial age by integrating gender and race information according to claim 3, wherein: The auxiliary module uses the cross entropy loss function to supervise the estimation task of auxiliary attributes of gender or race of face images. The loss function is expressed as: in, It is the probability of estimating the auxiliary attribute to be the true category e under the f-th sub-network of the main module, represents the probability that the auxiliary attribute of the i-th face image belongs to the true category e under the f-th sub-network of the main module, It refers to the probability that the auxiliary attribute of the i-th face image under the f-th sub-network in the main module belongs to category j.

6. The method for estimating facial age by integrating gender and race information according to claim 3, characterized in that: The deviation loss function of each sub-network of the main module considers the correlation between adjacent age categories while estimating the true age category of the face image. The deviation loss function of each sub-network of the main module is expressed as: in, is the face image x i The estimated probability of belonging to age category k, the bias term (ky i ) 2 As penalty factor, k is age category, y i is the true age value of the face image, and K is the total number of age categories.

7. The method for estimating facial age by integrating gender and race information according to claim 3, wherein: A loss function based on the absolute error between the predicted face age and the true value is introduced to make the predicted age closer to the true age. The loss function of the absolute error is expressed as: in, is the predicted age, y i is the true age value of the face image.

8. The method for estimating facial age by integrating gender and race information according to claim 7, wherein: Predicted age The calculation formula is: in, Indicates the age value is predicted in the f-th sub-network of the main module, represents the probability value of the predicted age being the true age value in the f-th sub-network of the main module, Indicates that the i-th face image multiplied by the f-th sub-network of the main module belongs to the corresponding auxiliary attribute The probability value of .

9. The method for estimating facial age by integrating gender and race information according to claim 8, wherein: The calculation formula for predicting age in the f-th sub-network of the main module is: in, is the probability that the face image belongs to age category k under the f-th sub-network of the main module.

10. A facial age estimation system using the facial age estimation method integrating gender and race information according to any one of claims 1 to 9, characterized in that: include: Image acquisition module, data preprocessing module, information fusion layer, convolutional neural network training module and output module. The convolutional neural network training module includes RepPSconv module and feature mapping module. The feature mapping module has main modules and auxiliary modules. An image acquisition module is used to acquire a collection of face images with face age, gender, and race labels, as well as a face image to be tested; The data preprocessing module is used to preprocess the data obtained by the image acquisition module, including face detection, face alignment and face cropping; The information fusion layer is used to add and fuse the gender and race information with the pixel values ​​of the face; The RepPSconv module is used to extract feature maps from images and use the PSconv operator to fuse information between feature maps at multiple scales at a fine-grained level. Feature mapping module, which is used to connect the extracted feature maps to the main module and auxiliary modules using a fully connected layer; The main module is used to estimate the age of the face image to be tested; Auxiliary module, used to optimize the results of the main module; The output module is used to output the age estimation result of the face image to be tested and call the camera to predict the face age in real time.

Images