Face image age estimation method based on global-to-local ordinal regression network

Abstract

The application discloses a face image age estimation method based on a global-to-local ordinal regression network. The application comprises the following steps: step 1, preprocessing a data set; step 2, extracting features of face images in the preprocessed data set; step 3, local age domain division; performing k-means clustering analysis on the extracted feature data of the entire data set, and dividing the entire data set into multiple local data domains according to the clustering result; step 4, constructing a global regression network and a local regression network; and step 5, age estimation. The application solves the problem of insufficient precision of an existing deep ordinal regression network in a face age estimation task. Without increasing the complexity of network design, the application effectively improves the accuracy of the ordinal regression network for the face age estimation task. The application has the advantages of high recognition accuracy and strong robustness.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision, and provides a face image age estimation method based on a global-to-local ordinal regression network. BACKGROUND

[0002] Age estimation is a method of predicting real age values or age groups based on face images, which has a wide range of applications in visual monitoring, human-computer interaction, social media, face retrieval, etc., and is an important topic in computer vision.

[0003] By regarding the age label as a numerical value, age estimation can be converted into a regression problem. However, ordinal regression is still challenging because there is no clear distinction between levels in many cases. For example, in face age estimation, due to numerous factors such as genes, diet, and lifestyle, there are large individual differences in the aging process of facial shape, size, and texture, and there is no clear aging feature in each age group. In order to solve this problem, the application provides a face image age estimation method based on a global-to-local ordinal regression network. The global regression network is responsible for preliminary prediction in the entire age range, and the local regression network learns the aging features in a specific age range and is responsible for age prediction in the specific age range. By combining the global regression network and the local regression network, the prediction accuracy is improved without increasing the complexity of network design, and excellent results are achieved on the AFAD dataset. SUMMARY

[0004] The application aims to solve the problem of low prediction accuracy of existing ordinal regression networks in the case of small differences in aging features.

[0005] The application firstly pre-processes the face images in the data set, converts the age label value in the original data set into an order label, and performs multi-scale face sub-block cropping on the face images, which can better extract global face information and local face information and extract more information closely related to age. The entire data set is subjected to k-means clustering analysis, and according to the clustering results, the entire data set is divided into multiple local data domains. Then the processed face images are input into the CNN network for feature extraction, and the application adopts ResNet-34 as the feature extraction backbone network. The face features extracted are subjected to age prediction by constructing a rank-consistent ordinal regression network. The results predicted by the global regression network are mapped into the local age domain, and the corresponding local regression network is used for age prediction to obtain the final prediction result. When constructing the ordinal regression network, by sharing the network parameters except the last layer of network, the problem of rank inconsistency in ordinal regression is solved, and the accuracy of the face age estimation task is improved.

[0006] In order to conveniently describe the content of the application, some terms are defined first.

[0007] Definition 1: Convolutional Neural Network. Convolutional Neural Network is a kind of feedforward neural network, which includes convolutional layer and pooling layer, and has excellent performance for large image processing.

[0008] Definition 2: Ordinal Regression. Ordinal regression is used to estimate the order of an object. By converting the regression problem into a series of binary classification problems, the ordinal regression result is composed of a series of binary classification results.

[0009] Definition 3: MAE index. MAE is the mean absolute error, which is used to measure the average absolute error between the predicted value and the true value, and is defined as follows: Where y i is the true label, is the model predicted label. The smaller the MAE is, the better the model is.

[0010] The technical solution adopted by the application to solve its technical problems is as follows:

[0011] Step 1: Pre-process the data set;

[0012] Get the AFAD data set, which is a data set composed of Asian face images, containing 165501 face images, with an age range of 15-45 years old. For the entire data set, randomly select 70% of the images as the training set, 10% of the images as the validation set, and 20% of the images as the test set. The size of all images is adjusted to 256x256x3 pixels. The age label of the data set is adjusted to 0-30 ordinal label;

[0013] Step 2: feature extraction;

[0014] The face image is subjected to feature extraction by a neural network; a ResNet-34 network is selected as the backbone network of the feature extraction network; each face image is subjected to random cropping to 256x256x3, 224x224x3 and 128x128x3 respectively and then input to the feature extraction network; the extracted features are fused and normalized to the range [-1, 1], and the feature data is saved as a tensor for use by the neural network;

[0015] Step 3: local age domain division;

[0016] The feature data extracted from the entire data set is subjected to k-means clustering analysis, and according to the clustering results, the entire data set is divided into multiple local data domains;

[0017] Step 4: constructing global regression network and local regression network;

[0018] The global regression network and the five local regression networks constructed each include two parts: a feature extraction network and a rank-consistent ordinal regression network;

[0019] The feature extraction network uses ResNet-34 as the backbone network, responsible for extracting age features;

[0020] The input of the rank-consistent ordinal regression network is the age feature, and the output is the age ordinal label;

[0021] Step 5: age estimation;

[0022] The global regression network and the local regression network are used to test the image, which is first input into the global regression network to obtain the global predicted age, and then the global predicted age is mapped to the age domain divided in step 2, and then the image is input into the corresponding local regression network to obtain the final prediction result; if it is mapped to two age intervals at the same time, the mean of the prediction results of the two local regression networks is taken as the final prediction age result.

[0023] Further, the feature extraction, after the face alignment and face image cropping of the face image in the data set after preprocessing, another two sizes are randomly cropped, and the images of the three sizes are uniformly adjusted to 224x224x3 size and then input into the ResNet-34 network for face feature extraction. The extracted features mainly include texture features, expression features, shape features, etc.

[0024] Further, the local age domain division divides the entire age range into multiple age stages with similar characteristics, and a corresponding local regression network is trained in each age stage; the entire face image dataset is subjected to k-means clustering, and is divided into five local age domains according to the clustering results, namely 15-19, 18-25, 23-29, 25-34 and 30-45.

[0025] Further, the global regression network and the local regression network are constructed, and each includes a feature extraction network and a rank consistency ordinal regression network. The features extracted by the feature extraction network are input into the rank consistency ordinal regression network. The rank consistency is realized by the weight parameters of the neural network except for the bias term of the last layer; the last layer of the rank consistency ordinal regression network is composed of k-1 binary classifiers, and the binary classifier can be represented by the following formula:

[0026]

[0027] wherein is the empirical probability of predicting the kth sample, and is obtained by the following formula:

[0028]

[0029] wherein, sigma is a sigmoid function, W is the weight parameter of the neural network except for the last layer, and b k is the bias term corresponding to the kth sample.

[0030] The output of the rank consistency ordinal regression network is the ordinal label composed of [1, 1, 1, 0, 0, …, 0], and the corresponding age label is obtained by the following formula:

[0031]

[0032] For the entire regression network, the loss function in the training process is represented by the following formula:

[0033]

[0034] wherein, lambda k is used to adjust the weight of the positive sample contribution to the loss.

[0035] The age estimation firstly inputs the test image into the global regression network, maps the prediction result to the dataset divided in step 2, and obtains the final prediction result by using the corresponding local regression network; if the two age intervals are simultaneously mapped, the mean value of the prediction results of the two local regression networks is taken as the final prediction age result.

[0036] The innovation of the present application lies in:

[0037] 1) In order to solve the problem that the existing ordinal regression network has low prediction accuracy when the difference between aging features is small, a global-to-local ordinal regression network is proposed, which improves the prediction accuracy without increasing the complexity of network design.

[0038] 2) In order to solve the problem that ordinal regression may have inconsistent ranks, rank consistency is achieved by sharing network parameters.

[0039] 3) For the AFAD dataset, k-means clustering analysis is used to divide the 15-45 year-old dataset into multiple age range crossing sub-datasets with similar features, and a specific local regression network is trained, and the experimental results show that this method achieves excellent results. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The network overall structure of the present application

[0041] Figure 2 The method flowchart of the present application. DETAILED DESCRIPTION

[0042] The present application will be further described below in combination with the drawings and examples.

[0043] The present application is a face image age estimation method based on global-to-local ordinal regression network, belonging to the field of image processing. It solves the problem of insufficient precision of existing deep ordinal regression network in face age estimation task. The present application adopts a two-layer ordinal regression network, wherein the global regression network is responsible for prediction in the entire age domain, and the local regression network is responsible for prediction in a specific age domain. The age value predicted by the global regression network is mapped into the local regression network, and the final prediction result is obtained by the specific regression network responsible for the local age domain. The present application effectively improves the accuracy of ordinal regression network for face age estimation task without increasing the complexity of network design. The advantage of the present application lies in high recognition accuracy and strong robustness.

[0044] As shown in Figure 1 and 2 , the face image age estimation method based on global-to-local ordinal regression network comprises the following steps:

[0045] Step 1: Preprocessing the dataset;

[0046] An AFAD dataset, which is a dataset composed of Asian face images, containing 165501 face images, age range from 15 to 45, is obtained. For the whole dataset, 70% of the images are randomly selected as the training set, 10% of the images are randomly selected as the validation set, and 20% of the images are randomly selected as the test set. The size of all images is adjusted to 256x256x3 pixels. The age label of the dataset is adjusted to the order label of 0-30;

[0047] Step 2: feature extraction;

[0048] Feature extraction is performed on the image by a neural network. ResNet-34 network is selected as the backbone network of the feature extraction network. Each image is randomly cropped to 256x256x3, 224x224x3 and 128x128x3 respectively and then input into the feature extraction network. The extracted features are fused and normalized to the range [-1, 1], and the feature data is saved as a tensor for use by the neural network.

[0049] Step 3: local age domain division

[0050] K-means clustering analysis is performed on the feature data extracted from the whole dataset, and according to the clustering results, the whole dataset is divided into multiple local data domains;

[0051] Step 4: construction of global and local regression networks;

[0052] The global and local regression networks constructed in this step include two parts. One is the feature extraction network in step 3, and the other is the rank consistency ordinal regression network. ResNet-34 network is selected as the backbone network of the feature extraction network, which is responsible for extracting age features. The input of the rank consistency ordinal regression network is the age feature, and the output is the age order label.

[0053] Step 5: age estimation;

[0054] In this step, the global and local regression networks described in step 4 are used. The test image is first input into the global regression network to obtain the global predicted age, which is then mapped to the age domain divided in step 2. Then the image is input into the corresponding local regression network to obtain the final prediction result. If it is mapped to two age intervals at the same time, the mean of the prediction results of the two local regression networks is taken as the final prediction age result.

[0055] The step 2 local age domain division divides the entire age range into multiple age stages with similar characteristics, and trains a corresponding local regression network in each age stage; the entire face image data set is clustered by k-means, and is divided into five local age domains according to the clustering results, which are 15-19, 18-25, 23-29, 25-34 and 30-45 respectively.

[0056] The step 3 feature extraction: after the image in the data set is preprocessed, face alignment and face image cropping are performed, and then two other sizes are randomly cropped; after the three sizes of images are uniformly adjusted to 224x224x3, they are input into the ResNet-34 network for face feature extraction; the extracted features mainly include texture features, expression features and shape features.

[0057] The step 4 constructs a global and local regression network, which includes a feature extraction network and a rank consistency ordinal regression network. The features extracted by the feature extraction network are input into the rank consistency ordinal regression network. The rank consistency is realized by the weight parameters of the neural network sharing except the last layer bias term; the last layer of the regression network is composed of k-1 binary classifiers, which can be represented by the following formula:

[0058]

[0059] Wherein is the predicted empirical probability of the kth sample, which is obtained by the following formula:

[0060]

[0061] Where σ is the sigmoid function, W is the weight parameter of the neural network except the last layer, and b k is the bias term corresponding to the kth sample.

[0062] The output of the regression network is an ordinal label composed of [1, 1, 1, 0, 0, …, 0], and the corresponding age label is obtained by the following formula:

[0063]

[0064] For the entire regression network, the loss function in the training process is represented by the following formula:

[0065]

[0066] Wherein, λ k is used to adjust the weight of the positive sample contribution to the loss.

[0067] The step 5 age estimation firstly inputs the test image into the global regression network, maps the prediction result of the global regression network to the data set divided in step 2, and obtains the final prediction result by using the corresponding local regression network; if the prediction result is mapped into two age intervals at the same time, the mean value of the prediction results of the two local regression networks is taken as the final prediction age result.

[0068] The network model is trained by using the network and loss function constructed in step 4, each network is trained for 200 rounds, the Batchsize is set to 128, the initial learning rate is set to 0.0005, the learning rate is adjusted to 0.0003 at the 80th round and to 0.0001 at the 150th round, the optimizer is an SGD optimizer, and the momentum is set to 0.9. During the training process, the model parameters saved after each round of training are used to verify the verification set, and the model parameters with the minimum MAE are saved. After the training is completed, the saved model parameters are used to test the test set.

[0069] The experimental results are shown in Table 1, and the MAE index of the age estimation result is reduced to 2.92 from the previous 3.23 for the AFAD data set.

[0070] Table 1 Table of performance data comparison

[0071]

[0072] Example 1:

[0073] The specific process of the network in the inference prediction stage is as follows: (1) input the real age of the face image as 32 years old; (2) perform image preprocessing on the face image, including random cropping, image flipping and the like; (3) input the preprocessed image into the feature extraction network to extract features, and save the extracted features as a feature tensor; (4) input the feature tensor obtained in (3) into the trained global regression network to obtain a rough prediction result of 30 years old; (5) according to the prediction result of the global regression network, map the features of the face image to the local age domains of 25-34 and 30-45, and input the features extracted in (3) into the corresponding local regression networks of the age domains of 25-34 and 30-45 respectively for prediction, and the results are 31 years old and 33 years old respectively, and the final result takes the mean value of the prediction results of the two local regression networks, i.e. (33+31) / 2=32 years old.

[0074] For the face image, the comparison of the network prediction result with other age estimation networks is as follows in Table 2:

[0075] Table 2 Comparison table of network prediction result and other age estimation networks

[0076] True age dLDLF CORF OR-CNN coGOL DRFs OL Reg-Ord Ours 32 36 29 33 30 31 30 31 32

[0077] While the preferred embodiments in the application have been described, additional modifications and changes can occur to those skilled in the art once they learn of the basic creative principles contained herein. Therefore, the appended claims are intended to cover all such modifications and changes as fall within the scope of the preferred embodiments in the application.

Claims

1. A face image age estimation method based on a global-to-local ordinal regression network, characterized in that Comprising the following steps: Step 1: preprocessing the data set; Step 2: extracting features from the preprocessed face images in the data set; Step 3: local age domain division; k-means clustering analysis is performed on the features extracted from the entire data set, and according to the clustering results, the entire data set is divided into multiple local data domains; Step 4: constructing a global regression network and a local regression network; Step 5: age estimation; Using the global regression network and the local regression network, the test image is first input into the global regression network to obtain the global predicted age, and the global predicted age is mapped into the age domain divided in step 2, and then the image is input into the corresponding local regression network to obtain the final prediction result; if it is mapped into two age intervals at the same time, the mean of the prediction results of the two local regression networks is taken as the final prediction age result; The constructed global regression network and five local regression networks each include two parts: a feature extraction network and a rank consistency ordinal regression network; the feature extraction network uses ResNet-34 as the backbone network, which is responsible for extracting age features; The input of the rank consistency ordinal regression network is the age feature, and the output is the age ordinal label; The features extracted by the feature extraction network are input into the rank consistency ordinal regression network; the rank consistency is realized by sharing the weight parameters of the neural network except the last layer bias term; the last layer of the rank consistency ordinal regression network is composed of k-1 binary classifiers, which can be represented by the following formula: where is the empirical probability of predicting the i-th sample, derived from the following equation: wherein σ is a sigmoid function, W is a weight parameter of the neural network except for the last layer, b k is a bias term corresponding to the kth binary classifier; The output of the rank consistency ordinal regression network is an ordinal label composed of [1, 1, 1, 0, 0, …, 0], and the corresponding age label is obtained by the following formula: For the entire regression network, the loss function during training is represented by the following formula: where λ k to adjust the weight of positive samples on loss contribution. 2.The face image age estimation method based on the global-to-local ordinal regression network of claim 1, wherein The data set of step 1 is the AFAD data set, which is a data set composed of Asian face images, containing 165501 face images, with an age range of 15-45 years old; for the entire data set, 70% of the images are randomly selected as the training set, 10% of the images are randomly selected as the validation set, and 20% of the images are randomly selected as the test set; all images are resized to 256x256x3 pixels; the age label of the data set is adjusted to a rank order label of 0-30. 3.The face image age estimation method based on the global-to-local ordinal regression network of claim 2, wherein Step 2 is implemented as follows: ResNet-34 network is selected as the backbone network of the feature extraction network; each face image is randomly cropped to 256x256x3, 224x224x3 and 128x128x3, respectively, and then input into the feature extraction network; the extracted features are fused and normalized to the range [-1, 1], and the feature data is saved as a tensor for use by the neural network. 4.The face image age estimation method based on the global-to-local ordinal regression network of claim 3, wherein The local age domain division of step 3 is implemented as follows: The entire age range is divided into multiple age ranges with similar features, and the corresponding local regression network is trained in each age range; the entire face image data set is k-means clustered, and according to the clustering results, it is divided into five local age domains, which are 15-19, 18-25, 23-29, 25-34 and 30-45. 5.The face image age estimation method based on the global-to-local ordinal regression network of claim 4, wherein The age estimation firstly inputs the test image into the global regression network, maps the prediction result of the global regression network to the data set divided in step 2, and obtains the final prediction result by using the corresponding local regression network; if the test image is mapped into two age intervals at the same time, the mean value of the prediction results of the two local regression networks is taken as the final prediction age result.

Images