Low-age face aging method based on age and key point guidance generation

Abstract

The application discloses a low-age face aging method based on age and key point guidance generation, and mainly solves the problems that the prior art cannot generate high-quality aging images for low-age faces and cannot simultaneously maintain the efficiency of a processing procedure and the accuracy of a target. The implementation scheme is as follows: a training set and a test set of face images of 0-20 years old are constructed, and a corresponding age label is given to each picture; key point detection is performed on the training set, and the detection result is converted into a Gaussian distribution graph; a face image generation adversarial network based on key point information guidance and cycle condition generation is constructed; a continuous age conversion loss is designed, and a network overall loss is constructed; the network is trained by using the training set and the Gaussian distribution graph until the overall loss converges; and the test set is loaded into the trained network to generate face images at each age stage. The application can generate higher-quality aging images for low-age faces and improve the naturalness of generated images.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image generation, and particularly relates to a low-age face aging method, which can be used to construct a face aging model of children and adolescents aged 1-20 years old. BACKGROUND

[0002] Face aging technology is based on facial features and automatically processes age to automatically predict facial features that age with age. Face aging methods have wide applications in many fields.

[0003] With the wide application of deep learning in the field of computer vision, generative adversarial network (GAN) has been gradually applied to the field of face aging image generation. The aging process of the face is complex and variable. The facial changes of children and adolescents are mainly growth and development, such as gradual uplift of the nasal bridge bone, widening of the lower jaw bone, and changes in facial shape due to the growth and arrangement of teeth. At the same time, with the growth of bones and soft tissues, the facial proportions will also change, for example, the size of the eyes relative to the whole face will decrease. In addition, the surge of hormones during puberty, such as growth hormone and sex hormone, will also promote the growth and remodeling of facial bone and muscle morphology. The facial aging of adults mainly involves skin aging, wrinkle formation, and redistribution of facial fat. The basic facial structure is basically fixed during puberty, and the aging speed of the facial structure after adulthood is relatively slow compared to the aging speed of the facial texture. At this time, the aging is mainly dominated by facial texture, such as the appearance of fine lines at the corners of the eyes, mouth, and forehead, skin relaxation of facial contours, and age spots caused by ultraviolet factors. These differences mean that different strategies need to be taken for different age groups.

[0004] In recent years, some experts and scholars have done some research on low-age face aging. Chandaliya et al. proposed a model named Conditional Perceptual Adversarial Variational Autoencoder (CPA-VAE) in the article Conditional perceptual adversarial variational autoencoder for age progression and regression on child face, aiming to perform age progression and regression tasks on child faces. The model uses a conditional variational autoencoder (CVAE) to generate facial images under the condition of a given child target age label, and measures the difference between the synthesized image and the real image in the high-level feature representation layer through a perceptual loss, which helps to generate more realistic and consistent facial feature images.

[0005] Deb et al. proposed a deep learning-based child face age-progression model in the article "Child face age-progression via deep feature aging", which requires a large amount of child face image data of different ages to train the model to learn the rules of facial changes with age, and the finally trained model can predict the future appearance of children by simulating the natural facial changes caused by age growth. However, this model is limited to using data containing pictures of the same individual at different ages as the training set, so it cannot be well generalized.

[0006] Lu et al. proposed a two-stage scheme in the article "Learning shape and texture progression for young child face aging", which divides aging into facial texture and geometric deformation, corresponding to geometric deformation generative adversarial network and texture transfer generative adversarial network respectively. In the final result presentation, the key points of the geometric deformation network are applied to the facial texture framework using the warping function change. Although this scheme has good results on the YAFD and UTKFace data sets, it has many parameter redundancy limitations due to its centralized customization of aging and training for different age groups, and the quality of the generated aged face is not high, so it cannot be widely used. SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art, and to propose a low-age face aging method based on age and key point guided generation for children under 20 years old, to integrate facial image features and key point information of the face, and to ensure that feature extraction can accurately focus on areas that change significantly with age, improving the effect of face aging generation task.

[0008] The technical idea to achieve the purpose of the present application is: by adopting a recurrent conditional generative adversarial network architecture, the facial image and key point information are fused, the model is accurately integrated with the facial image features and the key point information, the important facial areas are highlighted, the irrelevant information is suppressed, and the feature extraction can accurately focus on the areas that change significantly with age; by introducing a continuous age conversion loss, the network learning is promoted to maintain the consistency of the face image in age conversion, and the realism and credibility of the generated image are enhanced.

[0009] According to the above idea, the low-age face aging method based on age and key point guided generation of the present application is characterized in that it comprises the following steps:

[0010] (1) Obtain n pieces of face data of people aged 0-20 from a public face data set, divide them into a training set T and a test set M in a ratio of 9:1, and then divide the pictures in the training set and the test set into 5 categories according to age, i.e. 1-3 years old, 4-8 years old, 9-12 years old, 13-16 years old and 17-20 years old, which are defined as age labels 1-5 in turn, that is, each picture is assigned a corresponding age label;

[0011] (2) Construct a face image generation adversarial network APGAN based on key point information guidance and cycle condition generation, which includes a weight-shared forward and reverse generator G and a multi-task discriminator D:

[0012] (2a) Perform face key point detection on the original images in the training set T, convert the obtained face key point information into a Gaussian distribution graph, and normalize the Gaussian distribution graph;

[0013] (2b) Construct a generator G for adding aging features to the original face structure to generate high-quality aging images:

[0014] (2b1) Establish a face image and key point information attention fusion module FIKFAM for accurately integrating face image features and key point information and highlighting important face regions;

[0015] (2b2) Establish an encoder containing a feature extraction module, a convolution kernel with a size of 3x3, a FIKFAM module and 6 residual blocks;

[0016] (2b3) Establish a decoder containing two deconvolution layers, one convolution layer, three normalization layers and three ReLU activation layers, for generating output images from feature vectors;

[0017] (2b4) Add the outputs of the encoder and input them to the decoder to form the generator G;

[0018] (2c) Construct a multi-task learning discriminator D containing true and false image discrimination and age verification functions, for judging the authenticity of the image and evaluating whether the image reflects a specific age attribute;

[0019] (3) Design a continuous age conversion loss L CAT :

[0020]

[0021] Where x is the input picture, is the output image of the generator G, is the key point extracted from the picture p is the key point extracted from the picture x, and c a is the target age label;

[0022] (4) According to the discriminant prior loss L adv , age verification loss L cls , reconstruction cycle consistency loss L recon and continuous age conversion loss L CAT , the overall loss function min G max D L(G,D) is constructed:

[0023] min G max D L(G,D) = λ1L adv + λ2L cls + λ3L recon + λ4L CAT

[0024] Wherein, λ1, λ2, λ3, λ4 are weight coefficients of L adv , L cls , L recon , L CAT ;

[0025] (5) Using the back propagation algorithm, the face image generative adversarial network APGAN is trained by the training data set T until the overall loss function of the network converges, and a trained face generation model is obtained;

[0026] (6) The face image in the test data set M is loaded into the trained face generation model to generate face images at each age stage.

[0027] Compared with the prior art, the present application has the following advantages:

[0028] First, the face image generative adversarial network APGAN based on key point information guidance and cycle condition generation is constructed, which can realize the bidirectional conversion of age, and the face image and key point information attention fusion module of the network can efficiently combine and update the texture and structure information of the face image, and use the structure information to guide the update of the texture information, which not only preserves the facial identity features, but also learns the key point information of the face, which helps to ensure that the feature extraction is accurately focused on the area that changes significantly with age, and generates high-quality aging effect.

[0029] Second, the continuous age conversion loss CAT Loss is defined, which can solve the limitation of the reconstruction loss and enhance the ability of the network, so that it can not only maintain the identity consistency of the image, but also accurately and naturally convert between different age stages. At the same time, due to the introduction of the loss CAT Loss, the network can better understand and simulate the dynamic process of age change, improve the naturalness and diversity of the generated image, and enhance the realism and credibility of the generated image. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flow chart of the implementation of the present application;

[0031] Figure 2 is a schematic diagram of a face aging generation model constructed in the present application. DETAILED DESCRIPTION

[0032] The embodiments and effects of the present application are described in further detail below with reference to the accompanying drawings.

[0033] Referring to Figure 1 , the implementation steps of the present example are as follows:

[0034] Step 1. Construct a data set.

[0035] 1.1) Obtain n face data in the age range of 0-20 years from the UTKFace public data set as a data set:

[0036] Each image in the UTKFace data set contains face image, person gender, person age, and other label information, and only the age and gender information are used in the present example. The face age contained in the data set ranges from 1 year to 116 years, has a long age span, and is convenient for the model to learn the face aging characteristics at different age stages. In the present example, 4877 face images in the age range of 0-20 years are selected as a data set;

[0037] 1.2) Divide the data set into a training set T and a test set M in a ratio of 9:1, and then divide the images of the training set and the test set into 1-3 years, 4-8 years, 9-12 years, 13-16 years, and 17-20 years, a total of 5 categories;

[0038] 1.3) Assign a corresponding age label to each image of the training set T and the test set M:

[0039] Use one-hot encoding to assign a corresponding 1-5 age label to each image of the 5 age categories of the training set T and the test set M, that is, regard each age range as an independent category, and use a binary vector to represent the age label, which is represented as follows:

[0040] Label 1: [1, 0, 0, 0, 0], that is, when the age range is 1-3 years, the first column is 1 and the remaining columns are 0;

[0041] Label 2: [0, 1, 0, 0, 0], that is, when the age range is 4-8 years, the second column is 1 and the remaining columns are 0;

[0042] Label 3: [0, 0, 1, 0, 0], that is, when the age range is 9-12 years, the third column is 1 and the remaining columns are 0;

[0043] Label 4: [0, 0, 0, 1, 0], i.e. the fourth column is 1 when the age range is 13-16 years old, and the rest are 0;

[0044] Label 5: [0, 0, 0, 0, 1], i.e. the fifth column is 1 when the age range is 17-20 years old, and the rest are 0.

[0045] Step 2. Face key point detection is performed on the original images in the training set T, and the obtained face key point information is converted into a Gaussian distribution map, and the Gaussian distribution map is normalized.

[0046] 2.1) Input each image in the training set T into the CNN face key point detector based on nonlinear local regression in the existing Dlib library, and determine whether the face image exists;

[0047] If the face image exists, face key point detection is performed on the data in the data set, i.e. first using a deep learning method to detect the face in the input image, locating the face position and boundary box in the picture, and then using an existing pre-trained face key point detector to locate the key points in the face detected area to accurately locate the 68 important feature points in the face picture, wherein: 0-16 represents the contour of the face, 17-21 represents the contour of the left eyebrow, 22-26 represents the contour of the right eyebrow, 27-35 represents the contour of the nose and nose bridge, 36-41 represents the contour of the left eye, 42-47 represents the contour of the right eye, and 48-67 represents the contour of the mouth; then use all the feature points to form a two-dimensional matrix of 68x2 size, wherein the first dimension represents 68 key points, and the second dimension represents the coordinates of each key point in the image;

[0048] If it is determined that the face image does not exist, the picture is removed from the training set T, and the next image in the training set is continuously detected.

[0049] 2.2) Convert the face key point information into a Gaussian distribution map, and normalize the Gaussian distribution map:

[0050] Initialize a blank image: initialize a blank image of 128x128 size for each key point, a total of 68 images;

[0051] Apply a two-dimensional Gaussian function to the key point position of each key point on the corresponding blank image to calculate the corresponding Gaussian distribution map Q, and the calculation process is as follows:

[0052] where (x0, y0) is the coordinate of the key point, σ is the standard deviation of the Gaussian kernel, x and y are the coordinates on the image, and G(x, y) is a two-dimensional Gaussian function.

[0053] The Gaussian distribution graph Q is normalized to maintain consistency of image data and ensure that the values are within a certain range, obtaining a normalized Gaussian distribution graph Q'.

[0054] Step 3. Constructing a face image generation adversarial network APGAN based on key point information guidance and cycle condition generation, which includes a weight-shared forward and inverse generator G and a multi-task discriminator D.

[0055] Referring to Figure 2 , the implementation of this step includes the following:

[0056] 3.1) Constructing a generator G for adding aging features to the original facial structure to generate high-quality aging images:

[0057] 3.1.1) Establishing a facial image and key point information attention fusion module FIKFAM for accurately integrating facial image features and key point information and highlighting important facial regions:

[0058] 3.1.1a) Establishing a Module sub-module containing an image and key point upper and lower routes, a Sigmoid function layer, a multiplier, and an adder:

[0059] The upper image route and the lower key point route have the same structure, both containing two convolutional layers, two normalization layers, a ReLU activation layer, and a Sigmoid function layer. The specific structure is: first convolutional layer→first normalization layer→ReLU activation layer→second convolutional layer→second normalization layer;

[0060] The Sigmoid function layer is used to map the output Mask of the lower key point route to the range of (0, 1);

[0061] The multiplier is used to multiply the output of the lower key point route after the Sigmoid function layer with the output of the upper image route;

[0062] The adder is used to add the multiplication result to the input of the upper image route .

[0063] The lower key point route and the Sigmoid function layer are connected in series to form module 1, the upper image route and the multiplier are connected in series to form module 2, module 1 and module 2 are connected in parallel to form module 3, module 3 and the adder are connected in series to form module 4, and module 4 and the lower key point route are connected along the depth axis to form the Module sub-module.

[0064] 3.1.1b) Cascade M=6 sub-modules with the same architecture to form the face-image and keypoint information attention fusion module (FIKFAM);

[0065] 3.1.1c) The face-image and keypoint information attention fusion module (FIKFAM) precisely integrates face-image features and keypoint information and highlights important facial regions as follows:

[0066] Each Module in the FIKFAM gradually updates its input and to its output and where in the mthModule, its input is the face feature encoding and the position feature encoding and the output is the face feature encoding and the position feature encoding The update of each Module includes the following three steps:

[0067] First, generate the "face feature attention mask" Mask, i.e., first input the position feature encoding to the previous keypoint route, then generate the Mask through a Sigmoid function layer:

[0068]

[0069] where σ represents the Sigmoid function, conv point is the set of convolution, normalization, and ReLU activation layers of the previous keypoint route, represents the input position feature encoding of the mthModule;

[0070] The value of this Mask is between 0 and 1, indicating the importance of the corresponding pixel position in the generation process. Through the Mask, the model can identify and enhance the natural changes of facial key features such as eyes, nose, and mouth at different ages, while maintaining the coherence of the background and subtle features;

[0071] Second, update the face feature encoding , i.e., the input face feature encoding of the mthModule is transformed through the previous image route to After multiplying the transformed face feature encoding with the face feature attention mask Mask, add the face feature encoding of a Module to form a residual connection, i.e., realize the update of the face feature encoding :

[0072]

[0073] where represents element-wise multiplication, conv img is the convolution of the previous image route, and the residual connection helps to preserve the original facial feature encoding, which is crucial for the generation of the face aging conversion image;

[0074] In the third step, the position feature encoding is updated, that is, the position feature encoding is updated according to the updated facial feature encoding The position feature encoding is updated as follows:

[0075] The position feature encoding First, the position feature encoding is converted through the next key point route, and then the converted position feature encoding is concatenated with the updated facial feature encoding along the depth axis to obtain the updated position feature encoding

[0076] where || represents concatenation along the depth axis,

[0077] Each convolutional layer in each Module reduces the depth of the feature map by half, so that the depth of the feature map of the position feature encoding path and the facial feature encoding path in the next Module is consistent;

[0078] 3.1.2) An encoder containing a feature extraction module, a convolutional kernel with a size of 3x3, a FIKFAM module and 6 residual blocks is established to convert the target age label, the input image and the Gaussian distribution map corresponding to the input image into the facial feature encoding of the target age

[0079] The feature extraction module includes three convolutional layers, three normalization layers and three ReLU activation layers, and the structure is: first convolutional layer→first normalization layer→first ReLU activation layer→second convolutional layer→second normalization layer→second ReLU activation layer→third convolutional layer→third normalization layer→third ReLU activation layer.

[0080] The convolutional kernel has a step size of 1, a size of 3x3 and an edge padding size of 1;

[0081] The 6 residual blocks have the same structure, and each residual block is a convolutional layer with a step size of 1, a size of 3x3 and an edge padding size of 0;

[0082] The feature extraction module and the FIKFAM module are connected in series to form Module A, Module A and the convolutional kernel are connected in parallel to form Module B, and Module B and the 6 residual blocks are connected in series to form the encoder.

[0083] The encoder first fuses the target age label with the input image, keypoint gaussian map respectively, and extracts features through convolution layers stacked along the depth axis, combined with instance normalization and ReLU activation layers, to generate face feature encoding Position feature encoding Subsequently, the face feature encoding and the position feature encoding are sent into the FIKFAM module for fusion, outputting the position feature encoding and the face feature encoding generated by position guidance

[0084] 3.1.3) A decoder containing two deconvolution layers, one convolution layer, three normalization layers and three ReLU activation layers is established, and the structure is: the first deconvolution layer→the first normalization layer→the first ReLU activation layer→the second deconvolution layer→the second normalization layer→the second ReLU activation layer→the first convolution layer→the third normalization layer→the third ReLU activation layer;

[0085] The decoder is used to convert the face feature encoding and the position feature encoding output by the encoder into an output image, i.e. the position feature encoding in the encoder output is discarded, and the face feature encoding is retained, and then and the initial face feature encoding are merged and sent into the decoder to obtain an output image with the same size as the input.

[0086] 3.2) A multi-task learning discriminator D containing real and fake image discrimination and age verification functions is constructed to perform real and fake image discrimination and age verification:

[0087] Seven convolution kernels with increasing number of channels layer by layer and two fully connected layers FC1 and FC2 with different output parameters are selected and sequentially cascaded to form the multi-task learning discriminator D;

[0088] The first fully connected layer FC1 converts the extracted features into one-dimensional data representing the probability that the image is considered to be a real image, to identify whether the input image is provided by the dataset or generated by the generator, i.e. to realize the function of real and fake image discrimination, which is the core task of the discriminator;

[0089] The second fully connected layer FC2 maps the extracted features into 5-dimensional data equal to the number of age categories, and compares it with the target age label to evaluate whether the image reflects a specific age attribute, i.e. to realize age verification, which actually introduces an auxiliary classification task-age classification into the APGAN network, with the purpose of ensuring that the generated face image is not only visually realistic, but also consistent with the target age label in terms of aging or youth performance.

[0090] The discriminator shares parameters before proceeding to the FC1 and FC2 layers, so that it can output different tasks based on the same features. By introducing the age verification function, the discriminator can more effectively capture age-related features, and in turn generate face images that are more matched to the target age. This multi-task learning strategy not only enhances the discriminator's ability to distinguish between real and fake images, but also improves the accuracy of the generated image age attribute, promoting the generation of higher quality and more accurate age attribute face images.

[0091] Step 4. Designing continuous age conversion loss L CAT .

[0092] When dealing with tasks that require highly dynamic and complex conversion, such as age conversion, relying solely on reconstruction loss may cause the network to focus too much on restoring the input image, ignoring the diversity and complexity of the conversion process. Therefore, it is necessary to introduce CAT Loss, which enables the network to better understand and simulate the dynamic process of age change. Its specific implementation is:

[0093] 4.1) Given input image x and target age label c t , the output of the generator G is the generated image

[0094]

[0095] Where p is the key point extracted from the image x;

[0096] 4.2) Extract the key points from the generated image

[0097] 4.3) Given another target age label c a , and input it to the generator G together with the above image x, image key points key points p, to get the generated vector G(x, p, c a ) from the input image x to the target age c a and the generated vector from the input image to the target age c a .

[0098] 4.4) According to the two generated vectors obtained in 3.3), calculate the norm of the difference value to get the continuous age conversion loss L CAT :

[0099]

[0100] Step 5. Construct the overall loss function min G max D L(G,D).

[0101] 5.1) According to the original age input image x0, the target age c t , the real image x t of the target age, construct the discriminant prior loss L adv , as follows:

[0102]

[0103] Where G(x0,c t ) is the generated target age face image, D is the discriminator, E represents the data distribution expectation, P data represents the distribution of training data, x ~ P data represents that the training data x follows the P data distribution, x,c t ~ P data represents that x,c t follows the P data distribution, using this loss function can measure the closeness of the generated image and the original data set image, and optimize the discriminator.

[0104] 5.2) According to the original age face image x0, the original age label c0, the target age label c t , construct the age verification loss L cls , as follows:

[0105]

[0106] Where c0 ~ P data represents that c0 follows the P data distribution, P represents the conditional probability function, by optimizing the true and false image discriminant prior loss function and the age verification loss function, the generator can ensure that the generated face picture is not only real and reliable, but also can accurately reflect the corresponding age information, so as to generate high-quality face image.

[0107] 5.3) According to the original image x0, the key point Gaussian graph p0 corresponding to the original image, the original age label c0, the key point Gaussian graph p t corresponding to the target age image, and the target age label c t , construct the reconstruction cycle consistent loss L recon :

[0108] L recon =||G(G(x0,p0,c t )p t ,c0)-x0||

[0109] wherein G(x0, p0, c t ) is the original image and its key points, the target age face image generated by the target age label input generator, G(G(x0, p0, c t ) p t , c0) is the generated image and its key points, the new original age image reconstructed by the original age label input generator, the reconstruction cycle consistency loss function can realize the alignment of the input image and the output image at the pixel level, and ensure that the network can restore the original image, especially when the target label is set to the original label, which maximizes the preservation of personal identity features.

[0110] 5.4) After assigning different weights to the four losses, add them to obtain the overall loss of the network:

[0111] min G max D L(G,D)=λ1L adv +λ2L cls +λ3L recon +λ4L CAT

[0112] wherein λ1, λ2, λ3, λ4 are the weight coefficients of L adv , L cls , L recon , L CAT , respectively.

[0113] Step 6, using the back propagation algorithm, the face image generation adversarial network APGAN based on key point information guidance and cycle condition generation is trained by the training data set T.

[0114] (6.1) Set the hyperparameters of back propagation: set the initial learning rate to 0.0001, and the batch size to 16;

[0115] (6.2) Perform face key point detection on the images of the training set T, convert the obtained face key point information into a Gaussian distribution graph, and normalize it to obtain the normalized Gaussian distribution graph Q';

[0116] (6.3) Input the training set T and the Gaussian distribution graph Q' into the face image generation adversarial network APGAN based on key point information guidance and cycle condition generation, and generate the face aging output result;

[0117] (6.4) Calculate the value of the overall loss min G max D L1(G,D) according to the face aging output result, input the loss value into the Adam optimizer, and use the back propagation algorithm to update the gradient of the network parameters through the optimizer;

[0118] (6.5) Repeat (6.3)-(6.4) until the total loss function converges, and obtain the trained face image generation adversarial network.

[0119] Step 7, load the face images in the test data set M into the trained face image generation network APGAN to generate face images of each age stage.

[0120] (7.1) input c test images containing each age stage from the test set into the trained face image generation network APGAN, and obtain the generation results of each age stage corresponding to the c test images;

[0121] (7.2) traverse the test set, and repeat step (7.1) until the generation results corresponding to all images in the test set are output.

[0122] The effects of the present application can be further illustrated by the following simulation.

[0123] 1. Simulation conditions

[0124] The simulation platform of the present experiment is a computer with Intel Core i7-11800H CPU and 16GB memory, the operating system is Windows 10, and python 3.8 and Pytorch 1.7 are used to build and train the neural network model.

[0125] The data set used in the simulation is the UTKFace face data set, which is preprocessed and divided into training set and test set according to the ratio of 9:1.

[0126] The Face++ platform is used to generate aging picture age prediction, which provides a face analysis service based on face recognition technology, including age prediction function. When using this function, the user uploads or inputs a face photo, and Face++ will automatically detect the face in the photo and output the predicted age of the face.

[0127] 100 pictures in the test set are taken as test samples to generate their aging faces of each age stage and evaluate them.

[0128] Common evaluation of generation problems includes initial score IS, modal score MS, Wasserstein distance WD and Frechet initial distance FID, etc. In actual operation, evaluation indexes such as IS and MS cannot accurately evaluate the quality of generated images and sample diversity, and the WD distance is limited by the calculation cost. The performance evaluation used in the simulation of the present application is the following FID index:

[0129]

[0130] wherein μ data and are the mean and variance of the real samples, μ g and are the mean and variance of the generated samples.

[0131] The experimental results are evaluated by means of the Face++ API platform, which is divided into two evaluation methods, namely age accuracy and picture recognition rate.

[0132] 2. Simulation content and results

[0133] Simulation 1: The present application and several representative CAAE, StarGAN, D2AAE, ChildGAN and AgeGAN++ methods of the prior art are used to age the low-age faces of different age groups under the same experimental environment, and the performance evaluation index FID of each method is calculated, and the results are shown in Table 1.

[0134] Table 1 Performance evaluation results of different methods of the present application and the prior art for face aging of different age groups

[0135]

[0136]

[0137] As can be seen from Table 2, the FID results of the present application are significantly better than those of other algorithms in each age group. In the 1-3 year age group, the FID index of the present application is reduced by 60.18 compared with ChildGAN, in the 4-8 year age group, the FID index of the present application is reduced by 135.2 compared with ChildGAN, in the 9-12 year age group, the FID index of the present application is reduced by 152.67 compared with ChildGAN, in the 13-16 year age group, the FID index of the present application is reduced by 137.79 compared with ChildGAN, and in the 17-20 year age group, the FID index of the present application is reduced by 80.37 compared with ChildGAN.

[0138] Simulation 2: The present application and several representative CAAE, StarGAN, D2AAE, ChildGAN and AgeGAN++ methods of the prior art are used to age the low-age faces of different age groups under the same experimental environment, and the aged images and original images of each method are respectively transmitted into the Face++ platform for age prediction, and the average age difference between the aged images and the original images is calculated, that is, the age accuracy of each method, and the results are shown in Table 2.

[0139] Table 2 Comparison results of age accuracy of each method

[0140]

[0141] As can be seen from Table 2, the experimental results of this invention are superior to most existing algorithms. It ranks third in age accuracy in the 1-3 year old age group. This is because this invention introduces facial key points, so the accuracy is slightly lower in the 1-3 year old age group where the contour changes greatly. The age accuracy of this invention ranks second in other age groups, only lower than ChildGAN. Although the age accuracy of this invention is lower than ChildGAN, the aging images generated by ChildGAN often do not match the appearance of children, and their quality is far lower than that of this invention.

[0142] Simulation 3: The faces of young people of different ages were aged using the present invention and several existing representative methods such as CAAE, StarGAN, D2AAE, ChildGAN, and AgeGAN++ under the same experimental environment. The aged images were then transmitted to the Face++ platform for facial landmark localization, and the image recognition rate of each method was calculated. The results are shown in Table 3.

[0143] Table 3 Comparison of image recognition rates for each method

[0144] CAAE AgeGAN++ StarGAN ChildGAN The present invention Recognition accuracy 69％ 79％ 78.6％ 67.92％ 83.2％

[0145] As can be seen from Table 3, the image recognition accuracy of the present invention is better than that of other existing algorithms. That is, the face generated by the present invention has clear facial contours and features, and has higher image quality.

[0146] The above description is merely a specific example of the present invention and does not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

[0147] It should be noted that the step numbers in the specification and claims of this invention are only for the purpose of clearly describing the embodiments of this invention and facilitating understanding, and their order is not limited.

[0148] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

Claims

1. A low-age face aging method based on age and key point guided generation, characterized by, The method comprises the following steps: (1) obtaining n pieces of face data of people aged 0-20 from a public face data set, dividing the face data into a training set T and a test set M in a ratio of 9:1, and dividing the images in the training set and the test set into 5 categories according to age, i.e., 1-3 years old, 4-8 years old, 9-12 years old, 13-16 years old and 17-20 years old, and defining the 5 categories as age labels 1-5 in sequence, that is, assigning a corresponding age label to each image; (2) performing face key point detection on the original images in the training set T, converting the obtained face key point information into a Gaussian distribution graph, and normalizing the Gaussian distribution graph; (3) constructing a face image generation adversarial network APGAN based on key point information guidance and cycle conditional generation, which comprises a weight-shared forward and reverse generator G and a multi-task discriminator D: (3a) constructing the generator G for adding aging features to the original face structure to generate high-quality aging images: (3a1) establishing a face image and key point information attention fusion module FIKFAM for accurately integrating face image features and key point information and highlighting important face regions; (3a2) establishing an encoder comprising a feature extraction module, a convolution kernel with a size of 3x3, a FIKFAM module and 6 residual blocks; (3a3) establishing a decoder comprising two deconvolution layers, a convolution layer, three normalization layers and three ReLU activation layers for generating an output image from a feature vector; (3a4) adding the outputs of the encoder and inputting them into the decoder to form the generator G; (3b) constructing a multi-task learning discriminator D comprising a true and false image discrimination function and an age verification function for judging the authenticity of the image and evaluating whether the image reflects a specific age attribute; (4) Designing continuous age transition losses : ， in It is an input image. The output image of generator G, From the picture Key points extracted from From the picture Key points extracted from Target age tag; (5) According to the discriminative prior loss , the age verification loss , the reconstruction cycle consistency loss , and the continuous age transition loss , an overall loss function is constructed ; the calculation formulas are as follows: ， ， ， ， ， in, It is a target age image. It is the original age image. It is the original age tag. It is the target age tag. This is a Gaussian plot of key points corresponding to the original age. It is a Gaussian plot of key points extracted from the target age image generated from the middle. It is a generated face image of the target age. It is the generated target age image. It is the input image. The output image of generator G, From the picture Key points extracted from From the picture Key points extracted from It is the target age tag; They are Weighting coefficients; (6) training the face image generation adversarial network APGAN through the training data set T using a back propagation algorithm until the overall loss function of the network converges, and obtaining a trained face generation model; (7) loading the face images in the test data set M into the trained face generation model to generate face images of each age stage.

2. The method of claim 1, wherein, In step (1), the labels 1-5 are assigned to the images of the 5 age categories by using one-hot encoding, and the representation is as follows: Tag 1: i.e. when the age range is 1-3 years, the first column is 1 and the remaining columns are 0; Tag 2: i.e. when the age range is 4-8 years, the second column is 1, and the rest of the columns are 0; Tag 3: i.e. when the age range is 9-12 years, the third column is 1 and the remaining columns are 0; Tag 4: i.e. when the age range is 13-16 years, the fourth column is 1 and the remaining columns are 0; Tag 5: i.e. when the age range is between 17 and 20 years, the fifth column is 1 and the remaining columns are 0.

3. The method of claim 1, wherein, In step (2), the original images in the training set T are subjected to face key point detection, and the obtained face key point information is converted into a Gaussian distribution graph, and the Gaussian distribution graph is normalized, and the implementation steps include the following: (2a) inputting each image in the training set T into an existing CNN face key point detector based on nonlinear local regression to determine whether the face image exists; If the face image exists, 68 important feature points of the face in the image are located by a face key point detector to obtain a two-dimensional matrix of size 68 2, wherein the first dimension represents a total of 68 key points, and the second dimension represents the coordinates of each key point in the image. if it is determined that the face image does not exist, the image is removed from the training set T, and the next image in the training set is continuously detected; (2b) converting the face key point information into a Gaussian distribution graph and normalizing the Gaussian distribution graph: initializing blank images: initializing 68 blank images with a size of 128x128 and the same size as the original image for each key point; A two-dimensional Gaussian function is applied to the key point position on the corresponding blank image to calculate the corresponding Gaussian distribution map Q, and the calculation process is as follows: , wherein, is the coordinate of the key point, σ is the standard deviation of the Gaussian kernel, x and y are the coordinates on the image, is a two-dimensional Gaussian function; The Gaussian distribution map Q is normalized to maintain consistency of image data and ensure that the values are within a specific range.

4. The method of claim 1, wherein, In step (3a1), a face image and key point information attention fusion module FIKFAM is established, and the following is implemented: (3a1.1) A Module submodule is established, which includes an image and a key point upper and lower route, a Sigmoid function layer, a multiplier and an adder: The upper image route and the lower key point route have the same structure, both of which include two convolutional layers, two normalization layers, a ReLU activation layer and a Sigmoid function layer, and the specific structure is: first convolutional layer→first normalization layer→ReLU activation layer→second convolutional layer→second normalization layer; The Sigmoid function layer is used to map the output Mask of the lower key point route to the range of (0, 1); The multiplier is used to multiply the output of the lower key point route after the Sigmoid function layer with the output of the upper image route; The adder is used to add the multiplication result and the input of the previous image route addition (3a1.2) Six submodules Module with the same architecture are cascaded to form a face image and key point information attention fusion module FIKFAM.

5. The method of claim 1, wherein, In step (3a2), an encoder is established, which includes a feature extraction module, a convolution kernel, a FIKFAM module and six residual blocks, and the following is implemented: The feature extraction module includes three convolutional layers, three normalization layers and three ReLU activation layers, and the structure is: first convolutional layer→first normalization layer→first ReLU activation layer→second convolutional layer→second normalization layer→second ReLU activation layer→third convolutional layer→third normalization layer→third ReLU activation layer; The convolution kernel has a step size of 1, a size of 3x3 and an edge padding size of 1; The six residual blocks have the same structure, and each residual block is a convolutional layer with a step size of 1, a size of 3x3 and an edge padding size of 0; Module A is formed by connecting the feature extraction module and the FIKFAM module in series, Module B is formed by connecting the convolution kernel in parallel with Module A, and the encoder is formed by cascading the six residual blocks with Module B.

6. The method of claim 1, wherein, In step (3a3), a decoder is established, which includes two deconvolutional layers, a convolutional layer, three normalization layers and three ReLU activation layers, and the structure is: first deconvolutional layer→first normalization layer→first ReLU activation layer→second deconvolutional layer→second normalization layer→second ReLU activation layer→first convolutional layer→third normalization layer→third ReLU activation layer.

7. The method of claim 1, wherein, In step (3b), a multi-task learning discriminator D with true and false image discrimination and age verification functions is constructed, which is composed of seven convolution kernels with increasing channel numbers layer by layer and two fully connected layers FC1 and FC2 with different output parameters, and the true and false image discrimination and age verification are implemented as follows: The first fully connected layer FC1 converts the extracted features into one-dimensional data, which represents the probability that the image is considered to be a real image, to identify whether the input image is provided by the data set or generated by the generator, that is, to realize true and false image discrimination; The second fully connected layer FC2 maps the extracted features into 5-dimensional data equal to the number of age categories, and compares with the target age label to evaluate whether the image reflects a specific age attribute, i.e. to achieve age verification.

8. The method of claim 1, wherein, Step (6) uses the back propagation algorithm to train the face image generative adversarial network APGAN through the training data set T, and realizes the following: (6a) Set the hyperparameters of back propagation: set the initial learning rate to 0.0001, and the batch size to 16; (6b) performing face key point detection on the training set T, converting the obtained face key point information into a Gaussian distribution graph, and performing normalization on the Gaussian distribution graph to obtain a normalized Gaussian distribution graph ; (6c) Combine the images in the training set T with their corresponding Gaussian distribution maps The input is fed into the APGAN network to generate the face aging output; (6d) calculating the overall loss of the training set according to the face aging output result and updating the parameters of the network using the back propagation algorithm; (6e) Repeat (6c)~(6d) until the total loss function converges, and obtain the trained face image generative adversarial network.

9. The method of claim 1, wherein, Step (7) loads the face images in the test data set M into the trained face generative model to generate face images of each age stage, which is realized as follows: (7a) Take c test images from the test set, input the images into the trained APGAN network, load the trained model weights, and obtain the corresponding generated results of the c test images; (7b) Traverse the test set until the generated results corresponding to all images in the test set are output.

Images