Continuous face aging image generation method based on normalized stream model

Abstract

The invention discloses a continuous face aging image generation method based on a normalized stream model, and relates to the technical field of normalized streams, and the method comprises the following steps: S1, firstly, setting a specific age range because a selected FFHQ data set comprises a large-span face from an infant period to an old period; s2, constructing a data preprocessing module, and performing down-sampling on all input images with a fixed size; (S3); and constructing a reversible mapping module of the age attribute space and the prior hidden variable space. According to the method, a strategy of combining a normalized flow model with a continuous sampling advantage on a high-dimensional space and a StyleGAN framework with a strong generation capability is ingeniously combined, and a high-dimensional age attribute space is reversibly mapped to a continuously changing prior potential space; therefore, continuous age changes are accurately represented, fine-grained face changes along with the age changes are captured, the fine-grained face changes are paired with the expression ability of a pre-trained and fixed StyleGAN generator, and a powerful cross-age face image change framework is constructed.

Description

Technical Field

[0001] The invention relates to the technical field of normalized flow, and in particular to a method for generating continuous human face aging images based on a normalized flow model. Background Art

[0002] With the rapid development of computer technology, the field of computer vision has attracted much attention from all walks of life. Facial image analysis is one of the research hotspots in this field, and involves conveying important feature information such as identity, age, emotion and expression. However, facial features change with age. In criminal investigation and public security applications, the acquired facial images often have age differences from the faces to be compared. The cross-age domain transformation of faces, namely face aging, requires maintaining the accuracy of synthesis effects and identity information while generating cross-age faces, so as to render a specified face image with a specific age effect. However, due to the extremely high acquisition cost and the lack of face training samples of the same person over a long age span, face aging remains a very challenging task in computer vision.

[0003] In order to cope with the increasingly severe problem of cross-age changes in human faces, the rise of four deep generative models, including Generative Adversarial Net (GAN), Variational Auto-Encoder (VAE), Normalizing Flow and Diffusion Models, has led to unprecedented development in the synthesis of cross-age face images. Among these models, Conditional Generative Adversarial Network (CGAN) is a typical GAN ​​method that can generate target images with certain attributes by introducing prior information. In order to ensure identity consistency during image generation, the paper Face aging with identity-preserved conditional generative adversarial networks proposes an identity-preserving conditional GAN, which uses the identity module to retain identity information on the basis of CGAN to achieve face generation, but there are problems such as poor generation quality, unstable training, and little knowledge of the properties of the latent space. In the paper Only amatter of style: Age transformation using a style-based regression model, Alaluf et al. proposed an image-to-image translation method, which learns to directly encode real face images into the latent space of a pre-trained unconditional GAN ​​(such as StyleGAN) and operates under a given aging offset. Unlike methods that only operate in the latent space using a priori for the path that controls age, this method learns a more decoupled nonlinear path, which proves its end-to-end nature and combines the rich semantic latent space of StyleGAN to allow for deeper fine-grained editing of generated images.

[0004] The face aging model based on Generative Adversarial Network (GAN) is a typical method. It divides the face dataset into different age groups, inputs the face image to be transformed into the generator, and relies on the discriminator and classifier to map the output image to the face distribution of the specified age group. However, face images have the characteristic of changing smoothly and gradually with age. Since this method of dividing age into several discrete domains can only learn the changes of faces between different age groups, it cannot capture the characteristics of face images changing continuously with age. In contrast, other methods guide the model to perform more refined age transformation through pre-trained or synchronously trained refined age classifiers or regressors. However, this type of method overly relies on the training effect of the classifier and the large-scale data required for age classifier training, which brings a large data cost. Since Generative Adversarial Network (GAN) can only implicitly learn the distribution of the data space and does not give the probability distribution function of the data, in order to achieve the natural effect of smooth and gradual changes of face images with age, the model needs to be able to accurately model and characterize the distribution of age attribute variables and establish a two-way connection between the true distribution of the data and the prior distribution.

[0005] In summary, although existing face aging synthesis methods have made some progress, there are still problems such as inconsistency in facial identity as age increases, ambiguity in image mapping caused by changes in age attributes, and inability to capture continuous fine-grained aging features. Summary of the invention

[0006] The purpose of the present invention is to provide a method for generating continuous face aging images based on a normalized flow model to solve the problems of the existing face aging technology proposed in the above background technology, such as inconsistency in face identity, ambiguity in image mapping caused by changes in age attributes, and inability to capture continuous fine-grained aging features.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for generating continuous face aging images based on a normalized flow model, the method for generating continuous face aging images based on a normalized flow model comprising the following steps: S1: First, since the selected FFHQ dataset includes a wide range of faces from infancy to old age, a specific age range is set.

[0008] S2: Construct a data preprocessing module, downsample all input images to a fixed size, and perform normalization.

[0009] S3: Construct a reversible mapping module between the age attribute space and the prior latent variable space. Based on the principle of continuous probability distribution transformation, a reversible transformation function is constructed to transform the complex age attribute distribution in the high-dimensional space into a basic simple distribution, thereby accurately mapping the probability relationship between the prior distribution and the complex distribution and achieving maximum likelihood estimation. S4: Construct a feature alignment module based on encoder and decoder, use the decoder with feature pyramid, obtain three levels of feature vectors from the vector of the potential age space obtained by the reversible mapping module between the age attribute space and the prior latent variable space, and use three feature projection strategies for the three levels of feature vectors respectively, extract the style from the feature map, and finally obtain the face age attribute encoding aligned with the content style; S5: Construct an attention feature fusion module based on the attention mechanism, use two linear layers and a Sigmoid function to calculate the dynamic weights between content features and age features, and finally multiply the content features and age features with the shared weights and add them together to obtain the final feature vector input into StyleGAN.

[0010] S6: Construct a module combining a pre-trained and fixed StyleGAN generator and encoder, and input the 18 style vectors output by the attention feature fusion module into the generator to generate aging images of the required age.

[0011] S7: Input the face images in the test set into the trained model to obtain an aging image that changes continuously and smoothly with age.

[0012] Preferably, in S1, since the minimum age range of the FFHQ dataset is 0-2 and the maximum age range is 70-120, we select 1-100 as the general age range.

[0013] Preferably, in S2, the specific method of image preprocessing is to downsample all input images to We implemented it and adopted the VGG network as the pre-trained age classification network. We normalized the images with a mean of [0.485, 0.456, 0.406] and a variance of 1.

[0014] Preferably, in S3, it is assumed that the age attribute distribution is , the easy-to-sample prior latent space is , then the transformation relationship between the two in the flow model is , where it is assumed is reversible and The dimensions and The dimensions are the same, in order to fit a distribution , the change rules are as follows: in is a function exist The Jacobian matrix on , so that the determinant of the Jacobian matrix can be obtained trivially. is reversible, then It can also be directly based on Preferably, in S4, first, we use a decoder with a feature pyramid to obtain three levels of feature vectors from the vectors of the potential age space obtained from the reversible mapping module between the age attribute space and the prior latent variable space. And use three feature projection strategies for the three levels of feature vectors respectively to extract styles from the feature maps , , Finally, we get the face age attribute encoding that is aligned with the content style. : , , Then, the face age attribute is encoded The content style of the input image All 18 style vectors are learned together and input into the generator to generate the output image. Preferably, in S5, the content vector is fused using the attention feature fusion module. and age vector Dynamic weights are calculated to adaptively adjust the fusion weights of content features and style features to better capture the subtle differences between content features and style features, thereby improving the performance of the model.

[0015] Preferably, in S6, a fixed, pre-trained e4e encoder is used as a universal identity encoder to obtain the content feature latent space , combined with age attribute encoding , and are sent to StyleGAN to generate images. Furthermore, during the model training process, the loss function is: in, , , , and A weighting parameter to weigh different losses; is the loss for learning pixel-level similarity, is the loss for learning perceptual similarity, It is to ensure the consistency of the identity of the image while modifying the facial age attribute characteristics. is the label smoothing regularization loss as the classification loss, In order to achieve accurate modulation of latent variables, the maximum log-likelihood function in the normalized flow is introduced to constrain. The loss of adversarial learning between the generated face aging image and the existing real target age image is used to improve the quality of the model generated image. Compared with the prior art, the beneficial effects of the present invention are: the method cleverly combines the strategy of the normalized flow model with continuous sampling advantage in high-dimensional space and the StyleGAN framework with powerful generation ability, and accurately represents the continuous age change and captures the fine-grained facial changes with age by reversibly mapping the high-dimensional age attribute space to the continuously changing prior latent space, and pairs it with the expressive power of the pre-trained, fixed StyleGAN generator, thus constructing a powerful cross-age face image change framework; The present invention provides a method for generating continuous face aging images based on a normalized flow model, comprising: a reversible mapping module between an attribute space and a priori latent variable space, a feature alignment module based on an encoder and a decoder, a module for dynamically weighting content features and age features using an attention feature fusion mechanism, and a module for pairing a pre-trained and fixed StyleGAN generator with an encoder, wherein: The reversible mapping module between the attribute space and the prior latent variable space adopts the connection model structure in the NICE (Non-linear Independent Component Estimation) model, and is composed of multiple coupling layers and a scaling layer. The coupling layer uses the affine coupling method in the RealNVP (real-valued non-volume preserving) model, and uses random shuffling operations to effectively perform reversible mapping and Jacobian determinant calculations, realize reversible mapping between the attribute space and the prior latent variable space, and perform continuous sampling of the age attribute space, preventing image mapping ambiguity while capturing more continuous fine-grained features.

[0016] The feature alignment module based on the encoder and decoder uses a decoder with a feature pyramid to obtain three levels of feature vectors from the vector of the potential age space obtained from the reversible mapping module of the attribute space and the prior latent variable space, and uses three feature projection strategies to extract the style patterns of the bottom, middle and high layers from the feature map, and finally obtains 18 512-dimensional face age attribute codes aligned with the content style. The module that dynamically weights the content features and age features using the attention feature fusion mechanism uses two linear layers, two ReLU activation functions and a Sigmoid function to dynamically calculate the attention weight coefficient after the content features and age features are spliced, and finally the attention weight is added to the features after the content features and age features are multiplied in a shared form, and finally a fully fused feature is obtained.

[0017] The method uses a pre-trained and fixed StyleGAN generator and encoder pairing module to create an age encoder, and combines the input face image with the continuous high-dimensional age change distribution processed by the normalized stream and encodes it into a series of style vectors. These age-varying vectors and the face content vector obtained by the e4e encoder are then dynamically weighted through an attention feature fusion module, and finally input into StyleGAN together to generate an output image that more smoothly and accurately represents the required age transformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 (a) is the overall framework of the embodiment of the present invention; Figure 1 (b) is the details of the normalized flow architecture for implementing reversible mapping in an embodiment of the present invention; Figure 1 (c) is the details of the attention feature fusion module in the embodiment of the present invention; Figure 2 This is the visualization result of the embodiment of the present invention on the FFHQ dataset; Figure 3 This is the experimental table 1 of the present invention; Figure 4 This is Experimental Table 2 of the present invention. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0020] See also Figure 1-2 The present invention provides a technical solution: a method for generating continuous face aging images based on a normalized flow model, and the method for generating continuous face aging images based on a normalized flow model comprises the following steps: S1: First, since the selected FFHQ dataset includes a wide range of faces from infancy to old age, a specific age range is set.

[0021] S2: Construct a data preprocessing module, downsample all input images to a fixed size, and perform normalization.

[0022] S3: Construct a reversible mapping module between the age attribute space and the prior latent variable space. Based on the principle of continuous probability distribution transformation, a reversible transformation function is constructed to transform the complex age attribute distribution in the high-dimensional space into a basic simple distribution, thereby accurately mapping the probability relationship between the prior distribution and the complex distribution and achieving maximum likelihood estimation. S4: Construct a feature alignment module based on encoder and decoder, use the decoder with feature pyramid, obtain three levels of feature vectors from the vector of the potential age space obtained by the reversible mapping module between the age attribute space and the prior latent variable space, and use three feature projection strategies for the three levels of feature vectors respectively, extract the style from the feature map, and finally obtain the face age attribute encoding aligned with the content style; S5: Construct an attention feature fusion module based on the attention mechanism, use two linear layers and a Sigmoid function to calculate the dynamic weights between content features and age features, and finally multiply the content features and age features with the shared weights and add them together to obtain the final feature vector input into StyleGAN.

[0023] S6: Construct a module combining a pre-trained and fixed StyleGAN generator and encoder, and input the 18 style vectors output by the attention feature fusion module into the generator to generate an aging image of the required age. S7: Input the face images in the test set into the trained model to obtain an aging image that changes continuously and smoothly with age; In S1, since the minimum age range of the FFHQ dataset is 0-2 and the maximum age range is 70-120, we select 1-100 as the general age range.

[0024] In S2, the specific method of image preprocessing is to downsample all input images to We implemented it and adopted the VGG network as the pre-trained age classification network. We normalized the images with a mean of [0.485, 0.456, 0.406] and a variance of 1.

[0025] In S3, according to the established reversible mapping rule, we set an encoder consisting of three convolutional layers and four residual blocks. On the basis of the encoder, the age attribute variable Embedded in facial features. During the training process, face images and the randomly obtained target age For different data sets, we will preset an age range , making it easy to sample , the latent space meets Uniform distribution, as shown in the formula: in, are all 512-dimensional latent vectors related to age. Then, according to the change rules in the above normalized flow, the age attribute variable is obtained: like Figure 1As shown in (b), the present invention adopts the affine coupling method in RealNVP, so we will get the age attribute variable It is divided into two parts according to the dimension, namely and , one is fused with the shallow features of the face image in the form of age weight factors, and the other is fused with the deep features, and finally combined and input into the decoder.

[0026] In S4, a decoder with a feature pyramid is used to obtain a vector about the potential age space obtained by the reversible mapping module between the age attribute space and the prior latent variable space. The feature vector is obtained by three different feature alignment modules, in which the shallow features are aligned using average pooling and linear layers. Perform feature alignment to obtain feature style , the middle-level feature vector Through average pooling, The 2D convolution and linear layers get the feature pattern , and finally the high-level feature vector Through average pooling, The 2D convolution and linear layers get the feature pattern Finally, the three levels of styles are combined into 18 facial age attribute codes corresponding to the StyleGAN latent space. In S5, we use Attention Feature Fusion (AFF), which Figure 1 (c) is constructed by two linear layers, two ReLU activation functions and one Sigmoid activation function to calculate the dynamic attention weight: In S6, the face age attribute is encoded Content style obtained with the pre-trained e4e encoder The feature vector containing age and content learned jointly through the attention fusion mechanism is input into the pre-trained StyleGAN generator For generating high-resolution photorealistic faces from latent space ,in Represents the image size, thereby achieving an intermediate style representation that completes the translation from input pixels to output pixels.

[0027] Furthermore, we use Loss learns pixel-level similarity, using Loss learning perceived similarity: Where: F() represents the perceptual feature extractor. As humans age, their head shape changes naturally over time. Inspired by this, we perform and Loss constraints.

[0028] Identity preservation loss. In order to ensure the consistency of the identity of the image while modifying the facial age attribute features, the identity feature constraints are performed with the help of the pre-trained face recognition network: Where: R is the pre-trained Arcface face recognition network.

[0029] Classification loss. Although the maximum and minimum values ​​in each age range belong to two different categories, the degree of aging of the face is actually not much different. In order to meet this challenge, this paper uses label smoothing regularization loss as the classification loss. By "softening" the traditional one-hot type label, it can effectively suppress overfitting when calculating the loss value, so that face images that change more smoothly with age can be generated.

[0030] in, represents the classification cross entropy loss. The probability predicted by the classifier is expressed as Indicates that , is the classifier. The number of samples is , the number of categories is For the input sample , Indicates the target category The predicted probability of .

[0031] Flow loss. and The change rules between them can get the latent variable mapping loss:

[0032] GAN loss. The model feeds the generated aged image and the original target age image into the discriminator to form adversarial training. This paper adopts the loss form of LSGAN, where the loss of the discriminator is: The loss of the generator is: Total loss: The development language of the present invention adopts Python language, the system of the development environment adopts Ubuntu system, the deep learning framework adopted is PyTorch framework, and the dataset FFHQ is a high-quality face dataset, which contains 70,000 high-definition face images with a resolution of 1024×1024, and has a lot of changes in face attributes, with different ages, genders, races and other characteristics. The CelebA-HQ dataset is a high-quality version of the CelebA dataset, which contains 30,000 face images with a resolution of 1024×1024. Our model is mainly trained and verified on FFHQ, and evaluated on the CelebA-HQ dataset.

[0033] In order to verify the effectiveness of the above-mentioned method for generating continuous face aging images based on the normalized flow model, an experimental verification is carried out in the present invention. We use 95% of the dataset as the training set, and the remaining faces and CelebA-HQ as the test data. According to the age distribution of the FFHQ dataset, the age range of the FFHQ dataset is set to 1 to 100 years old, and the Adam optimizer is used to train the model.

[0034] like Figure 3 shown.

[0035] We use Face++'s online face analysis tool to evaluate the age of each age group on FFHQ and calculate the mean absolute error for each group of images. As shown in the table, we quantitatively compare the proposed model with LATS, CUSP and FADING on the FFHQ dataset. Lower MAE indicates that we have better aging accuracy, and we observe that the age errors generated by our model are closer to the real data in most age groups.

[0036] like Figure 4 shown.

[0037] In addition, we provide age-independent gender attribute preservation to evaluate the aging methods. It is noteworthy that we achieve the optimal gender preservation rate in all age groups except 15-19, 30-39, and 40-49, where the performance is suboptimal.

[0038] In S6, input the images in the test set and output the facial aging images you want for a specific age or even a wide age range. The visualization results on FFHQ are as follows: Figure 2 As shown, it is proved that the present invention establishes a continuous reversible mapping between the age attribute vector and the prior potential vector on the basis of maintaining identity consistency, realizes the controllable modulation of the age attribute, captures the fine-grained facial structure and texture features that change continuously with age, and obtains a smoother and more accurate facial aging image.

[0039] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for generating continuous face aging images based on a normalized flow model, characterized in that: The method for generating continuous face aging images based on the normalized flow model includes the following steps: S1: First, since the selected FFHQ dataset includes faces from infancy to old age, a specific age range is set. S2: Construct a data preprocessing module to downsample all input images to a fixed size and perform normalization. S3: Construct a reversible mapping module between the age attribute space and the prior latent variable space. Based on the principle of continuous probability distribution transformation, a reversible transformation function is constructed to convert the complex age attribute distribution in the high-dimensional space into a basic simple distribution, thereby accurately mapping the probability relationship between the prior distribution and the complex distribution and realizing maximum likelihood estimation; S4: Construct a feature alignment module based on an encoder and a decoder. Using a decoder with a feature pyramid, three levels of feature vectors are obtained from the vector of the potential age space obtained from the reversible mapping module between the age attribute space and the prior latent variable space, and three feature projection strategies are used for the three levels of feature vectors respectively to extract styles from the feature maps, and finally obtain the face age attribute encoding aligned with the content style; S5: Construct an attention feature fusion module based on the attention mechanism. Two linear layers and a Sigmoid function are used to calculate the dynamic weights between the content features and the age features. Finally, the content features and the age features are multiplied by the shared weights and then added to obtain the final feature vector input into StyleGAN. S6: Construct a module combining a pre-trained and fixed StyleGAN generator and encoder, and input the 18 style vectors output by the attention feature fusion module into the generator to generate an aging image of the required age. S7: Input the face images in the test set into the trained model to obtain an aging image that changes continuously and smoothly with age.

2. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that: In S1, since the minimum age range of the FFHQ dataset is 0-2 and the maximum age range is 70-120, we select 1-100 as the general age range.

3. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that : In S2, the specific method of image preprocessing is to downsample all input images to We implemented it and adopted the VGG network as the pre-trained age classification network. We normalized the images with a mean of [0.485, 0.456, 0.406] and a variance of 1.

4. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that : In S3, assume that the age attribute distribution is , the prior latent space that is easy to sample is , then the transformation relationship between the two in the flow model is , where it is assumed is reversible and The dimensions and The dimensions are the same, in order to fit a distribution , the change rules are as follows: in is a function exist The Jacobian matrix on , so that the determinant of the Jacobian matrix can be obtained trivially. is reversible, then It can also be directly based on Directly derived.

5. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that In S4, first, we use the decoder with feature pyramid to obtain three levels of feature vectors from the vectors of the latent age space obtained from the reversible mapping module between the age attribute space and the prior latent variable space: And use three feature projection strategies for the three levels of feature vectors respectively to extract styles from the feature maps , , , and finally get the face age attribute encoding aligned with the content style : Then, the face age attribute is encoded The content style of the input image All 18 style vectors are learned together and fed into the generator to produce the output image.

6. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that In S6, a fixed, pre-trained e4e encoder is used as a universal identity encoder to obtain the content feature latent space , combined with age attribute coding , and are sent to StyleGAN to generate images. Furthermore, during the model training process, the loss function is: in, , , , and Weighting parameters that weigh different losses; is the loss for learning pixel-level similarity, is the loss for learning perceptual similarity, It is to ensure the consistency of the identity of the image while modifying the facial age attribute characteristics. is the label smoothing regularization loss as the classification loss, In order to achieve accurate modulation of latent variables, the maximum log-likelihood function in the normalized flow is introduced to constrain. It is the loss of adversarial learning between the generated face aging images and the existing real target age images, which is used to improve the quality of the images generated by the model.

Images