A method for generating continuous face aging images based on a normalized flow model

Abstract

This invention discloses a method for generating continuous aging face images based on a normalized flow model, belonging to the field of normalized flow technology, and includes the following steps: S1: First, since the selected FFHQ dataset includes faces spanning a wide range from infancy to old age, a specific age range is set; S2: A data preprocessing module is constructed to downsample all input images to a fixed size; S3: A reversible mapping module is constructed between the age attribute space and the prior latent variable space. This method cleverly combines the normalized flow model, which has the advantage of continuous sampling in high-dimensional spaces, with the StyleGAN framework, which has powerful generative capabilities. By reversibly mapping the high-dimensional age attribute space to a continuously changing prior latent space, it accurately represents continuous age changes and captures fine-grained facial changes with age. This is then paired with the expressive power of a pre-trained, fixed StyleGAN generator to construct a powerful framework for cross-age face image changes.

Description

Technical Field

[0001] This invention relates to the field of normalized flow technology, specifically to a method for generating continuous aging face images based on a normalized flow model. Background Technology

[0002] With the rapid development of computer technology, the field of computer vision has attracted widespread attention. Facial image analysis is one of the research hotspots in this field, involving the transmission of important feature information such as identity, age, emotion, and expression. However, facial features change with age. In criminal investigation and public security applications, the acquired facial images often differ in age from the faces to be compared. For example, in predicting the current appearance of suspects or missing persons, or in conducting cross-age identity verification, cross-age domain transformation of faces becomes crucial. Cross-age domain transformation of faces, also known as face aging, requires maintaining the accuracy of the synthesis effect and identity information while generating cross-age faces, in order to render a specified facial image with a specific age effect. However, due to the extremely high cost of data acquisition and the lack of training samples of the same person's faces over a long period of time, face aging remains a highly challenging task in computer vision.

[0003] To address the increasingly serious problem of facial changes across age, the rise of four major deep generative models—Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Normalizing Flow, and Diffusion Models—has led to unprecedented advancements in the synthesis of cross-age facial images. Among these models, Conditional GANs (CGANs), as a typical GAN ​​method, can generate target images with certain attributes by incorporating prior information. To ensure identity consistency during image generation, the paper "Face aging with identity-preserved conditional generative adversarial networks" proposes an identity-preserving conditional GAN. This network, based on CGAN, utilizes an identity module to preserve identity information for face generation. However, it suffers from poor generation quality, unstable training, and limited knowledge of the properties of the latent space. In the paper "Only amatter of style: Age transformation using a style-based regression model," Alaluf et al. proposed an image-to-image translation method. This method learns to directly encode real face images into the latent space of a pre-trained unconditional GAN ​​(e.g., StyleGAN) and operates under a given aging offset. Unlike methods that operate solely using path priors controlling age in the latent space, this method learns a more decoupled nonlinear path. This method demonstrates its end-to-end properties and, combined with the rich semantic latent space of StyleGAN, allows for deeper, finer-grained editing of the generated images.

[0004] Facial aging models based on Generative Adversarial Networks (GANs) are a typical approach. They divide face datasets into different age groups and input the face images to be transformed into a generator. Relying on discriminators and classifiers, the output images are mapped onto the facial distribution of the specified age groups. However, facial images exhibit a smooth, gradual change with age. This method, which divides age into discrete domains, can only learn the changes in faces between different age groups and cannot capture the continuous changes in facial images over age. In contrast, other methods guide the model to perform more refined age transformations through pre-trained or synchronously trained refined age classifiers or regressors. However, these methods rely heavily on the training performance of the classifiers and the large amounts of data required for training the age classifiers, resulting in significant data costs. Since GANs can only implicitly learn the distribution of the data space and do not provide the probability distribution function of the data, to achieve a natural effect of smooth, gradual changes in facial images with age, the model needs to accurately model and represent the distribution of age attribute variables, establishing a bidirectional relationship between the true distribution of the data and the prior distribution.

[0005] In summary, although existing face aging synthesis methods have made some progress, they still suffer from problems such as inconsistency in face identity, ambiguity in image mapping caused by changes in age attributes, and inability to capture continuous fine-grained aging features as age increases. Summary of the Invention

[0006] The purpose of this invention is to provide a method for generating continuous aging face images based on a normalized flow model, in order to solve the problems of existing face aging techniques mentioned in the background art, such as inconsistency in face identity, ambiguity in image mapping caused by changes in age attributes, and inability to capture continuous fine-grained aging features.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for generating continuous aging face images based on a normalized flow model, the method comprising the following steps:

[0008] S1: First, since the selected FFHQ dataset includes a wide range of faces from infancy to old age, a specific age range is set.

[0009] S2: Construct a data preprocessing module to downsample all input images to a fixed size and perform normalization processing.

[0010] S3: Construct an invertible mapping module between the age attribute space and the prior latent variable space. Based on the principle of continuous probability distribution transformation, by constructing an invertible transformation function, the complex age attribute distribution in the high-dimensional space is transformed into a basic simple distribution, thereby accurately mapping the probability relationship between the prior distribution and the complex distribution and realizing maximum likelihood estimation.

[0011] S4: Construct a feature alignment module based on encoder and decoder. Utilize the decoder with feature pyramid to obtain feature vectors of three levels about the latent age space from the invertible mapping module between the age attribute space and the prior latent variable space. Apply three feature projection strategies to the feature vectors of these three levels respectively to extract the style from the feature map and finally obtain the face age attribute code aligned with the content style.

[0012] S5: Construct an attention-based feature fusion module, which uses two linear layers and a sigmoid function to calculate the dynamic weights between content features and age features. Finally, the content features and age features are multiplied by the shared weights and then added together to obtain the final feature vector of the input StyleGAN.

[0013] S6: Construct a module that combines a pre-trained and fixed StyleGAN generator and encoder, and inputs the 18 style vectors output by the attention feature fusion module into the generator to generate an aged image of the required age.

[0014] S7: Input the face images from the test set into the trained model to obtain aging images that change continuously and smoothly with age.

[0015] Preferably, in S1, since the minimum age range of the FFHQ dataset is 0-2 and the maximum age range is 70-120, we select 1-100 as the general age range.

[0016] Preferably, in S2, the specific method of image preprocessing is to downsample all input images to 256×256, and thus adopt the VGG network as the pre-trained age classification network. We use the mean [0.485, 0.456, 0.406] and the variance of 1 to normalize the images.

[0017] Preferably, in S3, it is assumed that the age attribute distribution is p w The easy-to-sample prior latent space is p. z In the flow model, the transformation relationship between the two is z = f(w), where f is assumed to be invertible and the dimensions of z and w are the same. This is to fit a distribution p. z The rules for change are as follows:

[0018]

[0019] in Let f be the Jacobian matrix of the function f over w. We use f to make the determinant of the Jacobian matrix trivial. Since f is invertible, w can also be directly derived from w = f. -1 (z) is obtained directly.

[0020] Preferably, in S4, firstly, we utilize a decoder with a feature pyramid to obtain the three-level feature vector F of the latent age space from the invertible mapping module between the age attribute space and the prior latent variable space. l F m F h Three feature projection strategies are applied to the feature vectors at these three levels respectively to extract patterns from the feature maps. The final result is a facial age attribute code aligned with the content style.

[0021] w l =Linear(AvgPool(F l ))

[0022] w m =Linear(Con2d(AvgPool(F m )))

[0023] w h =Linear(Con2d(AvgPool(F h )))

[0024] Then, the facial age attribute is encoded w e Together with the content style w of the input image, all 18 style vectors are learned and fed into the generator to produce the output image.

[0025] Preferably, in S5, the attention feature fusion module is used to combine the content vector w and the age vector w e Calculate dynamic weights and adaptively adjust the fusion weights of content features and style features to better capture subtle differences between content features and style features, thereby improving the model's performance.

[0026] Preferably, in S6, a fixed, pre-trained e4e encoder is used as a general identity encoder to obtain the content feature latent space W = e4e(X) ∈ R. 18×512 Combined with age attribute coding Both are fed into StyleGAN to generate images.

[0027] Furthermore, during model training, the loss function is:

[0028] L=λ pix L pix +λ lpips L LPIPS +λ id L ID +λ lsr L LS R+λ flow L flow +L GAN

[0029] Where, λ pix ,λ lpips , λ id , λ lsr and λ flow Weighting parameters to balance different losses;

[0030] L pix It is a loss function that learns pixel-level similarity, L LPIPS It is the loss of learning perceptual similarity, L ID This ensures the consistency of identity in an image while modifying facial age attributes. LSR The label smoothing regularization loss is used as the classification loss, L Flow To achieve accurate modulation of latent variables, the maximum log-likelihood function in the normalized flow is introduced as a constraint, L. GAN It is a loss function that uses adversarial learning to compare the generated aged face image with the existing real target age image, in order to improve the quality of the image generated by the model.

[0031] Compared with the prior art, the beneficial effects of the present invention are: the method cleverly combines the normalized flow model with the advantage of continuous sampling in high-dimensional space with the StyleGAN framework with powerful generative capabilities. By reversibly mapping the high-dimensional age attribute space to the continuously changing prior latent space, it accurately represents continuous age changes and captures fine-grained facial changes with age. It is then paired with the expressive power of the pre-trained, fixed StyleGAN generator to construct a powerful cross-age face image change framework.

[0032] This invention provides a method for generating continuous aging face images based on a normalized flow model, comprising: an invertible mapping module between the attribute space and the prior latent variable space; a feature alignment module based on the encoder and decoder; a module for dynamically weighting content features and age features using an attention feature fusion mechanism; and a module for pairing a pre-trained and fixed StyleGAN generator with the encoder.

[0033] The invertible mapping module between the attribute space and the prior latent variable space adopts the connection model structure in the NICE (Non-linear Independent Component Estimation) model, consisting of multiple coupling layers and a scaling layer. The coupling layers use the affine coupling method in the RealNVP (real-valued non-volume preserving) model and utilize random shuffling operations to effectively perform invertible mapping and Jacobian determinant calculation, realizing the invertible mapping between the attribute space and the prior latent variable space, and performing continuous sampling of the age attribute space. This prevents image mapping ambiguity while capturing more continuous fine-grained features.

[0034] The feature alignment module based on encoder and decoder uses a decoder with a feature pyramid to obtain feature vectors of three levels about the latent age space from the invertible mapping module between the attribute space and the prior latent variable space. It then uses three feature projection strategies to extract the style patterns of the bottom, middle and high layers from the feature map, and finally obtains 18 512-dimensional facial age attribute codes aligned with the content style.

[0035] The module that dynamically weights content features and age features using an attention feature fusion mechanism uses two linear layers, two ReLU activation functions, and a Sigmoid function to dynamically calculate the attention weight coefficients after concatenating the content features and age features. Finally, the attention weights are shared and added to the features resulting from multiplying the content features and age features respectively, to obtain fully fused features.

[0036] The method uses a pre-trained and fixed StyleGAN generator and encoder pairing module to create an age encoder. It combines the input face image with a continuous high-dimensional age change distribution after normalized stream processing and encodes it into a series of style vectors. Then, these age-varying vectors are dynamically weighted with the face content vectors obtained by the e4e encoder through an attention feature fusion module, and finally input together into StyleGAN to generate a smoother and more accurate output image that represents the desired age change. Attached Figure Description

[0037] Figure 1 (a) is the overall framework of an embodiment of the present invention;

[0038] Figure 1 (b) details the normalized flow architecture for implementing reversible mapping in an embodiment of the present invention;

[0039] Figure 1 (c) Details of the attention feature fusion module in the embodiments of the present invention;

[0040] Figure 2 This is a visualization of the embodiment of the present invention on the FFHQ dataset;

[0041] Figure 3 Table 1 is the experimental table for this invention;

[0042] Figure 4 Table 2 is the experimental table for this invention. Detailed Implementation

[0043] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0044] Please see Figure 1-2 This invention provides a technical solution: a method for generating continuous aging face images based on a normalized flow model, which includes the following steps:

[0045] S1: First, since the selected FFHQ dataset includes a wide range of faces from infancy to old age, a specific age range is set.

[0046] S2: Construct a data preprocessing module to downsample all input images to a fixed size and perform normalization processing.

[0047] S3: Construct an invertible mapping module between the age attribute space and the prior latent variable space. Based on the principle of continuous probability distribution transformation, by constructing an invertible transformation function, the complex age attribute distribution in the high-dimensional space is transformed into a basic simple distribution, thereby accurately mapping the probability relationship between the prior distribution and the complex distribution and realizing maximum likelihood estimation.

[0048] S4: Construct a feature alignment module based on encoder and decoder. Utilize the decoder with feature pyramid to obtain feature vectors of three levels about the latent age space from the invertible mapping module between the age attribute space and the prior latent variable space. Apply three feature projection strategies to the feature vectors of these three levels respectively to extract the style from the feature map and finally obtain the face age attribute code aligned with the content style.

[0049] S5: Construct an attention-based feature fusion module, which uses two linear layers and a sigmoid function to calculate the dynamic weights between content features and age features. Finally, the content features and age features are multiplied by the shared weights and then added together to obtain the final feature vector of the input StyleGAN.

[0050] S6: Construct a module that combines a pre-trained and fixed StyleGAN generator and encoder, and inputs the 18 style vectors output by the attention feature fusion module into the generator to generate an aged image of the required age.

[0051] S7: Input the face images from the test set into the trained model to obtain aging images that change continuously and smoothly with age;

[0052] In S1, since the minimum age range of the FFHQ dataset is 0-2 and the maximum age range is 70-120, we select 1-100 as the general age range.

[0053] In S2, the specific method of image preprocessing is to downsample all input images to 256×256, and thus adopt the VGG network as the pre-trained age classification network. We use the mean [0.485, 0.456, 0.406] and the variance of 1 to normalize the images.

[0054] In S3, based on the established reversible mapping rule, we set up an encoder containing three convolutional layers and four residual blocks. Building upon the encoder, we embed the age attribute variable w into the facial features. During training, the facial image X and the randomly obtained target age α are used. For different datasets, we preset an age range [α]. min ,α max This makes p easy to sample. z The latent space conforms to a uniform distribution of U(0,1), as shown in the equation:

[0055]

[0056] Among them, z α They are all 512-dimensional latent vectors related to age.

[0057] Then, based on the change rules in the normalized flow described above, the age attribute variable is obtained:

[0058] w α =f -1 (z α )

[0059] like Figure 1 As shown in (b), this invention adopts the affine coupling method in RealNVP, so we will obtain the age attribute variable w α It is split into two parts according to the dimension, namely and One feature is fused with shallow features of the face image, and the other is fused with deep features, both in the form of age weighting factors. Finally, they are combined and input into the decoder.

[0060] In S4, a decoder with a feature pyramid is used to obtain a vector about the latent age space from the invertible mapping module between the age attribute space and the prior latent variable space. This vector is then processed through three different feature alignment modules to obtain feature vectors. Average pooling and linear layers are used to align the shallow features F. l Feature alignment is performed to obtain feature patterns. Mid-layer feature vector F m Feature patterns are obtained through average pooling, 1×1 2D convolutions, and linear layers. Finally, the high-level feature vector F h Feature patterns are obtained through average pooling, 1×1 2D convolutions, and linear layers. Finally, the styles of these three levels are combined into 18 facial age attribute codes corresponding to the latent space of StyleGAN.

[0061] In S5, we employ Attention Feature Fusion (AFF), which... Figure 1 (c) is constructed using two linear layers, two ReLU activation functions, and one Sigmoid activation function. The dynamic attention weights are then calculated.

[0062] σ=AFF(ω,ω e )

[0063] In S6, the facial age attribute is encoded w e The feature vector containing age and content, learned jointly by the content style w obtained from the pre-trained e4e encoder and the content style w through an attention fusion mechanism, is input into the pre-trained StyleGAN generator G to generate high-resolution, realistic faces from the latent space. Where H×W represents the image size, thus enabling intermediate style representation to translate from input pixels to output pixels.

[0064] Furthermore, we use L pi x-loss learns pixel-level similarity, using L LPIPS Loss learning of perceived similarity:

[0065]

[0066] In the formula: F() represents the perceptual feature extractor. As humans age, their head shape naturally changes over time. Inspired by this, we perform L... pix and L LPIPS Loss constraints.

[0067] Identity preservation loss. To ensure consistency of identity in images while modifying facial age attributes, a pre-trained face recognition network is used to constrain identity features:

[0068]

[0069] In the formula: R is a pre-trained Arcface face recognition network.

[0070] Classification loss. Since the maximum and minimum values ​​in each age range belong to two different categories, but the degree of facial aging is actually not much different, this paper uses label smoothing regularization loss as the classification loss to address this challenge. By "softening" the traditional one-hot type labels, it can effectively suppress overfitting when calculating the loss value, enabling the generation of facial images that change more smoothly with age.

[0071]

[0072] in, This represents the classification cross-entropy loss. The probability predicted by the classifier is expressed by p. i It means that p i = C(G(X, α)), where C is the classifier. The number of samples is N, and the number of classes is n. For input sample j, p ij This represents the predicted probability of target category i.

[0073] Flow loss. According to p z and p w The rules governing the changes between these variables allow us to derive the latent variable mapping loss:

[0074]

[0075] GAN Loss. The model feeds the generated aged image and the original target age image into the discriminator, constituting adversarial training. This paper adopts the loss form of LSGAN, where the discriminator's loss is:

[0076]

[0077] The generator's loss is:

[0078]

[0079] Total losses:

[0080] L GAN =L D +L G

[0081] This invention is developed using Python, with an Ubuntu system as the development environment. The deep learning framework used is PyTorch. The FFHQ dataset is a high-quality face dataset containing 70,000 high-resolution (1024×1024) face images with diverse facial attributes, including age, gender, and ethnicity. The CelebA-HQ dataset is a high-quality version of the CelebA dataset, containing 30,000 1024×1024 resolution face images. Our model is primarily trained and validated on the FFHQ dataset, while evaluation is performed on the CelebA-HQ dataset.

[0082] To verify the effectiveness of the aforementioned method for generating continuous aging face images based on a normalized flow model, experimental verification was conducted in this invention. We used 95% of the dataset as the training set, and the remaining faces and CelebA-HQ data as test data. Considering the age distribution of the FFHQ dataset, the age range was set to 1 to 100 years, and the Adam optimizer was used to train the model.

[0083] like Figure 3 As shown.

[0084] We used Face++'s online face analysis tool to evaluate the age of images composed of each age group on FFHQ and calculated the mean absolute error (MAE) for each group of images. As shown in the table, we quantitatively compared our proposed model with LATS, CUSP, and FADING on the FFHQ dataset. The lower MAE indicates that we have better aging accuracy, and we observed that the age error generated by our model is closer to the real data in most age groups.

[0085] like Figure 4 As shown.

[0086] In addition, we provided age-independent sex retention to evaluate the aging approach. Notably, we achieved optimal sex retention rates in all age groups except the 15-19, 30-39, and 40-49 age groups, which were suboptimal.

[0087] In S6, input images from the test set, and output facial aging images for a specific age or even a wide age range. The visualization results on FFHQ are as follows. Figure 2 As shown, this invention demonstrates that, while maintaining identity consistency, it establishes a continuous and invertible mapping between the age attribute vector and the prior latent vector, thereby achieving controllable modulation of the age attribute and capturing fine-grained facial structure and texture features that change continuously with age, resulting in smoother and more accurate facial aging images.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for generating continuous aging face images based on a normalized flow model, characterized in that: The method for generating continuous aging face images based on the normalized flow model includes the following steps: S1: First, since the selected FFHQ dataset includes a wide range of faces from infancy to old age, a specific age range is set. S2: Construct a data preprocessing module to downsample all input images to a fixed size and perform normalization processing; S3: Construct an invertible mapping module between the age attribute space and the prior latent variable space. Based on the principle of continuous probability distribution transformation, by constructing an invertible transformation function, the complex age attribute distribution in the high-dimensional space is transformed into a basic simple distribution, thereby accurately mapping the probability relationship between the prior distribution and the complex distribution and realizing maximum likelihood estimation. S4: Construct a feature alignment module based on the encoder and decoder. Utilizing a decoder with a feature pyramid, it obtains feature vectors at three levels of the latent age space from the invertible mapping module between the age attribute space and the prior latent variable space. Three feature projection strategies are then applied to these three levels of feature vectors to extract patterns from the feature maps, ultimately obtaining a face age attribute code aligned with the content patterns. In S4, firstly, we utilize a decoder with a feature pyramid to obtain feature vectors F at three levels of the latent age space from the invertible mapping module between the age attribute space and the prior latent variable space. l F m F h Three feature projection strategies are applied to the feature vectors at these three levels respectively to extract patterns from the feature maps. The final result is a facial age attribute code aligned with the content style. w l =Linear(AvgPool(F l )) w m =Linear(Con2d(AvgPool(F m ))) w h =Linear(Con2d(AvgPool(F h ))) Then, the facial age attribute is encoded w e The content style of the input image is W = e4e(X)∈R 18×512 We learn all 18 style vectors together and input them into the generator to produce the output image, where X is the original input image; S5: Construct an attention-based feature fusion module, which uses two linear layers and a sigmoid function to calculate the dynamic weights between content features and age features. Finally, the content features and age features are multiplied by the shared weights and then added together to obtain the final feature vector of the input StyleGAN. S6: Construct a module combining a pre-trained and fixed StyleGAN generator and encoder. The 18 style vectors output from the attention feature fusion module are input into the generator to produce an aging image of the desired age. A fixed, pre-trained e4e encoder is used as a general identity encoder to obtain the content feature latent space W = e4e(X) ∈ R. 18 ×512 Combined with age attribute coding The images are then fed into StyleGAN to generate a single image. Furthermore, during model training, the loss function is: L=λ pix L pix +λ lpips L LPIPS +λ id L ID +λ lsr L LSR +λ flow L Flow +L GAN Where, λ pix ,λ lpips , λ id , λ lsr and λ flow To weigh the different losses; L pix It is a loss function that learns pixel-level similarity, L LPIPS It is the loss of learning perceptual similarity, L ID This ensures the consistency of identity in an image while modifying facial age attributes. LSR The label smoothing regularization loss is used as the classification loss, L Flow To achieve accurate modulation of latent variables, the maximum log-likelihood function in the normalized flow is introduced as a constraint, L. GAN It is a loss that combines the generated aged face image with the existing real target age image for adversarial learning, in order to improve the quality of the image generated by the model; S7: Input the face images from the test set into the trained model to obtain aging images that change continuously and smoothly with age.

2. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that: In S1, since the minimum age range of the FFHQ dataset is 0-2 and the maximum age range is 70-120, we select 1-100 as the general age range.

3. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that... In S2, the specific method of image preprocessing is to downsample all input images to 256×256, and thus adopt the VGG network as the pre-trained age classification network. We use the mean [0.485, 0.456, 0.406] and the variance of 1 to normalize the images.

4. The method for generating continuous face aging images based on a normalized flow model according to claim 1, characterized in that... In S3, assume the age attribute distribution is p w The easy-to-sample prior latent space is p. z In the flow model, the transformation relationship between the two is z = f(w), where f is assumed to be invertible and the dimensions of z and w are the same. This is to fit a distribution p. z The rules for change are as follows: in Let f be the Jacobian matrix of the function f over w. We use f to make the determinant of the Jacobian matrix trivial. Since f is invertible, w can also be directly obtained from w = f. -1 (z) is obtained directly.

Images