Portrait age method based on hierarchical optimization and semantic graph editing

Abstract

The invention provides a portrait age-making method based on hierarchical optimization and semantic graph editing, and relates to the technical field of image age-making simulation.According to the portrait age-making method based on hierarchical optimization and semantic graph editing, an input portrait is accurately embedded into a potential space through hierarchical optimization and iteration of latent code vector gradients, fine depiction of skin details (such as pores, wrinkles and spots) is achieved, and the image age-making effect is improved. Therefore, the identity confidence of the generated result is greatly improved; meanwhile, redundant calculation parameters are effectively reduced through a layered optimization strategy, and the system operation efficiency is improved. The invention also designs a semantic graph editing method based on BiseNET semantic segmentation, and the method carries out customized semantic editing on a portrait non-age region and optimizes a finally generated result. Meanwhile, a threshold mask replacement strategy is introduced, a virtual semantic region is generated according to preset probability distribution, the rigid constraint of traditional semantic segmentation on boundary features of a non-age region is broken through, the region affected by shielding can be subjected to reasonable age processing, and natural continuity of a final image is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of portrait aging simulation technology, specifically a portrait aging method based on hierarchical optimization and semantic graph editing. Background Technology

[0002] Facial aging technology is a deep learning-based facial image processing technique designed to simulate the natural changes in a face over time. This technology has applications in entertainment fields such as medical aesthetics and film production. Furthermore, it demonstrates significant value in public safety, specifically in the following ways: in solving cold cases, the technology can age photos of suspects taken at the time of the crime to generate images of their possible appearance years later, providing clues for their capture; in the recovery of abducted children, the technology can use photos taken when the child went missing to generate predicted images of their adult appearance through age simulation, providing visual comparison for identification.

[0003] However, previous facial aging techniques have been limited by the difficulty of collecting training samples. Obtaining high-quality, time-aligned facial image datasets covering a wide age range and individuals with the same identity is extremely difficult, which makes facial aging a challenging task in the field of computer vision to this day.

[0004] With the development of generative adversarial networks in the field of image synthesis, significant progress has been made in the research of aging simulation based on generative adversarial networks.

[0005] Existing methods mainly fall into two categories: facial attribute decoupling and image translation. Facial attribute decoupling is often inspired by the StyleGAN series of works, and its core lies in exploring the latent space within StyleGAN to achieve age conversion. Specifically, it involves finding a subspace in the latent space that contains semantic labels related to age, moving the latent encoding along editable directions that control age changes within this subspace to alter facial features, and finally reconstructing a high-quality aged portrait using a StyleGAN generator. This type of method typically achieves good results in editing binary attributes, such as the presence or absence of glasses or hairstyle, but it is limited by the uncontrollable scale of editing, the lack of corresponding age labels, and the non-linear changes in age features. Therefore, it cannot guarantee the accuracy of identity features and age during age conversion. Image translation treats portrait age conversion as a conditional image-to-image translation task. Its core idea is to give a target age as a conditional input, and then train the network to convert the input portrait into the target age group as the output. However, such methods have weak potential spatial decoupling capabilities, severely distort simulated texture features, lack realism, and cannot effectively simulate wrinkle texture representations around areas occluded by non-age-related factors (such as hats, glasses, hair, etc.), resulting in poor performance of the generated portraits in terms of detail expression, especially subtle age feature representations. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to propose a portrait aging method based on hierarchical optimization and semantic graph editing, comprising: Step 1: Obtain face image facial images Inputting the latent code into the trained e4e encoder yields the latent code. ; Step 2: Encoding the latent potential The process involves processing and optimization to obtain the final structural style code, which in turn yields the target reconstructed face image. Step 3: Input the reconstructed face image into the BiseNET semantic segmentation network to obtain the semantic map. ; Step 4: Based on semantic graph Construct the identity latent code C; Step 5: Set the target age based on the facial image. Constructing age latent codes Age code The age code and the identity code C are optimized to obtain the optimized age code and the optimized identity code. The optimized age code and the optimized identity code are then merged to obtain the hybrid code. Step 6: Construct the target identity age latent code based on the hybrid latent code; Step 7: Input the target's age latent code into the StyleGAN2 generator to obtain the age-rendered portrait corresponding to the target age.

[0007] Optionally, the trained e4e encoder described in step 1 is then processed based on a first loss function. The first loss function obtained by updating the parameters. Calculated using the following formula: ; in, Indicates pixel loss, The weight parameters represent the pixel loss. Indicates perceived loss. The weight parameters represent the perceptual loss. This indicates the loss in principal component analysis. The weighting parameters represent the loss in principal component analysis; Wherein, the pixel loss Calculated using the following formula: ; in, This represents the number of pixels for the high-resolution component. This represents the number of pixels in the low-resolution component. For facial images, To use bicubic interpolation pairs The downsampled image is a low-resolution image of size s, which is 50% of the original image. Indicated by The high-resolution image generated as input, Indicated by The low-resolution image generated as input, Denotes the squared Frobenius norm of a matrix; Wherein, the perceived loss Calculated using the following formula: ; in, This indicates the first extraction from the pre-trained VGG-19 network. Layer feature map, Indicates the first The number of channels in the layer feature map. Indicates the first The height of the layer feature map, Indicates the first The width of the layer feature map, VGG-19 extracts feature maps in the channel and spatial location The activation value at that location; Wherein, the principal component analysis loss Calculated using the following formula: ; in, For extracting from millions of potential codes n Principal component basis vectors This represents the first principal component in the basis vectors. Represents the first basis vector. n Principal components, belong p , Represents the basis vectors of the first i Principal components, μ The mean vector of the latent space. For the i The eigenvalues ​​of the principal components , This is the activation function.

[0008] Optionally, step 2 specifically includes: Step 2.1: Encode the latent potential Input the StyleGAN2 generator, obtain the output of the first m style blocks of the StyleGAN2 generator as the structure tensor F, replace the first m vectors of the latent encoding with the structure tensor to obtain the initial structure style encoding; input the initial structure style encoding into the StyleGAN2 generator to obtain the reconstructed face image; Step 2.2: Calculate the second loss function The initial structural style code is modified to minimize the second loss function, resulting in the final structural style code, which in turn yields the target reconstructed face image.

[0009] Optionally, the second loss function is calculated in step 2.2. Specifically, this is achieved through the following formula: ; in, Represents feature loss, The weight parameters represent the feature loss. Represents gradient loss, The weight parameters represent the gradient loss. Indicates visual perception loss. Weight parameters representing visual perception loss; Wherein, the feature loss Calculated using the following formula: ; in, Indicates the number of feature layers involved in the calculation. This indicates that the StyleGAN2 generator is at the 1st... The feature map extracted from the generated result by the layer. No. Reference feature map corresponding to the layer; Wherein, the gradient loss Calculated using the following formula: ; Where N represents the total number of pixels. and These represent the Sobel gradient operators in the horizontal and vertical directions, respectively. Indicates position The vector values ​​output by the StyleGAN2 generator at that location; Wherein, the visual perception loss Calculated using the following formula: ; in, , These represent the first generation of the StyleGAN2 generator. The height and width of the feature map output by the layer. The first generation of the StyleGAN2 generator The weights of the feature maps output by the layer. This represents the normalization layer of the StyleGAN2 generator along the channel dimension. In spatial location Activation function, Indicates the first Layer in spatial location superior, and The channel-wise squared difference vector of the normalized eigenvectors. This represents the reconstructed face image.

[0010] Optionally, step 4 specifically includes: Step 4.1: Define hair and eyes as non-age-based semantic categories, and then analyze the semantic graph. The incomplete semantic map is obtained by processing the regions where the non-age-related semantic categories are located. Specifically, this is achieved through the following formula: ; in, This indicates the region in the semantic graph where non-age-based semantic categories are located; Step 4.2: Fill in the skin region in the incomplete semantic map to obtain the final filled semantic map; Step 4.3: Set the first latent code Based on the formula that minimizes cross-entropy loss, for the first latent code Adjustments are made to minimize the cross-entropy loss, resulting in the second latent code. ; The formula for minimizing the cross-entropy loss is expressed as: ; in, This represents the StyleGAN2 generator. Indicates input of the first latent code The obtained semantic map is located at ( i,j The feature value of channel category c at point ) This represents the final filled semantic graph. Indicates the position ( i,j The characteristic value of channel category c at point (). Step 4.4: Based on the second latent code The identity latent code C is constructed using the following formula: ; in, Represents a hybrid structure tensor. Indicates mixed-style encoding. and All are weight parameters. This indicates the second latent code. Input a StyleGAN2 generator, output the first m layers of style blocks from the StyleGAN2 generator. This represents the m-th feature vector of the final structural style encoding.

[0011] Optionally, step 5 specifically includes: Step 5.1: Set target age Using existing age classifiers, Stitched together with a face image in the form of constant channels In the process, obtain facial images of the target age. Specifically, it is expressed by the following formula: ; Step 5.2: Extract the facial image of the target age. The input is an age encoder, and then the feature map output by the age encoder is downsampled through 18 style mapping blocks to obtain the age latent code. Specifically, it is expressed by the following formula: ; in, Indicates an age encoder. Represents a style mapping block; Step 5.3: Age Code Optimize with identity latent code C to obtain optimized age latent code and optimized identity latent code; Step 5.4: Merge the optimized age latent code and the optimized identity latent code to obtain the hybrid latent code.

[0012] Optionally, step 5.3 specifically includes: Step 5.3.1: Calculate age loss Specifically, this is achieved through the following formula: ; in, G represents the pre-trained age classifier, and G denotes the StyleGAN2 generator. Step 5.3.2: Calculate identity loss Specifically, this is achieved through the following formula: ; in, This represents the identity features extracted by the pre-trained ArcFace network. ,in, The estimated age represented by a facial image. Represents the weighting function. ; Step 5.3.3: Calculate the cyclic loss Specifically, this is achieved through the following formula: ; Where E represents the age encoder; Age code Optimization was performed to reduce age loss. and cyclic loss Minimum, while also being related to age potential Optimize with identity latent code C to reduce identity loss The minimum value yields the optimized age potential code and the optimized identity potential code.

[0013] Optionally, step 6 specifically includes: Step 6.1: Input the hybrid latent code into the StyleGAN2 generator and the BiseNET semantic segmentation network in sequence to obtain the semantic graph of the hybrid latent code; Step 6.2: Based on the semantic graph of the hybrid latent code, calculate the bicubic downsampling soft mask, wherein the bicubic downsampling soft mask includes... and Specifically, it is calculated using the following formula: ; in, Indicates an indicator function, Position in the semantic graph representing the hybrid latent code The corresponding semantic category, and Indicates the location The soft mask at the location, where K represents the semantic category; Step 6.3: Construct the soft mask structure tensor based on the bicubic downsampling soft mask. The soft mask structure tensor Expressed as: ; in, Indicates will Input to the StyleGAN2 generator, the position of the feature map output from the first m style blocks of the StyleGAN2 generator ( i,j The eigenvalue at () is a position; Calculate the target structure tensor Specifically, this is achieved through the following formula: ; Calculate the target style vector Specifically, this is achieved through the following formula: ; in, express The mixed style vector of the Kth category, The weight of the region containing semantic category K; The target structure tensor and the target style vector Constitute the target latent code The target latent code Expressed as: ; Step 6.4: Calculate the third loss function based on the constraints of cross-entropy loss and style loss, and apply it to the target latent code. Adjustments are made to minimize the third loss function, thus obtaining the target's identity age potential code.

[0014] Optionally, in step 6.4, a third loss function is calculated using the constraints of cross-entropy loss and style loss, specifically implemented through the following formula: ; in, This represents the third loss function. Represents a multi-class cross-entropy function. Represents semantic segmentation, Represents the target semantic graph. The weight parameters represent the style loss function. The style loss function represents the style loss function. Calculated using the following formula: ; in, For the Gram matrix of layer feature map This indicates that a masking operation is performed on the region corresponding to semantic category K in the semantic graph of the hybrid latent code.

[0015] The beneficial effects of adopting the above technical solution are as follows: This invention precisely embeds the input portrait image into the latent space by hierarchically optimizing the gradient of the latent code vector, achieving fine depiction of skin details (such as pores, wrinkles, and blemishes), thereby significantly improving the identity confidence of the generated result. Simultaneously, the hierarchical optimization strategy effectively reduces redundant computational parameters, improving system efficiency. This invention also designs a semantic graph editing method based on BiseNET semantic segmentation. This method performs customized semantic editing on non-age-related regions of the portrait, optimizing the final generated result. Furthermore, a threshold mask substitution strategy is introduced to generate virtual semantic regions with a preset probability distribution, breaking through the rigid constraints of traditional semantic segmentation on the boundary features of non-age-related regions. This allows occluded regions to also receive reasonable age-related processing, ensuring the natural coherence of the final image. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a portrait aging method based on hierarchical optimization and semantic graph editing in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another portrait aging method based on hierarchical optimization and semantic graph editing in an embodiment of the present invention. Figure 3This image shows the experimental results of a face aging method based on hierarchical optimization and semantic graph editing in an embodiment of the present invention. (a1) is the input face image; (a2), (a3), (a4), (a5), and (a6) are the face images of the person in (a1) at ages 0, 10, 20, 30, and 50, respectively; (b1) is the input face image; (b2), (b3), (b4), (b5), and (b6) are the face images of the person in (b1) at ages 0, 10, 20, 30, and 50, respectively; and (c1) is the input face image; (c2), (c3), (c4), (c5), and (c6) are the face images of the person in (c1) at ages 0, 10, 20, 30, and 50, respectively. The input face image is (d1), and (d2), (d3), (d4), (d5), and (d6) are the face images of the person in (d1) at the ages of 0, 10, 20, 30, and 50, respectively. The input face image is (e1), and (e2), (e3), (e4), (e5), and (e6) are the face images of the person in (e1) at the ages of 0, 10, 20, 30, and 50, respectively. The input face image is (f1), and the face images of the person in (f2), (f3), (f4), (f5), and (f6) are the face images of the person at the ages of 0, 10, 20, 30, and 50, respectively. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0018] To address the problems existing in current technologies, this invention provides a portrait aging method based on hierarchical optimization and semantic graph editing, combining... Figure 1 and Figure 2 This may include the following steps: Step 1: Obtain face image facial images The latent code is obtained by inputting it into the trained e4e encoder (Designing an Encoder for StyleGAN Image Manipulation). The latent encoding It consists of 18 style vectors that can be optimized independently.

[0019] The trained e4e encoder maps the input face to the generator's W+ space for coarse-grained optimization, which is used to quickly fit the overall structure and style features of the face.

[0020] The trained e4e encoder is trained based on a first loss function. The first loss function obtained by updating the parameters. Calculated using the following formula: ; in, Indicates pixel loss, The weight parameters represent the pixel loss. Indicates perceived loss. The weight parameters represent the perceptual loss. This indicates the loss in principal component analysis. The weighting parameters represent the loss in principal component analysis; Wherein, the pixel loss Calculated using the following formula: ; in, This represents the number of pixels for the high-resolution component. This represents the number of pixels in the low-resolution component. For facial images, To perform bicubic interpolation on The downsampled image is a low-resolution image of size s, which is 50% of the original image. Indicated by The high-resolution image generated as input, Indicated by The low-resolution image generated as input, The squared Frobenius norm of a matrix is ​​represented by subtracting each element of the matrix point by point, squaring the results, and then summing them. Pixel loss is calculated through frequency division, ensuring the generated result balances global structure and local detail, resulting in a visually coherent and textured final product.

[0021] Among them, the bicubic interpolation function Used to ensure geometric consistency of multi-scale supervision.

[0022] Wherein, the perceived loss Calculated using the following formula: ; in, This indicates the first extraction from the pre-trained VGG-19 network. Layer feature map, Indicates the first The number of channels in the layer feature map. Indicates the first The height of the layer feature map, Indicates the first The width of the layer feature map, VGG-19 extracts feature maps in the channel and spatial location The activation value at the specified location; this loss term effectively suppresses pixel-level overfitting through high-level semantic feature matching.

[0023] Wherein, the principal component analysis loss Calculated using the following formula: ; in, For extracting from millions of potential codes n Principal component basis vectors This represents the first principal component in the basis vectors. Represents the basis vectors of the first n Principal components, belong p , Represents the basis vectors of the first i Principal components, μ The mean vector of the latent space. For the i The eigenvalues ​​of the principal components , This is the activation function used to ensure the regular projection of the underlying encoding.

[0024] Since the W+ latent space suffers from insufficient spatial detail representation and unstable individual feature retention, the coarse-grained latent encoding obtained in step 1 is optimized with fine-grained precision and mapped to the larger FS latent space. The structure tensor and style encoding are used to adjust local detail features to improve the accuracy of facial identity feature representation, which is specifically achieved through step 2.

[0025] Step 2: Encoding the latent potential The process involves processing and optimization to obtain the final structural style code, which in turn yields the target reconstructed face image. Step 2.1: Encode the latent potential Input the StyleGAN2 generator, obtain the output of the first m layers of style blocks of the StyleGAN2 generator as the structure tensor F, replace the first m layers of vectors of the latent encoding with the structure tensor to capture facial information details (such as moles, wrinkles, etc.) to obtain the initial structure style encoding; input the initial structure style encoding into the StyleGAN2 generator to obtain the reconstructed face image; Step 2.2: Calculate the second loss function The initial structural style code is modified to minimize the second loss function, resulting in the final structural style code, which in turn yields the target reconstructed face image.

[0026] Among them, the second loss function is calculated. Specifically, this is achieved through the following formula: ; in, Represents feature loss, The weight parameters represent the feature loss. Represents gradient loss, The weight parameters represent the gradient loss. Indicates visual perception loss. Weight parameters representing visual perception loss; Wherein, the feature loss Calculated using the following formula: ; in, This indicates the number of feature layers involved in the calculation, totaling from... arrive There are 5 layers in total, so the accumulated squared error is divided by... This is equivalent to taking the average of these 5 layers. This indicates that the StyleGAN2 generator is at the 1st... The feature map extracted from the generated result by the layer. No. The reference feature map corresponding to the layer comes from the original high-resolution map, and... They have the same shape and are used for comparison.

[0027] Wherein, the gradient loss Calculated using the following formula: ; Where N represents the total number of pixels. and These represent the Sobel gradient operators in the horizontal and vertical directions, respectively. Indicates position The vector values ​​output by the StyleGAN2 generator at that location; Wherein, the visual perception loss Calculated using the following formula: ; in, , These represent the first generation of the StyleGAN2 generator. The height and width of the feature map output by the layer. The first generation of the StyleGAN2 generator The weights of the feature maps output by the layer. This represents the normalization layer of the StyleGAN2 generator along the channel dimension. In spatial location Activation function, Indicates the first Layer in spatial location superior, and The channel-wise squared difference vector of the normalized eigenvectors. This represents the reconstructed face image.

[0028] Step 3: Input the reconstructed face image into the BiseNET semantic segmentation network to obtain the semantic map. ; Step 4: Based on semantic graph Construct the identity latent code C; Specifically, the non-age-related semantic categories in the semantic map are retrieved, and the corresponding semantic regions are masked to obtain an incomplete semantic map. This incomplete semantic map is then repaired through interpolation and finally input into a differentiable generator to invert the latent encoding of the repaired semantic map, thereby reconstructing the latent encoding in S2. The differentiable generator is implemented by minimizing the cross-entropy loss using a GAN-based inversion method.

[0029] Step 4.1: Define hair and eyes as non-age-based semantic categories, and then analyze the semantic graph. The incomplete semantic map is obtained by processing the regions where the non-age-related semantic categories are located. Specifically, this is achieved through the following formula: ; in, This indicates the region in the semantic graph where non-age-based semantic categories are located; Step 4.2: Fill in the skin region in the incomplete semantic map to obtain the final filled semantic map; Specifically, an image inpainting technique based on the fastmarching method is used to effectively utilize the areas near the boundary that are already labeled as skin. The process propagates from the outside inwards, filling in uncovered pixels to form a semantically coherent skin region. After filling, small region culling and morphological closure operations are performed to remove occasional floating fragments and seal internal holes. Specifically, this process employs two operations: small region culling to remove floating pseudo-pixel fragments; and morphological closure to fill any small holes that may exist within the skin, ensuring that the repaired skin region has continuous boundaries and a closed shape. This results in a closed shape with continuous boundaries, generating the final infill semantic map.

[0030] Step 4.3: Set the first latent code Based on the formula that minimizes cross-entropy loss, for the first latent code Adjustments are made to minimize the cross-entropy loss, resulting in the second latent code. ; The formula for minimizing the cross-entropy loss is expressed as: ; in, This represents the StyleGAN2 generator. Indicates input of the first latent code The obtained semantic map is located at ( i,j The feature value of channel category c at point ) This represents the final filled semantic graph. Indicates the position ( i,j The characteristic value of channel category c at point (). Step 4.4: Based on the second latent code The identity latent code C is constructed using the following formula: ; in, Represents a hybrid structure tensor. Indicates mixed-style encoding. and All are weight parameters. and , This indicates the second latent code. Input a StyleGAN2 generator, output the first m layers of style blocks from the StyleGAN2 generator. This represents the m-th feature vector of the final structural style encoding.

[0031] Step 5: Set the target age based on the facial image. Constructing age latent codes Age code The age code and the identity code C are optimized to obtain the optimized age code and the optimized identity code. The optimized age code and the optimized identity code are then merged to obtain the hybrid code. Step 5.1: Set target age Using existing age classifiers, Stitched together with a face image in the form of constant channels In the process, obtain facial images of the target age. Specifically, it is expressed by the following formula: ; Among them, facial images It has 4 channels.

[0032] Step 5.2: Extract the facial image of the target age. The input age encoder extracts feature maps at three different spatial scales. These feature maps are then downsampled using 18 style mapping blocks, resulting in 18 vectors that correspond one-to-one with 18 vectors in the W+ latent space, thus yielding the age latent code. Specifically, it is expressed by the following formula: ; in, Indicates an age encoder. Represents a style mapping block; The age encoder is derived from the DEX model proposed in the 2015 paper "Deep Expectation of apparent age from single image".

[0033] Step 5.3: Age Code Optimize with identity latent code C to obtain optimized age latent code and optimized identity latent code; Step 5.3.1: Calculate age loss Specifically, this is achieved through the following formula: ; in, G represents the pre-trained age classifier, and G denotes the StyleGAN2 generator. Step 5.3.2: Calculate identity loss Specifically, this is achieved through the following formula: ; in, This represents the identity features extracted by the pre-trained ArcFace (Additive Angular Margin Loss for Deep Face Recognition) network. ,in, The estimated age represented by a facial image. Represents the weighting function. Where a weight of 1 indicates that the age has not changed, and as the weight increases... The increase, It will decrease monotonically accordingly, reaching a minimum of 0.5.

[0034] Step 5.3.3: Calculate the cyclic loss Specifically, this is achieved through the following formula: ; Where E represents the age encoder; Age code Optimization was performed to reduce age loss. and cyclic loss Minimum, while also being related to age potential Optimize with identity latent code C to reduce identity loss The minimum value yields the optimized age potential code and the optimized identity potential code.

[0035] Among them, age loss is used to detect the accuracy of age transformation. By minimizing the squared error between the target age and the estimated age of the generated image, the network finds an appropriate "age line" in the latent space, thus finely driving the transition of the face from the original age to the target age. Identity loss ensures that the network strictly preserves identity features when the age difference is small, while allowing moderate deviation when the age difference is large, to balance recognizability and the naturalness of age change. Cyclic loss is used if the network can accurately return to the target age after two round trips. This indicates that potential paths and age control maintain both structural stability and reversibility, effectively compensating for the weak supervision problem caused by label scarcity.

[0036] Step 5.4: Merge the optimized age latent code and the optimized identity latent code to obtain the hybrid latent code.

[0037] Step 6: Construct the target identity age latent code based on the hybrid latent code; Step 6.1: Input the hybrid latent code into the StyleGAN2 generator and the BiseNET semantic segmentation network in sequence to obtain the semantic graph of the hybrid latent code; Step 6.2: Based on the semantic graph of the hybrid latent code, calculate the bicubic downsampling soft mask, wherein the bicubic downsampling soft mask includes... and Specifically, it is calculated using the following formula: ; in, Indicates an indicator function, Position in the semantic graph representing the hybrid latent code The corresponding semantic category, and Indicates the location The soft mask at the location, where K represents the semantic category; In this invention, there are two semantic categories: semantic category 1 and semantic category 2. In the calculation formula for the double cubic downsampling soft mask, when the position of the hybrid latent code in the semantic graph... When the corresponding semantic category is semantic category 1, , When the position of the hybrid latent code in the semantic graph When the corresponding semantic category is semantic category 2, , When the position of the hybrid latent code in the semantic graph When the corresponding semantic categories are semantic category 1 and semantic category 2, , .

[0038] thus, The value of is within the range of [0,1], at which point from It is safe to replicate the corresponding features in the middle, and then... and As the proportional weight for copying categories from the corresponding latent encoding, a soft mask structure tensor is constructed. Specifically, this is achieved through step 6.3.

[0039] Step 6.3: Construct the soft mask structure tensor based on the bicubic downsampling soft mask. The soft mask structure tensor Expressed as: ; in, Indicates will Input to the StyleGAN2 generator, the position of the feature map output from the first m style blocks of the StyleGAN2 generator ( i,j The eigenvalue at () is a position; Calculate the target structure tensor Specifically, this is achieved through the following formula: ; Calculate the target style vector Specifically, in terms of style expression vector fusion, a linear fusion strategy is adopted to represent the style expression vector in each semantic region as a weighted average of the reference maps of each semantic category, which is achieved through the following formula: ; in, express The mixed style vector of the Kth category, The weights of the regions containing semantic category K satisfy the normalization constraint. ; The target structure tensor and the target style vector Constitute the target latent code The target latent code Expressed as: ; Step 6.4: Calculate the third loss function using the constraints of cross-entropy loss and style loss, specifically through the following formula: ; in, This represents the third loss function. Represents a multi-class cross-entropy function. Represents semantic segmentation, Represents the target semantic graph. The weight parameters represent the style loss function. The style loss function represents the style loss function. Calculated using the following formula: ; in, For the The Gram matrix of the layer feature map is used to characterize the global covariance information between the channels of that layer, i.e. In the formula It is the VGG network's first Layer activation mapping, This indicates that a masking operation is performed on the region corresponding to semantic category K in the semantic graph of the hybrid latent code. This occurs on layers relu1_2, relu2_2, relu3_3, and relu4_3 of VGG-16.

[0040] Among them, cross-entropy loss and style loss are used to constrain the semantic segmentation results of the generated human image to make them consistent with the target mask at the region level. The style loss function is used to maintain the consistency of style features between the generated image and the input image, and to ensure that different style elements in the image can be synthesized naturally.

[0041] target latent code Adjustments are made to minimize the third loss function, thus obtaining the target's identity age potential code.

[0042] In conjunction with the above technical solutions, the present invention was implemented on the FFHQ dataset, and the results are as follows: Figure 3Where (a1) is the input face image, (a2), (a3), (a4), (a5), and (a6) are the face images of the person in (a1) at ages 0, 10, 20, 30, and 50, respectively; (b1) is the input face image, (b2), (b3), (b4), (b5), and (b6) are the face images of the person in (b1) at ages 0, 10, 20, 30, and 50, respectively; and (c1) is the input face image, (c2), (c3), (c4), (c5), and (c6) are the face images of the person in (c1) at ages 0, 10, 20, 30, and 50, respectively. Face images, (d1) is the input face image, (d2), (d3), (d4), (d5) and (d6) are the face images of the person in (d1) at the ages of 0, 10, 20, 30 and 50 respectively, (e1) is the input face image, (e2), (e3), (e4), (e5) and (e6) are the face images of the person in (e1) at the ages of 0, 10, 20, 30 and 50 respectively, (f1) is the input face image, (f2), (f3), (f4), (f5) and (f6) are the face images of the person in (f6) at the ages of 0, 10, 20, 30 and 50 respectively.

[0043] The FFHQ dataset was collected and made public by NVIDIA's research team.

[0044] Step 7: Input the target's age latent code into the StyleGAN2 generator to obtain the age-rendered portrait corresponding to the target age.

[0045] This invention was developed using Python, with PyTorch as the deep learning framework and Anaconda as the software environment. During training, three NVIDIA GeForce GTX 3090 Ti GPUs were used on a cloud platform; during testing, an Intel(R)core(TM) i7-9750H CPU was used. To balance training stability and GPU memory capacity, the data was divided into multiple batches for training, with a batch size of 9 and a total of 300 training epochs. The Adam optimizer was used in the experiments, with hyperparameters set to β1=0 and β2=0.999. An adversarial loss was applied in a non-saturating form, and an r1 regularization term was introduced to enhance training stability. The initial learning rate was set to... The learning rate of the mapping network was further reduced to 0.01 times its initial value after the 50th and 100th training epochs. The generator weights employed an exponential moving average (EMA) to smooth parameter updates. The training datasets included the officially provided FFHQ dataset and a self-built Asian portrait dataset. Validation was performed on 5% of the samples from each of the CACD, CelebA, FFHQ, and the self-built Asian portrait dataset.

[0046] This invention utilizes Megvii Technology's Face++ online face analysis platform to evaluate the age prediction accuracy and identity authentication confidence of the generated images. For European (Eur) and Asian (As) portraits, this invention is quantitatively compared with existing mainstream models CUSP, HRFAE, LATS, IPCGAN, SAM, and FADING. Tables 1 and 2 show the test results. The experimental results demonstrate that this invention significantly outperforms the aforementioned comparative methods in both age prediction accuracy and identity retention confidence.

[0047] Table 1 Age Prediction Assessment Table

[0048] Table 2 Identity Confidence Table

[0049] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A portrait aging method based on hierarchical optimization and semantic graph editing, characterized in that, include: Step 1: Obtain face image facial images Inputting the latent code into the trained e4e encoder yields the latent code. ; Step 2: Encoding the latent potential The process involves processing and optimization to obtain the final structural style code, which in turn yields the target reconstructed face image. Step 3: Input the reconstructed face image into the BiseNET semantic segmentation network to obtain the semantic map. ; Step 4: Based on semantic graph Construct the identity latent code C; Step 5: Set the target age based on the facial image. Constructing age latent codes Age code The age code and the identity code C are optimized to obtain the optimized age code and the optimized identity code. The optimized age code and the optimized identity code are then merged to obtain the hybrid code. Step 6: Construct the target identity age latent code based on the hybrid latent code; Step 7: Input the target's age latent code into the StyleGAN2 generator to obtain the age-rendered portrait corresponding to the target age.

2. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 1, characterized in that, The e4e encoder trained in step 1 is then processed based on the first loss function. The first loss function obtained by updating the parameters. Calculated using the following formula: ； in, Indicates pixel loss, The weight parameters represent the pixel loss. Indicates perceived loss. The weight parameters represent the perceptual loss. This indicates the loss in principal component analysis. The weighting parameters represent the loss in principal component analysis; Wherein, the pixel loss Calculated using the following formula: ； in, This represents the number of pixels for the high-resolution component. This represents the number of pixels in the low-resolution component. For facial images, To use bicubic interpolation pairs The downsampled image is a low-resolution image of size s, which is 50% of the original image. Indicated by The high-resolution image generated as input, Indicated by The low-resolution image generated as input, Denotes the squared Frobenius norm of a matrix; Wherein, the perceived loss Calculated using the following formula: ； in, This indicates the first extraction from the pre-trained VGG-19 network. Layer feature map, Indicates the first The number of channels in the layer feature map. Indicates the first The height of the layer feature map, Indicates the first The width of the layer feature map, VGG-19 extracts feature maps in the channel and spatial location The activation value at that location; Wherein, the principal component analysis loss Calculated using the following formula: ； in, For extracting from millions of potential codes n Principal component basis vectors This represents the first principal component in the basis vectors. Represents the basis vectors of the first n Principal components, belong p , Represents the basis vectors of the first i Principal components, μ The mean vector of the latent space. For the i The eigenvalues ​​of the principal components , This is the activation function.

3. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 1, characterized in that, Step 2 specifically includes: Step 2.1: Encode the latent potential Input the StyleGAN2 generator, obtain the output of the first m style blocks of the StyleGAN2 generator as the structure tensor F, replace the first m vectors of the latent encoding with the structure tensor to obtain the initial structure style encoding; input the initial structure style encoding into the StyleGAN2 generator to obtain the reconstructed face image; Step 2.2: Calculate the second loss function The initial structural style code is modified to minimize the second loss function, resulting in the final structural style code, which in turn yields the target reconstructed face image.

4. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 3, characterized in that, In step 2.2, the second loss function is calculated. Specifically, this is achieved through the following formula: ； in, Represents feature loss, The weight parameters represent the feature loss. Represents gradient loss, The weight parameters represent the gradient loss. Indicates visual perception loss. Weight parameters representing visual perception loss; Wherein, the feature loss Calculated using the following formula: ； in, Indicates the number of feature layers involved in the calculation. This indicates that the StyleGAN2 generator is at the 1st... The feature map extracted from the generated result by the layer. No. Reference feature map corresponding to the layer; Wherein, the gradient loss Calculated using the following formula: ； Where N represents the total number of pixels. and These represent the Sobel gradient operators in the horizontal and vertical directions, respectively. Indicates position The vector values ​​output by the StyleGAN2 generator at that location; Wherein, the visual perception loss Calculated using the following formula: ； in, , These represent the first generation of the StyleGAN2 generator. The height and width of the feature map output by the layer. The first generation of the StyleGAN2 generator The weights of the feature maps output by the layer. This represents the normalization layer of the StyleGAN2 generator along the channel dimension. In spatial location Activation function, Indicates the first Layer in spatial location superior, and The channel-wise squared difference vector of the normalized eigenvectors. This represents the reconstructed face image.

5. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 1, characterized in that, Step 4 specifically includes: Step 4.1: Define hair and eyes as non-age-based semantic categories, and then analyze the semantic graph. The incomplete semantic map is obtained by processing the regions where the non-age-related semantic categories are located. Specifically, this is achieved through the following formula: ； in, This indicates the region in the semantic graph where non-age-based semantic categories are located; Step 4.2: Fill in the skin region in the incomplete semantic map to obtain the final filled semantic map; Step 4.3: Set the first latent code Based on the formula that minimizes cross-entropy loss, for the first latent code Adjustments are made to minimize the cross-entropy loss, resulting in the second latent code. ; The formula for minimizing the cross-entropy loss is expressed as: ； in, This represents the StyleGAN2 generator. Indicates input of the first latent code The obtained semantic map is located at ( i,j The feature value of channel category c at point ) This represents the final filled semantic graph. Indicates the position ( i,j The characteristic value of channel category c at point (). Step 4.4: Based on the second latent code The identity latent code C is constructed using the following formula: ； in, Represents a hybrid structure tensor. Indicates mixed-style encoding. and All are weight parameters. This indicates the second latent code. Input a StyleGAN2 generator, output the first m layers of style blocks from the StyleGAN2 generator. This represents the m-th feature vector of the final structural style encoding.

6. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Set target age Using existing age classifiers, Stitched together with a face image in the form of constant channels In the process, obtain facial images of the target age. Specifically, it is expressed by the following formula: ； Step 5.2: Extract the facial image of the target age. The input is an age encoder, and then the feature map output by the age encoder is downsampled through 18 style mapping blocks to obtain the age latent code. Specifically, it is expressed by the following formula: ； in, Indicates an age encoder. Represents a style mapping block; Step 5.3: Age Code Optimize with identity latent code C to obtain optimized age latent code and optimized identity latent code; Step 5.4: Merge the optimized age latent code and the optimized identity latent code to obtain the hybrid latent code.

7. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 6, characterized in that, Step 5.3 specifically includes: Step 5.3.1: Calculate age loss Specifically, this is achieved through the following formula: ； in, G represents the pre-trained age classifier, and G denotes the StyleGAN2 generator. Step 5.3.2: Calculate identity loss Specifically, this is achieved through the following formula: ； in, This represents the identity features extracted by the pre-trained ArcFace network. ,in, The estimated age represented by a facial image. Represents the weighting function. ; Step 5.3.3: Calculate the cyclic loss Specifically, this is achieved through the following formula: ； Where E represents the age encoder; Age code Optimization was performed to reduce age loss. and cyclic loss Minimum, while also being related to age potential Optimize with identity latent code C to reduce identity loss The minimum value yields the optimized age potential code and the optimized identity potential code.

8. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 1, characterized in that, Step 6 specifically includes: Step 6.1: Input the hybrid latent code into the StyleGAN2 generator and the BiseNET semantic segmentation network in sequence to obtain the semantic graph of the hybrid latent code; Step 6.2: Based on the semantic graph of the hybrid latent code, calculate the bicubic downsampling soft mask, wherein the bicubic downsampling soft mask includes... and Specifically, it is calculated using the following formula: ； in, Indicates an indicator function, Position in the semantic graph representing the hybrid latent code The corresponding semantic category, and Indicates the location The soft mask at the location, where K represents the semantic category; Step 6.3: Construct the soft mask structure tensor based on the bicubic downsampling soft mask. The soft mask structure tensor Expressed as: ； in, Indicates will Input to the StyleGAN2 generator, the position of the feature map output from the first m style blocks of the StyleGAN2 generator ( i,j The eigenvalue at () is a position; Calculate the target structure tensor Specifically, this is achieved through the following formula: ； Calculate the target style vector Specifically, this is achieved through the following formula: ； in, express The mixed style vector of the Kth category, The weight of the region containing semantic category K; The target structure tensor and the target style vector Constitute the target latent code The target latent code Expressed as: ； Step 6.4: Calculate the third loss function based on the constraints of cross-entropy loss and style loss, and apply it to the target latent code. Adjustments are made to minimize the third loss function, thus obtaining the target's identity age potential code.

9. The portrait aging method based on hierarchical optimization and semantic graph editing according to claim 8, characterized in that, In step 6.4, the third loss function is calculated using the constraints of cross-entropy loss and style loss, specifically through the following formula: ； in, This represents the third loss function. Represents a multi-class cross-entropy function. Represents semantic segmentation, Represents the target semantic graph. The weight parameters represent the style loss function. The style loss function represents the style loss function. Calculated using the following formula: ； in, For the Gram matrix of layer feature map This indicates that a masking operation is performed on the region corresponding to semantic category K in the semantic graph of the hybrid latent code.

Images