Method for training an image generation model and related apparatuses

Abstract

Embodiments of the present application relate to the technical field of image processing, and disclose a method for training an image generation model and related apparatuses. A pre-trained age detection model is used to detect the age of each real face image to obtain a predicted age, and an age difference between the predicted age and a real age is calculated. The age difference and the real face image are input into an image generation network for fusion to generate a predicted face image affected by the age difference. The predicted age can reflect the universal age characteristics of the real face image, and the age difference can reflect the individual age characteristics (older or younger) of the input image. Thus, the age difference can guide the image generation network to learn the individual age characteristics of the input image. The trained image generation model can accurately generate an age change image that conforms to the individual age characteristics and the expected age based on the age difference between the expected age and the real age.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and related apparatus for training an image generation model. Background Technology

[0002] As machine learning technology matures, the types of services based on it are also increasing. For example, computer devices can use machine learning to process facial images and obtain age-changing images. The initial age of a face in the original image differs from the second age in the age-changing image, but both correspond to the same identity. This service is widely needed in various scenarios. For instance, it can be used to predict what a missing child will look like several years from now, aiding in their search; or to predict a suspect's appearance many years into their life, based on old photos of them when they were young; in film and television production, it can be used to predict an actor's appearance in old age, helping makeup artists to create the desired look; and during leisure time, people may want to use current images to recall photos from their childhood.

[0003] Computer devices typically process input facial images and predictive ages using machine learning models to generate age-varying images that accurately reflect the person's age. These machine learning models are trained on facial images from different age groups. When processing facial images using this method, feature changes are determined solely by age group, often resulting in identical feature changes for individuals. That is, for users A and B of the same age, the feature changes are the same as their images change from one age to another; the feature changes are singular. However, in real life, each person's age-related feature changes are influenced by individual factors and are therefore specific. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a method and related apparatus for training an image generation model, wherein the trained image generation model can generate age change images that conform to the individual's age characteristics and expected age.

[0005] In a first aspect, embodiments of this application provide a method for training an image generation model. The image generation network includes an encoder, a mapping transformation module, a fusion module, and a decoding module, comprising:

[0006] Acquire several real face images, each labeled with the real age, and the real ages of the several real face images cover a preset age range;

[0007] A pre-trained age detection model is used to detect the age of several real face images, and the predicted age corresponding to each real face image is obtained.

[0008] The real face image and the age difference corresponding to the real face image are input into the image generation network to obtain the predicted face image. The age difference is the difference between the predicted age and the real age. The real face image is encoded by the encoder to obtain the encoding result. The age difference is mapped and transformed by the mapping and transformation module to obtain the age vector. The encoding result and the age vector are fused by the fusion module. The fusion result is input into the decoding module to decode to obtain the predicted face image.

[0009] The loss function is used to calculate the loss between the predicted face image and the real face image. Based on the loss sums corresponding to several real face images, the image generation network is iteratively trained until convergence, thus obtaining the image generation model.

[0010] In some embodiments, the decoding module includes a decoder with N branches, one decoder corresponding to one age group;

[0011] The fusion result is input into the decoding module for decoding to obtain the predicted face image, including:

[0012] Based on the predicted age range, determine the target decoder from N decoders;

[0013] The fusion result is input into the target decoder for decoding to obtain the predicted face image.

[0014] In some embodiments, the method further includes: determining the number of branches N of the decoder based on the error between the maximum age value of a preset age range and the age detection model.

[0015] In some embodiments, the mapping conversion module performs age difference mapping conversion in the following manner:

[0016] The age difference is calculated using a mapping function to obtain an age vector; the mapping function includes a linear fully connected network.

[0017] In some embodiments, the fusion module fuses the encoding result and the age vector in the following manner:

[0018] The encoded result and the age vector are multiplied and fused to obtain the fused result.

[0019] In some embodiments, the loss function includes adversarial loss, reconstruction loss, and perceptual style loss, wherein the adversarial loss reflects whether the predicted face image is a real face, the reconstruction loss reflects the pixel difference between the predicted face image and the real face image, and the perceptual style loss reflects the difference between the feature map of the predicted face image and the feature map of the real face image.

[0020] In some embodiments, the loss function includes:

[0021] in,

[0022]

[0023]

[0024]

[0025] Where Loss is the loss function. To combat the losses, To reconstruct the loss, To perceive style loss, α i Let represent the on / off state of the decoder in the i-th branch, N be the number of decoding branches, I be the real face image, I′ be the predicted face image, D(I) be the probability that the discriminator identifies the real face image as real, t be the age difference, G(I,t) be the predicted face image generated by the image generation network based on the real face image and the age difference, and D(G(I,t)) be the probability that the discriminator identifies the predicted face image as real. This is the i-th feature map extracted from a real face image by the feature extraction network. The i-th feature map is obtained by the feature extraction network from the predicted face image.

[0026] Secondly, this application provides a method for generating an age-changing image, including:

[0027] Obtain the test face image, the true age of the face in the test face image, and the expected age;

[0028] The test face image and the age difference between the real age and the expected age are input into the image generation model to obtain an age change image. The age change image reflects the age of the person and is consistent with the expected age. The image generation model is trained using the same method as the image generation model training method described in the first aspect.

[0029] Thirdly, this application provides an electronic device, including:

[0030] At least one processor, and

[0031] A memory that is communicatively connected to at least one processor, wherein,

[0032] The memory stores instructions that can be executed by at least one processor, such that the instructions are executed by at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0033] To address the aforementioned technical problems, in a fourth aspect, this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer device to perform the method described in the first aspect above.

[0034] The beneficial effects of this application's embodiments are as follows: Unlike existing technologies, the method for training an image generation model provided in this application first constructs an image generation network including an encoder, a mapping and transformation module, a fusion module, and a decoding module. Then, it acquires several real face images, each labeled with its actual age, and these real face images cover a preset age range. A pre-trained age detection model is used to detect the age of these real face images, obtaining the predicted age corresponding to each real face image. The real face image and the age difference corresponding to the real face image are input into the image generation network to obtain the predicted face image. Finally, a loss function is used to calculate the loss between the predicted face image and the real face image, and the image generation network is iteratively trained based on the sum of the losses corresponding to the several real face images until convergence, resulting in the image generation model. Here, the age difference is the difference between the predicted age and the actual age. The real face image is encoded by the encoder to obtain the encoded result, the age difference is mapped and transformed by the mapping and transformation module to obtain the age vector, the encoded result and the age vector are fused by the fusion module, and the fused result is input into the decoding module for decoding to obtain the predicted face image.

[0035] In this embodiment, a pre-trained age detection model is used to predict age, and the age difference between the predicted age and the actual age is calculated. This age difference, along with a real face image, is then input into an image generation network for fusion, generating a predicted face image influenced by the age difference. Since the age detection model is trained on a large number of face images, its output predicted age reflects the universal age characteristics of real face images, while the age difference reflects the individual's unique age characteristics (looking older or younger). Therefore, the age difference can guide the image generation network to learn the individual's unique age characteristics in the input image. Several image groups are used to train the image generation network in the above manner. Based on backpropagation of the loss sum, the predicted face images corresponding to each image group are constrained to continuously approach the real face image. Thus, the image generation network can accurately reconstruct the individual's unique age characteristics in the input image. Therefore, the trained image generation model can accurately generate an age change image that matches both the individual's unique age characteristics and the expected age based on the age difference between the expected and actual ages. In other words, when predicting aging or tracing back to youth, the age characteristics in the test face image are changed based on the individual's current unique age characteristics, ensuring that the changes in age characteristics are not uniform. Attached Figure Description

[0036] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0037] Figure 1 This is a schematic diagram illustrating application scenarios of the age change prediction system in some embodiments of this application;

[0038] Figure 2 This is a schematic diagram of the structure of an electronic device in some embodiments of this application;

[0039] Figure 3 This is a flowchart illustrating the method for training an image generation model in some embodiments of this application;

[0040] Figure 4 This is a schematic diagram of the structure of the image generation network in some embodiments of this application;

[0041] Figure 5 This is a schematic diagram of the structure of the image generation network in some embodiments of this application;

[0042] Figure 6 This is a flowchart illustrating a method for generating images of age changes in some embodiments of this application. Detailed Implementation

[0043] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0046] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0047] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0048] To facilitate understanding of the methods provided in the embodiments of this application, the terms used in the embodiments of this application will first be introduced:

[0049] (1) Neural Network

[0050] A neural network can be composed of neural units, specifically understood as a neural network with input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. Neural networks with many hidden layers are called deep neural networks (DNNs). The work of each layer in a neural network can be described by the mathematical expression y = a(W·x + b). From a physical perspective, the work of each layer in a neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space: 1. Dimensionality increase / decrease; 2. Magnification / scaling; 3. Rotation; 4. Translation; 5. "Bending". Operations 2 and 3 are performed by "W·x", operation 4 by "+b", and operation 5 by "a()". The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within that class. W is the weight matrix of each layer in the neural network, where each value represents the weight of a neuron in that layer. This matrix W determines the spatial transformation from the input space to the output space, as described above; that is, the W of each layer in the neural network controls how the space is transformed. The purpose of training the neural network is to ultimately obtain the weight matrices of all layers of the trained neural network. Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0051] It should be noted that in the embodiments of this application, the models used for machine learning tasks are essentially neural networks. Common components in neural networks include convolutional layers, pooling layers, normalization layers, and deconvolutional layers. By assembling these common components in neural networks, a model is designed. When the model parameters (weight matrices of each layer) are determined such that the model error meets a preset condition or the number of model parameters is adjusted to reach a preset threshold, the model converges.

[0052] The convolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform convolution operations on the image. The purpose of convolution is to extract different features from the input image. The first convolutional layer may only extract some low-level features such as edges, lines, and corners, while deeper convolutional layers can iteratively extract more complex features from low-level features.

[0053] A deconvolutional layer is used to map a low-dimensional space to a high-dimensional space while preserving the connections / patterns between them (the connections during convolution). A deconvolutional layer is configured with multiple convolutional kernels, each with a corresponding stride, to perform deconvolution operations on the image. Generally, framework libraries used for designing neural networks (such as the PyTorch library) have a built-in `upsumple()` function, which allows for low-dimensional to high-dimensional spatial mapping.

[0054] Pooling layers mimic the human visual system, reducing the dimensionality of data and representing images with higher-level features. Common pooling operations include max pooling, mean pooling, random pooling, median pooling, and combined pooling. Typically, pooling layers are periodically inserted between convolutional layers in neural networks to achieve dimensionality reduction.

[0055] The normalization layer is used to normalize all neurons in the intermediate layer to prevent gradient explosion and gradient vanishing.

[0056] (2) Loss Function

[0057] During neural network training, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight matrix of each layer (usually initialized before the first update, pre-configuring parameters for each layer). For example, if the prediction is too high, the weight matrix is ​​adjusted to predict a lower value, and this process continues until the neural network accurately predicts the target value. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the basis of the loss function or objective function. These are crucial equations for measuring the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the neural network becomes a process of minimizing this loss.

[0058] (3) Generative Adversarial Networks

[0059] Generative Adversarial Networks (GANs) typically consist of a generator (G) and a discriminator (D). Unsupervised learning is achieved through a game-like interaction between the generator and the discriminator. The generator randomly samples data from the latent space as input, and its output should closely mimic real samples from the training set. The discriminator takes either real samples or the generator's output as input and aims to distinguish the generator's output from all input samples, including real samples, as closely as possible. The generator, in turn, tries to deceive the discriminator. This adversarial relationship between the generator and the discriminator allows for continuous parameter adjustments, ultimately generating highly realistic images and completing the model's training.

[0060] Before introducing the embodiments of this application, a brief introduction will be given to the age-related image processing method based on machine learning known to the inventors of this application, so as to facilitate the understanding of the embodiments of this application later.

[0061] In some schemes, a conditional generative adversarial network (GAN) is trained using multiple images of different age groups with pre-labeled ages as a training set. This GAN comprises an image generator (G), an image discriminator (D), an age estimation network (AEN), and an identity recognition network (FRN). G is trained to generate aged images; specifically, it automatically and effectively generates aged images by inputting young images and preset age conditions. D is used to verify whether the generated aged images are genuine, ensuring their deceptiveness. AEN reduces the discrepancy between the age of the generated aged images and the preset value, while FRN ensures consistency in the identity of the person in the generated image. During model training, the training set is grouped according to age groups, allowing the GAN to learn the aging features corresponding to each age group, thus realizing the aging changes in the face.

[0062] In this scheme, aging features are learned from each set of images, ensuring that these features are determined solely by age group and that individuals exhibit identical aging characteristics. Therefore, for each set of test data, the feature changes in facial aging are the same. However, in real life, each person's aging feature changes are influenced by individual factors and are specific. Therefore, this scheme does not consider individual differences and cannot accurately generate images showing signs of aging.

[0063] To address the aforementioned problems, this application provides a method for training an image generation model, a method for generating images showing age changes, and related apparatus. First, an image generation network is constructed, comprising an encoder, a mapping and transformation module, a fusion module, and a decoding module. Then, the image generation network is trained using several real face images and their corresponding age differences to obtain the image generation model. Specifically, a pre-trained age detection model is used to detect the age of each real face image to obtain a predicted age. The age difference between the predicted age and the actual age is calculated, and the age difference and the real face images are input together into the image generation network for fusion to generate a predicted face image affected by the age difference. Since the age detection model is trained on a large number of face images, its output predicted age can reflect the universal age characteristics of real face images, while the age difference can reflect the individual's unique age characteristics (older or younger) in the input image. Therefore, the age difference can guide the image generation network to learn the individual's unique age characteristics in the input image. The image generation network is trained using several image groups in the manner described above. Based on backpropagation of the loss sum, the predicted face images corresponding to each image group are constrained to continuously approximate the real face images. Thus, the image generation network can accurately reconstruct the individual's unique age characteristics from the input image. Therefore, the trained image generation model can accurately generate age change images that match both the expected and actual ages based on the age difference between the expected and actual ages. In other words, when predicting aging or tracing back to youth, the age characteristics in the test face image are adjusted based on the individual's current unique age characteristics, ensuring that the changes in age characteristics are not uniform.

[0064] The following describes exemplary applications of the electronic devices provided in this application for training image generation models or generating age-changing images. It is understood that the electronic devices can both train image generation models and use such image generation models to generate age-changing images.

[0065] The electronic device provided in this application embodiment can be a server, such as a server deployed in the cloud. When the server is used to train an image generation model, it iteratively trains the image generation network using the training set provided by other devices or those skilled in the art, determines the final model parameters, and then configures the image generation network with these final model parameters to obtain the image generation model. The training set includes several real face images, each labeled with its real age, and the real ages of these real face images cover a preset age range. When the server is used to generate age-changing images, it calls the built-in image generation model to perform corresponding calculations on test face images provided by other devices or users, the real age of the faces in the test face images, and the expected age, generating age-changing images that match the expected age.

[0066] The electronic devices provided in some embodiments of this application can be various types of terminals such as laptops, desktop computers, or mobile devices. When the terminal is used to train an image generation model, those skilled in the art input a prepared training set into the terminal and design an image generation network on the terminal. The terminal uses the training set to iteratively train the image generation network to determine the final model parameters. The image generation network is then configured with these final model parameters to obtain the image generation model. When the terminal is used to generate age-changing images, it calls the built-in image generation model to perform corresponding calculations on the user-input test face image, the actual age of the face in the test face image, and the expected age, generating an age-changing image that matches the expected age.

[0067] As an example, see Figure 1 , Figure 1 This is a schematic diagram of an application scenario for the system for predicting age changes provided in this application embodiment. The terminal 10 is connected to the server 20 via a network, which can be a wide area network or a local area network, or a combination of both.

[0068] Terminal 10 can be used to acquire training sets and construct image generation networks. For example, those skilled in the art can download prepared training sets on the terminal and construct the network structure of the image generation network. It is understood that terminal 10 can also be used to acquire test face images, the actual age of the face in the test face image, and the expected age. For example, a user inputs a test face image, the actual age of the face in the test face image, and the expected age through an input interface. After input, the terminal automatically acquires the test face image, the actual age of the face in the test face image, and the expected age. Alternatively, terminal 10 may have a camera to capture test face images, and then the user inputs the actual age and expected age through the input interface of terminal 10.

[0069] In some embodiments, terminal 10 locally executes the method for training an image generation model provided in this application to train the designed image generation network using a training set, determine the final model parameters, and then configure the image generation network with the final model parameters to obtain the image generation model. In some embodiments, terminal 10 may also send a training set and a constructed image generation network stored on the terminal by a person skilled in the art to server 20 via a network. Server 20 receives the training set and the image generation network, trains the designed image generation network using the training set, determines the final model parameters, and then sends the final model parameters to terminal 10. Terminal 10 saves the final model parameters so that the image generation network can be configured with the final model parameters to obtain the image generation model.

[0070] In some embodiments, terminal 10 locally executes the method for generating age change images provided in this application embodiment to provide users with an age change image generation service. It calls a built-in image generation model to perform corresponding calculations on the user-inputted test face image, the actual age of the face in the test face image, and the expected age, generating an age change image that matches the expected age. In some embodiments, terminal 10 can also send the user-inputted test face image, the actual age of the face in the test face image, and the expected age to server 20 via a network. Upon receiving the test face image, the server 20 calls its built-in image generation model to perform corresponding calculations on the test face image, the actual age of the face in the test face image, and the expected age, generating an age change image that matches the expected age, and then sends the age change image to terminal 10. After receiving the age change image, terminal 10 displays the age change image on its own display interface, allowing users to view the effect of the face in the test face image changing its age according to the expected age.

[0071] The structure of the electronic device in the embodiments of this application is described below. Figure 2 This is a schematic diagram of the structure of an electronic device 500 in an embodiment of this application. The electronic device 500 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together through a bus system 540. It is understood that the bus system 540 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0072] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0073] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0074] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0075] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0076] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks.

[0077] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0078] Display module 553 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., a display screen, a speaker, etc.) associated with user interface 530;

[0079] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0080] As can be understood from the above, the method for training an image generation model and the method for generating age-changing images provided in the embodiments of this application can be implemented by various types of electronic devices with computing capabilities, such as smart terminals and servers.

[0081] The following describes the method for training an image generation model provided in this application, using exemplary applications and implementations of the server provided in the embodiments of this application. See also Figure 3 , Figure 3 This is a flowchart illustrating the method for training an image generation model provided in an embodiment of this application. Please refer to... Figure 4 Figure 4 is a schematic diagram of the image generation network structure. For example... Figure 4 As shown, the image generation network, which serves as the network structure of the image generation model, includes an encoder, a mapping and transformation module, a fusion module, and a decoding module. The encoder, mapping and transformation module, fusion module, and decoding module are all pre-configured neural networks, possessing neural network components (convolutional layers, deconvolutional layers, pooling layers, or activation functions, etc.). The basic structure and principles of neural networks have been explained in detail in "Introduction to Terminology (1)," and will not be elaborated upon here. The image generation network can be constructed by those skilled in the art on a neural network design platform on a terminal (e.g., a computer) and then sent to a server.

[0082] In some embodiments, the intra-layer structure (convolution kernel, stride, etc.), inter-layer connection structure, and layer combination of convolutional layers, deconvolutional layers, or pooling layers in the encoder, mapping conversion module, fusion module, and decoding module can be set to obtain a specific image generation network.

[0083] Those skilled in the art will understand that the encoder includes multiple downsampling convolutional layers, where the size of the output feature map decreases with each downsampling convolutional layer. The decoding module includes multiple upsampling convolutional layers, where the size of the output feature map increases with each upsampling convolutional layer. Those skilled in the art can configure parameters such as the kernel size and stride of each downsampling and upsampling convolutional layer according to actual needs.

[0084] Similarly, the structure of the mapping transformation module can be set so that the mapping transformation model can be used to perform spatial transformation processing on the feature data so that the size of the transformed feature data meets the input requirements of the next level; the structure of the fusion module can also be set so that the fusion module can be used to fuse at least two feature data so that the fusion result can reflect the features in at least two feature data.

[0085] Please refer to it again. Figure 3 The method S100 may specifically include the following steps:

[0086] S10: Obtain several real face images, each real face image is labeled with the real age, and the real ages of the several real face images cover a preset age range.

[0087] Any real face image is a face image taken at the person's actual age. For example, if real face image 1# is a photo of a person taken when they are 46 years old, then the real age labeled for real face image 1# is 46 years old.

[0088] The real ages of these real face images cover a preset age range. In some embodiments, the preset age range can be 1-100 years old, so that there are face images of people aged 1-100 years old among these real face images. Each age corresponds to multiple real face images of different people.

[0089] It is worth noting that these multiple real facial images can be composed of real facial images of different people, as long as the ages cover a preset age range (e.g., 1-100 years old). It is not necessary to have facial images of the same person at different ages, which effectively reduces the difficulty of data collection. Those skilled in the art can set the preset age range according to the actual situation, such as 1-90 years old. Those skilled in the art can obtain facial photos of several individuals of different ages from various publicly available facial databases and label them with the actual age at the time the photos were taken.

[0090] In some implementations, to accelerate the convergence speed of model training, real face images can be preprocessed to adjust the faces in the real face images to be frontal, and the real face images can be normalized to uniform size.

[0091] In some embodiments, preprocessing of the real facial image specifically includes:

[0092] A facial landmark algorithm is used to obtain the center coordinates of the left and right eyeballs in a real face image;

[0093] Calculate the angle between the line connecting the centers of the left and right eyeballs in a real face image and the horizontal direction;

[0094] Using the center coordinates of the left and right eyeballs in the real face image as the base point, rotate the real face image by the aforementioned angle;

[0095] The face region is extracted from the rotated real face image and resized to a preset resolution to obtain the preprocessed real face image.

[0096] Based on the facial landmark algorithm, several key points are identified in the real face image, including points in areas such as eyebrows, eyes, nose, mouth, and facial contours. From these key points, the center coordinates of the left and right eyeballs in the real face image can be obtained, and the angle θ between the line connecting the center coordinates of the left and right eyeballs and the horizontal direction can be calculated.

[0097] It is understandable that the included angle θ is the angle at which the face deviates from a frontal view. In order to adjust the face in the real face image to a frontal view, the real face image is rotated according to the above included angle θ, with the center coordinates of the left and right eyeballs in the real face image as the base point, so as to obtain a frontal view.

[0098] Specifically, the rotated real face image can be calculated using the following formula:

[0099]

[0100] Where (x,y) are the two-dimensional coordinates of the pixels in the real face image before rotation, and (x',y') are the two-dimensional coordinates of the pixels in the real face image after rotation.

[0101] Based on the premise that the face in the rotated real-face image is a frontal view, the facial region is cropped from the rotated real-face image and resized to a preset resolution to obtain a preprocessed real-face image. Thus, the size of the preprocessed real-face image is equal to the preset resolution.

[0102] In some embodiments, the preset resolution can be 1024*1024. In other embodiments, the preset resolution can be set by those skilled in the art according to actual conditions, and no limitation is made here.

[0103] In this embodiment, by aligning the facial features of the preprocessed real face image using the above preprocessing method, the model can learn features better and converge more effectively.

[0104] It is understood that the real face images in the following description may be preprocessed real face images, and for the sake of convenience, they will be referred to as real face images below.

[0105] S20: Use a pre-trained age detection model to detect the age of several real face images and obtain the predicted age corresponding to each real face image.

[0106] The age detection model is pre-trained using a neural network and training set by those skilled in the art, and is therefore existing technology in this field, so it will not be described in detail here. The age detection model can detect the age of an input face image. Therefore, by inputting several real face images into the age detection model, the predicted age corresponding to each real face image can be obtained.

[0107] Since age detection models are trained on a large number of facial images, their predicted ages can reflect the universal age characteristics of real facial images. Here, universal age characteristics refer to the age-appropriate features that most people's faces exhibit under normal circumstances (e.g., the degree of wrinkles, sagging, or drooping is age-appropriate).

[0108] S30: Input the real face image and the age difference corresponding to the real face image into the image generation network to obtain the predicted face image.

[0109] Here, the age difference is the difference between the predicted age and the actual age. The actual face image is encoded by the encoder to obtain the encoding result. The age difference is mapped and transformed by the mapping and transformation module to obtain the age vector. The encoding result and the age vector are fused by the fusion module. The fusion result is input into the decoding module for decoding to obtain the predicted face image.

[0110] After obtaining the predicted age, the difference between the predicted age and the real age is calculated to obtain the age difference. Then, the real face image and the age difference are input into the image generation network. The image generation network generates the predicted face image based on the real face image and the age difference.

[0111] Since age detection models are trained on a large number of facial images, their predicted ages reflect the universal age characteristics of real facial images. Therefore, age differences can reflect the individual's unique age characteristics (looking older or younger) in real facial images. Consequently, age differences can guide image generation networks to learn the individual age characteristics of faces in real facial images. Here, individual age characteristics refer to age characteristics influenced by individual factors, possessing specificity, such as looking older, younger, or having looser skin and more wrinkles.

[0112] Specifically, please refer to Figure 5 The process begins with inputting a real face image into an encoder for downsampling feature extraction, yielding the encoded result. The age difference is then input into a mapping transformation model, converting the numerical value into an age vector. Since the age difference reflects an individual's unique age characteristics in a real face image, the age vector also reflects these characteristics. The encoded result and the age vector are then fused into a fusion module to obtain a fused feature map that reflects both facial features and unique age characteristics. Finally, the fused feature map is input into a decoding module for upsampling decoding, resulting in the predicted face image. Based on the fusion result reflecting both facial features and unique age characteristics, the image generation network can learn these unique age characteristics and generate predicted face images influenced by them.

[0113] In some implementations, please refer to Figure 5 The decoding module includes N branches of decoders, with each decoder corresponding to an age range. For example, if each age range is 10 years, and the preset age range is 1-100 years, then the decoding module includes 10 branches of decoders.

[0114] Each branch's decoder comprises multiple upsampling convolutional layers, and the layer structures of these 10 branches can be identical. Each decoder upsamples the input fusion result (fusion feature map) to obtain a predicted face image with universal and individual age features. Each branch focuses on learning the universal and individual age features for its corresponding age group.

[0115] In some embodiments, the aforementioned method S100 further includes: determining the number of branches N of the decoder based on the error between the maximum age value of the preset age range and the age detection model.

[0116] Understandably, the age detection model is pre-trained and has prediction errors, typically within ±5 years. Therefore, we can use 5 years as an age range. When the preset age range is 1-100, the maximum age is 100, thus determining the number of decoder branches N = 100 / 5 = 20. That is, (1,5) corresponds to one decoder, [5,10) corresponds to one decoder, and so on, [95,100) corresponds to one decoder.

[0117] In some embodiments, a gated switch α may be used. i (i = 1, ..., N) are used to control the input of the fusion result, when α i =0 indicates that the decoder of the i-th branch is in the off state, when α i =1 indicates that the decoder of the i-th branch is in the open state, and the fusion result is input into the decoder of the i-th branch.

[0118] In this embodiment, the number of decoder branches N is determined based on the error of the age detection model and the preset age range. Using the error as the age range can reduce the impact of the error of the age detection model and help the decoders of each branch generate more accurate predicted face images.

[0119] In this embodiment, the aforementioned "fusion result input to decoding module for decoding to obtain predicted face image" includes:

[0120] Based on the predicted age range, the target decoder is determined from N decoders; the fusion result is input into the target decoder for decoding to obtain the predicted face image.

[0121] For example, if the predicted age is 48 years old, the age range is (45, 50]. Therefore, the decoder corresponding to the age range (45, 50) is selected as the target decoder. The gating switch of the target decoder is turned on, while the gating switches of other branch decoders are turned off and do not participate in training. The fusion result is input into the target decoder for upsampling. The target decoder focuses on learning the universal age features and individual age features of the (45, 50) age range, and outputs a predicted face image with universal and individual age features.

[0122] In this embodiment, a multi-branch decoding module is constructed, with each branch's decoder focusing on training its own age group. It focuses on learning the universal age characteristics and individual age characteristics of the age group corresponding to its branch. The training of each age group will not interfere with each other, which can accelerate model convergence and improve model accuracy.

[0123] In some embodiments, the aforementioned mapping transformation module performs mapping transformation on the age difference in the following manner: it calculates the age difference using a mapping function to obtain an age vector; wherein the mapping function includes a linear fully connected network.

[0124] The mapping function is used to map the age difference to a spatial vector. For example, the mapping function can be a linear fully connected network. Linear fully connected networks are existing technology in this field and will not be described in detail here.

[0125] For example, the age difference t is calculated using the following formula and transformed into an age vector V. t .

[0126] V t =f(t)

[0127] In this embodiment, a mapping function is used to spatially map the age difference and transform it into an age vector. Thus, the age vector, which can represent individual age characteristics, can be easily fused with the encoding results.

[0128] In some embodiments, the aforementioned fusion module fuses the encoding result and the age vector in the following manner: multiplying the encoding result and the age vector to obtain the fusion result.

[0129] Here, multiplication fusion refers to multiplying corresponding positions while maintaining the same resolution. The age vector is processed through deconvolution to transform it into an age difference feature map with the same resolution as the encoded result. Then, it is multiplied corresponding positions with the encoded result (encoded feature map) to obtain the fused result. In this embodiment, multiplication fusion ensures that the fused result includes individual age features of the age difference.

[0130] S40: The loss function is used to calculate the loss between the predicted face image and the real face image. Based on the loss sums corresponding to several real face images, the image generation network is iteratively trained until convergence, thus obtaining the image generation model.

[0131] Here, the loss function can be configured by those skilled in the art in the terminal. The configured loss function is sent to the server along with the image generation network. After the server processes and obtains each predicted face image, it uses the loss function to calculate the loss between each real face image and the predicted face image. Based on the loss, the image generation network is iteratively trained until the image generation network converges, and the image generation model is obtained.

[0132] Understandably, the smaller the difference between each real face image and the predicted face image, the more similar the predicted face image and the real face image will be, indicating that the predicted face image can accurately reproduce individual age characteristics. Therefore, the model parameters of the aforementioned image generation network can be adjusted based on the differences between each predicted face image and the real face image, and the image generation network can be iteratively trained. That is, the differences are backpropagated, causing the predicted face image output by the image generation network to continuously approximate the real face image until the image generation network converges, resulting in the image generation model.

[0133] In some embodiments, the Adam algorithm is used to optimize the model parameters. For example, the number of iterations is set to 100,000, the initial learning rate is set to 0.001, the learning rate weight decay is set to 0.0005, and the learning rate decays to 1 / 10 of its original value every 1,000 iterations. The learning rate, loss, and input can be fed into the Adam algorithm to obtain the adjusted model parameters output by the Adam algorithm. These adjusted model parameters are used for the next training iteration until training is complete. Then, the converged model parameters of the image generation network are output, thus obtaining the image generation model.

[0134] In this embodiment, an image generation network is constructed, comprising an encoder, a mapping and transformation module, a fusion module, and a decoding module. Then, the image generation network is trained using several real face images and their corresponding age differences to obtain an image generation model. Specifically, a pre-trained age detection model is used to detect the age of each real face image to obtain a predicted age. The age difference between the predicted age and the real age is calculated, and this age difference, along with the real face image, is input into the image generation network for fusion to generate a predicted face image influenced by the age difference. Since the age detection model is trained on a large number of face images, its output predicted age reflects the universal age characteristics of real face images, while the age difference reflects the individual's unique age characteristics (older or younger). Therefore, the age difference can guide the image generation network to learn the individual's unique age characteristics in the input image. By training the image generation network with several image groups in the above manner, and using backpropagation based on the loss sum, the predicted face image corresponding to each image group is constrained to continuously approach the real face image. Thus, the image generation network can accurately reconstruct the individual's unique age characteristics in the input image. Therefore, the trained image generation model can accurately generate age change images that match one's own individual age characteristics and expected age based on the age difference between the expected age and the actual age. In other words, when predicting aging or tracing back to youth, it changes the age characteristics in the test face image based on the individual's current individual age characteristics, so that the changes in age characteristics are not uniform.

[0135] In some embodiments, the loss function includes adversarial loss, reconstruction loss, and perceptual style loss, wherein the adversarial loss reflects whether the predicted face image is a real face, the reconstruction loss reflects the pixel difference between the predicted face image and the real face image, and the perceptual style loss reflects the difference between the feature map of the predicted face image and the feature map of the real face image.

[0136] Those skilled in the art will understand that adversarial loss is the loss in predicting whether a face image corresponds to a real face image. A large adversarial loss indicates a significant difference between the distribution of the predicted face image and the distribution of the real face image; conversely, a small adversarial loss indicates a smaller difference or similarity between the distribution of the predicted face image and the distribution of the real face image. Here, image distribution refers to the distribution of various parts of the face image, such as facial features.

[0137] Reconstruction loss is the pixel difference between the predicted face image and the real face image. A large reconstruction loss indicates a significant difference between the pixels in the predicted and real face images, while a small reconstruction loss indicates a smaller difference. Understandably, the smaller the reconstruction loss, the closer the pixels are to the real face image, resulting in less distortion.

[0138] Perceptual style loss compares the feature map obtained by convolving the predicted face image with the feature map obtained by convolving the real face image, so that the high-level information (content and global structure) is similar.

[0139] In some embodiments, the loss function includes:

[0140] in,

[0141]

[0142]

[0143]

[0144] Where Loss is the loss function. To combat the losses, To reconstruct the loss, To perceive style loss, α i Let represent the on / off state of the decoder in the i-th branch, N be the number of decoding branches, I be the real face image, I′ be the predicted face image, D(I) be the probability that the discriminator identifies the real face image as real, t be the age difference, G(I,t) be the predicted face image generated by the image generation network based on the real face image and the age difference, and D(G(I,t)) be the probability that the discriminator identifies the predicted face image as real. This is the i-th feature map extracted from a real face image by the feature extraction network. The i-th feature map is obtained by the feature extraction network from the predicted face image.

[0145] In some embodiments, convolutional neural networks such as VGG can be used to downsample real face images and extract five feature maps. Similarly, convolutional neural networks such as VGG can be used to downsample the predicted face image and extract five feature maps.

[0146] Therefore, by iteratively training the image generation network based on the differences calculated by loss functions including adversarial loss, reconstruction loss, and perceptual style loss, the predicted face images can be constrained to continuously approach the real face images in terms of distribution, pixel features, and multi-scale features. This is beneficial to improving the performance of the trained image generation model in predicting age changes.

[0147] In summary, by designing the above-described image generation network structure, which includes an encoder, a mapping and transformation module, a fusion module, and a decoding module, the image generation network is trained using several real face images and their corresponding age differences to obtain an image generation model. Specifically, a pre-trained age detection model is used to detect the age of each real face image to obtain a predicted age. The age difference between the predicted age and the real age is calculated, and this age difference, along with the real face image, is input into the image generation network for fusion to generate a predicted face image influenced by the age difference. Since the age detection model is trained on a large number of face images, its output predicted age reflects the universal age characteristics of real face images, while the age difference reflects the individual's unique age characteristics (too old or too young). Therefore, the age difference can guide the image generation network to learn the individual's unique age characteristics in the input image. By training the image generation network with several image groups in the above manner, and using backpropagation based on the loss sum, the predicted face image corresponding to each image group is constrained to continuously approach the real face image. Thus, the image generation network can accurately reconstruct the individual's unique age characteristics in the input image. Therefore, the trained image generation model can accurately generate age change images that match one's own individual age characteristics and expected age based on the age difference between the expected age and the actual age. In other words, when predicting aging or tracing back to youth, it changes the age characteristics in the test face image based on the individual's current individual age characteristics, so that the changes in age characteristics are not uniform.

[0148] After training an image generation model using the method provided in this application, the model can be used to generate images showing age changes. The method for generating images showing age changes provided in this application can be implemented by various types of electronic devices with computational processing capabilities, such as smart terminals and servers.

[0149] The method for generating age-changing images provided in this application is described below with reference to exemplary applications and implementations of the terminal provided in the embodiments of this application. See also Figure 6 , Figure 6 This is a schematic flowchart of a method for generating images showing age changes provided in an embodiment of this application. Method S200 includes the following steps:

[0150] S201: Obtain the test face image, the true age of the face in the test face image, and the expected age.

[0151] The aging prediction assistant or youth-tracking assistant (application software) built into the terminal (e.g., a smartphone) acquires a test facial image, its corresponding real age, and expected age. This test facial image can be captured by the terminal or input by the user. It is understood that the test facial image includes a face. The real age and expected age can be input by the user, for example, a real age of 20 and an expected age of 30, or a real age of 50 and an expected age of 30.

[0152] S202: Input the test face image and the age difference between the actual age and the expected age into the image generation model to obtain an age change image. This age change image reflects a person's age in accordance with the expected age; wherein, the image generation model is trained using any of the image generation model training methods described in the above embodiments.

[0153] After obtaining the real age and expected age, the terminal calculates the age difference between them. Then, it inputs the test face image and the age difference into an image generation model. After processing, the model outputs an age change image. This image reflects a person's age in a way that aligns with their expected age. When the expected age is greater than the real age, the age change image is a predicted image of aging; when the expected age is less than the real age, it is a youthful image.

[0154] The image generation model is trained using any of the methods described above. The aging prediction assistant or youth-tracing assistant within the terminal includes this image generation model, which is then invoked to predict aging or trace youth. It is understood that this image generation model is trained using any of the methods described above and has the same structure and function as the image generation model in the above embodiments, and will not be described in detail here.

[0155] This application also provides a computer-readable storage medium storing computer-executable instructions for causing an electronic device to execute the method for training an image generation model provided in this application, for example... Figure 3-5 The method for training the image generation model shown, or the method for generating age-changing images provided in the embodiments of this application, for example, such as... Figure 6 The method for generating images of age changes is shown.

[0156] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0157] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0158] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0159] As an example, executable instructions can be deployed to execute on a single computing device (including devices such as smart terminals and servers), or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0160] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. When executed by a computer, the program instructions cause the computer to perform the method for training an image generation model or the method for generating age-changing images as described in the foregoing embodiments.

[0161] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training an image generation model, characterized in that, The image generation network includes an encoder, a mapping and transformation module, a fusion module, and a decoding module, including: Acquire several real face images, each of which is labeled with a real age, and the real ages of the several real face images cover a preset age range; A pre-trained age detection model is used to detect the age of the several real face images to obtain the predicted age corresponding to each real face image; The real face image and the age difference corresponding to the real face image are input into the image generation network to obtain a predicted face image; wherein, the age difference is the difference between the predicted age and the real age, the real face image is encoded by the encoder to obtain the encoding result, the age difference is mapped and transformed by the mapping and transformation module to obtain the age vector, the encoding result and the age vector are fused by the fusion module, and the fusion result is input into the decoding module to decode to obtain the predicted face image; The loss function is used to calculate the loss between the predicted face image and the real face image. Based on the sum of the losses corresponding to the several real face images, the image generation network is iteratively trained until convergence, thus obtaining the image generation model.

2. The method according to claim 1, characterized in that, The decoding module includes N branches of decoders, with each decoder corresponding to an age group; The fusion result is input into the decoding module for decoding to obtain the predicted face image, including: Based on the predicted age range, the target decoder is determined from the N decoders; The fusion result is input into the target decoder for decoding to obtain the predicted face image.

3. The method according to claim 2, characterized in that, The method further includes: determining the number of branches N of the decoder based on the error between the maximum age value of the preset age range and the age detection model.

4. The method according to any one of claims 1-3, wherein the mapping conversion module performs mapping conversion on the age difference in the following manner: The age difference is calculated using a mapping function to obtain the age vector; wherein, The mapping function includes a linear fully connected network.

5. The method according to any one of claims 1-3, characterized in that, The fusion module fuses the encoding result and the age vector in the following manner: The encoding result and the age vector are multiplied and fused to obtain the fusion result.

6. The method according to claim 2 or 3, characterized in that, The loss function includes adversarial loss, reconstruction loss, and perceptual style loss, wherein the adversarial loss reflects whether the predicted face image is the real face, the reconstruction loss reflects the pixel difference between the predicted face image and the real face image, and the perceptual style loss reflects the difference between the feature map of the predicted face image and the feature map of the real face image.

7. The method according to claim 6, characterized in that, The loss function includes: in, Where Loss is the loss function. For the aforementioned resistance loss, For the reconstruction loss, For the perceptual style loss, α i Let represent the on / off state of the decoder for the i-th branch, N be the number of decoding branches, I be the real face image, I′ be the predicted face image, D(I) be the probability that the discriminator identifies the real face image as real, t be the age difference, G(I,t) be the predicted face image generated by the image generation network based on the real face image and the age difference, and D(G(I,t)) be the probability that the discriminator identifies the predicted face image as real. This refers to the i-th feature map extracted from a real face image by the feature extraction network. The i-th feature map is obtained by the feature extraction network from the predicted face image.

8. A method for generating images showing changes in age, characterized in that, include: Obtain a test face image, the actual age of the face in the test face image, and the expected age; The test face image and the age difference between the actual age and the expected age are input into the image generation model to obtain the age change image, wherein the age change image reflects the age of the person and is consistent with the expected age; wherein the image generation model is trained using the method of training an image generation model as described in any one of claims 1-7.

9. An electronic device, characterized in that, include: At least one processor, and The memory communicatively connected to the at least one processor, wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer device to perform the method as described in any one of claims 1-8.

Images