Training of a face attribute editing model, face attribute editing method and device

Abstract

The application discloses a kind of face attribute editing model training, face attribute editing method and device, it is related to computer technology field.The method comprises the following steps: obtaining original image and target attribute;Original image and target attribute are used as the input of initial face attribute editing model, and first image is obtained;Original image and the original attribute of original image are used as the input of initial face attribute editing model, and second image is obtained;First image and the original attribute of original image are used as the input of initial face attribute editing model, and third image is obtained;Based on the difference between first image, second image and third image and original image, the network parameters of initial face attribute editing model are determined, and target face attribute editing model is obtained.The method reduces the dependence on training sample, solves the technical problem that a large number of image pairs are required as training samples in related technologies, reduces the cost and reduces the influence of data accuracy on model accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a training method and apparatus for a facial attribute editing model, a method for editing facial attributes, and an apparatus for such editing. Background Technology

[0002] Facial attribute editing involves altering one or more attributes of a face, such as its pose or age. It's a type of facial editing widely used in image processing and video editing. For example, editing the age attribute of a face image can create the illusion of facial aging, potentially helping to locate missing persons. In entertainment, facial expressions can be edited to make photos or videos look more natural. In the workplace, facial expression editing can be used in web conferencing, reducing buffering and stuttering by storing reconstructed faces based on facial features.

[0003] Currently, two main methods are used in related technologies to edit facial attributes. The first algorithm typically involves finding key points in the facial image and then using these key points to edit the face manually or through mesh deformation. The second algorithm uses a pre-trained facial image generation model trained through adversarial learning to infer the direction of latent vectors, thereby changing the attributes. This can be divided into two methods: direct adversarial training and inferring the direction of latent vectors from the generation model.

[0004] However, the first method often results in unnatural and distorted faces due to stretching and deformation of the image, leading to harsh and unnatural results. The second method requires a large dataset of paired faces and is prone to changing face IDs during use; that is, although the attribute changes to the target attribute, the face may not be the same person's. Similarly, using inverse latent vectors to change the direction can lead to attribute entanglement, where modifying one attribute severely affects others. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, embodiments of the present invention provide a training method, apparatus, electronic device and medium for a face attribute editing model, a face attribute editing method, and a medium.

[0006] In a first aspect, embodiments of the present invention provide a method for training a facial attribute editing model, comprising: Obtain the original image and target attributes; The original image and the target attributes are used as input to the initial face attribute editing model to obtain the first image; The original image and its original attributes are used as input to the initial face attribute editing model to obtain the second image; The original attributes of the first image and the original image are used as input to the initial face attribute editing model to obtain the third image; Based on the differences between the first image, the second image, the third image, and the original image, the network parameters of the initial face attribute editing model are determined, and the target face attribute editing model is obtained.

[0007] Optionally, determining the network parameters of the initial face attribute editing model based on the differences between the first image, the second image, the third image, and the original image includes: An identity loss function, a reconstruction loss function, and a cycle consistency loss function are determined; the identity loss function is used to determine the difference between the first image and the original image, the reconstruction loss function is used to determine the difference between the second image and the original image, and the cycle consistency loss function is used to determine the difference between the third image and the original image. Based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function, the network parameters of the initial face attribute editing model are determined.

[0008] Optionally, determining the network parameters of the initial face attribute editing model based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function includes: Based on the identity loss function, the reconstruction loss function, the cycle consistency loss function, and the adversarial loss function, the network parameters of the initial face attribute editing model are determined using adversarial learning.

[0009] Optionally, the initial face attribute editing model adopts an encoder-decoder structure, wherein the encoder includes a feature extractor, an identity encoder, a condition generator, and a feature mapping network; The feature extractor is used to obtain the first intermediate feature vector of the image input to the initial face attribute editing model; The identity encoder is used to obtain the identity feature vector of the face in the image input to the initial face attribute editing model based on the first intermediate feature vector; The condition generator is used to convert the target attribute or the original attribute into an attribute feature vector; The feature mapping network is used to obtain a second intermediate feature vector of the image input to the initial face attribute editing model based on the first intermediate feature vector, and to obtain a target feature vector based on the second intermediate feature vector and the attribute feature vector; The decoder is used to decode the identity feature vector and the target feature vector.

[0010] Optionally, determining the difference between the first image and the original image using the identity loss function includes: The identity features of the first image are obtained using the identity encoder, and the identity features of the original image are also obtained using the identity encoder. Based on the identity loss function, the difference between the identity features of the first image and the identity features of the original image is determined, thereby determining the difference between the first image and the original image.

[0011] Secondly, embodiments of the present invention provide a method for editing facial attributes, including: Obtain the face image to be edited; Determine the editable attributes of the face image to be edited and the target values ​​corresponding to the editable attributes; The image of the face to be edited and the target value corresponding to the attribute to be edited are used as inputs to the target face attribute editing model to obtain the output image of the target face attribute editing model. The output image is the desired image after the attribute to be edited of the image of the face to be edited is edited.

[0012] Optionally, obtaining the face image to be edited includes: obtaining the image to be edited; using a face detection model to determine the face in the image to be edited to obtain a first face image; performing face alignment on the face image to be edited to obtain a second face image; and removing the background from the second face image to obtain the face image to be edited.

[0013] Thirdly, embodiments of the present invention provide a training apparatus for a facial attribute editing model, comprising: The first acquisition module is used to acquire the original image and target attributes; The generation module is used to take the original image and the target attributes as input to an initial face attribute editing model to obtain a first image; take the original image and the original attributes of the original image as input to the initial face attribute editing model to obtain a second image; and take the first image and the original attributes of the original image as input to the initial face attribute editing model to obtain a third image. An iterative module is used to determine the network parameters of the initial face attribute editing model based on the differences between the first image, the second image, the third image and the original image, and to obtain the target face attribute editing model.

[0014] Optionally, the iterative module is used to: determine an identity loss function, a reconstruction loss function, and a cycle consistency loss function; the identity loss function is used to determine the difference between the first image and the original image, the reconstruction loss function is used to determine the difference between the second image and the original image, and the cycle consistency loss function is used to determine the difference between the third image and the original image; and based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function, determine the network parameters of the initial face attribute editing model.

[0015] Optionally, the iterative module is used to: determine the network parameters of the initial face attribute editing model by using adversarial learning based on the identity loss function, the reconstruction loss function, the cycle consistency loss function, and the adversarial loss function.

[0016] Fourthly, embodiments of the present invention provide a facial attribute editing device, comprising: The second acquisition module is used to acquire the face image to be edited; The determining module is used to determine the editable attributes of the face image to be edited and the target values ​​corresponding to the editable attributes; The editing module is used to take the face image to be edited and the target value corresponding to the attribute to be edited as input to the target face attribute editing model, and obtain the output image of the target face attribute editing model. The output image is the desired image after the attribute to be edited of the face image to be edited is edited.

[0017] Optionally, the second acquisition module is further configured to: acquire an image to be edited; use a face detection model to determine the face in the image to be edited to obtain a first face image; perform face alignment on the image to be edited to obtain a second face image; and remove the background from the second face image to obtain the image to be edited.

[0018] Fifthly, embodiments of the present invention provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a training method or a face attribute editing method of any embodiment of the present invention.

[0019] In a sixth aspect, embodiments of the present invention provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a training method or a face attribute editing method for a face attribute editing model according to any embodiment of the present invention.

[0020] One embodiment of the above invention has the following advantages or beneficial effects: The training method for the face attribute editing model in this invention involves obtaining a first image by using the original image and target attributes as inputs to the initial face attribute editing model; obtaining a second image by using the original image and its original attributes as inputs to the initial face attribute editing model; obtaining a third image by using the first image and its original attributes as inputs to the initial face attribute editing model; and then, based on the differences between the first image, the second image, the third image, and the original image, iteratively training the network parameters of the initial face attribute editing model to obtain the target face attribute editing model. This method reduces the dependence on training samples, solves the technical problem in related technologies that require a large number of image pairs as training samples when training the model, reduces costs, and mitigates the impact of data accuracy on model accuracy.

[0021] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0022] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 A flowchart illustrating the training method of the face attribute editing model according to an embodiment of the present invention is shown; Figure 2 A schematic diagram of the structure of the initial face attribute editing model according to an embodiment of the present invention is shown; Figure 3 The diagram shows the effect of face attribute editing using the target face attribute editing model of this invention. Figure 4 A flowchart illustrating the face attribute editing method according to an embodiment of the present invention is shown; Figure 5 A schematic diagram of a sub-process of the face attribute editing method according to an embodiment of the present invention is shown; Figure 6 A schematic diagram of the structure of a training device for a face attribute editing model according to an embodiment of the present invention is shown; Figure 7 A schematic diagram of the structure of the face attribute editing device according to an embodiment of the present invention is shown; Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0023] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0024] Figure 1 A flowchart illustrating the training method of the face attribute editing model according to an embodiment of the present invention is shown. Figure 1 As shown, the training method includes: Step S101: Obtain the original image and target attributes.

[0025] The original image can be any face image from the training dataset. In this embodiment, the face images in the training dataset can be pre-annotated, for example, with labels indicating gender, age, expression, skin color, etc. The labeled gender, age, expression, skin color, etc., are the original attributes of the face image. Face images in the training dataset have the same original attributes, such as the same gender and belonging to the same age group. The target attribute is the desired attribute after editing one or more original attributes of the original image; for example, the target attribute could be (20-29 years old, male).

[0026] Step S102: Use the original image and the target attribute as input to the initial face attribute editing model to obtain the first image.

[0027] Step S103: Use the original image and its original attributes as input to the initial face attribute editing model to obtain the second image.

[0028] Step S104: Use the original attributes of the first image and the original image as input to the initial face attribute editing model to obtain the third image.

[0029] Step S105: Based on the differences between the first image, the second image, the third image and the original image, determine the network parameters of the initial face attribute editing model to obtain the target face attribute editing model.

[0030] For steps S102-S104, the initial face attribute editing model can be a pre-built model. By iteratively optimizing the network parameters of the initial face attribute editing model, the final face attribute editing model can be obtained.

[0031] In an optional embodiment, the initial face attribute editing model can employ an encoder-decoder structure. The encoder encodes the data input to the initial face attribute editing model to obtain the features of the input data, and the decoder decodes the features of the input data to obtain the output image.

[0032] As a specific embodiment, the encoder may include a feature extractor, an identity encoder, a condition generator, and a feature mapping network.

[0033] The feature extractor can be used to obtain the first intermediate feature vector of the image input to the initial face attribute editing model. The feature extractor can be a convolutional neural network.

[0034] An identity encoder can be used to extract features from the first intermediate feature vector to obtain the identity feature vector of the image input to the initial face attribute editing model.

[0035] A condition generator is used to transform the attribute information (such as target or original attributes) of the initial face attribute editing model into attribute feature vectors. The condition generator can be a multilayer perceptron (MLP). A multilayer perceptron (MLP) is a type of artificial neural network with a directed structure that maps a set of input vectors to a set of output vectors.

[0036] The feature mapping network can be used to extract features from the first intermediate feature vector again, obtaining a second intermediate feature vector from the image input to the initial face attribute editing model. Then, based on the second intermediate feature vector and the attribute feature vector, the target feature is obtained. In an optional embodiment, the condition generator converts the attribute information of the input initial face attribute editing model into a one-dimensional attribute feature vector. The feature mapping network first extracts features from the first intermediate feature vector to obtain a one-dimensional second intermediate feature vector. Then, the second intermediate feature vector and the attribute feature vector are concatenated. The concatenated long vector is then passed through a multilayer perceptron to obtain the target feature vector.

[0037] The decoder can decode the identity feature vector and the target feature vector to obtain the output image.

[0038] In step S102, the original image and target attributes are input into the original face attribute editing model to obtain a first image. The first image is an image that matches the target attributes while preserving identity features. In step S103, the original image and its original attributes are input into the original face attribute editing model to reconstruct the original image, obtaining a second image. The second image is a reconstructed image; if the input attribute information is the same as the original attribute information, the generated image should also be the same as the original image. In step S104, the first image and the original attributes of the original image are input into the original face attribute editing model to obtain a third image. The third image is a cyclically reconstructed image, using the first image generated with the target attributes as input and the original attributes of the original image as conditions to generate an image identical to the original image.

[0039] For step S105, after obtaining the first image, the second image, and the third image, the network parameters of the initial face attribute editing model can be iteratively optimized based on the differences between the first image, the second image, the third image, and the original image to obtain the final target face attribute editing model.

[0040] To determine the differences between the first, second, and third images and the original image, the following loss functions can be constructed: an identity loss function, a reconstruction loss function, and a cycle consistency loss function. Then, based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function, the network parameters of the initial face attribute editing model are determined. The identity loss function is used to determine the difference between the first image and the original image, the reconstruction loss function is used to determine the difference between the second image and the original image, and the cycle consistency loss function is used to determine the difference between the third image and the original image. The identity loss function is used to ensure that the generated first image is consistent with the original image in all attributes except for the original attributes, avoiding attribute entanglement. When using the identity loss function to determine the difference between the first image and the original image, an identity encoder is needed to obtain the identity features of the first image and the original image respectively. Then, the identity loss function is used to determine the difference between the identity features of the first image and the identity features of the original image. The difference between the identity features of the first image and the identity features of the original image is the difference between the first image and the original image.

[0041] As a concrete example, the identity loss function is shown in the following equation:

[0042] The reconstruction loss function is shown in the following equation:

[0043] The cycle consistency loss function is shown in the following equation:

[0044] in, Represents the first image. Indicates the identity features of the first image. Indicates the identity features of the original image. This represents the second image. This represents the third image.

[0045] The training method for the face attribute editing model in this invention involves obtaining a first image by using the original image and target attributes as inputs to the initial face attribute editing model; obtaining a second image by using the original image and its original attributes as inputs to the initial face attribute editing model; obtaining a third image by using the first image and its original attributes as inputs to the initial face attribute editing model; and then, based on the differences between the first image, the second image, the third image, and the original image, iteratively training the network parameters of the initial face attribute editing model to obtain the target face attribute editing model. This method reduces the dependence on training samples and solves related problems. This technology addresses the technical problem of needing a large number of image pairs as training samples when training models, reduces costs, and mitigates the impact of data accuracy on model accuracy.

[0046] In an optional embodiment, the initial face attribute editing model can be used as the generator network of a generative adversarial network (GAN) and trained through generative adversarial learning. The GAN comprises a generator network and a discriminator network; in this embodiment, the initial face... The attribute editing model serves as the generator network within a generative adversarial network (GAN). During training, a GAN fixes one network (either the generator or the discriminator) while updating the parameters of the other, iterating alternately. Therefore, the loss function in this embodiment also includes an adversarial loss function. As a specific example, the adversarial loss function is shown below:

[0047] After determining the identity loss function, reconstruction loss function, cycle consistency loss function, and adversarial loss function, the final loss function in this embodiment can be obtained based on these functions to iteratively optimize the network parameters of the initial face attribute editing model. As a specific example, the final loss function determined based on the identity loss function, reconstruction loss function, cycle consistency loss function, and adversarial loss function is shown in the following equation:

[0048] The training method for the facial attribute editing model in this invention trains the initial facial attribute editing model through adversarial learning. It only requires the original image and the desired target attribute as input to the initial facial attribute editing model, avoiding the problem of using a large number of paired images as training samples in related technologies. This reduces the need for dataset organization, lowers costs, and reduces the impact of training samples on model accuracy. The identity loss function, reconstruction loss function, and cycle consistency loss function ensure the orientation of the generated image. Modifying specific attributes of the original image does not affect other attributes, and there is no attribute entanglement. This ensures that the generated image and the original image are of the same person.

[0049] To facilitate understanding of the training method of the facial attribute editing model in this embodiment of the invention, a female face image aged 3-6 years is used as the original image, with the original attribute being (3-6 years old, female), and (20-29 years old, male) as the target attribute for illustration. Figure 2 As shown, the initial face attribute editing model in this embodiment includes a feature extractor, an identity encoder, a condition generator, a feature mapping network, and a decoder. The condition generator and the feature mapping network are... Figure 2 The Feacture Conversion section.

[0050] First, facial images of females aged 3-6 years and target attributes are input into the original facial attribute editing model. The feature extractor extracts features from the input facial images of females aged 3-6 years, obtaining a first intermediate feature vector. This first intermediate feature vector is copied and input into both the identity encoder and the feature mapping network. The identity encoder extracts features from the first intermediate feature vector, obtaining identity feature vectors that are invariant to identity, facial angle, and expression. The conditional generator converts the target attributes into one-dimensional conditional feature vectors. The feature mapping network first converts the first intermediate feature vector into a second intermediate feature vector, concatenates the second intermediate feature vector with the conditional feature vector to obtain a long vector, and then inputs this long vector into a multilayer perceptron to obtain the target feature vector. The decoder decodes the target feature vector and the identity feature vector to obtain the first image. Next, facial images of females aged 3-6 years and their original attributes are input into the original facial attribute editing model to obtain the second image. The first image and the original attributes are then input into the original facial attribute editing model to obtain the third image. Then, using the final loss function determined based on the identity loss function, reconstruction loss function, cycle consistency loss function, and adversarial loss function, the network parameters of the initial face attribute editing model are iteratively optimized to obtain the final target face attribute editing model. As an example, the effect of using this target face attribute editing model to simultaneously edit gender and age attributes is shown below. Figure 3 As shown.

[0051] Figure 4 A flowchart of a face attribute editing method according to an embodiment of the present invention is shown, as follows: Figure 4 As shown, the method includes: Step S401: Obtain the face image to be edited.

[0052] In this embodiment, the face image to be edited can be a face image that has undergone some preprocessing, such as a face image with background elements removed.

[0053] Step S402: Determine the editable attributes of the face image to be edited and the target values ​​corresponding to the editable attributes.

[0054] In an optional embodiment, the user can select the editable attributes and target values ​​of the face image to be edited via a user interface. For example, the editable attributes may include age, gender, and facial expression. The target value for the editable attribute is the expected value of the editable attribute, such as an age of 50-55 years old. After detecting the user's selection, the user-selected attribute and its value are used as the editable attributes and target values ​​of the face image to be edited.

[0055] Step S403: Using the face image to be edited and the target value corresponding to the attribute to be edited as input to the target face attribute editing model, obtain the output image of the target face attribute editing model. The output image is the desired image after editing the attribute to be edited in the face image to be edited. The target face attribute editing model is based on... Figure 1 The model trained using the method shown is shown.

[0056] In optional embodiments, such as Figure 5 As shown, the image of the face to be edited can be obtained according to the following process: Step S501: Obtain the image to be edited. The image to be edited can be an image that includes other elements of the face to be edited.

[0057] Step S502: Using a face detection model, determine the faces in the image to be edited to obtain a first face image; Step S503: Perform face alignment on the face image to be edited to obtain a second face image; Step S504: Remove the background from the second face image to obtain the face image to be edited.

[0058] The embodiments of the present invention can automatically edit facial attributes of any image including a face without requiring manual image preprocessing, thus expanding the scope of application.

[0059] Figure 6 A schematic diagram of a training device for a facial attribute editing model according to an embodiment of the present invention is shown. Figure 6 As shown, the training device 600 for the face attribute editing model includes: The first acquisition module 601 is used to acquire the original image and target attributes; The generation module 602 is used to obtain a first image by taking the original image and the target attributes as input to an initial face attribute editing model; to obtain a second image by taking the original image and the original attributes of the original image as input to the initial face attribute editing model; and to obtain a third image by taking the first image and the original attributes of the original image as input to the initial face attribute editing model. The iteration module 603 is used to determine the network parameters of the initial face attribute editing model based on the differences between the first image, the second image, the third image and the original image, and to obtain the target face attribute editing model.

[0060] Optionally, the iterative module is used to: determine an identity loss function, a reconstruction loss function, and a cycle consistency loss function; the identity loss function is used to determine the difference between the first image and the original image, the reconstruction loss function is used to determine the difference between the second image and the original image, and the cycle consistency loss function is used to determine the difference between the third image and the original image; and based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function, determine the network parameters of the initial face attribute editing model.

[0061] Optionally, the iterative module is used to: determine the network parameters of the initial face attribute editing model by using adversarial learning based on the identity loss function, the reconstruction loss function, the cycle consistency loss function, and the adversarial loss function.

[0062] Figure 7 A schematic diagram of a face attribute editing device according to an embodiment of the present invention is shown. Figure 7 As shown, the face attribute editing device 700 includes: The second acquisition module 701 is used to acquire the face image to be edited; The determining module 702 is used to determine the editable attributes of the face image to be edited and the target values ​​corresponding to the editable attributes; The editing module 703 is used to take the face image to be edited and the target value corresponding to the attribute to be edited as input to the target face attribute editing model, and obtain the output image of the target face attribute editing model. The output image is the desired image after the attribute to be edited of the face image to be edited is edited.

[0063] Optionally, the second acquisition module is further configured to: acquire an image to be edited; use a face detection model to determine the face in the image to be edited to obtain a first face image; perform face alignment on the image to be edited to obtain a second face image; and remove the background from the second face image to obtain the image to be edited.

[0064] The above-described apparatus can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0065] This invention also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804. The processor 801, communication interface 802, and memory 803 communicate with each other via the communication bus 804. Memory 803 is used to store computer programs; When the processor 801 executes the program stored in the memory 803, it performs the following steps: acquiring an original image and target attributes; using the original image and target attributes as input to an initial face attribute editing model to obtain a first image; using the original image and its original attributes as input to the initial face attribute editing model to obtain a second image; using the first image and its original attributes as input to the initial face attribute editing model to obtain a third image; and based on the differences between the first image, the second image, the third image, and the original image, determining the network parameters of the initial face attribute editing model to obtain a target face attribute editing model.

[0066] The communication bus 804 mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 804 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0067] Communication interface 802 is used for communication between the aforementioned terminal and other devices.

[0068] The memory 803 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor 801.

[0069] The processor 801 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0070] In another embodiment of the present invention, a computer-readable medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the training method or the face attribute editing method of any of the above embodiments.

[0071] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the training method or the face attribute editing method of any of the above embodiments.

[0072] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0073] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0074] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A training method for a facial attribute editing model, characterized in that, include: Obtain the original image and target attributes; The original image and the target attributes are used as input to the initial face attribute editing model to obtain the first image; The first image is an image that matches the target attribute in terms of attributes while preserving the identity features; The original image and its original attributes are used as input to an initial face attribute editing model to obtain a second image; wherein, the second image is the image that should be identical to the original image when the input attribute information is the same as the original attributes; The original attributes of the first image and the original image are used as input to an initial face attribute editing model to obtain a third image; wherein, the third image is generated using the first image as input and the original attributes as conditions, and is consistent with the original image. Based on the differences between the first image, the second image, and the third image and the original image, the network parameters of the initial face attribute editing model are determined, and the target face attribute editing model is obtained. The step of determining the network parameters of the initial face attribute editing model based on the differences between the first image, the second image, the third image, and the original image includes: An identity loss function, a reconstruction loss function, and a cycle consistency loss function are determined; the identity loss function is used to determine the difference between the first image and the original image, the reconstruction loss function is used to determine the difference between the second image and the original image, and the cycle consistency loss function is used to determine the difference between the third image and the original image. Based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function, the network parameters of the initial face attribute editing model are determined; The original images are from a pre-constructed training dataset, and multiple face images in the training dataset have the same original attributes. The initial face attribute editing model adopts an encoder-decoder structure; The encoder includes a feature extractor, an identity encoder, a condition generator, and a feature mapping network; The feature extractor is used to obtain the first intermediate feature vector of the image input to the initial face attribute editing model; The identity encoder is used to obtain the identity feature vector of the face in the image input to the initial face attribute editing model based on the first intermediate feature vector; The condition generator is used to convert the target attribute or the original attribute into an attribute feature vector; The feature mapping network is used to obtain a second intermediate feature vector of the image input to the initial face attribute editing model based on the first intermediate feature vector, and to obtain a target feature vector based on the second intermediate feature vector and the attribute feature vector; The decoder is used to decode the identity feature vector and the target feature vector.

2. The method according to claim 1, characterized in that, The process of determining the network parameters of the initial face attribute editing model based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function includes: Based on the identity loss function, the reconstruction loss function, the cycle consistency loss function, and the adversarial loss function, the network parameters of the initial face attribute editing model are determined using adversarial learning.

3. The method according to claim 2, characterized in that, Determining the difference between the first image and the original image using the identity loss function includes: The identity features of the first image are obtained using the identity encoder, and the identity features of the original image are also obtained using the identity encoder. Based on the identity loss function, the difference between the identity features of the first image and the identity features of the original image is determined, thereby determining the difference between the first image and the original image.

4. A method for editing facial attributes, characterized in that, include: Obtain the face image to be edited; Determine the editable attributes of the face image to be edited and the target values ​​corresponding to the editable attributes; The image of the face to be edited and the target value corresponding to the attribute to be edited are used as inputs to the target face attribute editing model to obtain the output image of the target face attribute editing model. The output image is the desired image after the attribute to be edited of the image of the face to be edited is edited. The target face attribute editing model is the target face attribute editing model according to any one of claims 1 to 3.

5. The method according to claim 4, characterized in that, The process of obtaining the face image to be edited includes: Get the image to be edited; Using a face detection model, the faces in the image to be edited are identified to obtain a first face image; The face image to be edited is aligned to obtain a second face image; Remove the background from the second face image to obtain the face image to be edited.

6. A training device for a facial attribute editing model, characterized in that, include: The first acquisition module is used to acquire the original image and target attributes; The generation module is used to take the original image and the target attributes as input to the initial face attribute editing model to obtain the first image; The first image is an image that matches the target attribute in terms of attributes while preserving the identity features; the original image and the original attributes of the original image are used as input to the initial face attribute editing model to obtain the second image; The second image is the image that should be the same as the original image when the input attribute information is the same as the original attribute; the original attributes of the first image and the original image are used as input to the initial face attribute editing model to obtain the third image; The third image is generated by using the first image as input and the original attributes as conditions, resulting in an image consistent with the original image. An iterative module is used to determine the network parameters of the initial face attribute editing model based on the differences between the first image, the second image, the third image and the original image, and to obtain the target face attribute editing model; The iterative module is used to: determine an identity loss function, a reconstruction loss function, and a cycle consistency loss function; the identity loss function is used to determine the difference between the first image and the original image, the reconstruction loss function is used to determine the difference between the second image and the original image, and the cycle consistency loss function is used to determine the difference between the third image and the original image; and based on the identity loss function, the reconstruction loss function, and the cycle consistency loss function, determine the network parameters of the initial face attribute editing model. The original images are from a pre-constructed training dataset, and multiple face images in the training dataset have the same original attributes. The initial face attribute editing model adopts an encoder-decoder structure; The encoder includes a feature extractor, an identity encoder, a condition generator, and a feature mapping network; The feature extractor is used to obtain the first intermediate feature vector of the image input to the initial face attribute editing model; The identity encoder is used to obtain the identity feature vector of the face in the image input to the initial face attribute editing model based on the first intermediate feature vector; The condition generator is used to convert the target attribute or the original attribute into an attribute feature vector; The feature mapping network is used to obtain a second intermediate feature vector of the image input to the initial face attribute editing model based on the first intermediate feature vector, and to obtain a target feature vector based on the second intermediate feature vector and the attribute feature vector; The decoder is used to decode the identity feature vector and the target feature vector.

7. A facial attribute editing device, characterized in that, include: The second acquisition module is used to acquire the face image to be edited; The determining module is used to determine the editable attributes of the face image to be edited and the target values ​​corresponding to the editable attributes; The editing module is used to take the face image to be edited and the target value corresponding to the attribute to be edited as input to the target face attribute editing model, and obtain the output image of the target face attribute editing model. The output image is the desired image after the attribute to be edited of the face image to be edited is edited. The target face attribute editing model is the target face attribute editing model according to any one of claims 1 to 3.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-3 or 4-5.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-3 or 4-5.

Images