A face synthesis model training method and device, a storage medium and an electronic device
By adjusting facial images and training a face synthesis model with the goal of minimizing differences, the problem of synthesizing realistic faces was solved, and the model's ability to identify deepfakes was improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
- Filing Date
- 2022-12-26
- Publication Date
- 2026-06-02
AI Technical Summary
How to train a model to synthesize more realistic faces and improve the detection effectiveness of deepfake models.
By acquiring facial images, determining facial regions, adjusting them, and then using them as target images to synthesize with the original images, the results are input into the model to be trained. The training objective is to minimize the difference between the optimized image and the original image.
It improves the difference between the foreground and background of the synthesized face image, thus enhancing the model's recognition performance.
Smart Images
Figure CN116245773B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computers, and in particular to a method, apparatus, storage medium, and electronic device for training a face synthesis model. Background Technology
[0002] With the continuous development of internet technology and machine learning technology, deepfake technology is also constantly evolving.
[0003] One of the more well-known techniques in deepfakes is face synthesis, which combines the faces of different people into a new face. For example, by using part of one person's face as the foreground and another person's face as the background, the foreground is overlaid on the background to synthesize a new face. In addition to being used in film, entertainment and other fields, this technique has also been used by many criminals for illegal activities, causing great harm to social security. Therefore, the identification of deepfakes is very necessary. The training samples for deepfake identification models are faces synthesized using face synthesis technology. The more realistic the face synthesized using face synthesis technology, the better the identification effect of the deepfake identification model trained on it will be.
[0004] Therefore, how to train a model to synthesize more realistic faces is an urgent problem to be solved. Summary of the Invention
[0005] This specification provides a face model training method, apparatus, storage medium, and electronic device to at least partially solve the aforementioned problems existing in the prior art.
[0006] The following technical solution is adopted in this specification:
[0007] This specification provides a method for training a face synthesis model, the method comprising:
[0008] Acquire facial images;
[0009] The region containing the face in the face image is determined as the foreground region;
[0010] The face image is adjusted, and the image in the foreground region of the adjusted face image is used as the target image;
[0011] The target image is combined with the face image before adjustment to synthesize a face, and the resulting synthesized image is used as a training sample.
[0012] The training samples and the foreground region are input into the face synthesis model to be trained to obtain the optimized image output by the face synthesis model to be trained.
[0013] The face synthesis model to be trained is trained with the goal of minimizing the difference between the optimized image and the original face image.
[0014] Optionally, the facial image may be adjusted, specifically including:
[0015] Perform frequency domain transformation and / or spatial domain transformation on the face image.
[0016] Optionally, the frequency domain transformation includes at least one of smoothing transformation and fuzzy transformation;
[0017] The spatial transformation includes at least one of brightness transformation, contrast transformation, and saturation transformation.
[0018] Optionally, the target image and the unadjusted face image are combined to form a face image, specifically including:
[0019] Replace the image in the foreground region of the original face image with the target image.
[0020] Optionally, the face synthesis model to be trained includes at least a feature extraction layer, a feature fusion layer, and an output layer;
[0021] The training samples and the foreground region are input into the face synthesis model to be trained to obtain the optimized image output by the face synthesis model to be trained, specifically including:
[0022] The training samples and the foreground region are input into the face synthesis model to be trained, and the features of the training samples in the foreground region and the features in the background region are extracted by the feature extraction layer of the face synthesis model to be trained.
[0023] The features of the training sample in the foreground region and the features in the background region are input into the feature fusion layer of the face synthesis model to be trained, so as to obtain the fused features of the training sample in the foreground region and the features in the non-foreground region output by the feature fusion layer;
[0024] The fused features are input into the output layer of the face synthesis model to be trained to obtain an optimized image output by the output layer based on the fused features.
[0025] Optionally, the method further includes:
[0026] Obtain the first face image and the second face image;
[0027] The region where the face is located in the second face image is determined as the foreground region of the second face image;
[0028] The image in the foreground region of the second face image is determined as the target image;
[0029] The target image and the first face image are combined to form a face image, and the resulting composite image is used as the image to be processed.
[0030] The foreground regions of the image to be processed and the second face image are input into a trained face synthesis model to obtain the processed image output by the trained face synthesis model.
[0031] This specification provides an apparatus for training a face synthesis model, the apparatus comprising:
[0032] The module acquires facial images;
[0033] The determination module is used to determine the region where the face is located in the face image, as the foreground region;
[0034] The adjustment module is used to adjust the face image and use the image in the foreground region of the adjusted face image as the target image.
[0035] The first synthesis module is used to synthesize the target image and the face image before adjustment, and use the synthesized image as a training sample.
[0036] The second synthesis module is used to input the training samples and the foreground region into the face synthesis model to be trained, and obtain the optimized image output by the face synthesis model to be trained.
[0037] The training module is used to train the face synthesis model to be trained with the goal of minimizing the difference between the optimized image and the face image before adjustment.
[0038] Optionally, the adjustment module is specifically used to perform frequency domain transformation and / or spatial domain transformation on the face image.
[0039] Optionally, the frequency domain transformation includes at least one of smoothing transformation and fuzzy transformation;
[0040] The spatial transformation includes at least one of brightness transformation, contrast transformation, and saturation transformation.
[0041] Optionally, the first synthesis module is specifically used to replace the image in the foreground region of the face image before adjustment with the target image.
[0042] Optionally, the face synthesis model to be trained includes at least a feature extraction layer, a feature fusion layer, and an output layer;
[0043] The second synthesis module is specifically used to: input the training sample and the foreground region into the face synthesis model to be trained; extract the features of the training sample in the foreground region and the features in the background region through the feature extraction layer of the face synthesis model to be trained; input the features of the training sample in the foreground region and the features in the background region into the feature fusion layer of the face synthesis model to be trained, and obtain the fused features of the training sample in the foreground region and the features in the non-foreground region output by the feature fusion layer; input the fused features into the output layer of the face synthesis model to be trained, and obtain the optimized image output by the output layer based on the fused features.
[0044] Optionally, the device further includes:
[0045] The application module is used to acquire a first face image and a second face image; determine the region where the face is located in the second face image as the foreground region of the second face image; determine the image in the foreground region of the second face image as the target image; synthesize the target image and the first face image, and use the resulting synthesized image as the image to be processed; input the image to be processed and the foreground region of the second face image into a trained face synthesis model to obtain the processed image output by the trained face synthesis model.
[0046] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0047] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.
[0048] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:
[0049] In the face synthesis model training method provided in this application, a face image is acquired and adjusted. The face region of the adjusted face image is used as the target image. The face synthesis image obtained by combining the target image and the face image before adjustment is used as the training sample and input into the face synthesis model to be trained. The training objective is to minimize the difference between the optimized image output by the model and the face image before adjustment.
[0050] As can be seen from the above method, this method first adjusts and synthesizes the face image to obtain training samples, and then trains the face synthesis model to be trained with the goal of minimizing the difference between the output image of the model and the face image before adjustment. The face synthesis model trained by this method can make the difference between the foreground and background of the face synthesis image smaller. Attached Figure Description
[0051] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:
[0052] Figure 1 This is a flowchart illustrating a face synthesis model training method provided in this specification.
[0053] Figure 2A This is a schematic diagram of a human face image A provided in this specification;
[0054] Figure 2B This is a schematic diagram of a human face image B provided in this specification;
[0055] Figure 2C This is a schematic diagram of a composite image of faces A and B provided in this specification;
[0056] Figure 3 This is a schematic diagram of a synthesized target image and a face image before adjustment, as provided in this specification.
[0057] Figure 4 This is a schematic diagram of a face synthesis model training device provided in this specification;
[0058] Figure 5 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation
[0059] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.
[0060] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.
[0061] Figure 1This document provides a flowchart illustrating a face synthesis model training method, which includes the following steps:
[0062] S100: Acquire face image.
[0063] The face synthesis model training method provided in this manual can be executed by any electronic device with computing capabilities, such as a server or terminal. This method is used to train a face synthesis model, and the trained face synthesis model can synthesize face images of different people into a better face image.
[0064] The goal is to synthesize facial images from different individuals into a better-looking composite image, minimizing the differences between the images from different people. Typically, in a composite image, the foreground region is derived from one person's face, while the background region is from another person's face. Figure 2A As shown, the shaded area at the bottom of the image of A's face is A's face (i.e., Figure 2A The foreground area shown is the main image of B, and the other areas are the background areas. The shaded area at the bottom of B's face image is B's face (i.e., the foreground area shown). Figure 2B (As shown in the foreground area), the composite image of faces A and B is created by replacing the lower shadow area of the lower shadow area in the face image B with the lower shadow area in the face image of face A, as shown in the foreground area. Figure 2C As shown, the foreground region of the face synthesis images A and B is the shadow part of the foreground region of A, and the background region is the image of B's face image excluding the foreground region. The foreground and background of such face synthesis images usually differ. To facilitate training, this method uses the same face image to simulate such face synthesis images with differences between the foreground and background.
[0065] Specifically, this method first acquires a face image, which is an image with a face as the main feature. Whether the image has a face as the main feature can be detected by face detection models or algorithms such as Multi-task convolutional neural network (MTCNN).
[0066] S102: Determine the region where the face is located in the face image as the foreground region.
[0067] To determine the region where a face is located in a face image, specifically, the region where the face is located can be determined by the feature points of the face.
[0068] S104: Adjust the face image and use the image in the foreground region of the adjusted face image as the target image.
[0069] As mentioned above, this method simulates a face synthesis image where the foreground and background differ, requiring adjustments to the face synthesis image so that the adjusted face image is different from the original face image.
[0070] The main difference between different faces may be the difference between the frequency domain and the spatial domain of the image.
[0071] Specifically, adjusting a face image involves performing frequency domain transformation and / or spatial domain transformation on the face image. Frequency domain transformation includes at least one of smoothing transformation and blurring transformation, while spatial domain transformation includes at least one of brightness transformation, contrast transformation, and saturation transformation.
[0072] For example, filtering can be applied to facial images, such as mean filtering, Gaussian filtering, or adjusting the brightness, contrast, and saturation of the facial image.
[0073] The adjusted face image differs from the original face image. The image in the foreground region of the adjusted face image is used as the target image for the following step S106.
[0074] S106: Perform face synthesis between the target image and the face image before adjustment, and use the resulting synthesized image as a training sample.
[0075] The target image is combined with the original face image to create a face composite image. Specifically, for example... Figure 3 As shown, the face image of C is the face image before adjustment, and the face image of C1 is the face image after adjustment. The target image of C1 is the lower shadow area of the face image of C1, which is the foreground area image. By combining the target image of C1 with the face image before adjustment, the image in the foreground area of the face image before adjustment can be replaced by the target image. That is, the image in the foreground area of C1 (i.e., the image in the lower shadow area of the face image of C) is replaced by the image in the foreground area of C (i.e., the image in the lower shadow area of the face image of C).
[0076] As described in step S104, the target image is the image in the foreground region of the adjusted face image. Therefore, there is a difference between the target image and the face image before adjustment. The composite image obtained by combining the target image and the face image before adjustment is the training sample, which is a composite face image with differences between the foreground and background.
[0077] S108: Input the training samples and the foreground region into the face synthesis model to be trained to obtain the optimized image output by the face synthesis model to be trained.
[0078] Optionally, the face synthesis model to be trained includes at least a feature extraction layer, a feature fusion layer, and an output layer.
[0079] The training samples and the foreground region are input into the face synthesis model to be trained. The feature extraction layer of the face synthesis model to be trained can extract the features of the training samples in the foreground region and the background region. Then, the features of the training samples in the foreground region and the background region are input into the feature fusion layer of the face synthesis model to be trained. The fused features of the training samples in the foreground region and the non-foreground region are obtained from the output of the feature fusion layer. Finally, the fused features are input into the output layer of the face synthesis model to be trained to obtain the optimized image output by the output layer based on the fused features.
[0080] S110: The face synthesis model to be trained is trained with the goal of minimizing the difference between the optimized image and the face image before adjustment.
[0081] Since the purpose of training a face synthesis model is to minimize the difference between the foreground and background of the synthesized image output by the model, and the foreground of the training sample is the face region of the adjusted face image, while the background is the face image before adjustment, and there is no difference between the foreground and background of the face image before adjustment, if we want the difference between the foreground and background of the synthesized image output by the model to be minimized, we can train the face synthesis model with the goal of minimizing the difference between the optimized image and the face image before adjustment.
[0082] As can be seen from the above method, this method uses a person's face image to simulate a composite image of two different people's faces. By adjusting the face image and then fusing the adjusted face image with the original face image, it simulates a composite image of two different people's faces. In fact, it simulates the differences between the images from different faces in the composite image of two different people's faces, that is, it simulates the differences between the foreground and background of the composite image of two different people's faces. The simulated composite image is input into the face synthesis model to be trained. In order to minimize the difference between the foreground and background of the simulated composite image, the training objective is to minimize the difference between the output image of the model and the original face image. The face synthesis model trained using this method can make the difference between the foreground and background of the composite image smaller.
[0083] Optionally, the number of face images acquired according to step S100 is not less than one. For ease of training, after acquiring multiple face images, as described in steps S102 to S106, a target image can be determined for each acquired face image, and the target image can be combined with the face image before adjustment to synthesize a face. The resulting synthesized image is then used as a training sample. Thus, after determining multiple training samples, the face synthesis model to be trained can be trained.
[0084] Specifically, after acquiring multiple face images, for each face image, the region where the face is located in the face image is determined as the foreground region of the face image. The face image is then adjusted, and the image in the foreground region of the adjusted face image is used as the target image of the face image. The target image of the face image is then combined with the face image before adjustment to synthesize a face image, and the resulting synthesized image is used as a training sample.
[0085] Optionally, after training the face synthesis model according to the steps described in S100-110 above, the trained face synthesis model can be applied as follows: first, obtain a first face image and a second face image; then, determine the region where the face is located in the second face image as the foreground region of the second face image, and determine the image in the foreground region of the second face image as the target image; synthesize the target image and the first face image; use the resulting synthesized image as the image to be processed; finally, input the image to be processed and the foreground region of the second face image into the trained face synthesis model to obtain the processed image output by the trained face synthesis model, thus obtaining the synthesized image of the first face image and the second face image.
[0086] The above describes one or more embodiments of the face synthesis model training method provided in this specification. Based on the same idea, this specification also provides a corresponding face synthesis model training device, as shown in Figure 2.
[0087] Figure 4 This specification provides a schematic diagram of a face synthesis model training device, which specifically includes:
[0088] Module 401 acquires the face image;
[0089] The determining module 402 is used to determine the region where the face is located in the face image as the foreground region;
[0090] The adjustment module 403 is used to adjust the face image and use the image in the foreground region of the adjusted face image as the target image.
[0091] The first synthesis module 404 is used to synthesize the target image and the face image before adjustment, and use the resulting synthesized image as a training sample.
[0092] The second synthesis module 405 is used to input the training samples and the foreground region into the face synthesis model to be trained, and obtain the optimized image output by the face synthesis model to be trained.
[0093] Training module 406 is used to train the face synthesis model to be trained with the goal of minimizing the difference between the optimized image and the face image before adjustment.
[0094] Optionally, the adjustment module 403 is specifically used to perform frequency domain transformation and / or spatial domain transformation on the face image.
[0095] Optionally, the frequency domain transformation includes at least one of smoothing transformation and fuzzy transformation;
[0096] The spatial transformation includes at least one of brightness transformation, contrast transformation, and saturation transformation.
[0097] Optionally, the first synthesis module 404 is specifically used to replace the image in the foreground region of the face image before adjustment with the target image.
[0098] Optionally, the face synthesis model to be trained includes at least a feature extraction layer, a feature fusion layer, and an output layer;
[0099] The second synthesis module 405 is specifically used to: input the training sample and the foreground region into the face synthesis model to be trained; extract the features of the training sample in the foreground region and the features in the background region through the feature extraction layer of the face synthesis model to be trained; input the features of the training sample in the foreground region and the features in the background region into the feature fusion layer of the face synthesis model to be trained, and obtain the fused features of the training sample in the foreground region and the features in the non-foreground region output by the feature fusion layer; input the fused features into the output layer of the face synthesis model to be trained, and obtain the optimized image output by the output layer based on the fused features.
[0100] Optionally, the device further includes:
[0101] Application module 407 is used to acquire a first face image and a second face image; determine the region where the face is located in the second face image as the foreground region of the second face image; determine the image in the foreground region of the second face image as the target image; synthesize the target image and the first face image to obtain the synthesized image as the image to be processed; input the image to be processed and the foreground region of the second face image into a trained face synthesis model to obtain the processed image output by the trained face synthesis model.
[0102] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided face synthesis model method.
[0103] This instruction manual also provides Figure 5 The diagram shows the structure of the electronic device. Figure 5At the hardware level, the autonomous driving device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for various operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The described face synthesis model method. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.
[0104] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.
[0105] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0106] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0107] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.
[0108] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0109] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0113] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0114] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0115] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0116] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0117] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0118] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0119] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.
Claims
1. A method for training a face synthesis model, the method comprising: Acquire facial images; The region containing the face in the face image is determined as the foreground region; The face image is adjusted, and the image in the foreground region of the adjusted face image is used as the target image; The target image is combined with the face image before adjustment to synthesize a face, and the resulting synthesized image is used as a training sample. The training samples and the regions where the faces are located in the unadjusted face images are input into the face synthesis model to be trained, and the optimized image output by the face synthesis model to be trained is obtained. The face synthesis model to be trained is trained with the goal of minimizing the difference between the optimized image and the original face image.
2. The method as described in claim 1, wherein adjusting the facial image specifically includes: Perform frequency domain transformation and / or spatial domain transformation on the face image.
3. The method as described in claim 2, wherein the frequency domain transformation includes at least one of smoothing transformation and fuzzy transformation; The spatial transformation includes at least one of brightness transformation, contrast transformation, and saturation transformation.
4. The method as described in claim 1, wherein the target image and the unadjusted face image are combined to form a face image, specifically comprising: Replace the image in the foreground region of the original face image with the target image.
5. The method as described in claim 1, wherein the face synthesis model to be trained includes at least a feature extraction layer, a feature fusion layer, and an output layer; The training samples and the regions containing faces in the unadjusted face images are input into the face synthesis model to be trained to obtain the optimized image output by the face synthesis model, specifically including: The training samples and the regions where the faces are located in the face images before adjustment are input into the face synthesis model to be trained. The feature extraction layer of the face synthesis model to be trained extracts the features of the training samples in the foreground region and the features in the background region. The features of the training sample in the foreground region and the features in the background region are input into the feature fusion layer of the face synthesis model to be trained, so as to obtain the fused features of the training sample in the foreground region and the features in the non-foreground region output by the feature fusion layer; The fused features are input into the output layer of the face synthesis model to be trained to obtain an optimized image output by the output layer based on the fused features.
6. The method of claim 1, further comprising: Obtain the first face image and the second face image; The region where the face is located in the second face image is determined as the foreground region of the second face image; The image in the foreground region of the second face image is determined as the target image; The target image and the first face image are combined to form a face image, and the resulting composite image is used as the image to be processed. The foreground regions of the image to be processed and the second face image are input into a trained face synthesis model to obtain the processed image output by the trained face synthesis model, which is used as the synthesized image of the first face image and the second face image.
7. An apparatus for training a face synthesis model, the apparatus comprising: The module acquires facial images; The determination module is used to determine the region where the face is located in the face image, as the foreground region; The adjustment module is used to adjust the face image and use the image in the foreground region of the adjusted face image as the target image. The first synthesis module is used to synthesize the target image and the face image before adjustment, and use the synthesized image as a training sample. The second synthesis module is used to input the training samples and the regions where the faces are located in the face images before adjustment into the face synthesis model to be trained, so as to obtain the optimized image output by the face synthesis model to be trained. The training module is used to train the face synthesis model to be trained with the goal of minimizing the difference between the optimized image and the face image before adjustment.
8. The apparatus of claim 7, wherein the adjustment module is specifically used to perform frequency domain transformation and / or spatial domain transformation on the face image.
9. The apparatus of claim 8, wherein the frequency domain transformation includes at least one of smooth transformation and fuzzy transformation; The spatial transformation includes at least one of brightness transformation, contrast transformation, and saturation transformation.
10. The apparatus of claim 7, wherein the first synthesis module is specifically configured to replace the image in the foreground region of the face image before adjustment with the target image.
11. The apparatus of claim 7, wherein the face synthesis model to be trained includes at least a feature extraction layer, a feature fusion layer, and an output layer; The second synthesis module is specifically used to: input the regions where the faces are located in the training samples and the unadjusted face images into the face synthesis model to be trained; extract the features of the training samples in the foreground region and the background region through the feature extraction layer of the face synthesis model to be trained; input the features of the training samples in the foreground region and the background region into the feature fusion layer of the face synthesis model to be trained, and obtain the fused features of the training samples in the foreground region and the non-foreground region output by the feature fusion layer; input the fused features into the output layer of the face synthesis model to be trained, and obtain the optimized image output by the output layer based on the fused features.
12. The apparatus of claim 7, further comprising: The application module is used to acquire the first face image and the second face image; The region where the face is located in the second face image is determined as the foreground region of the second face image; the image in the foreground region of the second face image is determined as the target image. The target image and the first face image are combined to form a face image, and the resulting composite image is used as the image to be processed. The foreground regions of the image to be processed and the second face image are input into a trained face synthesis model to obtain the processed image output by the trained face synthesis model.
13. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.
14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any one of claims 1 to 6.