Face shape transformation model training, face shape transformation method and related device
By generating an adversarial network, the first generation network and the second generation network are used to realize face shape transformation, which solves the problem that face shape transformation in the prior art is difficult to keep other information on the face unchanged, and realizes accurate face shape transformation and expression maintenance.
Patent Information
- Application Number
- CN202110728844.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-29
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to accurately realize face shape transformation without changing other face information, especially when it is difficult to obtain training sample pairs with obvious correspondence.
By using the method of generating an adversarial network, the target image is generated using the first generation network and the second generation network respectively, so that it is determined to belong to the corresponding face shape, and the sample image and the transformed image are determined to be the same image, thereby realizing the face shape transformation.
On the basis of ensuring that the image after the face shape is changed can be judged as the corresponding face shape, control the changes to other face information as little as possible, so as to achieve the same person and the same expression before and after the conversion.
Smart Images

Figure CN113379594B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, specifically the field of computer vision and deep learning technology, and can be applied to scenarios such as face image processing, and in particular to a face shape transformation model training and face shape transformation method, as well as corresponding devices, electronic devices, computer-readable storage media and computer program products. Background Art
[0002] Face attribute editing technology involves many different editing contents, such as removing or adding glasses, removing or adding bangs, hair color editing, facial features editing, and makeup editing. However, there is also a category of editing that involves changing the face shape, such as face shape editing, age editing when getting younger or changing from a child to an older person, and face shape changes when changing faces.
[0003] How to accurately achieve face shape transformation without changing other facial information is the difficulty of face shape editing in face attribute editing. Summary of the invention
[0004] The embodiments of the present disclosure provide a face shape transformation model training, a face shape transformation method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product.
[0005] In a first aspect, an embodiment of the present disclosure proposes a face shape transformation model training method, comprising: obtaining a first generative network and a second generative network based on a first face shape feature set and a second face shape feature set; using the first generative network to generate a first target image corresponding to a first sample image, and controlling the first target image to be judged as belonging to a second face shape; using the second generative network to generate a second target image corresponding to the second sample image, and controlling the second target image to be judged as belonging to the first face shape; using the second generative network and the first generative network to generate a first transformation image corresponding to the first target image and a second transformation image corresponding to the second target image respectively; controlling the first sample image and the first transformation image, and the second sample image and the second transformation image to be judged as the same image; outputting a generative adversarial network that meets preset requirements as a target face shape transformation model; wherein the first sample image and the second sample image belong to a first face shape and a second face shape respectively, and the generative adversarial network includes the first generative network and the second generative network.
[0006] In a second aspect, an embodiment of the present disclosure proposes a face shape transformation model training device, comprising: a generative network training unit, configured to obtain a first generative network and a second generative network based on a first face shape feature set and a second face shape feature set; a first primary transformation unit, configured to generate a first target image corresponding to a first sample image using the first generative network, and control the first target image to be judged to belong to a second face shape; a second primary transformation unit, configured to generate a second target image corresponding to a second sample image using the second generative network, and control the second target image to be judged to belong to the first face shape; a secondary transformation unit, configured to generate a first transformation image corresponding to the first target image and a second transformation image corresponding to the second target image using the second generative network and the first generative network respectively; an identical image control unit, configured to control the first sample image and the first transformation image, and the second sample image and the second transformation image to be judged to be identical images; a target face shape transformation model output unit, configured to output a generative adversarial network that meets preset requirements as a target face shape transformation model; wherein the first sample image and the second sample image belong to the first face shape and the second face shape respectively, and the generative adversarial network includes the first generative network and the second generative network.
[0007] In a third aspect, an embodiment of the present disclosure proposes a face shape transformation method, including: obtaining a face image to be transformed; calling a target face shape transformation model to transform the face image to be transformed, and the face shape transformation model is obtained according to the face shape transformation model training method described in any implementation method of the first aspect.
[0008] In a fourth aspect, an embodiment of the present disclosure proposes a face shape transformation device, comprising: a face image acquisition unit to be transformed, configured to acquire a face image to be transformed; a model call processing unit, configured to call a target face shape transformation model to transform the face image to be transformed, and the face shape transformation model is obtained according to a face shape transformation model training device as described in any implementation method in the second aspect.
[0009] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the face shape transformation model training method described in any implementation manner in the first aspect or the face shape transformation method described in any implementation manner in the third aspect.
[0010] In a sixth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement a face shape transformation model training method as described in any implementation method in the first aspect or a face shape transformation method as described in any implementation method in the third aspect when executed.
[0011] In the seventh aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the face shape transformation model training method described in any implementation manner in the first aspect or the face shape transformation method described in any implementation manner in the third aspect.
[0012] The face shape transformation model training and face shape transformation method provided by the embodiments of the present disclosure, on the basis of ensuring that the image after the first face shape transformation can be identified as the corresponding face shape, also additionally control the sample image and the image after the second face shape transformation to be identified as the same image. Therefore, when it is difficult to obtain a training sample pair with an obvious corresponding relationship, not only can the image after the face shape transformation be controlled to pass the face shape identification, but also the change of other facial information can be controlled as little as possible, so as to achieve the same person and the same expression before and after the transformation.
[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0015] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;
[0016] Figure 2 A flowchart of a face shape transformation model training method provided by an embodiment of the present disclosure;
[0017] Figure 3 A flowchart of another face shape transformation model training method provided by an embodiment of the present disclosure;
[0018] Figure 4 A schematic diagram of the structure of a stylegan2 model provided in an embodiment of the present disclosure;
[0019] Figure 5 A schematic diagram of the structure of a facial key point migration network provided in an embodiment of the present disclosure;
[0020] Figure 6 An example diagram of key point migration provided by an embodiment of the present disclosure;
[0021] Figure 7 A structural block diagram of a face shape transformation model training device provided by an embodiment of the present disclosure
[0022] Figure 8 A structural block diagram of a face shape transformation device provided by an embodiment of the present disclosure;
[0023] Fig. 9 A schematic diagram of the structure of an electronic device suitable for executing a face shape transformation model training method and / or a face shape transformation method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.
[0025] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.
[0026] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, electronic device, and computer-readable storage medium for training a face recognition model and recognizing a face of the present application can be applied.
[0027] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0028] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various applications for realizing information communication between the terminal devices 101, 102, 103 and the server 105 can be installed, such as face image editing applications, model training applications, online social applications, etc.
[0029] Terminal devices 101, 102, 103 and server 105 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.; when terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here.
[0030] The server 105 can provide various services through various built-in applications. For example, a face image editing application that can provide face shape editing services to users can achieve the following effects when the server 105 runs the face image editing application: first, the face image to be transformed is received from the terminal devices 101, 102, and 103 through the network 104; then, the target face shape transformation model is called to transform the face image to be transformed. Furthermore, the transformation result can be returned to the terminal devices 101, 102, and 103.
[0031] Among them, the face shape transformation model can be trained by the face shape transformation model training application built into the server 105 according to the following steps: first, based on the first face shape feature set and the second face shape feature set, a first generation network and a second generation network are obtained; then, the first generation network is used to generate a first target image corresponding to the first sample image, and the first target image is controlled to be judged to belong to the second face shape; at the same time, the second generation network is used to generate a second target image corresponding to the second sample image, and the second target image is controlled to be judged to belong to the first face shape; next, the second generation network and the first generation network are used to generate a first transformation image corresponding to the first target image and a second transformation image corresponding to the second target image respectively; next, the first sample image and the first transformation image, the second sample image and the second transformation image are controlled to be judged as the same image; finally, the generative adversarial network that meets the preset requirements is output as the target face shape transformation model, the first sample image and the second sample image belong to the first face shape and the second face shape respectively, and the generative adversarial network includes the first generation network and the second generation network.
[0032] Since training the face shape transformation model requires more computing resources and stronger computing power, the face shape transformation model training method provided in the subsequent embodiments of the present application is generally performed by the server 105 with stronger computing power and more computing resources. Accordingly, the face shape transformation model training device is generally also set in the server 105. However, it should also be pointed out that when the terminal devices 101, 102, and 103 also have computing power and computing resources that meet the requirements, the terminal devices 101, 102, and 103 can also complete the above-mentioned various operations handed over to the server 105 through the face shape transformation model training application installed thereon, and then output the same result as the server 105. Correspondingly, the face shape transformation model training device can also be set in the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also not include the server 105 and the network 104.
[0033] Of course, the server used to train the face shape transformation model may be different from the server that calls the trained face shape transformation model for use. In particular, the face shape transformation model trained by the server 105 can also be used to obtain a lightweight face shape transformation model suitable for being placed in the terminal devices 101, 102, and 103 through model distillation, that is, it is possible to flexibly choose to use the lightweight face shape transformation model in the terminal devices 101, 102, and 103 or the more complex face shape transformation model in the server 105 according to the recognition accuracy actually required.
[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0035] Please refer to Figure 2 , Figure 2 A flowchart of a face shape transformation model training method provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:
[0036] Step 201: obtaining a first generation network and a second generation network based on a first face shape feature set and a second face shape feature set;
[0037] This step is intended to be performed by the execution subject of the face shape transformation model training method (e.g. Figure 1The server 105 shown constructs two sample relationship pairs based on the first face feature set and the second face feature set, so that the two generation networks can learn the corresponding relationships therefrom respectively. For example, when the first face feature set is used as the input sample and the second face feature set is used as the output sample, the generation network can be trained to learn from the sample relationship pair the transformation relationship representing the transformation from the first face feature to the second face feature. If the input-output relationship is opposite, the transformation relationship representing the transformation from the long second face feature to the first face feature can be learned therefrom.
[0038] It should be noted that the first face shape feature set and the second face shape feature set are feature sets that characterize the respective face shapes extracted from the first face shape image set and the second face shape image set, respectively. The face shape features can be represented in a variety of ways, such as the face shape outer contour, facial features, face shape key points, etc. At the same time, it should be emphasized that the first face shape image set and the second face shape image set are obtained independently, and there is no dependency or causal relationship between the two, nor is one face shape image transformed based on another face shape image. For example, a preset number of first face shape images and second face shape images can be directly generated by a random face image generation model to obtain the first face shape image set and the second face shape image set. The random face image generation model can learn face shape distinguishing features based on a small number of samples pre-labeled with face shapes, and then randomly generate a large number of specified face shape images under the guidance of the face shape distinguishing features. After that, face shape features can be extracted from the first face shape image set and the second face shape image set to obtain the first face shape feature set and the second face shape feature set.
[0039] Taking the face feature set representing the face feature as an example, the above step of obtaining the face feature set can be specifically as follows: extracting face key points from the first face image set and the second face image set respectively to obtain the first face key point set and the second face key point set.
[0040] Among them, the first face shape and the second face shape generally refer to two face shapes that are considered to be different, and do not specifically refer to two face shapes. The specific face shapes referred to by the first face shape and the second face shape can be determined by oneself according to actual conditions.
[0041] It should also be noted that although the sample pairs obtained by training the first generative network and the second generative network are both the first face feature set and the second face feature set, the training processes are independent of each other. Due to the characteristics of neural networks and pre-training algorithms, the two transformations learned from two sample pairs with opposite input samples and output samples are usually not directly reversible relationships, and the transformation relationships obtained by training the same sample pair multiple times may also be different.
[0042] Step 202: using the first generation network to generate a first target image corresponding to the first sample image, and controlling the first target image to be judged as belonging to the second face shape;
[0043] Based on step 201, this step aims to convert the first sample image belonging to the first face shape into the first target image by the above-mentioned execution subject using the first generation network. Since the first generation network has learned the transformation relationship from the first face shape to the second face shape, it is only when the first target image is judged to belong to the second face shape that it can be preliminarily considered that the first generation network has an available face shape conversion function.
[0044] However, it should be noted that since it is difficult to obtain sample pairs with a clear corresponding relationship, although the transformation relationship from the first face shape to the second face shape can be learned from the independently obtained first face shape feature set and the second face shape feature set, in actual use, it is necessary not only to ensure that the face shape is transformed, but also to ensure that other facial features in the image are changed as little as possible. This cannot be achieved by only focusing on the features of the face shape. Precisely because the sample pairs used to train the generative network are unlikely to include a pair of face images in which only the face shape is transformed, other methods are needed to achieve this goal.
[0045] Step 203: using a second generation network to generate a second target image corresponding to the second sample image, and controlling the second target image to be judged to belong to the first face shape;
[0046] Based on step 201, this step aims to convert the second sample image belonging to the second face shape into a second target image by the above-mentioned execution subject using the second generation network. Since the second generation network has learned the transformation relationship from the second face shape to the first face shape, only when the second target image is judged to belong to the first face shape can it be preliminarily considered that the second generation network has a usable face shape conversion function.
[0047] Step 204: Generate a first transformed image corresponding to the first target image and a second transformed image corresponding to the second target image by using the second generation network and the first generation network respectively;
[0048] On the basis of step 202 and step 203, this step aims to convert the first target image belonging to the second face shape into a first transformation image that should belong to the first face shape by the above-mentioned execution subject using the second generation network, and at the same time convert the second target image belonging to the first face shape into a second transformation image that should belong to the second face shape by using the first generation network.
[0049] That is, the content described in this step is: based on the face shape of the original image of one face shape obtained through the transformation relationship, the face shape of the image after the first face shape transformation is judged as the corresponding face shape, and then the image after the second face shape transformation is obtained through another transformation relationship opposite to it.
[0050] Since the current first generation network and the second generation network can only ensure that the face shape of the first transformed image is finally determined to belong to the first face shape and the face shape of the second transformed image is finally determined to belong to the second face shape, but other parameters except the face shape are no longer within the determination, for example, other facial parameters may be greatly adjusted during the conversion process, so that although it can be determined to be the desired face shape after the change, there are significant changes in expression and facial details. For practical use, a model that only meets the face shape requirements is obviously not enough, and it is also necessary to control other parameters to achieve the preservation of other information of the face.
[0051] Step 205: Control the first sample image and the first transformed image, the second sample image and the second transformed image to be determined as the same image;
[0052] That is, this step controls the image after two transformations so that there will not be obvious differences in facial details due to the two transformations, and it can still be judged as belonging to the same image. In this way, facial details (such as face-related information, expression information, key point information, etc. used to identify whether they belong to the same user) can be maintained. There are many ways to achieve the control purpose of this step. For example, a discriminant network for whether the image belongs to the same person can be added to the generative adversarial network, and the discriminant network can be used to guide the adjustment of the transformation relationship in the generative network. It can also be achieved by adding a special loss function, etc.
[0053] Step 206: Output the generative adversarial network that meets the preset requirements as a target face transformation model.
[0054] On the basis of step 205, this step aims to enable the above-mentioned execution subject to output the generative adversarial network that converges or meets the training end condition as the desired target face transformation model.
[0055] Specifically, the preset requirements mainly refer to the requirements for when the model is considered to be in convergence or meets the conditions for the end of training, for example: when the value of the loss function is less than the preset value, the number of iterative training times exceeds the preset number, the difference in the value of the loss function in two adjacent iterations is less than the preset value, etc.
[0056] The face shape transformation model training method provided by the embodiment of the present disclosure, on the basis of ensuring that the image after the first face shape transformation can be identified as the corresponding face shape, further controls the sample image and the image after the second face shape transformation to be identified as the same image. Therefore, when it is difficult to obtain a training sample pair with an obvious corresponding relationship, not only can the image after the face shape transformation be controlled to pass the face shape identification, but also the change of other facial information can be controlled as little as possible, so as to achieve the same person and the same expression before and after the transformation.
[0057] Please refer to Figure 3 , Figure 3A flowchart of another face shape transformation image training method provided by an embodiment of the present disclosure, wherein process 300 includes the following steps:
[0058] Step 301-1: using the first face shape feature set and the second face shape feature set as input samples and output samples for training a generative network, respectively, to obtain a first generative network;
[0059] That is, the first generation network can learn the transformation relationship from the first facial feature to the second facial feature from the sample pair.
[0060] Step 302-1: Generate a first target image corresponding to the first sample image using a first generation network;
[0061] Based on step 301-1, this step aims to generate a first target image corresponding to the first sample image through the first generation network by the above-mentioned execution subject, that is, to try to transform the face shape of the first sample image with the help of the learned transformation relationship from the first facial feature to the second facial feature.
[0062] Step 303-1: The second discriminant network determines whether the first target image belongs to the second face shape. If it is determined to belong to the second face shape, step 304-1 is executed. Otherwise, the transformation relationship learned by the first generating network is readjusted until the output first target is determined to belong to the second face shape ( Figure 3 In the example, the method is represented by redirecting to step 301-1;
[0063] On the basis of step 302-1, this step aims to determine whether the face shape of the first target image belongs to the second face shape by the above-mentioned execution subject through the second discriminant network in the generative adversarial network.
[0064] The second discriminant network having the above-mentioned discriminant capability may be constructed in a manner of: extracting second facial features based on a second facial feature set, and constructing a second discriminant network based on the second facial features.
[0065] Step 304-1: Generate a first transformed image corresponding to the first target image using a second generation network;
[0066] On the basis of step 303 - 1 , this step aims to convert the first target image into the first transformed image by the above-mentioned execution subject through the second generation network of the generative adversarial network.
[0067] Different from the first generation network, the transformation relationship learned by the second generation network is exactly opposite to that of the first generation network, that is, the transformation relationship learned is from the second facial features to the first facial features.
[0068] Step 305-1: The third discriminant network determines whether the first sample image and the first transformed image are the same. If they are determined to be the same image, execute step 307; otherwise, execute step 306;
[0069] On the basis of step 304 - 1 , this step aims at the execution subject to determine whether the first sample image and the first transformed image are the same image through the third discriminant network.
[0070] Different from the second discriminant network used to determine whether the image belongs to the second face shape and the first discriminant network used to determine whether the image belongs to the first face shape, the third discriminant network in this step is used to determine whether two images are the same image, and the discrimination criterion can be obtained based on the preset same image discrimination rule.
[0071] A specific identification step can be:
[0072] Obtain at least one of a difference in facial key points, a difference in expression, and a difference in face identity information between the first sample image and the first transformed image;
[0073] In response to at least one of the difference in facial key points, the difference in expression, and the difference in face identity information being smaller than the corresponding preset difference, it is determined that the first sample image and the first transformed image belong to the same image.
[0074] The preset differences corresponding to differences in facial key points, the preset differences corresponding to differences in facial expressions, and the preset differences corresponding to differences in facial identity information may be the same or different, but are usually different due to different categories. This implementation method uses up to three parameters to distinguish the same image at the same time, covering multiple angles and improving the accuracy of the distinction results as much as possible.
[0075] Step 301-2: using the second face shape feature set and the first face shape feature set as input samples and output samples for training another generative network, respectively, to obtain a second generative network;
[0076] That is, the second generation network can learn the transformation relationship from the second facial features to the first facial features from the sample pair.
[0077] Step 302-2: Generate a second target image corresponding to the second sample image using a second generation network;
[0078] Step 303-2: The first discriminant network determines whether the second target image belongs to the first face shape. If it is determined to belong to the first face shape, step 304-2 is executed. Otherwise, the transformation relationship learned by the second generating network is readjusted until the output second target image is determined to belong to the first face shape ( Figure 3 In the example, the method is represented by redirecting to step 301-2;
[0079] The first discriminant network having the above-mentioned discriminant capability may be constructed in a manner of: extracting first face features based on a first face feature set, and constructing a first discriminant network based on the first face features.
[0080] Step 304-2: Generate a second transformed image corresponding to the second target image using the first generation network;
[0081] This step aims to convert the second target image into a second transformed image by the above-mentioned execution subject through the first generation network of the generative adversarial network.
[0082] Step 305-2: The third discriminant network determines whether the second sample image and the second transformed image are the same. If they are determined to be the same image, execute step 307; otherwise, execute step 306;
[0083] Step 301-2 to step 305-2 is another set similar to step 301-1 to step 305-1, in which the input-output relationship in the sample pair is opposite, the face shape of the initial image is different, and the discrimination capabilities of the generation network and the discrimination network are adaptively adjusted accordingly.
[0084] Step 306: Adjust the parameters of the current cycle loss function;
[0085] This step is based on the fact that the judgment result of step 305-1 is that the first sample image and the first transformed image are not the same image or the judgment result of step 305-2 is that the second sample image and the second transformed image are not the same image, and is intended to adjust the two generation networks by adjusting the parameters of the current cycle loss function by the above-mentioned execution subject, so that the image generated by the adjusted generation network can pass the judgment of step 305-1 and step 305-2 ( Figure 3 denoted by pointing to step 301-1 and step 301-2).
[0086] That is, this step attempts to solve the above problem by introducing a new cycle loss function. The reason why it is called a cycle loss function is that the first sample image is transformed twice by the first transformation and the second transformation, and finally returns to the first face shape to which it belongs, which is equivalent to experiencing a face shape transformation cycle. Therefore, the cycle loss function is a loss function used in the present disclosure to control the first sample image and the first transformation image to be judged as belonging to the same image, and the second sample image and the second transformation image to be belonging to the same image.
[0087] Step 307: Outputting the generative adversarial network having the cyclic loss function that meets the preset requirements as the target face shape transformation model.
[0088] This step is based on the fact that the judgment result of step 305-1 is that the first image and the fifth image are the same image or the judgment result of step 305-2 is that the third image and the sixth image are the same image, and aims to output the model with the cyclic loss function that meets the preset requirements as the target face transformation model by the above-mentioned execution subject.
[0089] The preset requirements for setting the cyclic loss function may include multiple requirements, such as: when the value is less than a preset value, the number of iterative training exceeds a preset number, the difference between the loss function values of two adjacent iterations is less than a preset value, etc. For example, a generative adversarial network with a cyclic loss function whose actual value is less than a preset value is output as a target face shape transformation model.
[0090] That is, the generative adversarial network in this embodiment includes a first generative network, a second generative network, a first discriminative network, a second discriminative network and a third discriminative network.
[0091] This embodiment provides a specific implementation method for obtaining the first generation network and the second generation network through steps 301-1 and 301-2, so that they can learn different transformation relationships respectively; through steps 303 and 305, the discrimination task is specifically implemented through the discriminant network in the generative adversarial network; and through steps 306 and 307, a cyclic loss function is specifically selected to control the face image to maintain other image content unchanged before and after the face shape conversion.
[0092] It should be understood that there is no causal and dependent relationship between the above-mentioned parts in this embodiment, and the specific implementation methods of each part can be combined with the embodiment of process 200 to form different independent embodiments. This embodiment only exists as a limited embodiment that simultaneously includes multiple specific implementation parts.
[0093] On the basis of any of the above embodiments, in order to obtain a sufficient number of first face feature sets and second face feature sets, a variety of methods can also be used. For example, firstly mark a small number of face images of the face category to train a classifier, and then use the classifier and the face random generator to generate the required number of sample images of the corresponding face shape. Finally, the face features are extracted from the sample set of each face shape to form a face feature set. For example, a face key point set, a face feature set, a face outer contour set, and so on. The classifier and the face random generator can also be implemented using a variety of specific models, such as the common stylegan2 model and other identical or similar models, which are not listed here one by one.
[0094] The above embodiments describe how to train a face shape transformation model from various aspects. In order to highlight the effect of the trained face shape transformation model from the actual usage scenario as much as possible, the present disclosure also specifically provides a solution for using the trained face shape transformation model to solve practical problems. A face shape transformation method includes the following steps:
[0095] Obtain the face image to be transformed;
[0096] The target face shape transformation model is called to perform face shape transformation on the face image to be transformed.
[0097] Furthermore, the execution subject may also return the face shape transformed image output by the model to the user who initiated the transformation task, or upload it to a certain network address according to the user's instructions.
[0098] In order to deepen the understanding, the present disclosure also provides a specific implementation scheme in combination with the specific application scenario of the need for adult and child face shape transformation. The implementation scheme is generally divided into several parts: 1) training sample generation model; 2) using the trained sample generation model to generate adult and child face image sets that meet the requirements; 3) extracting adult face key point sets and child face key point sets from the adult and child face image sets respectively; 4) training a face key point migration model for realizing the exchange of adult and child face shapes based on the adult face key point set and the child face key point set. The following is an explanation of each part:
[0099] 1) Training sample generation model
[0100] In this embodiment, the scheme generated by stylegan2 is used to generate child images and adult images. First, a large amount of data is used to train the stylegan2 model; then a small number of images are randomly generated using the trained stylegan2 model; next, two groups of adult and child data are obtained through manual annotation, and then a linear classifier is trained based on the two groups of data to obtain a classification hyperplane (the normal vector of the classification hyperplane is the attribute axis of the two groups of adult and child data, that is, one side of the attribute axis generates adult images and the other side generates child images).
[0101] The specific architecture of stylegan2 is as follows Figure 4 As shown, Figure 4The left side shows a fully connected latent code mapping network, which aims to map z from 512 dimensions to 8*512 dimensions w. Then w passes through A in the decoder structure shown on the right to obtain the mod coefficient of the weight of the corresponding layer in the decoder. Next, the corresponding layer weight is modulated and demodulated to obtain the corresponding weight. The next step is to calculate the corresponding convolution, and the result is added to B (noise) on the right to be sent to the next layer until the final 1024×1024 image is obtained. The training process directly judges the authenticity of the final generated image. After the training converges, the final stylegan2 model is obtained.
[0102] Use the obtained stylegan2 model to randomly generate a small number of pictures (about 4000 pictures), and manually annotate the corresponding latent code z with adult and child labels. Use a linear support vector machine to train a classifier on the latent code z. Obtain the classification hyperplane, and then obtain the normal vector of the hyperplane, which is the direction vector of the adult and child attribute axis.
[0103] 2) Generate a set of adult and child face images that meet the requirements:
[0104] A large amount of data is randomly generated using the classification hyperplane of the stylegan2 model. At the same time, the classifier trained in the previous step is used to discriminate the attributes. The desired number of adult face images and child face images can be generated. In this embodiment, 20,000 images of each type are required.
[0105] Randomly generate latent code z and use the trained support vector machine classifier to classify the latent code z. When an adult needs to be generated and the classification result is a child, z is moved toward the adult direction according to the direction vector of the adult-child attribute axis. When a child needs to be generated and the classification result is an adult, z is moved toward the child direction according to the direction vector of the adult-child attribute axis. In this way, the required 20,000 adult images and 20,000 child images can be generated as needed.
[0106] 3) Face key point extraction:
[0107] Extract key points from the generated child and adult face images. The key point extraction model can use an open source face key point detection model based on dlib (a C++ open source toolkit containing machine learning algorithms).
[0108] 4) Facial key point migration model training:
[0109] The generated facial key point data is used to train the facial key point transfer model. The extracted facial key points, the corresponding adult and child labels, the generator based on the fully connected design, and the discriminator are used to train the key point transfer model.
[0110] The overall training method used is cyclegan design. The specific structure is as follows Figure 5 As shown. The encoder, decoder and MLP (Multi-Layer Perception) are all fully connected networks. Encoder is responsible for encoding the input facial key points to obtain xfeature (x feature), and MLP is responsible for encoding the condition (feedback) to obtain cfeature (c feature). Then xfeature and cfeature are concat (fused) together and sent to the decoder for decoding to obtain the facial key points corresponding to the condition. Condition can take [0,1] or [1,0]. Among them, [0,1] means that the goal is to change an adult into a child, and [1,0] means to migrate the child's key points to an adult. The final output of the network is not to directly output the corresponding key points, but the relative movement distance of the original key points when migrating to the target key points. The discriminator is also composed of a fully connected network. The training method adopts the cyclegan (cycle generative adversarial) method. Specifically, the loss function includes reconstruction loss, regularization loss of offset distance, adversarial loss (these three are all commonly used cycle functions of generative adversarial networks) and cycle loss (that is, the cycle loss function described in the above embodiment of the present application).
[0111] Figure 6 The migration results of some key points are shown. The first row describes the transformation of adult face key points to child face key points from left to right, and the second row describes the transformation of child face key points to adult face key points from left to right.
[0112] Further references Figure 7 and Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides a face shape transformation model training device embodiment and a face shape transformation device embodiment, the face shape transformation model training device embodiment and Figure 2 The embodiment of the face shape transformation model training method shown corresponds to the embodiment of the face shape transformation device, and the embodiment of the face shape transformation method corresponds to the embodiment of the face shape transformation device. The above-mentioned device can be specifically applied to various electronic devices.
[0113] like Figure 7As shown, the face shape transformation model training device 700 of this embodiment may include: a generation network training unit 701, a first primary transformation unit 702, a second primary transformation unit 703, a secondary transformation unit 704, and a target face shape transformation model output unit 705. The generative network training unit 701 is configured to obtain a first generative network and a second generative network based on a first face feature set and a second face feature set; the first primary transformation unit 702 is configured to generate a first target image corresponding to the first sample image using the first generative network, and control the first target image to be judged as belonging to the second face shape; the second primary transformation unit 703 is configured to generate a second target image corresponding to the second sample image using the second generative network, and control the second target image to be judged as belonging to the first face shape; the secondary transformation unit 704 is configured to generate a first transformation image corresponding to the first target image and a second transformation image corresponding to the second target image using the second generative network and the first generative network respectively; the same image control unit 704 is configured to control the first sample image and the first transformation image, and the second sample image and the second transformation image to be judged as the same image; the target face shape transformation model output unit 705 is configured to output the generative adversarial network that meets the preset requirements as a target face shape transformation model; wherein the first sample image and the second sample image belong to the first face shape and the second face shape respectively, and the generative adversarial network includes the first generative network and the second generative network.
[0114] In this embodiment, in the face shape transformation model training device 700, the specific processing of the generation network training unit 701, the first primary transformation unit 702, the second primary transformation unit 703, the secondary transformation unit 704, and the target face shape transformation model output unit 705 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201 - 205 in the corresponding embodiment are not repeated here.
[0115] In some optional implementations of this embodiment, the network training generating unit 701 may be further configured to:
[0116] The first face shape feature set and the second face shape feature set are used as input samples and output samples for training a generative network, respectively, to obtain a first generative network;
[0117] The second face shape feature set and the first face shape feature set are respectively used as input samples and output samples for training another generative network to obtain a second generative network.
[0118] In some optional implementations of this embodiment, the face shape transformation model training device 700 may further include:
[0119] A first discriminant network construction unit is configured to extract a first face shape feature based on the first face shape feature set, and construct a first discriminant network based on the first face shape feature;
[0120] A second discriminant network construction unit is configured to extract a second face shape feature based on a second face shape feature set, and construct a second discriminant network based on the second face shape feature;
[0121] The third discriminant network construction unit is configured to construct a third discriminant network based on a preset same image discrimination rule.
[0122] In some optional implementations of this embodiment, the first primary transformation unit 702 may include a first control subunit configured to control the first target image to be determined to belong to the second face shape, and the first control subunit may be further configured to:
[0123] Controlling the first target image to be discriminated by the second discriminant network as belonging to the second face shape;
[0124] The second primary transformation unit 703 may include a second control subunit configured to control the second target image to be determined to belong to the first face shape, and the second control subunit may be further configured to:
[0125] Control the second target image to be judged by the first discriminant network to belong to the first face shape.
[0126] In some optional implementations of this embodiment, the same image control unit 704 may be further configured to:
[0127] Controlling the first sample image and the first transformed image to be judged as the same image by the third discriminant network;
[0128] The second sample image and the second transformed image are controlled to be discriminated as the same image by the third discriminant network.
[0129] In some optional implementations of this embodiment, the target face shape transformation model output unit 705 may be further configured to:
[0130] Based on the first generative network, the second generative network, the first discriminative network, the second discriminative network and the third discriminative network, a target generative adversarial network is constructed;
[0131] The target generative adversarial network that meets the preset requirements is output as a target face transformation model.
[0132] In some optional implementations of this embodiment, the face shape transformation model training device 700 may further include:
[0133] A face shape image set random generation unit is configured to generate a preset number of first face shape images and second face shape images respectively through a face image random generation model to obtain a first face shape image set and a second face shape image set;
[0134] The face shape feature extraction unit is configured to extract face shape features from the first face shape image set and the second face shape image set respectively to obtain a first face shape feature set and a second face shape feature set.
[0135] In some optional implementations of this embodiment, the face shape feature extraction unit may be further configured as follows:
[0136] Face key points are extracted from the first face shape image set and the second face shape image set respectively to obtain a first face shape key point set and a second face shape key point set.
[0137] In some optional implementations of this embodiment, the same image control unit 704 may be further configured to:
[0138] By adjusting the parameters of a preset cyclic loss function, the first sample image and the first transformed image, and the second sample image and the second transformed image are controlled to be determined as the same image;
[0139] Correspondingly, the target face shape transformation model output unit 705 can be further configured as follows:
[0140] The generative adversarial network with a cyclic loss function that satisfies the preset exit condition is output as a target face transformation model.
[0141] In some optional implementations of this embodiment, the same image control unit 704 may include a first subunit configured to control the first sample image and the first transformed image to be determined as the same image, and the first subunit may be further configured to:
[0142] Obtain at least one of a difference in facial key points, a difference in expression, and a difference in face identity information between the first sample image and the first transformed image;
[0143] In response to at least one of the difference in facial key points, the difference in expression, and the difference in face identity information being smaller than the corresponding preset difference, it is determined that the first sample image and the first transformed image are the same image.
[0144] like Figure 8As shown, the face shape transformation device 800 of this embodiment may include: a face image acquisition unit 801 to be transformed, and a model calling processing unit 802. The face image acquisition unit 801 to be transformed is configured to acquire the face image to be transformed; the model calling processing unit 802 is configured to call the target face shape transformation model to transform the face image to be transformed; and the target face shape transformation model is obtained according to the face shape transformation model training device 700.
[0145] In this embodiment, in the face shape transformation device 800, the specific processing of the face image acquisition unit 801 to be transformed and the model call processing unit 802 and the technical effects brought by them can respectively correspond to the relevant descriptions in the method embodiment, and will not be repeated here.
[0146] The present embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The face shape transformation model training apparatus and the face shape transformation apparatus provided by the present embodiment, on the basis of ensuring that the image after the first face shape transformation can be identified as the corresponding face shape, also additionally control the sample image and the image after the second face shape transformation to be identified as the same image. Thus, when it is difficult to obtain a training sample pair with an obvious corresponding relationship, not only can the image after the face shape transformation be controlled to pass the face shape identification, but also the change of other facial information can be controlled as little as possible, so as to achieve the same person and the same expression before and after the transformation.
[0147] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the face shape transformation model training method and / or face shape transformation method described in any of the above embodiments when executing.
[0148] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the face shape transformation model training method and / or face shape transformation method described in any of the above embodiments when executed.
[0149] The embodiments of the present disclosure provide a computer program product, which, when executed by a processor, can implement the face shape transformation model training method and / or face shape transformation method described in any of the above embodiments.
[0150] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0151] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0152] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0153] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the face shape transformation model training method and / or the face shape transformation method. For example, in some embodiments, the face shape transformation model training method and / or the face shape transformation method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the face shape transformation model training method and / or the face shape transformation method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the face shape transformation model training method and / or the face shape transformation method in any other appropriate manner (for example, by means of firmware).
[0154] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0155] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0156] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0158] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0159] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.
[0160] The technical solution of the disclosed embodiment, on the basis of ensuring that the image after the first face shape transformation can be identified as the corresponding face shape, further controls the sample image and the image after the second face shape transformation to be identified as the same image. Therefore, when it is difficult to obtain a training sample pair with an obvious corresponding relationship, not only can the image after the face shape transformation be controlled to pass the face shape identification, but also the change of other facial information can be controlled as little as possible, so as to achieve the same person and the same expression before and after the transformation.
[0161] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0162] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A face shape transformation model training method, comprising: Based on the first face shape feature set and the second face shape feature set, a first generative network and a second generative network are obtained; Using the first generation network to generate a first target image corresponding to the first sample image, and controlling the first target image to be judged as belonging to a second face shape; generating a second target image corresponding to the second sample image by using the second generation network, and controlling the second target image to be judged to belong to the first face shape; Generate a first transformed image corresponding to the first target image and a second transformed image corresponding to the second target image by using the second generation network and the first generation network respectively; Controlling the first sample image and the first transformed image, and the second sample image and the second transformed image to be determined as the same image; The generative adversarial network that meets preset requirements is output as a target face shape transformation model; wherein the first sample image and the second sample image belong to the first face shape and the second face shape respectively, and the generative adversarial network includes the first generation network and the second generation network.
2. The method according to claim 1, wherein: The method of obtaining a first generation network and a second generation network based on the first face shape feature set and the second face shape feature set includes: Using the first face shape feature set and the second face shape feature set as input samples and output samples for training a generative network, respectively, to obtain the first generative network; The second face shape feature set and the first face shape feature set are used as input samples and output samples for training another generative network, respectively, to obtain the second generative network.
3. The method according to claim 1, further comprising: Extracting a first face feature based on the first face feature set, and constructing a first discriminant network based on the first face feature; Extracting a second face shape feature based on the second face shape feature set, and constructing a second discriminant network based on the second face shape feature; A third discriminant network is constructed based on the preset same image discrimination rules.
4. The method according to claim 3, wherein: The controlling the first target image to be judged as belonging to a second face shape comprises: Controlling the first target image to be discriminated by the second discriminant network as belonging to the second face shape; The controlling the second target image to be judged as belonging to the first face shape comprises: Control the second target image to be judged by the first discriminant network to belong to the first face shape.
5. The method according to claim 3, wherein: The controlling the first sample image and the first transformed image, and the second sample image and the second transformed image to be determined as the same image, comprises: Controlling the first sample image and the first transformed image to be discriminated as the same image by the third discriminant network; The second sample image and the second transformed image are controlled to be discriminated as the same image by the third discriminant network.
6. The method according to claim 3, wherein: The step of outputting the generative adversarial network that meets preset requirements as a target face shape transformation model includes: Based on the first generative network, the second generative network, the first discriminative network, the second discriminative network and the third discriminative network, construct a target generative adversarial network; The target generative adversarial network that meets the preset requirements is output as the target face transformation model.
7. The method according to claim 1, further comprising: Generate a preset number of first face shape images and second face shape images respectively through a face image random generation model to obtain a first face shape image set and a second face shape image set; Face shape features are extracted from the first face shape image set and the second face shape image set respectively to obtain the first face shape feature set and the second face shape feature set.
8. The method according to claim 7, wherein: The extracting face shape features from the first face shape image set and the second face shape image set to obtain the first face shape feature set and the second face shape feature set comprises: Face shape key points are extracted from the first face shape image set and the second face shape image set respectively to obtain the first face shape key point set and the second face shape key point set.
9. The method according to claim 1, wherein: The controlling the first sample image and the first transformed image, and the second sample image and the second transformed image to be determined as the same image, comprises: By adjusting the parameters of a preset cyclic loss function, the first sample image and the first transformed image, and the second sample image and the second transformed image are controlled to be determined as the same image; Correspondingly, the method of outputting the generative adversarial network that meets the preset requirements as a target face shape transformation model includes: The generative adversarial network having a cyclic loss function that satisfies a preset exit condition is output as the target face shape transformation model.
10. The method according to any one of claims 1 to 9, wherein: The controlling the first sample image and the first transformed image to be determined as the same image comprises: Obtaining at least one of a difference in facial key points, a difference in facial expression, and a difference in facial identity information between the first sample image and the first transformed image; In response to at least one of the face key point difference, the expression difference, and the face identity information difference being smaller than a corresponding preset difference, it is determined that the first sample image and the first transformed image are the same image.
11. A face shape transformation method, comprising: Obtain the face image to be transformed; Calling a target face shape transformation model to transform the face image to be transformed; wherein the target face shape transformation model is obtained based on the face shape transformation model training method according to any one of claims 1 to 10.
12. A face shape transformation model training device, comprising: A generative network training unit is configured to obtain a first generative network and a second generative network based on the first face feature set and the second face feature set; A first primary transformation unit is configured to generate a first target image corresponding to the first sample image using the first generation network, and control the first target image to be judged as belonging to a second face shape; A second primary transformation unit is configured to generate a second target image corresponding to the second sample image using the second generation network, and control the second target image to be judged to belong to the first face shape; a secondary transformation unit, configured to generate a first transformation image corresponding to the first target image and a second transformation image corresponding to the second target image by using the second generation network and the first generation network respectively; an identical image control unit configured to control the first sample image and the first transformed image, and the second sample image and the second transformed image to be determined as identical images; The target face transformation model output unit is configured to output the generative adversarial network that meets the preset requirements as the target face transformation model; wherein the first sample image and the second sample image belong to the first face shape and the second face shape respectively, and the generative adversarial network includes the first generation network and the second generation network.
13. The device according to claim 12, wherein: The generation network training unit is further configured to: Using the first face shape feature set and the second face shape feature set as input samples and output samples for training a generative network, respectively, to obtain the first generative network; The second face shape feature set and the first face shape feature set are used as input samples and output samples for training another generative network, respectively, to obtain the second generative network.
14. The apparatus according to claim 12, further comprising: a first discriminant network construction unit, configured to extract a first face feature based on the first face feature set, and construct a first discriminant network based on the first face feature; a second discriminant network construction unit, configured to extract a second face shape feature based on the second face shape feature set, and construct a second discriminant network based on the second face shape feature; The third discriminant network construction unit is configured to construct a third discriminant network based on a preset same image discrimination rule.
15. The device according to claim 14, wherein: The first primary transformation unit includes a first control subunit configured to control the first target image to be determined as belonging to a second face shape, and the first control subunit is further configured to: Controlling the first target image to be discriminated by the second discriminant network as belonging to the second face shape; The second primary transformation unit includes a second control subunit configured to control the second target image to be determined as belonging to the first face shape, and the second control subunit is further configured to: Control the second target image to be judged by the first discriminant network to belong to the first face shape.
16. The device according to claim 14, wherein: The same image control unit is further configured to: Controlling the first sample image and the first transformed image to be discriminated as the same image by the third discriminant network; The second sample image and the second transformed image are controlled to be discriminated as the same image by the third discriminant network.
17. The device according to claim 14, wherein: The target face shape transformation model output unit is further configured to: Based on the first generative network, the second generative network, the first discriminative network, the second discriminative network and the third discriminative network, construct a target generative adversarial network; The target generative adversarial network that meets the preset requirements is output as the target face transformation model.
18. The apparatus according to claim 12, further comprising: A face shape image set random generation unit is configured to generate a preset number of first face shape images and second face shape images respectively through a face image random generation model to obtain a first face shape image set and a second face shape image set; The face shape feature extraction unit is configured to extract face shape features from the first face shape image set and the second face shape image set respectively to obtain the first face shape feature set and the second face shape feature set.
19. The device according to claim 18, wherein: The face shape feature extraction unit is further configured to: Face shape key points are extracted from the first face shape image set and the second face shape image set respectively to obtain the first face shape key point set and the second face shape key point set.
20. The device according to claim 12, wherein: The same image control unit is further configured to: By adjusting the parameters of a preset cyclic loss function, the first sample image and the first transformed image, and the second sample image and the second transformed image are controlled to be determined as the same image; Correspondingly, the target face shape transformation model output unit is further configured as follows: The generative adversarial network having a cyclic loss function that satisfies a preset exit condition is output as the target face shape transformation model.
21. The device according to any one of claims 12 to 20, wherein: The same image control unit includes a first subunit configured to control the first sample image and the first transformed image to be determined as the same image, and the first subunit is further configured to: Obtaining at least one of a difference in facial key points, a difference in facial expression, and a difference in facial identity information between the first sample image and the first transformed image; In response to at least one of the face key point difference, the expression difference, and the face identity information difference being smaller than a corresponding preset difference, it is determined that the first sample image and the first transformed image are the same image.
22. A face shape changing device, comprising: A face image acquisition unit to be transformed, configured to acquire a face image to be transformed; The model calling processing unit is configured to call a target face shape transformation model to perform face shape transformation on the face image to be transformed; wherein the target face shape transformation model is obtained based on the face shape transformation model training device according to any one of claims 12 to 21.
23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the face shape transformation model training method described in any one of claims 1-10 and / or the face shape transformation method described in claim 11.
24. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the face shape transformation model training method described in any one of claims 1 to 10 and / or the face shape transformation method described in claim 11.
