Face image generation method and device, electronic equipment and storage medium

Abstract

The application provides a face image generation method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining first classification features of an initial face image by a first sub-classification model of a first neural network obtained through training; determining parameters of a first sub-flow model of the first neural network based on the first classification features, and inputting the target face image into the first sub-flow model with the determined parameters for forward operation to obtain initial features of the initial face image; obtaining first condition information input by a user, adjusting the initial features based on the first condition information; inputting the adjusted initial features into the first sub-flow model with the determined parameters for inverse operation to obtain a first target face image corresponding to the adjusted initial features, so that the target face image can be quickly obtained according to the initial face image, the user's face adjusting operation is simplified, and the user can quickly obtain the attribute features that he or she wants.

Description

Technical Field

[0001] This application relates to the field of facial image generation technology, and more particularly to a method, apparatus, electronic device, and storage medium for generating facial images. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Currently, character customization is widely used across various platforms and systems, playing a significant role in MMO games. Many players spend considerable time exploring character customization systems to achieve their ideal facial image or model. However, while current technologies offer a wide range of attribute settings, players often struggle to quickly obtain the desired attributes. For instance, creating a blond, bearded 50-year-old male might require users to meticulously configure specific facial parameters and features step-by-step, making character customization a complex process and difficult to achieve the desired facial image. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and storage medium for generating facial images.

[0005] To achieve the above objectives, this application provides a method for generating a face image, comprising:

[0006] The first classification feature of the initial face image is obtained by training the first sub-classification model of the first neural network;

[0007] Based on the first classification features, the parameters of the first sub-stream model of the first neural network are determined, and the target face image is input into the first sub-stream model after the parameters are determined for forward operation to obtain the initial features of the initial face image.

[0008] Obtain first conditional information input by the user, and adjust the initial features based on the first conditional information; wherein, the first conditional information and the first classification feature belong to the same type;

[0009] The adjusted initial features are input into the first sub-stream model after the parameters are determined for inverse operation to obtain the first target face image corresponding to the adjusted initial features.

[0010] Based on the same inventive concept, an exemplary embodiment of this application also provides a facial image generation apparatus, comprising:

[0011] The classification feature extraction module obtains the first classification features of the initial face image through the first sub-classification model of the first neural network obtained through training;

[0012] an image feature extraction module configured to determine parameters of a first sub-flow model of the first neural network based on the first classification feature, and input the target face image into the first sub-flow model with the determined parameters for forward operation to obtain initial features of the initial face image;

[0013] an adjustment module configured to obtain first condition information input by a user, and adjust the initial features based on the first condition information, wherein the first condition information and the first classification feature belong to the same type;

[0014] an image generation module configured to input the adjusted initial features into the first sub-flow model with the determined parameters for inverse operation to obtain a first target face image corresponding to the adjusted initial features.

[0015] Based on the same inventive concept, the example embodiments of the present application further provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable by the processor, and the processor implements the face image generation method as described above when executing the program.

[0016] Based on the same inventive concept, the example embodiments of the present application further provide a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the face image generation method as described above.

[0017] As can be seen from the above, the face image generation method, device, electronic device, and storage medium provided by the present application obtain first classification features of an initial face image through a first sub-classification model of a first neural network obtained by training; determine parameters of a first sub-flow model of the first neural network based on the first classification features, and input the target face image into the first sub-flow model with the determined parameters for forward operation to obtain initial features of the initial face image; obtain first condition information input by a user, and adjust the initial features based on the first condition information, wherein the first condition information and the first classification feature belong to the same type; input the adjusted initial features into the first sub-flow model with the determined parameters for inverse operation to obtain a first target face image corresponding to the adjusted initial features, so that the target face image can be quickly obtained from the initial face image, simplifying the user's face pinching operation, and at the same time, the finally generated target face image is adjusted through the condition information input by the user, so that the user can quickly obtain the attribute features he wants. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, the drawings in the following description only are the embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0019] Figure 1 A flowchart of a face image generation method according to an embodiment of the application;

[0020] Figure 2 A flowchart of another face image generation method according to an embodiment of the application;

[0021] Figure 3 A flowchart of still another face image generation method according to an embodiment of the application;

[0022] Figure 4 A structural diagram of a face image generation device according to an embodiment of the application;

[0023] Figure 5 A structural diagram of a specific electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0024] The principles and spirits of the application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and implement the application, and do not limit the scope of the application in any way. On the contrary, these embodiments are provided to make the application more thorough and complete, and to fully convey the scope of the application to those skilled in the art.

[0025] According to the embodiments of the application, a face image generation method, device, electronic device and storage medium are provided.

[0026] In this document, it should be understood that any number of elements in the drawings are used for illustration only and not limitation, and any naming is only for distinction and does not have any limiting meaning.

[0027] The principles and spirits of the application will be described below with reference to several representative embodiments of the application. SUMMARY

[0029] In the related art, there are generally three methods for obtaining a face image through a face pinching system. The first method is to use skeletal animation, the second method is to use morph animation, and the third method is to pinch a face texture. The face pinching system using skeletal animation requires binding a plurality of bones at each detailed part of the face, moving the bones to drive the face grid vertex to move, and feeding back the movement of the grid vertex to another set of skeletal skin for expression animation. The face pinching system using morph animation requires first making a plurality of face models in extreme conditions, mixing the extreme models by adjusting parameters (weights) to generate new face grid vertices; after the face grid vertices are generated, they can be fed back to the skeletal skin for expression animation, or they can continue to be mixed with the extreme models representing the expression to generate expression animation. Finally, another form of face pinching is to pinch the face texture, such as the thickness of the eyebrows and beard, the color of the lips, the height of the eyebrows, etc. These do not require the grid model to change, but can be achieved by changing the texture. The texture pinching is similar to the morph animation, and is also mixed according to the parameters of a plurality of textures in extreme conditions. However, for users, regardless of which implementation method in the related art is used, a large number of parameter adjustments are required to obtain a face image that they are satisfied with. In addition, for artists, a large amount of face model making is required for subsequent fusion, which is a huge amount of work.

[0030] To solve the above problems, the present application provides a face image generation method, specifically comprising:

[0031] The first classification feature of the initial face image is obtained through the first sub-classification model of the first neural network obtained by training; the parameters of the first sub-flow model of the first neural network are determined based on the first classification feature, and the target face image is input into the first sub-flow model after the parameters are determined for forward operation to obtain the initial feature of the initial face image; the first condition information input by the user is obtained, and the initial feature is adjusted based on the first condition information; wherein the first condition information and the first classification feature belong to the same type; the adjusted initial feature is input into the first sub-flow model after the parameters are determined for inverse operation to obtain the first target face image corresponding to the adjusted initial feature, so that the target face image can be quickly obtained from the initial face image, the user's face pinching operation is simplified, and the finally generated target face image is adjusted through the condition information input by the user, so that the user can quickly obtain the attribute feature he wants.

[0032] After introducing the basic principles of the present application, the various non-limiting embodiments of the present application will be specifically introduced below.

[0033] Overview of application scenarios

[0034] In some specific application scenarios, the face image generation method of the present application can be applied to various platforms or systems involving the generation of face images.

[0035] In some specific application scenarios, the face image generation method of the present application can be applied to the setting of the face image of a character in a game system, or an APP with a face image generation function.

[0036] The face image generation method according to the exemplary embodiments of the present application will be described below in conjunction with specific application scenarios. It should be noted that the above-mentioned application scenarios are only shown for the purpose of facilitating the understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0037] Exemplary method

[0038] Reference Figure 1 The embodiments of the present application provide a face image generation method, comprising the following steps:

[0039] S101, obtaining first classification features of an initial face image by a first sub-classification model of a first neural network obtained by training.

[0040] In specific implementation, the initial face image is input into the first sub-classification model of the first neural network obtained by training to extract the first classification features of the initial face image. The initial face image is generally provided by a user, for example, the user can use his own face image as the initial face image, or use some face images he likes as the initial face image. Optionally, the face image can be a human face image or an animal face image, which is not limited.

[0041] It should be noted that the first neural network is mainly divided into a first sub-classification model and a first sub-flow model, that is, the first neural network is composed of a classification model and a flow model, and the structure of the sub-classification model is not limited here, and any classification model structure in the related art can be selected as the structure of the first sub-classification model, for example, a convolutional neural network (CNN), a fully connected neural network (FCN), and a generative adversarial network (GAN), etc. The sub-classification model is mainly used to extract the first classification feature of the input initial image, and the first type feature represents a certain feature of the face image, for example, hair color, skin color, age, etc. of the face image. The specific representation of which feature can be set according to the needs, and through the training of the first neural network model, the first sub-classification model learns to extract the feature. For example, the first type feature represents the hair color feature, and the corresponding first sub-classification model can extract different hair color features, that is, the process of training the first sub-classification model can be regarded as a training process of a classification model capable of classifying hair color.

[0042] In S102, parameters of the first sub-flow model of the first neural network are determined based on the first classification feature, and the target face image is input into the first sub-flow model after the parameters are determined for forward operation to obtain the initial feature of the initial face image.

[0043] In specific implementation, after obtaining the first classification feature, the parameters of the first sub-flow model of the first neural network are determined according to the first classification feature, and then the target face image is input into the first sub-flow model after the parameters are determined for forward operation to obtain the initial feature of the initial face image.

[0044] It should be noted that the first neural network is mainly divided into a first sub-classification model and a first sub-flow model, wherein the first sub-flow model is equivalent to a flow model, and the specific structure of the first sub-flow model is not limited here, and any flow model structure in the related art can be selected as the structure of the first sub-flow model, for example, a standard flow model (Normlizing Flow), an autoregressive flow model (Autoregressive Flow), and a glow model (Generative Flow with Invertible 1*1 Convolutions), etc.

[0045] It should be noted that generally, the parameters of the neural network model are fixed after the training is completed, but in this embodiment, the parameters of the sub-flow model are constantly changed by the first classification feature, that is, different first classification features are obtained by inputting different initial face images each time, so that the parameters of the sub-flow model are different each time. The parameters of the sub-flow model are controlled by the output of the sub-classification model, which can further affect the initial features of the initial face image output by the sub-flow model. That is, the initial features output by the sub-flow model are limited by the first classification feature as a prior condition.

[0046] In some embodiments, determining the parameters of the first sub-flow model of the first neural network based on the first classification feature specifically comprises:

[0047] obtaining a mapping relationship between the parameters of the first sub-flow model determined in the process of training the first neural network and the first classification feature;

[0048] determining the parameters of the first sub-flow model based on the first classification feature and the mapping relationship.

[0049] In specific implementation, when training the first neural network model, a preset relationship (function) between the parameters of a first sub-flow model and the first classification feature can be fitted first, and then a large amount of sample data is used for training and optimization, so as to finally obtain the mapping relationship between the parameters of the first sub-flow model and the first classification feature. After obtaining the preset relationship, the parameters of the first sub-flow model can be determined according to the first classification feature and the mapping relationship.

[0050] In some embodiments, the first sub-flow model comprises a plurality of process processing layers; determining the parameters of the first sub-flow model based on the first classification feature and the mapping relationship specifically comprises:

[0051] obtaining a target mapping relationship between the parameters of each process processing layer determined in the process of training the first neural network and the first classification feature;

[0052] determining the parameters of each process processing layer based on the first classification feature and the target mapping relationship;

[0053] determining the parameters of the first sub-flow model based on the parameters of all the process processing layers.

[0054] In a specific implementation, since the flow model generally includes multiple flow processing layers (Flow Block), and the number of parameters of different flow processing layers of the same flow model is different, in general, the number of parameters corresponding to the multiple flow processing layers gradually decreases from front to back. Therefore, in order to accurately determine the parameters of the first sub-flow model according to the first classification feature and the mapping relationship, the target mapping relationship between the parameters of each flow processing layer and the first classification feature can be determined respectively, and then the first classification feature and the target mapping relationship determine the parameters of each flow processing layer.

[0055] In S103, first condition information input by a user is obtained, and the initial feature is adjusted based on the first condition information; wherein the first condition information and the first classification feature belong to the same type.

[0056] In a specific implementation, the first condition information input by the user obtained can be condition information of a certain category input by the user in the form of text, for example, "hair color is yellow". Alternatively, multiple options for a certain category can be pre-set, for example, yellow, black, and white hair color options are pre-set, and then the user selects one from the options as the first condition information. After obtaining the first condition information, the initial feature is adjusted according to the first condition information, that is, a certain type of feature can be adjusted according to the user's preference. For example, the initial hair color of the face image input by the user is black, but the user wants to adjust it to yellow, so the first condition information can be input to achieve the adjustment.

[0057] It should be noted that in the embodiment, the first condition information input by the user obtained corresponds to the first classification feature output by the sub-classification model, that is, the first classification feature output by the sub-classification model is a feature related to hair color, and the information input by the user related to the adjustment of the hair color is obtained. Further, the accuracy of subsequent feature adjustment is ensured.

[0058] In some embodiments, the first condition information input by the user is obtained, and the initial feature is adjusted based on the first condition information, specifically including:

[0059] determining a preset feature corresponding to the first condition information;

[0060] iteratively adjusting the initial feature multiple times;

[0061] determining multiple first target results in which the loss of the perceptual loss function is in a preset range from the results of multiple iterative adjustments, and determining a second target result in which the cross-entropy loss with the preset feature is the smallest from the multiple first target results;

[0062] adjusting the initial feature to the second target result.

[0063] In a specific implementation, when the initial feature is adjusted based on the first condition information, a preset feature corresponding to the first condition information is determined first. Optionally, a plurality of preset features corresponding to a plurality of condition information can be set in advance, so that when a certain condition information is received, the preset feature corresponding to the condition information can be determined. Then, the initial feature is adjusted multiple times, and a plurality of first target results with a perceptual loss function loss in a preset range are determined from the results of the multiple iterations. Optionally, the preset range can be set as needed. The adjusted initial feature and the initial feature before adjustment can maintain a certain similarity through the screening of the perceptual loss function, so as to avoid that when the adjusted initial feature generates a face image again, not only a certain feature changes, but also the whole is completely different from the original face image. Finally, a second target result most similar to the preset feature corresponding to the first condition information is found through a cross-entropy loss function (Cross Entropy) to meet the adjustment needs of the user.

[0064] In some embodiments, the first condition information is color feature information; the first condition information input by the user is obtained, and the initial feature is adjusted based on the first condition information, specifically including:

[0065] A spatial feature vector corresponding to each of three primary colors is obtained; wherein the three primary colors include red, green and blue;

[0066] A first relative position of a color feature corresponding to the first condition information and a spatial feature vector corresponding to each of the three primary colors is determined;

[0067] The initial feature is adjusted multiple times;

[0068] A plurality of first target results with a perceptual loss function loss in a preset range are determined from the results of the multiple iterations;

[0069] A second relative position of each of the first target results and the spatial feature vector corresponding to each of the three primary colors is determined, and a cross-entropy loss of the first relative position and the second relative position is determined;

[0070] From all the target cross-entropy losses, a cross-entropy loss with the smallest loss is determined, and the initial feature is adjusted to the first target result corresponding to the cross-entropy loss with the smallest loss.

[0071] In a specific implementation, when the first condition information is color feature information, in order to meet the requirement of the user that the current feature is adjusted to any color, the color features of all colors can be defined by three primary colors (three-element color), that is, the spatial feature vectors corresponding to the three primary colors are preset, each color is defined as a relative position to the spatial feature vectors corresponding to the three primary colors, different colors have different relative positions to the spatial feature vectors corresponding to the three primary colors, then the first target result closest to the first relative position of the color feature corresponding to the first condition information is found, and the first target result can be used as the adjusted initial feature.

[0072] In S104, the adjusted initial feature is input into the first sub-flow model with determined parameters for inverse operation to obtain a first target face image corresponding to the adjusted initial feature.

[0073] In a specific implementation, since the flow model itself is reversible, that is, the output can be obtained by input through forward operation, and the input can be obtained by output through inverse operation. Therefore, the first target face image corresponding to the adjusted initial feature can be obtained by inputting the adjusted initial feature into the first sub-flow model with determined parameters for inverse operation.

[0074] In some embodiments, after obtaining the first target face image corresponding to the adjusted initial feature, the method further includes:

[0075] obtaining a second classification feature of the first target face image by using a second sub-classification model of a second neural network obtained through training;

[0076] determining parameters of a second sub-flow model of the second neural network based on the second classification feature, and inputting the first target face image into the second sub-flow model with the determined parameters for forward operation to obtain a target feature of the first target face image;

[0077] obtaining second condition information input by the user, and adjusting the target feature based on the second condition information; wherein the second condition information and the second classification feature belong to the same type;

[0078] inputting the adjusted target feature into the second sub-flow model for inverse operation to obtain a second target face image corresponding to the adjusted target feature.

[0079] In implementation, the second neural network has substantially the same structure as the first neural network, and is composed of a classification model and a flow model. The main difference between the two is that the classification features obtained by the sub-classification model are different. For example, when the first classification feature is hair color feature, the second classification feature can be age feature. By taking the output of the first neural network as the input of the second neural network, the second target face image is finally obtained, which can meet the user's adjustment of two different types of features. For example, the adjustment of hair color can be realized by the first neural network, and then the face image with adjusted hair color is input into the second neural network for age adjustment, so that the simultaneous adjustment of hair color and age is realized.

[0080] In some embodiments, a third neural network similar to the first neural network described above can be arranged after the second neural network, and a plurality of neural networks similar to the first neural network described above can be arranged after the third neural network. Each neural network can correspond to the adjustment of a type of feature, so that the user's adjustment of multiple types of features can be met. For example, the first neural network can be used to adjust the hair color of the face image, the second neural network can be used to adjust the age of the face image, and the third neural network can be used to adjust the gender of the face image. Figure 2 Fig. 4 is a flowchart of another method for generating a face image according to an embodiment of the present application. In this method, the input face image can be processed by N neural networks in turn, and the changes and adjustments are made continuously, so that the finally output face image can meet the user's adjustment of multiple types of features of the face image.

[0081] In some embodiments, the loss function for training the first neural network or the second neural network includes a cross-entropy loss function, a margin loss function, and a perceptual loss function.

[0082] In implementation, the loss function for training the first neural network or the second neural network includes a cross-entropy loss function, a margin loss function, and a perceptual loss function. The cross-entropy loss is used to ensure the classification loss. In order to ensure that the classes cannot overlap, that is, the feature distribution is as far away as possible in the space, a margin loss function is added. Optionally, the distance between the means of any two distributions can be set to be greater than 6 sigma. Finally, in order to ensure that the face does not change significantly due to the network, and to ensure that it is the face of the same object, a perceptual loss function is added. Optionally, the weights assigned to each loss function can be set as needed without limitation. Optionally, in addition to the above three loss functions, other loss functions can be added as needed, which are not limited herein.

[0083] In some embodiments, after obtaining the first target face image corresponding to the adjusted initial feature, the method further includes:

[0084] convert the first target face image into a three-dimensional face model based on a three-dimensional morphable model.

[0085] In particular implementation, when the target face image is obtained, the first target face image is converted into a three-dimensional face model based on a three-dimensional morphable model, which is used in subsequent production. Optionally, the three-dimensional morphable model can be selected according to the needs of any three-dimensional morphable model in related technologies, which is not limited, for example, 3DMM or MeInGame.

[0086] It should be noted that the method of any one of the embodiments applied in the process of generating the first target face image can be applied in the process of generating the second target face image, and the subsequent processing method of the first target face image can also be applied to the second target face image.

[0087] Reference Figure 3 In another embodiment of the face image generation method provided by the present application, the initial face image is first input into a sub-classification model, and the classification features of the initial face image are output from the sub-classification model to a sub-flow model to adjust the parameters of the sub-flow model. After adjusting the parameters of the sub-flow model, the initial face image is input into the sub-flow model, and the initial features of the initial face image are output from the sub-flow model. At this time, the condition information input by the user is obtained, the initial features are adjusted according to the condition information, and then the adjusted initial features are input into the sub-flow model to obtain the target face image by reverse output.

[0088] The face image generation method provided by the present application obtains the first classification features of the initial face image through the first sub-classification model of the first neural network obtained by training; determines the parameters of the first sub-flow model of the first neural network based on the first classification features, and inputs the target face image into the first sub-flow model after determining the parameters for forward operation to obtain the initial features of the initial face image; obtains the first condition information input by the user, adjusts the initial features based on the first condition information; wherein the first condition information and the first classification features belong to the same type; inputs the adjusted initial features into the first sub-flow model after determining the parameters for inverse operation to obtain the first target face image corresponding to the adjusted initial features, so that the target face image can be quickly obtained from the initial face image, the user's face operation is simplified, and the target face image generated finally is adjusted through the condition information input by the user, so that the user can quickly obtain the attribute features he wants.

[0089] Exemplary device

[0090] Based on the same inventive concept, the application also provides a face image generation device corresponding to the method of any of the above embodiments.

[0091] Reference Figure 4 The face image generation device comprises:

[0092] The classification feature extraction module 201 obtains first classification features of the initial face image through a first sub-classification model of the first neural network obtained by training;

[0093] The image feature extraction module 202 determines parameters of a first sub-flow model of the first neural network based on the first classification features, and inputs the target face image into the first sub-flow model after the parameters are determined for forward operation, to obtain initial features of the initial face image;

[0094] The adjustment module 203 obtains first condition information input by a user, and adjusts the initial features based on the first condition information; wherein the first condition information and the first classification features belong to the same type;

[0095] The image generation module 204 inputs the adjusted initial features into the first sub-flow model after the parameters are determined for inverse operation, to obtain a first target face image corresponding to the adjusted initial features.

[0096] For the convenience of description, the above system is described in various modules in terms of functions. Of course, the functions of the modules can be implemented in the same or multiple software and / or hardware when the application is implemented.

[0097] The system of the above embodiments is used to implement the face image generation method of any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0098] Based on the same inventive concept, the application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the face image generation method of any of the above embodiments when executing the program.

[0099] Figure 5 A more specific hardware structure of an electronic device is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0100] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided by the embodiments of the present specification.

[0101] The memory 1020 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the relevant program codes are saved in the memory 1020 and called and executed by the processor 1010.

[0102] The input / output interface 1030 is configured to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input devices can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices can include a display, a speaker, a vibrator, an indicator light, etc. Figure 5

[0103] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to implement the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.), or can realize communication through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.). Figure 5

[0104] The bus 1050 includes a channel for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0105] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only include the components necessary to implement the solutions of the embodiments of the present specification, and does not have to include all the components shown in the figure. Figure 5

[0106] ​​​The electronic device of the above embodiment is used to implement the face image generation method of the corresponding any one of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0107] Exemplary program product

[0108] Based on the same inventive concept, corresponding to the method of any one of the above embodiments, the present application also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the face image generation method of any one of the above embodiments.

[0109] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0110] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to perform the face image generation method of any one of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0111] Those skilled in the art should understand that the discussion of any one of the above embodiments is only exemplary, and is not intended to imply that the scope (including claims) of the present application is limited to these examples; the above embodiments or technical features between different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of the different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in details.

[0112] Additionally, to simplify the description and discussion, and so as not to obscure the embodiments of the application being presented, the well-known functions or constructions of integrated circuit (IC) chips and other components can or can not be shown in the figures and will be omitted as not to unnecessarily obscure the embodiments of the application being presented. Moreover, the apparatus can be shown in block diagram form in order to avoid obscuring the embodiments of the application, and this also acknowledges the fact that the details in regards to the implementation of such block diagram apparatus are highly dependent on the platform within which the application is to be implemented (i.e., such details should be well within the purview of one of ordinary skill in the art to consider given the specific application). Where specific details are set forth in order to describe an illustrative embodiment of the application, it will be apparent to one of ordinary skill in the art that the application can be practiced without, or with variations of, these specific details. Thus, the description is to be considered as illustrative and not restrictive, and the scope of the application should be determined not with reference to the above description, but should be given to the appended claims.

[0113] While the application has been described in connection with specific embodiments thereof, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0114] Embodiments of the application are intended to cover all such alternatives, modifications and variations as falling within the scope of the appended claims. Accordingly, any omission, modification, equivalent replacement, improvement, etc. made in the spirit and principle of the embodiments of the application should be included in the scope of protection of the application.

Claims

1. A method of generating a face image, characterized by, The method comprises the following steps: obtaining a first classification feature of an initial face image by a first sub-classification model of a first neural network obtained through training; determining parameters of a first sub-flow model of the first neural network based on the first classification feature, and inputting the initial face image into the first sub-flow model with the determined parameters for forward operation to obtain an initial feature of the initial face image; the parameters of the first sub-flow model are controlled to change by the first classification feature, and the first classification feature is different according to different initial face images, so that the parameters of the sub-flow model are different each time; obtaining first condition information input by a user, and adjusting the initial feature based on the first condition information; wherein the first condition information and the first classification feature belong to the same type; inputting the adjusted initial feature into the first sub-flow model with the determined parameters for inverse operation to obtain a first target face image corresponding to the adjusted initial feature.

2. The method of claim 1, wherein, After obtaining the first target face image corresponding to the adjusted initial feature, the method further comprises: obtaining a second classification feature of the first target face image by a second sub-classification model of a second neural network obtained through training; determining parameters of a second sub-flow model of the second neural network based on the second classification feature, and inputting the first target face image into the second sub-flow model with the determined parameters for forward operation to obtain a target feature of the first target face image; obtaining second condition information input by a user, and adjusting the target feature based on the second condition information; wherein the second condition information and the second classification feature belong to the same type; inputting the adjusted target feature into the second sub-flow model for inverse operation to obtain a second target face image corresponding to the adjusted target feature.

3. The method of claim 1, wherein, determining parameters of a first sub-flow model of the first neural network based on the first classification feature, specifically comprising: obtaining a mapping relationship between the parameters of the first sub-flow model determined in the process of training the first neural network and the first classification feature; determining the parameters of the first sub-flow model based on the first classification feature and the mapping relationship.

4. The method of claim 3, wherein, The first sub-flow model comprises a plurality of flow processing layers; determining the parameters of the first sub-flow model based on the first classification feature and the mapping relationship, specifically comprising: obtaining a target mapping relationship between the parameters of each flow processing layer determined in the process of training the first neural network and the first classification feature; determining the parameters of each flow processing layer based on the first classification feature and the target mapping relationship; determining the parameters of the first sub-flow model based on the parameters of all the flow processing layers.

5. The method of claim 1, wherein, obtaining first condition information input by a user, and adjusting the initial feature based on the first condition information, specifically comprising: determining a preset feature corresponding to the first condition information; iteratively adjusting the initial feature for multiple times; determine a plurality of first target results from the results of the multiple iterations of adjustment, wherein a loss of a perceptual loss function of the plurality of first target results is within a preset range, and determine a second target result from the plurality of first target results, wherein a cross-entropy loss of the second target result with respect to the preset feature is the smallest; adjust the initial feature to the second target result.

6. The method of claim 1, wherein, The first condition information is color feature information; obtaining the first condition information input by a user, and adjusting the initial feature based on the first condition information, specifically comprising: obtaining a spatial feature vector corresponding to each of three primary colors; wherein the three primary colors include red, green and blue; determining a first relative position of a color feature corresponding to the first condition information and a spatial feature vector corresponding to each of the three primary colors; performing multiple iterations of adjustment on the initial feature; determining a plurality of first target results from the results of the multiple iterations of adjustment, wherein a loss of a perceptual loss function of the plurality of first target results is within a preset range; determining a second relative position of each of the first target results and the spatial feature vector corresponding to each of the three primary colors, and determining a target cross-entropy loss of the first relative position and the second relative position; determining a target cross-entropy loss with the smallest loss from all the target cross-entropy losses, and adjusting the initial feature to a first target result corresponding to the target cross-entropy loss with the smallest loss.

7. The method of claim 2, wherein, The loss function of the first neural network or the second neural network includes a cross-entropy loss function, a margin loss function and a perceptual loss function.

8. The method of claim 1, wherein, After obtaining the first target face image corresponding to the adjusted initial feature, the method further comprises: converting the first target face image into a three-dimensional face model based on a three-dimensional morphable model.

9. An apparatus for generating a face image, characterized by comprising: Comprise: a classification feature extraction module that obtains a first classification feature of an initial face image through a first sub-classification model of a first neural network obtained by training; an image feature extraction module that determines parameters of a first sub-stream model of the first neural network based on the first classification feature, and inputs the initial face image into the first sub-stream model with the determined parameters for forward operation to obtain an initial feature of the initial face image; The parameters of the first sub-stream model are changed by the first classification feature, and the first classification feature is different according to different initial face images, resulting in different parameters of the sub-stream model each time; an adjustment module that obtains first condition information input by a user, and adjusts the initial feature based on the first condition information; The first condition information and the first classification feature belong to the same type; an image generation module that inputs the adjusted initial feature into the first sub-stream model with the determined parameters for inverse operation to obtain a first target face image corresponding to the adjusted initial feature.

10. An electronic device, comprising: The non-transitory computer readable storage medium stores computer instructions for causing a computer to execute the method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions for causing a computer to execute the method according to any one of claims 1 to 8.

Images