Model training method, face generation method and device
Through iterative training to generate the network, the face-pinching parameters are automatically generated, which solves the problem that the 3D face model cannot be directly applied to games or metacosmic scenes in the existing technology, and realizes automatic face-pinching parameters generation, reducing the cost of manual labeling.
Patent Information
- Application Number
- CN202411036887.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to apply 3D face models directly generated by artificial intelligence models to games or metacosmic scenes, and requires a large number of artificial configuration of facial parameters.
By determining the similarity between real face images and virtual face images as a loss function, iterative training generates the network, automatically generates face pinching parameters, and then applying these parameters to generate a face pinching model suitable for 3D engines.
Without manual annotation, the cost of manual annotation is reduced, the problem that 3D face models cannot be directly applied to games or metaverse scenes is solved, and the automatic face-pinching parameter generation is realized.
Smart Images

Figure CN119006953A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, specifically to the field of computer vision technology, and especially to a model training method, a face generation method, a model training device, a face generation device, an electronic device and a medium. Background Art
[0002] Nowadays, computers can use neural network structures such as CNN to automatically generate images with new styles, and even generate 3D reconstruction results from a single facial image, thanks to the development of deep learning in recent years. However, due to problems such as model structure and art materials, direct image reconstruction of 3D models is often not directly applicable to games and metaverse scenes. Nowadays, more methods are based on changing the skeleton parameters of existing 3D models.
[0003] A standard workflow for creating 3D characters requires configuring a large number of facial parameters. The game engine takes these detailed facial parameters as input and then generates a 3D face model. Summary of the invention
[0004] In order to solve the problem of how to correspond real face images and face pinching parameters, a model training method, a face generation method, a model training device, a face generation device, an electronic device and a medium are provided.
[0005] According to a first aspect, a model training method is provided, comprising: determining a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generation network; using a face recognition network to determine the similarity between the virtual face image and the real face image, and using the similarity as a second loss function; iteratively training the face pinching parameters of the pre-trained generation network according to the second loss function to obtain training samples; and training a face pinching parameter generation network according to a training set composed of the training samples to obtain a face pinching parameter generation network.
[0006] The above model training method automatically generates a large number of true value training samples through iterative training, so that the face pinching parameter generation network can be trained without manual labeling. This method greatly reduces the cost of manual labeling.
[0007] In combination with the first aspect, in an embodiment of the present application, after using a face recognition network to determine the similarity between the virtual face image and the real face image, and using the similarity as a second loss function, the method also includes: using a face segmentation network to determine the preset parts corresponding to the real face image and the virtual face image; determining the similarity between the preset parts of the virtual face image and the preset parts of the real face image, and using the similarity as a third loss function, so as to iteratively train the face pinching parameters of the pre-trained generation network according to the second loss function and the third loss function to obtain training samples.
[0008] In combination with the first aspect, in an embodiment of the present application, the preset part includes at least one of a nose, a mouth, eyes and eyebrows.
[0009] In combination with the first aspect, in an embodiment of the present application, before determining the real face image and the virtual face image, the method also includes: pre-training the generation network; wherein the pre-training method includes: generating a face pinching effect map corresponding to random face pinching parameters based on a 3D engine; and pre-training the generation network using the face pinching parameters and the face pinching effect map.
[0010] In combination with the first aspect, in an embodiment of the present application, the loss function in the pre-training process includes a first loss function and a feature loss function.
[0011] According to a second aspect, a face generation method is provided, comprising: receiving a real face image; processing the image through a face pinching parameter generation network obtained by the model training method described in the first aspect to obtain face pinching parameters; applying the face pinching parameters to a 3D face model to generate a face.
[0012] The above-mentioned face generation method can generate face-pinching parameters based on real face images, and then apply the face-pinching parameters to the 3D face model built and rendered by the 3D engine to generate a face, thereby solving the problem that the 3D model directly generated by the artificial intelligence model cannot be directly applied to existing games or some metaverse scenes.
[0013] According to a third aspect, a model training device is provided, comprising: a first unit, used to determine a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generation network; a second unit, used to determine the similarity between the virtual face image and the real face image using a face recognition network, and use the similarity as a second loss function; a third unit, used to iteratively train the face pinching parameters of the pre-trained generation network according to the second loss function to obtain training samples; and a fourth unit, used to train a face pinching parameter generation network according to a training set composed of the training samples to obtain a face pinching parameter generation network.
[0014] According to a fourth aspect, a face pinching parameter generation device is provided, comprising: a fifth unit, for receiving a real face image; a sixth unit, for processing through a face pinching parameter generation network obtained by the model training method described in the first aspect, to obtain face pinching parameters, so as to apply the face pinching parameters to a 3D face model.
[0015] According to the fifth aspect, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method as in any one of the embodiments of the first aspect or the second aspect.
[0016] According to a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method of any embodiment of the first aspect or the second aspect is implemented.
[0017] It is understandable that the model training device provided in the third aspect, the face generation device provided in the fourth aspect, the electronic device provided in the fifth aspect, and the computer-readable storage medium provided in the sixth aspect are used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0019] Figure 1 is an exemplary system architecture diagram to which some embodiments of the present application may be applied;
[0020] Figure 2 is a flow chart of an embodiment of a model training method according to the present application;
[0021] Figure 3 is a schematic diagram of an application scenario of the model training method according to the present application;
[0022] Figure 4 is a flow chart of an embodiment of a face generation method according to the present application;
[0023] Figure 5 is a structural schematic diagram of an embodiment of a model training device according to the present application;
[0024] Figure 6 is a structural schematic diagram of an embodiment of a face generation device according to the present application;
[0025] Figure 7It is a block diagram of an electronic device used to implement the model training method or face generation method of the embodiment of the present application. DETAILED DESCRIPTION
[0026] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0028] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the model training method, model training device or face generation method, face generation device of the present application can be applied.
[0029] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0030] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as video applications, live broadcast applications, instant messaging tools, email clients, social platform software, etc.
[0031] The terminal devices 101, 102, 103 here can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.
[0032] The server 105 may be a server that provides various services, such as a background server that provides support for the terminal devices 101, 102, and 103. The background server may analyze and process the received data such as real face images, and feed back the processing results to the terminal device.
[0033] It should be noted that the model training method or face pinching parameter generation method provided in the embodiments of the present application can be executed by the server 105 or the terminal devices 101, 102, 103. Accordingly, the model training device or the face pinching parameter generation device can be set in the server 105 or the terminal devices 101, 102, 103.
[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0035] Continue to refer Figure 2 , shows a process 200 according to an embodiment of a model training method of the present application. The model training method comprises the following steps:
[0036] Step 201, determining a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generation network.
[0037] In this embodiment, the execution entity (e.g. Figure 1 The server or terminal device shown in the figure can generate a virtual face image by calling a pre-trained generation network. The generation network can be an upsampling encoding network.
[0038] In some embodiments, the dimension of the face pinching parameters is 90, and it is upsampled layer by layer through 8 stages in the generative network G(x). Each stage includes a transposed convolution layer (convtranspose2d), a batch normalization layer (bn), and an activation layer (re lu), and finally outputs a 3-channel 512*512 virtual face image.
[0039] The specific process of pre-training the generation network includes: generating a face pinching effect map corresponding to the random face pinching parameters based on a 3D engine (such as unity); and pre-training the generation network using the face pinching parameters and the face pinching effect map.
[0040] During the pre-training process, the first loss function Loss1 is used to constrain the ambiguity of the result.
[0041] Loss1=argmin(G(x)-Engi ne(x))
[0042] Among them, G(x) represents the generation network that inputs face pinching parameters and outputs face images, and Eng i ne(x) represents the process of inputting face pinching parameters and rendering images on the Unity platform.
[0043] At the same time, in the process of pre-training, in order to ensure the consistency of the local content of the generated results, the vgg16 perceptual loss calculation (Perceptua l Losses) is introduced, and the intermediate features are taken as the loss function. The l of the feature loss function feat The formula is as follows:
[0044]
[0045] Among them, y^ is the face generated by G(x), y is the face rendered by Eng i ne(x), j is the middle layer, C j H j W j Represents the feature map size of the jth layer.
[0046] Step 202: Use a face recognition network to determine the similarity between the virtual face image and the real face image, and use the similarity as a second loss function.
[0047] In this embodiment, the execution subject uses the face recognition network to extract 256-dimensional face embedding representations of the real face image and the virtual face image respectively. Then the face embedding representation is used to calculate the cosine distance between the real face image and the virtual face image as the similarity. And the cosine distance is used as the second loss function Loss2.
[0048] Loss2=cos(Fe(y^),Fe(yi))
[0049] Among them, Fe represents the extraction of 256-dimensional embedded features, y^ represents the face generated by G(x), and yi represents the real face.
[0050] Step 203: Iteratively train the pre-trained generation network using the second loss function to obtain a sample generation network.
[0051] In this embodiment, the above-mentioned execution subject uses the second loss function to iteratively train the face pinching parameters of the pre-trained generation network to obtain a sample generation network.
[0052] In some other embodiments of the present application, the model training method also includes iteratively training the generation network through a second loss function and a third loss function, and the step of determining the third loss function includes: determining the preset parts corresponding to the real face image and the virtual face image; using the face segmentation network to determine the similarity between the preset parts of the virtual face image and the preset parts of the real face image, and using the similarity as the third loss function, so as to iteratively train the pre-trained generation network according to the second loss function and the third loss function to obtain the generation network.
[0053] In some examples, the preset parts may include a nose, a mouth, eyes, and eyebrows.
[0054] In some better examples, the weight distribution ratios of nose, mouth, eyes and eyebrows are 0.3, 0.3, 0.2, and 0.2 respectively.
[0055] In the example of the present application, the above-mentioned execution subject respectively segments the preset parts of the real face image and the virtual face image through the face segmentation network, and then calculates the similarity in each corresponding segmented area, and uses the similarity as the third loss function. After the second loss function and the third loss function are determined, the face pinching parameter is used as the iteration object to iteratively train the generation network to obtain the true value image corresponding to the face pinching parameter, and the face pinching image and its corresponding true value sample are used as a set of training samples.
[0056] Step 204: training the face pinching parameter generation network according to the training set composed of the training samples to obtain the face pinching parameter generation network.
[0057] In this embodiment, the execution subject obtains multiple training samples by performing steps 201, 202 and 203 for multiple times, thereby forming a training set. The training set is then used to train the face pinching parameter generation network.
[0058] In some examples, the face pinching parameter generation network is a downsampling encoding network.
[0059] The model training method provided in the above-mentioned embodiment of the present application automatically generates a large number of true value training samples through iterative training, so that the face pinching parameter generation network can be trained without manual labeling. This method greatly reduces the cost of manual labeling.
[0060] Continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an application scenario of the model training method according to this embodiment. Figure 3In the application scenario, the execution subject 301 uses a neural network to process the real face image 302 and the virtual face image to obtain a second loss function and a third loss function 303, and then uses the second loss function and the third loss function 303 to iterate the face pinching parameters of the generation network to obtain a training sample 304. Then, the face pinching parameter generation network is trained through the training samples, so that the face pinching parameter generation network after training can generate corresponding face pinching parameters according to the real face image.
[0061] Further references Figure 4 , which shows a process 400 of an embodiment of the face generation method of the present application. The process 400 includes the following steps:
[0062] Step 401, receiving a real face image.
[0063] In this embodiment, the execution subject (eg Figure 1 The server or terminal device shown in the figure can receive a real face image input by a user, such as a photo, and then pre-process the photo by aligning, rotating, cropping, etc., so as to improve the accuracy of the subsequent face pinching parameter generation model.
[0064] Step 402, the face pinching parameters are generated by the face pinching parameter generation network obtained by the model training method and processed to obtain the face pinching parameters.
[0065] It can be understood that the training method of the face pinching parameter generation network can refer to step 201, step 202, step 203 and step 204, which will not be repeated here.
[0066] Step 403: Apply the face pinching parameters to the 3D face model to generate a face.
[0067] In this embodiment, the 3D face model can be a game engine, such as Unity, UnrealEngine, CryEngine, etc. This application does not impose any special restrictions on this.
[0068] The above-mentioned face generation method can generate face-pinching parameters based on real face images, and then apply the face-pinching parameters to the 3D face model built and rendered by the 3D engine to generate a face, thereby solving the problem that the 3D model directly generated by the artificial intelligence model cannot be directly applied to existing games or metaverse scenes.
[0069] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a model training device, which is similar to Figure 2 Corresponding to the method embodiment shown in FIG. 1 , in addition to the features described below, the device embodiment may also include Figure 2 The device can be applied to various electronic devices.
[0070] like Figure 5 As shown, the model training device 500 of this embodiment includes: a first unit 501, used to determine a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generation network; a second unit 502, used to determine the similarity between the virtual face image and the real face image by using a face recognition network, and use the similarity as a second loss function; a third unit 503, used to iteratively train the face pinching parameters of the pre-trained generation network according to the second loss function to obtain training samples; a fourth unit 504, used to train the face pinching parameter generation network according to the training set composed of the training samples to obtain the face pinching parameter generation network.
[0071] In this embodiment, the specific processing of the model training device 500 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of the steps in the corresponding embodiments are not repeated here.
[0072] In some optional implementations of the present embodiment, the model training device 500 also includes an eighth unit, which is used to use a face segmentation network to determine the preset parts corresponding to the real face image and the virtual face image; determine the similarity between the preset parts of the virtual face image and the preset parts of the real face image, and use the similarity as a third loss function to iteratively train the face pinching parameters of the pre-trained generation network according to the second loss function and the third loss function to obtain training samples.
[0073] In some optional implementations of this embodiment, the preset part includes at least one of a nose, a mouth, eyes, and eyebrows.
[0074] In some optional implementations of this embodiment, the model training device 500 also includes a ninth unit for pre-training the generation network; wherein the pre-training method includes: generating a face pinching effect map corresponding to the random face pinching parameters based on a 3D engine; and pre-training the generation network using the face pinching parameters and the face pinching effect map.
[0075] In some optional implementations of this embodiment, the loss function in the pre-training process includes a first loss function and a feature loss function.
[0076] Further references Figure 6 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a face generation device, which is similar to Figure 4Corresponding to the method embodiment shown in FIG. 1 , in addition to the features described below, the device embodiment may also include Figure 4 The device can be applied to various electronic devices.
[0077] like Figure 6 As shown, the model training device 600 of this embodiment includes: a fifth unit 601, used to receive a real face image; a sixth unit 602, used to process through a face pinching parameter generation network obtained according to a model training method to obtain face pinching parameters; a seventh unit 603, used to apply the face pinching parameters to a 3D face model to generate a face.
[0078] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
[0079] like Figure 7 , is a block diagram of an electronic device according to a model training method or a face generation method according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0080] like Figure 7 As shown, the electronic device includes: one or more processors 701, memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as required. The processor can process the instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 701 is taken as an example.
[0081] The memory 702 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to execute the model training method or face generation method provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to execute the model training method or face generation method provided in the present application.
[0082] The memory 702, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the model training method or face generation method in the embodiments of the present application. The processor 701 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 702, that is, implementing the model training method or face generation method in the above method embodiments.
[0083] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created for use by the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0084] The electronic device of the model training method or face generation method may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.
[0085] The input device 703 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 704 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.
[0086] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0087] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0089] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0090] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0091] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0092] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor.
[0093] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device: determines a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generation network; determines the similarity between the virtual face image and the real face image using a face recognition network, and uses the similarity as a second loss function; iteratively trains the face pinching parameters of the pre-trained generation network according to the second loss function to obtain training samples; trains the face pinching parameter generation network according to the training set composed of the training samples to obtain the face pinching parameter generation network. Alternatively, receive a real face image; process the face pinching parameter generation network obtained by the model training method to obtain face pinching parameters; apply the face pinching parameters to a 3D face model to generate a face.
[0094] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.
Claims
1. A model training method, comprising: Determining a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generative network; Determine the similarity between the virtual face image and the real face image using a face recognition network, and use the similarity as a second loss function; Iteratively training the face pinching parameters of the pre-trained generation network according to the second loss function to obtain training samples; The face pinching parameter generation network is trained according to the training set composed of the training samples to obtain the face pinching parameter generation network.
2. The method according to claim 1, wherein: After determining the similarity between the virtual face image and the real face image using a face recognition network and using the similarity as a second loss function, the method further includes: Determine the preset parts corresponding to the real face image and the virtual face image using a face segmentation network; Determine the similarity between the preset part of the virtual face image and the preset part of the real face image, and use the similarity as the third loss function to iteratively train the face pinching parameters of the pre-trained generation network according to the second loss function and the third loss function to obtain training samples.
3. The method according to claim 2, wherein: The preset parts include at least one of a nose, a mouth, eyes and eyebrows.
4. The method according to claim 1, wherein: Before determining the real face image and the virtual face image, the method further includes: Pre-training the generating network; wherein the pre-training method includes: Generate face pinching effect pictures corresponding to random face pinching parameters based on 3D engine; The generation network is pre-trained using the face pinching parameters and the face pinching effect graph.
5. The method according to claim 4, wherein: The loss function in the pre-training process includes the first loss function and the feature loss function.
6. A method for generating a human face, comprising: Receiving a real face image; Processing the face pinching parameter generation network obtained by the model training method according to any one of claims 1 to 5 to obtain the face pinching parameters; The face pinching parameters are applied to the 3D face model to generate a face.
7. A model training device, comprising: A first unit is used to determine a real face image and a virtual face image, wherein the virtual face image is generated by inputting a set of random face pinching parameters into a pre-trained generation network; A second unit is used to determine the similarity between the virtual face image and the real face image by using a face recognition network, and use the similarity as a second loss function; A third unit is used to iteratively train the face pinching parameters of the pre-trained generation network according to the second loss function to obtain training samples; The fourth unit is used to train the face pinching parameter generation network according to the training set composed of the training samples to obtain the face pinching parameter generation network.
8. A face generation device, comprising: The fifth unit is used for receiving a real face image; A sixth unit is configured to process the face pinching parameter generation network obtained by the model training method according to any one of claims 1 to 5 to obtain the face pinching parameter; The seventh unit is used to apply the face pinching parameters to the 3D face model to generate a face.
9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5 or the method according to claim 6.
10. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 or the method according to claim 6 is implemented.
Citation Information
Cited By
3D digital human intelligent face pinching system based on photos
CN119810281A
