Method, device, electronic device and storage medium for generating face model

Abstract

The present invention discloses a method, device, electronic device and storage medium for generating a face model. The method comprises obtaining a first mixed expression model under a first specification, retopologically transferring a first target expression model in the first mixed expression model to a mixed expression model under a second specification, obtaining a second mixed expression model including an expression base model corresponding to a first identity type, splitting the second mixed expression model into a third mixed expression model under the first specification, obtaining a fourth mixed expression model corresponding to a second identity type, performing feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information, and constructing a target expression model based on the target feature information. By adopting the embodiments of the present invention, the face modeling capability of the second identity type can be improved, and the technical problem that the identity type of face data of the existing face parameterized model is relatively single can be solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device, electronic device and storage medium for generating a face model. Background Art

[0002] The current 3D facial reconstruction solution is implemented using basic facial parametric model technology. The specific implementation method is: using the facial parametric model as the standard facial model, using a single or multiple facial images, obtaining the facial parameter model coefficients of the facial image through an algorithm, and deforming the neutral face of the standard facial model to obtain the 3D facial model corresponding to the facial image. Existing commonly used facial parametric models include: ARKit, which has 61 blendshapes (expression model data); the facewarehouse facial model, which has 11,000 vertices and is composed of 47 expressions photographed by 150 models; and the FLAME facial model, which has 5,023 vertices simulated from 33,000 head data, 4 joints, and 3 PCA parameter coefficients: shape, pose, and expression.

[0003] However, ARkit lacks a facial shape dimension and is primarily used for animation-driven products, such as emoji, but cannot be used for 3D reconstruction of faces. Facewarehouse used Kinect's RGBD camera to capture 150 individuals aged 7 to 80 from different regions or ethnicities, including neutral expressions and 19 other expressions. However, the final facial model only had 47 blendshapes, demonstrating slightly weaker representation capabilities. FLAME, learned from 33,000 facial data, has stronger representation capabilities, but the identity types of the facial data are relatively single, and its ability to model faces of other identity types is poor. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, electronic device and storage medium for generating a face model to solve the technical problem that the identity type of face data in existing face parameterized models is relatively single.

[0005] In a first aspect, an embodiment of the present invention provides a method for generating a face model, comprising:

[0006] Acquire a first mixed expression model under a first specification of a preset first parameterized model, wherein the first mixed expression model includes a first preset number of expression base models under the first specification;

[0007] Retopologically repositioning the first target expression model in the first mixed expression model onto a mixed expression model under a second specification to obtain a second mixed expression model, wherein the second specification is an expression model specification of a preset second parameterized model, and the second mixed expression model includes an expression base model corresponding to the first identity type;

[0008] Splitting the second mixed expression model into the first preset number of expression base models under the first specification to obtain a third mixed expression model;

[0009] Acquire a fourth mixed expression model corresponding to the second identity type, where the fourth mixed expression model is an expression base model under the first specification, and the fourth mixed expression model includes the first preset number of expression base models;

[0010] Performing feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information corresponding to each expression base model in the third mixed expression model and the fourth mixed expression model;

[0011] A target expression model is constructed according to the target feature information, wherein the target expression model includes an expression base model corresponding to the feature information of the first identity type and the second identity type.

[0012] In a second aspect, an embodiment of the present invention provides a device for generating a face model, comprising:

[0013] A first acquisition module is configured to acquire a first mixed expression model under a first specification of a preset first parameterized model, wherein the first mixed expression model includes a first preset number of expression base models under the first specification;

[0014] a topology module, configured to retopologically retopologically transform the first target expression model in the first mixed expression model onto a mixed expression model under a second specification to obtain a second mixed expression model, wherein the second specification is an expression model specification of a preset second parameterized model, and the second mixed expression model includes an expression base model corresponding to the first identity type;

[0015] a splitting module, configured to split the second mixed expression model into the first preset number of expression base models under the first specification to obtain a third mixed expression model;

[0016] A second acquisition module is configured to acquire a fourth mixed expression model corresponding to the second identity type, wherein the fourth mixed expression model is an expression base model under the first specification, and the fourth mixed expression model includes the first preset number of expression base models;

[0017] a decomposition module, configured to perform feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information corresponding to each expression base model in the third mixed expression model and the fourth mixed expression model;

[0018] A construction module is used to construct a target expression model according to the target feature information, wherein the target expression model includes an expression base model corresponding to the feature information of the first identity type and the second identity type.

[0019] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and runnable on the processor, wherein the processor implements the steps in any one of the above-mentioned methods for generating a face model when executing the computer program.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned methods for generating a face model are implemented.

[0021] An embodiment of the present invention provides a method, device, electronic device and storage medium for generating a facial model. The method performs feature decomposition processing on a third mixed expression model corresponding to a first identity type and a fourth mixed expression model corresponding to a second identity type, thereby obtaining target feature information containing the first identity type and the second identity type, and then constructing a target expression model based on the target feature information. A facial parameterized model for the second identity type can be constructed, thereby improving the facial modeling capability of the second identity type and solving the technical problem that the identity type of facial data of the existing facial parameterized model is relatively single. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a method for generating a face model provided by an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a first target expression model provided by an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of obtaining any expression base model provided by an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of a retopology provided by an embodiment of the present invention;

[0026] Figure 5 1 is a schematic diagram of a retopology result of an expression base model of a neutral expression provided by an embodiment of the present invention;

[0027] Figure 6 This is a structural diagram of a device for generating a face model provided by an embodiment of the present invention;

[0028] Figure 7 This is another structural diagram of the device for generating a face model provided by an embodiment of the present invention;

[0029] Figure 8 This is a schematic diagram of a third structure of the apparatus for generating a face model provided by an embodiment of the present invention;

[0030] Figure 9 This is a schematic structural diagram of an electronic device provided by an embodiment of the present invention;

[0031] Figure 10 This is another structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0033] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0034] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0035] In related technologies, commonly used facial parametric models include: ARKit, which has 61 blendshapes (expression model data); the facewarehouse face model, which has 11,000 vertices and is composed of 47 expressions photographed by 150 models; the FLAME face model, which has 5,023 vertices simulated from 33,000 head data, 4 joints, and 3 PCA parameter coefficients of shape, pose, and expression.

[0036] However, ARkit lacks a facial shape dimension and is primarily used for animation-driven products, such as emoji, but cannot be used for 3D reconstruction of faces. Facewarehouse used Kinect's RGBD camera to capture 150 individuals aged 7 to 80 from different regions or ethnicities, including neutral expressions and 19 other expressions. However, the final facial model only had 47 blendshapes, demonstrating slightly weaker representation capabilities. FLAME, learned from 33,000 facial data, has stronger representation capabilities, but the identity types of the facial data are relatively single, and its ability to model faces of other identity types is poor.

[0037] In order to solve the technical problems existing in the related art, the embodiment of the present invention provides a method for generating a face model, see Figure 1 , Figure 1 1 is a flow chart of a method for generating a face model provided by an embodiment of the present invention, the method comprising steps 101 to 106;

[0038] Step 101: Acquire a first mixed expression model under a first specification of a preset first parameterized model, where the first mixed expression model includes a first preset number of expression base models under the first specification.

[0039] In this embodiment, the first parameterized model provided in this embodiment can be a high-precision parameterized facial model. For example, the first parameterized model provided in this embodiment can be a parameterized model of Metahuman lod1. Adopting the expression model specification of the Metahuman parameterized model, i.e., the first specification, can shorten the time required for subsequent application development. Furthermore, the larger number of vertices in lod1 compared to the most precise lod0 facial model can significantly reduce computational costs during facial reconstruction while also providing higher accuracy.

[0040] In this embodiment, a Metahuman digital human is randomly selected as the first parametric model, and expression base models corresponding to multiple different expressions are derived from the first parametric model to obtain a first mixed expression model. Specifically, this embodiment can use Bridge software to export data supported by Maya, and then derive the corresponding first mixed expression model from the data supported by Maya.

[0041] As an optional embodiment, the first preset number provided in this embodiment may be any number of expression base models of different expressions included in the first parameterized model. For example, the first preset number provided in this embodiment may be 218.

[0042] Step 102, retopology the first target expression model in the first mixed expression model to the mixed expression model under the second specification to obtain a second mixed expression model, the second specification is the expression model specification of the preset second parameterized model, and the second mixed expression model includes the expression base model corresponding to the first identity type.

[0043] In this example, see Figure 2 , Figure 2 is a schematic diagram of the first target expression model provided by an embodiment of the present invention. Specifically, Figure 2 As shown, the first target expression model provided in this embodiment is an expression model corresponding to the neutral expression in the first mixed expression model, that is, an expression model corresponding to the neutral expression without any expression.

[0044] For the first mixed expression model, it can be represented by A=[A0,A1,…Ai,…An], where A0 is the neutral expression in the first mixed expression model, i.e., the first target expression model, and Ai represents the offset relative to the neutral expression. For any expression base model with an expression, it can be obtained by adding the offset to the first target expression model corresponding to the neutral expression. For details, see Figure 3 , Figure 3 This is a schematic diagram of obtaining any expression base model provided by an embodiment of the present invention, such as Figure 3 As shown, by adding the offset Ai of the expression required by the user to the first expression model of the neutral expression, the expression base model corresponding to the expression required by the user can be obtained.

[0045] In this embodiment, the second parameterized model may be a parameterized model containing expression base models corresponding to multiple different identity types. For example, the second parameterized model may be a Facewarehouse parameterized model. The Facewarehouse parameterized model is constructed by collecting data from 150 individuals aged 7 to 80 from different regions or ethnic groups (i.e., the first identity type). Specifically, the data is collected using a Kinect RGBD camera to capture these individuals' neutral expressions and 19 other expressions, thereby constructing a Facewarehouse parameterized model containing expression base models corresponding to multiple individuals of different first identity types.

[0046] Specifically, the step provided in this embodiment of retopologically ...

[0047] Among them, the second target expression model provided by this embodiment is a second parameterized model, that is, the expression base model corresponding to the neutral expression in the Facewarehouse parameterized model. The expression key points provided by this embodiment can be facial key points on the expression base model used to characterize facial expressions, such as facial key points such as the corners of the eyes, the corners of the nose, and the corners of the mouth. This embodiment calculates the first mapping relationship between the first expression key point and the second expression key point, which can be determined by calculating the offset between the first expression key point and the second expression key point that represent the same expression key point. In this way, the first expression key point in the first target expression model can be deformed to the position of the second expression key point through the determined first mapping relationship, thereby completing the process of retopologically retopologically transferring the first target expression model to the mixed expression model under the second specification to obtain the second mixed expression model.

[0048] In some embodiments, see Figure 4 and Figure 5 , Figure 4 This is a schematic diagram of a retopology provided by an embodiment of the present invention. Figure 5 FIG. 1 is a schematic diagram of the retopology result of the neutral expression base model provided by an embodiment of the present invention. Figure 4 and Figure 5 As shown, the specific method provided by this implementation for retopologically ... Figure 4 The points on the expression base model on the right side of the middle are used to calculate the expression key points on the Facewarehouse parameterized model and the corresponding expression key points on the expression base model of the neutral expression of the Metahuman parameterized model (i.e. Figure 4 The mapping relationship between the points on the expression base model on the left); secondly, the deformation algorithm is used to deform the expression base model of Metahuman's neutral expression into a Facewarehouse parameterized model, and the following is obtained: Figure 5The neutral expression base model is shown; and then the expression base models corresponding to other expressions are obtained in the same manner, thereby obtaining multiple Metahuman lod1 expression base models of 150 individuals (ie, the first identity type), which is the second mixed expression model provided in this embodiment.

[0049] The deformation algorithm provided in this embodiment may be a mesh deformation transfer algorithm, or other algorithms capable of deforming expression key points on an expression base model, which are not listed here one by one.

[0050] Step 103 : split the second mixed expression model into a first preset number of expression base models under the first specification to obtain a third mixed expression model.

[0051] In this embodiment, after obtaining the second mixed expression model, to facilitate editing or adjusting the expression model in the same software, such as Maya, this embodiment further requires converting the second mixed expression model into expression base models containing a predetermined number of expressions under the first specification. Specifically, this embodiment can utilize a splitting algorithm to split the multiple expression base models in the second mixed expression model into 218 Metahuman-defined expressions, thereby obtaining a third mixed expression model containing the first predetermined number, i.e., 218 expression base models.

[0052] The splitting algorithm provided in this embodiment may be an example based facial rigging algorithm, or other algorithms capable of performing expression splitting on an expression base model, which are not listed here one by one.

[0053] Step 104 : Acquire a fourth mixed expression model corresponding to the second identity type, where the fourth mixed expression model is an expression base model under the first specification, and includes a first preset number of expression base models.

[0054] In this embodiment, to address the technical issue of existing facial parametric models with relatively single identity types in facial data, this embodiment provides a fourth mixed expression model corresponding to a second identity type. The second identity type provided in this embodiment can be the identity type required by the user. Specifically, the steps for obtaining the fourth mixed expression model corresponding to the second identity type provided in this embodiment may include: collecting multiple expression images of the second identity type; performing three-dimensional reconstruction based on the expression images to obtain a mixed expression model corresponding to the second identity type; and retopologically repositioning the first target expression model onto the mixed expression model corresponding to the second identity type to obtain the fourth mixed expression model.

[0055] Specifically, this embodiment uses a 4D data acquisition system constructed from nine high-speed industrial cameras to capture 200 individuals, including neutral expressions and a preset number of other expressions, for example, 20. Reality capture software is used to perform a 3D reconstruction of the nine facial expressions of each individual. Wrap4D software is then used to retopologize Metahuman's neutral expressions onto the 21 expressions of the 200 individuals, resulting in 21 Metahuman lod1 expression base models for the 200 individuals. Subsequently, to facilitate editing or adjusting the expression models within the same software, such as Maya, this embodiment also requires converting the 21 Metahuman lod1 expression base models for the 200 individuals into 218 expression base models based on the first specification, thereby generating a fourth hybrid expression model. Specifically, the example-based facial rigging algorithm can also be used to split the 21 expression base models into 218 Metahuman-defined expressions, thereby generating the fourth hybrid expression model.

[0056] As an optional embodiment, in order to improve the accuracy of the expression base model in the fourth mixed expression model, the step of performing three-dimensional reconstruction based on the expression picture to obtain a mixed expression model corresponding to the second identity type provided in this embodiment may include: performing three-dimensional reconstruction based on the expression picture to obtain an initial mixed expression model; removing the expression base models that do not correspond to the expression picture in the initial mixed expression model to obtain a mixed expression model corresponding to the second identity type.

[0057] By removing the expression base models that do not correspond to the expression pictures in the initial mixed expression model obtained by three-dimensional reconstruction, that is, removing the expression base models that are not the specified expressions, the accuracy of the expression base models in the fourth mixed expression model can be effectively improved, thereby improving the accuracy of the subsequent construction of the final target expression model.

[0058] Step 105 : performing feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information corresponding to each expression base model in the third mixed expression model and the fourth mixed expression model.

[0059] In this embodiment, all the expression base models in the third mixed expression model and the fourth mixed expression model are first mixed, that is, the expression base models corresponding to the 150 individuals representing the first identity type and the 200 individuals representing the second identity type are mixed, and then the mixed expression base models are subjected to feature decomposition processing to obtain the target feature information corresponding to the mixed expression base models.

[0060] Among them, the target feature information provided by this embodiment includes identity feature information corresponding to the first identity type and the second identity type, and also includes expression feature information of expressions corresponding to all individuals. The feature classification processing provided by this embodiment can be performed using matrix singular value decomposition (SVD). Specifically, the 218 Metahuman expression base models of 350 individuals in the third mixed expression model and the fourth mixed expression model can be first assembled into a data tensor with a rank of 3 (7556 vertices * 350 identity types * 218 expressions), and then SVD is used to decompose the identity feature information and expression feature information for the subsequent construction of the target expression model.

[0061] Step 106 : constructing a target expression model according to the target feature information, wherein the target expression model includes an expression base model corresponding to the feature information of the first identity type and the second identity type.

[0062] In this embodiment, after obtaining the identity feature information corresponding to the first identity type and the second identity type, and the target feature information of the expression feature information, a target expression model with diversified identity types of the facial data can be constructed, thereby solving the technical problem that the identity type of the facial data of the existing facial parametric model is relatively single.

[0063] In some embodiments, to increase the versatility of the target expression model provided by this embodiment, after constructing the target expression model based on target feature information, the method for generating a facial model provided by this embodiment may further include: providing multiple expression controllers on the target expression model; and establishing a second mapping relationship between the expression controllers and each expression base model in the target expression model to drive the target expression model according to the expression controllers. Specifically, this embodiment may construct 116 controllers on the target expression model and establish a second mapping relationship between the controllers and 218 expressions. By dragging the controllers in any Unreal Engine, the 218 expression coefficients can be obtained from the second mapping relationship, thereby providing convenience for the annotation of expression data.

[0064] As an optional embodiment, in order to provide a target expression model with high-precision facial texture, after the step of constructing a target expression model based on target feature information, the facial model generation method provided in this embodiment may also include: obtaining high-precision facial model data, the facial model data including a high-precision texture map of the facial model; replacing the texture map onto the target expression model to obtain a target expression model containing a high-precision texture.

[0065] By replacing the texture map in the acquired high-precision face model data onto the target expression model, the target expression model can have high-precision face texture, further improving the accuracy of the target expression model provided by this embodiment.

[0066] In some embodiments, the step of replacing the texture map onto the target expression model to obtain the target expression model containing high-precision texture provided in this embodiment may include: determining a third mapping relationship between the vertices of the face model and the target expression model; according to the third mapping relationship, scaling the texture map onto the target expression model to obtain the target expression model containing high-precision texture.

[0067] Specifically, by determining the vertices of the face model and the target expression model, and determining the third mapping relationship between the vertices of the two models, the range of the face model mapped to the point surface of the target expression model can be determined, so that the face texture corresponding to this part of the vertices can be extracted and scaled to the texture size of the target expression model to be replaced to the corresponding position of the target expression model, thereby obtaining a target expression model with high-precision texture.

[0068] In summary, an embodiment of the present invention provides a method for generating a facial model, the method comprising obtaining a first mixed expression model under a first specification, retopologically transferring a first target expression model in the first mixed expression model to a mixed expression model under a second specification, obtaining a second mixed expression model including an expression base model corresponding to a first identity type, splitting the second mixed expression model into a third mixed expression model under the first specification, obtaining a fourth mixed expression model corresponding to a second identity type, performing feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information, and constructing a target expression model based on the target feature information. By adopting the embodiment of the present invention, the face modeling capability of the second identity type can be improved, and the technical problem that the identity type of the face data of the existing face parameterized model is relatively single can be solved.

[0069] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a device for generating a face model. The device for generating a face model can be implemented as an independent entity or integrated into an electronic device, such as a terminal. The terminal may include a mobile phone, a tablet computer, etc.

[0070] See Figure 6 , Figure 6 FIG. 1 is a structural diagram of a device for generating a face model according to an embodiment of the present invention, Figure 6 As shown, the face model generation device 600 provided by the embodiment of the present invention includes: a first acquisition module 601, a topology module 602, a splitting module 603, a second acquisition module 604, a decomposition module 605 and a construction module 606;

[0071] The first acquisition module 301 is used to acquire a first mixed expression model under a first specification of a preset first parameterized model, where the first mixed expression model includes a first preset number of expression base models under the first specification.

[0072] Topology module 602 is used to retopology the first target expression model in the first mixed expression model to the mixed expression model under the second specification to obtain a second mixed expression model. The second specification is the expression model specification of the preset second parameterized model, and the second mixed expression model includes the expression base model corresponding to the first identity type.

[0073] The splitting module 603 is configured to split the second mixed expression model into a first preset number of expression base models under a first specification to obtain a third mixed expression model.

[0074] The second acquisition module 604 is configured to acquire a fourth mixed expression model corresponding to the second identity type, where the fourth mixed expression model is an expression base model under the first specification and includes a first preset number of expression base models.

[0075] The decomposition module 605 is configured to perform feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information corresponding to each expression base model in the third mixed expression model and the fourth mixed expression model.

[0076] The construction module 606 is used to construct a target expression model according to the target feature information. The target expression model includes an expression base model corresponding to the feature information of the first identity type and the second identity type.

[0077] In some embodiments, the topology module 602 provided in this embodiment is specifically used to: determine a first expression key point on a first target expression model in a first parameterized model, and a second expression key point on a second target expression model in a preset second parameterized model; calculate a first mapping relationship between the first expression key point and the second expression key point; and according to the first mapping relationship, deform the first target expression model to a second mixed expression model under a second specification.

[0078] In some embodiments, the second acquisition module 604 provided in this embodiment is specifically used to: collect multiple expression pictures of the second identity type; perform three-dimensional reconstruction based on the expression pictures to obtain a mixed expression model corresponding to the second identity type; and retopology the first target expression model to the mixed expression model corresponding to the second identity type to obtain a fourth mixed expression model.

[0079] In some embodiments, the second acquisition module 604 provided in this embodiment is further specifically used to: perform three-dimensional reconstruction based on the expression picture to obtain an initial mixed expression model; remove the expression base model that does not correspond to the expression picture in the initial mixed expression model to obtain a mixed expression model corresponding to the second identity type.

[0080] In some embodiments, see Figure 7 , Figure 7 FIG. 1 is another structural diagram of a device for generating a face model according to an embodiment of the present invention. Figure 7 As shown, the face model generation device 600 provided by the embodiment of the present invention further includes: a setting module 607 and a building module 608;

[0081] The setting module 607 is used to set a plurality of expression controllers on the target expression model.

[0082] The establishing module 608 is used to establish a second mapping relationship between the expression controller and each expression base model in the target expression model, so as to drive the target expression model according to the expression controller.

[0083] In some embodiments, see Figure 8 , Figure 8 This is a schematic diagram of the third structure of the apparatus for generating a face model provided by an embodiment of the present invention, such as Figure 8 As shown, the face model generation device 600 provided by the embodiment of the present invention further includes: a third acquisition module 609 and a replacement module 610;

[0084] The third acquisition module 609 is configured to acquire high-precision face model data, where the face model data includes a texture map of the high-precision face model.

[0085] The replacement module 610 is used to replace the texture map onto the target expression model to obtain a target expression model with high-precision texture.

[0086] In some embodiments, the replacement module 610 provided in this embodiment is specifically used to: determine a third mapping relationship between the vertices of the face model and the target expression model; and scale the texture map to the target expression model according to the third mapping relationship to obtain a target expression model containing high-precision texture.

[0087] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also be found in the beneficial effects in the previous method embodiments, which will not be repeated here.

[0088] Also, see Figure 9 , Figure 9 This is a structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device may be a mobile terminal such as a smart phone, a tablet computer, or the like. Figure 9 As shown, the electronic device 900 includes a processor 901 and a memory 902. The processor 901 is electrically connected to the memory 902.

[0089] The processor 901 is the control center of the electronic device 900. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or loading applications stored in the memory 902 and calling data stored in the memory 902, it executes various functions of the electronic device 900 and processes data, thereby monitoring the electronic device 900 as a whole.

[0090] In this embodiment, the processor 901 in the electronic device 900 will load the instructions corresponding to the processes of one or more applications into the memory 902 according to the following steps, and the processor 901 will run the application stored in the memory 902, thereby implementing any step in the face model generation method provided in the above embodiment.

[0091] The electronic device 900 can implement the steps in any embodiment of the method for generating a face model provided in the embodiments of the present invention. Therefore, it can achieve the beneficial effects that can be achieved by any method for generating a face model provided in the embodiments of the present invention. Please refer to the previous embodiments for details and will not be repeated here.

[0092] See Figure 10 , Figure 10 is another structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 10 As shown, Figure 10 The electronic device 1000 is a mobile terminal such as a smart phone or a laptop computer.

[0093] RF circuit 1010 is used to receive and transmit electromagnetic waves, converting them into electrical signals, thereby communicating with a communication network or other devices. RF circuit 1010 may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, memory, and the like. RF circuit 1010 can communicate with various networks such as the Internet, an intranet, or a wireless network, or with other devices via a wireless network. Such wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The wireless networks may utilize various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g, and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messaging, and any other suitable communication protocols, including those currently undeveloped.

[0094] The memory 1020 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method for generating a face model in the above-mentioned embodiment. The processor 1080 executes various functional applications and generates a face model by running the software programs and modules stored in the memory 1020.

[0095] The memory 1020 may include a high-speed random access memory (RAM) and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1020 may further include a memory remotely located relative to the processor 1080, and such remote memory may be connected to the electronic device 1000 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The input unit 1030 can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control. Specifically, the input unit 1030 may include a touch-sensitive surface 1031 and other input devices 1032. The touch-sensitive surface 1031, also known as a touch display or touchpad, can detect user touch operations on or near it (for example, operations performed by a user using a finger, stylus, or any other suitable object or accessory on or near the touch-sensitive surface 1031) and drive corresponding connected devices according to a pre-set program. Optionally, the touch-sensitive surface 1031 may include a touch detection device and a touch controller. The touch detection device detects the user's touch direction and detects the signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 1080. It can also receive and execute commands from the processor 1080. In addition, the touch-sensitive surface 1031 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 1031, the input unit 1030 can also include other input devices 1032. Specifically, the other input devices 1032 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, and the like.

[0097] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device 1000. These graphical user interfaces can be composed of graphics, text, icons, videos, or any combination thereof. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like. Furthermore, the touch-sensitive surface 1031 can cover the display panel 1041. When the touch-sensitive surface 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. The processor 1080 then provides a corresponding visual output on the display panel 1041 based on the type of touch event. Although the touch-sensitive surface 1031 and the display panel 1041 are shown in the figure as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface 1031 and the display panel 1041 can be integrated to implement input and output functions.

[0098] The electronic device 1000 may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may generate an interrupt when the flip cover is closed or closed. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the electronic device 1000 can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0099] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the electronic device 1000. The audio circuit 1060 converts received audio data into electrical signals and transmits them to the speaker 1061, which then converts them into sound signals for output. Meanwhile, the microphone 1062 converts collected sound signals into electrical signals, which are then received by the audio circuit 1060 and converted into audio data. The audio data is then processed by the output processor 1080 and transmitted to, for example, another terminal via the RF circuit 1010. Alternatively, the audio data is output to the memory 1020 for further processing. The audio circuit 1060 may also include an earphone jack to allow communication between an external headset and the electronic device 1000.

[0100] The electronic device 1000 can help users receive requests, send information, etc. through a transmission module 1070 (e.g., a Wi-Fi module), providing users with wireless broadband Internet access. Although the transmission module 1070 is shown in the figure, it is understandable that it is not a required component of the electronic device 1000 and can be omitted as needed without changing the essence of the invention.

[0101] Processor 1080 is the control center of electronic device 1000. It connects all components of the phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 1020 and accessing data stored in memory 1020, it executes various functions of electronic device 1000 and processes data, thereby providing overall monitoring of the electronic device. Optionally, processor 1080 may include one or more processing cores. In some embodiments, processor 1080 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 1080.

[0102] The electronic device 1000 also includes a power supply 1090 (e.g., a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 1080 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 1090 can also include any components such as one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0103] Although not shown, the electronic device 1000 further includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal further includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured so that one or more processors execute the one or more programs to implement any step of the method for generating a face model provided in the above embodiment.

[0104] During specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments and will not be repeated here.

[0105] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be accomplished through instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium storing a plurality of instructions that, when executed by a processor, can implement any step in the method for generating a face model provided in the above embodiments.

[0106] The storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0107] Since the instructions stored in the storage medium can execute the steps in any embodiment of the method for generating a face model provided in the embodiments of the present invention, the beneficial effects that can be achieved by any method for generating a face model provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0108] The above is a detailed introduction to the method, device, electronic device, and storage medium for generating a face model provided by the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application. Moreover, for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating a face model, characterized in that: include: Acquire a first mixed expression model under a first specification of a preset first parameterized model, wherein the first mixed expression model includes a first preset number of expression base models under the first specification; Retopologically repositioning the first target expression model in the first mixed expression model onto a mixed expression model under a second specification to obtain a second mixed expression model, wherein the second specification is an expression model specification of a preset second parameterized model, and the second mixed expression model includes an expression base model corresponding to the first identity type; Splitting the second mixed expression model into the first preset number of expression base models under the first specification to obtain a third mixed expression model; Acquire a fourth mixed expression model corresponding to the second identity type, where the fourth mixed expression model is an expression base model under the first specification, and the fourth mixed expression model includes the first preset number of expression base models; Performing feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information corresponding to each expression base model in the third mixed expression model and the fourth mixed expression model; A target expression model is constructed according to the target feature information, wherein the target expression model includes an expression base model corresponding to the feature information of the first identity type and the second identity type.

2. The method according to claim 1, wherein The step of retopologically ... Determining a first expression key point on a first target expression model in the first parameterized model and a second expression key point on a second target expression model in a preset second parameterized model; Calculating a first mapping relationship between the first expression key point and the second expression key point; According to the first mapping relationship, the first target expression model is deformed into a second mixed expression model under a second specification.

3. The method according to claim 1, wherein The obtaining of the fourth mixed expression model corresponding to the second identity type includes: Collect multiple expression pictures of the second identity type; Performing three-dimensional reconstruction based on the expression picture to obtain a mixed expression model corresponding to the second identity type; The first target expression model is retopologically added to the mixed expression model corresponding to the second identity type to obtain a fourth mixed expression model.

4. The method according to claim 3, wherein The performing three-dimensional reconstruction based on the expression picture to obtain a mixed expression model corresponding to the second identity type includes: Performing three-dimensional reconstruction based on the expression picture to obtain an initial mixed expression model; The expression base models that do not correspond to the expression picture in the initial mixed expression model are removed to obtain a mixed expression model corresponding to the second identity type.

5. The method according to claim 1, wherein After the step of constructing a target expression model according to the target feature information, the method further comprises: Setting a plurality of expression controllers on the target expression model; A second mapping relationship is established between the expression controller and each expression base model in the target expression model, so as to drive the target expression model according to the expression controller.

6. The method according to claim 1, wherein After the step of constructing a target expression model according to the target feature information, the method further comprises: Acquire high-precision face model data, wherein the face model data includes a texture map of the high-precision face model; The texture map is replaced onto the target expression model to obtain a target expression model containing high-precision texture.

7. The method according to claim 6, wherein The step of replacing the texture map onto the target expression model to obtain a target expression model containing a high-precision texture comprises: Determining a third mapping relationship between vertices of the face model and the target expression model; According to the third mapping relationship, the texture map is scaled onto the target expression model to obtain a target expression model containing a high-precision texture.

8. A device for generating a face model, characterized in that: include: A first acquisition module is configured to acquire a first mixed expression model under a first specification of a preset first parameterized model, wherein the first mixed expression model includes a first preset number of expression base models under the first specification; a topology module, configured to retopologically retopologically transform the first target expression model in the first mixed expression model onto a mixed expression model under a second specification to obtain a second mixed expression model, wherein the second specification is an expression model specification of a preset second parameterized model, and the second mixed expression model includes an expression base model corresponding to the first identity type; a splitting module, configured to split the second mixed expression model into the first preset number of expression base models under the first specification to obtain a third mixed expression model; A second acquisition module is configured to acquire a fourth mixed expression model corresponding to the second identity type, wherein the fourth mixed expression model is an expression base model under the first specification, and the fourth mixed expression model includes the first preset number of expression base models; a decomposition module, configured to perform feature decomposition processing on the third mixed expression model and the fourth mixed expression model to obtain target feature information corresponding to each expression base model in the third mixed expression model and the fourth mixed expression model; A construction module is used to construct a target expression model according to the target feature information, wherein the target expression model includes an expression base model corresponding to the feature information of the first identity type and the second identity type.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 7 are implemented.

Images