Facial expression model generation method and device, electronic device, and storage medium

Abstract

The application provides a facial expression model generation method and device, an electronic device and a storage medium. The method comprises the following steps: determining an expressionless character model of a target character; determining a target expressionless reference model of a target reference character and a plurality of target expression reference models of different expressions of the target reference character from a model library according to facial features of the expressionless character model; and generating a plurality of expression character models of different expressions corresponding to the expressionless character model according to morphing data between each target expression reference model and the target expressionless reference model. The method can automatically generate a plurality of expression character models of different expressions of the target character, improve the generation efficiency of the expression character model, and determine the corresponding target expressionless reference model through the facial features of the expressionless character model, thereby improving the accuracy of the expression character model in expressing the expression and improving the generation effect of the expression character model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods and apparatus for generating facial expression models, electronic devices, and storage media. Background Technology

[0002] Blend Shape is a commonly used method for compositing facial expressions. It's a technique that combines the deformation of a single mesh into a combination of many predefined shapes. Each predefined shape is a deformation of a circle, and the mesh for each deformation is stored as a series of vertex positions. Interpolation between different vertex positions yields the deformation effect. Therefore, each deformation typically represents an extreme case of that deformation.

[0003] Blend Shape records the displacement of each vertex on the model. Therefore, in the process of creating facial expression models, each time a facial expression model is created, a copy of the expressionless source model is needed. Then, based on the copied source model, modifications are made according to the desired expression to obtain the facial expression model of the source model. Finally, the facial expression model is added as the deformation target of the source model. After that, adjusting the blending parameters can achieve deformation between the source model and the deformation target.

[0004] Nowadays, the quality requirements for games, animation, and film are getting higher and higher, and animated characters need to be more diverse, that is, more characters and more expressions are required. The existing methods of creating facial expression models require artists to repeatedly copy the expressionless source models of each character and manually adjust the copied source models to obtain the corresponding facial expression models. This process is labor-intensive and inefficient.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] In view of the above problems, this application is made to provide a method, apparatus, electronic device, and storage medium for generating facial expression models that overcomes or at least partially solves the above problems, including:

[0007] A method for generating an facial expression model, the method comprising:

[0008] Determine the expressionless character model of the target character;

[0009] Based on the facial features of the expressionless character model, a target expressionless reference model of a target reference character matching the facial features and multiple target expression reference models with different expressions are determined from the model library.

[0010] Based on the deformation data between each target facial expression reference model and the target expressionless reference model, multiple facial expression character models with different expressions are generated corresponding to the expressionless character model.

[0011] An apparatus for generating facial expression models, the apparatus comprising:

[0012] The target character determination module is used to determine the expressionless character model of the target character;

[0013] The reference character determination module is used to determine, from the model library, a target expressionless reference model of a target reference character that matches the facial features of the expressionless character model, and multiple target expression reference models with different expressions;

[0014] The expression model generation module is used to generate multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model.

[0015] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method for generating an expression model as described above.

[0016] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for generating an expression model as described above.

[0017] This application has the following advantages:

[0018] In this embodiment, a blank expression character model of the target character is determined; based on the facial features of the blank expression character model, a target blank expression reference model of the target reference character matching the facial features and multiple target expression reference models with different expressions are determined from the model library; based on the deformation data between each target expression reference model and the target blank expression reference model, multiple expression character models with different expressions corresponding to the blank expression character model are generated; this can achieve automated generation of multiple expression character models with different expressions of the target character, improving the generation efficiency of expression character models; furthermore, determining the corresponding target blank expression reference model through the facial features of the blank expression character model can improve the accuracy of the generated expression character model in expressing expressions, thus improving the generation effect of the expression character model. Attached Figure Description

[0019] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figures 1 to 4 This is a schematic diagram of the existing Blend Shape manufacturing process;

[0021] Figure 5 This is a flowchart illustrating the steps of a method for generating an expression model according to an embodiment of this application.

[0022] Figure 6 This is a schematic diagram of the target expressionless reference model in an example of Embodiment 1 of this application;

[0023] Figure 7 This is a schematic diagram of the target facial expression reference model in an example of Embodiment 1 of this application;

[0024] Figure 8 This is a schematic diagram of an expressionless character model in an example of Embodiment 1 of this application;

[0025] Figure 9 This is a schematic diagram of an expression character model in an example of Embodiment 1 of this application;

[0026] Figure 10 This is a flowchart illustrating the steps involved in generating an facial expression character model in an example of Embodiment 1 of this application.

[0027] Figure 11 This is a structural block diagram of an expression model generation device according to an embodiment of this application. Detailed Implementation

[0028] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0029] Blend Shape is a technique that transforms a single mesh into combinations of many predefined shapes. For example... Figure 1 As shown, Figure 1 It includes a source model and two deformation targets, which are described below. Figure 1 This example illustrates the basic workflow of creating a Blend Shape. First, a source model A needs to be created. Then, source model A is copied, and it is manually adjusted according to requirements to obtain the desired result. Figure 1 The deformed target 1 and deformed target 2 are shown; for example... Figure 2 As shown, with source model A (i.e., pCube1) selected, click the Create Deformation button in the Deformation Editor to create a Blend Shape for source model A; and so on. Figure 3 As shown, with deformation target 1 (i.e., pCube2) and deformation target 2 (i.e., pCube3) selected, click the "Add Target" button to add deformation target 1 and deformation target 2 to the created Blend Shape. At this point, the Blend Shape has stored the displacement of each vertex of the original model A relative to deformation target 1 and deformation target 2, respectively. Then you can proceed as follows... Figure 4 The deformation between the source model A and deformation target 1 and deformation target 2 is achieved by controlling the mixed weights.

[0030] It is evident that existing technologies using Blend Shape to create deformable models require artists to pre-create the source models and all deformable targets. Taking games as an example, games typically have a large number of characters and numerous facial expressions. Generally, different characters share the same facial expression models; for instance, character A has expressions like laughing, closing eyes, and opening mouth, while character B also has these expressions. Using existing Blend Shape technology, artists must manually create the source models and facial expression models for all characters, resulting in a large workload and low efficiency.

[0031] In view of this, this application provides a method for generating expression models. After determining the expressionless character model of the target character to be generated, a target expressionless reference model that matches the facial features of the expressionless character model is determined from the model library, as well as multiple target expression reference models with different expressions corresponding to the target expressionless reference model. Then, based on the deformation data between each target expression reference model and the target expressionless reference model, multiple expression character models with different expressions of the expressionless character model are automatically generated. This can improve the generation efficiency and effect of expression character models.

[0032] Reference Figure 5 The diagram illustrates a flowchart of a method for generating an facial expression model according to an embodiment of this application. In this embodiment, the method may include the following steps:

[0033] Step 501: Determine the expressionless character model of the target character.

[0034] In this embodiment of the application, the target character can refer to a character in the game whose facial expression model is to be generated. The expressionless model can be considered as the source model designed by the artist for the target character. For ease of distinction, the expressionless model of the target character is referred to as the expressionless character model, and the facial expression model of the target character is referred to as the facial expression character model.

[0035] Step 502: Based on the facial features of the expressionless character model, determine from the model library the target expressionless reference model of the target reference character that matches the facial features, as well as multiple target expression reference models with different expressions.

[0036] A reference character can be a character in the same game that differs from the target character and has already had its expressionless and facial models generated, or it can be a character in another game whose expressionless and facial models have already been generated. Furthermore, the reference character's topological structure is identical to the target character's; that is, the reference character's model and the target character's model have the same number of vertices, and the connections between the vertices are the same. For ease of distinction, the expressionless model of the reference character is denoted as the expressionless reference model, and the facial model of the reference character is denoted as the facial reference model.

[0037] The model library stores at least one reference model group for a reference character. Each reference model group includes a neutral expression reference model and multiple expression reference models for different characters. It should be noted that the expression model in this embodiment may or may not be a specific model structure, but rather represented by the expression name and corresponding deformation data; this deformation data is the deformation data generated when the neutral expression reference model is transformed into the corresponding expression reference model. The reference model groups stored in the model library can be manually imported by the user, for example, by triggering an import control in the interface and selecting a pre-made reference model group to import into the model library. The reference model groups stored in the model library can also include reference model groups automatically imported through automation, for example, after generating an expression model related to a character, automatically importing the character's expression model and neutral expression model into the model library to enrich the model library.

[0038] The facial features of a neutral character model can be used to represent the facial features of a target character. A target reference character can be considered a reference character whose facial features match those of the target character; that is, the facial features of the neutral character model of the target reference character match the facial features of the neutral character model of the target character. Determining the target reference model group through facial feature matching can improve the effectiveness of subsequent automatic generation of the target character's facial expression model.

[0039] Step 503: Based on the deformation data between each target expression reference model and the target expressionless reference model, generate multiple expression character models with different expressions corresponding to the expressionless character model.

[0040] The target facial expression reference model can be considered as obtained by deformation of the target expressionless reference model. Therefore, there is deformation data between each target facial expression reference model and the target expressionless reference model. This deformation data can include the displacement data between each vertex of the target facial expression reference model and the corresponding vertex of the target expressionless reference model; or, it can also include the displacement data between the feature vertices of the target facial expression reference model and the corresponding feature vertices of the target expressionless reference model. The number of feature vertices of the model is less than the total number of vertices of the model.

[0041] Since the facial features of the expressionless character model match those of the target expressionless reference model, after obtaining the deformation data corresponding to the target expression reference model, the vertices or feature vertices of the expressionless character model can be moved according to the deformation data to automatically transform the expressionless character model into the corresponding expression character model.

[0042] This application embodiment determines a blank expression character model of a target character; based on the facial features of the blank expression character model, it determines a target blank expression reference model of a target reference character matching the facial features and multiple target expression reference models with different expressions from a model library; based on the deformation data between each target expression reference model and the target blank expression reference model, it generates multiple expression character models with different expressions corresponding to the blank expression character model; this can achieve automated generation of multiple expression character models with different expressions of a target character, improving the generation efficiency of expression character models; furthermore, by determining the corresponding target reference model group through the facial features of the blank expression character model, it can improve the accuracy of the generated expression character model in expressing expressions, thus improving the generation effect of the expression character model.

[0043] The method for generating the facial expression model in this exemplary embodiment will be further explained below.

[0044] In step 501, the expressionless character model of the target character is determined.

[0045] Taking games as an example, in the game design stage, it is necessary to design and create an expressionless model for each character in the game, and then use the expression model generation method provided in this application to automatically generate the expression models of each character.

[0046] The target character can refer to a character in the game whose emoticon model needs to be generated. When there are multiple characters whose emoticon models need to be generated, the target character whose emoticon model needs to be generated can be determined in a certain order, and the emoticon model generation method provided in this application embodiment can be executed sequentially to generate the emoticon model of the target character. Alternatively, all characters whose emoticon models need to be generated can be taken as target characters, and the emoticon model generation method provided in this application embodiment can be executed in parallel to generate the emoticon model of each target character.

[0047] In step 502, based on the facial features of the expressionless character model, a target expressionless reference model of the target reference character matching the facial features and multiple target expression reference models with different expressions are determined from the model library.

[0048] In this embodiment, the topological structure of the expressionless reference model of the reference character stored in the model library is the same as that of the target character. The facial features of the model are related to the relative positional relationships between the vertices (or feature vertices) of the model. When the facial features of two models match, it can be considered that the relative positional relationships between the vertices (or feature vertices) of the two models are the same or approximately the same.

[0049] In this embodiment, the facial features of the expressionless character model can refer to the facial features of the expressionless character model's face, used to represent the face shape of the target character. It should be noted that the face shape in this application includes the facial contour and facial features. The inventors have discovered that characters with the same or similar facial shapes have similar expressions; for example, the distance between the position of the upper eyelid in character A's closed-eye expression and the position of the upper eyelid when character A is expressionless (i.e., with eyes open normally) is D1; ​​when character B has the same facial shape as character A, the distance between the position of the upper eyelid in character B's closed-eye expression and the position of the upper eyelid when character B is expressionless (i.e., with eyes open normally) is also D1.

[0050] This application embodiment determines the target expressionless reference model of the target reference character by using the facial features of the expressionless character model of the target character, and then determines the target expression reference models of multiple different expressions of the target reference character, so that the effect of the generated expression model is close to the ideal effect.

[0051] In an optional embodiment of this application, the above-mentioned determination of a target expressionless reference model for a target reference character matching the facial features of the expressionless character model from a model library may include:

[0052] Determine the character distance between each character vertex of the expressionless character model and the character center of the expressionless character model, and the reference distance between each reference vertex of the expressionless reference model of each reference character in the model library and the reference center of the corresponding expressionless reference model.

[0053] The scaling factor of the corresponding expressionless reference model is determined based on the character distance and the reference distance, and the corresponding expressionless reference model is scaled according to the scaling factor to obtain the scaled expressionless reference model.

[0054] Determine the values ​​of the affine transformation matrices between the expressionless character model and each scaled expressionless reference model;

[0055] The minimum value is determined from the multiple values, and the expressionless reference model corresponding to the minimum value is determined as the target expressionless reference model.

[0056] In this embodiment, after determining the expressionless character model of the target character, the distance between each vertex of the expressionless character model and the center of the model can be calculated. For easy distinction, the vertices of the expressionless character model are recorded as character vertices, the center of the expressionless character model is recorded as the character center, and the distance between each character vertex of the expressionless character model and the character center is recorded as the character distance.

[0057] The model library stores reference model groups for multiple reference characters. Each reference model group includes a neutral expression reference model for that character and multiple expression reference models with different facial expressions. For each neutral expression reference model stored in the model library, the distances between each vertex of the neutral expression reference model and the model center are calculated. For ease of distinction, the vertices of the neutral expression reference model are designated as reference vertices, the model center of the neutral expression reference model is designated as the reference center, and the distances between each reference vertex of the neutral expression reference model and the reference center are designated as reference distances.

[0058] After obtaining the character distances corresponding to each character vertex in the expressionless character model and the reference distances corresponding to each reference vertex in the expressionless reference model, since the topological structures of the expressionless character model and the expressionless reference model are the same, the ratio between the character distances of each character vertex in the expressionless character model and the reference distances of the corresponding reference vertices in the expressionless reference model can be calculated. Finally, the average of all ratios is calculated to obtain the mean ratio. This mean ratio is used as the scaling factor for the expressionless reference model to scale it, resulting in the scaled expressionless reference model. At this point, the scaled expressionless reference model is closest to the expressionless character model.

[0059] The triangles corresponding to the expressionless character model and the expressionless reference model can be obtained through affine transformation. Therefore, the affine transformation matrices of the expressionless character model and the expressionless reference model can be constructed and solved to obtain the values ​​corresponding to the affine transformation matrices. It should be noted that in this embodiment, the affine transformation matrix includes the sub-affine transformation matrices of each triangle. The value corresponding to the affine transformation matrix can represent the sum or average of the sub-values ​​of all sub-affine transformation matrices, where the sub-value of the sub-affine transformation matrix is ​​the absolute value of the determinant of the affine transformation matrix of each triangle.

[0060] The smaller the value, the higher the facial shape matching degree between the expressionless reference model and the expressionless character model. Therefore, after obtaining the values ​​corresponding to each expressionless reference model, the minimum value can be determined, and the expressionless reference model corresponding to the minimum value can be determined as the target expressionless reference model.

[0061] Optionally, to reduce computational cost, the aforementioned character vertices can refer to the feature vertices in the expressionless character model, and correspondingly, the reference vertices can refer to the feature vertices in the expressionless reference model.

[0062] To further improve the matching degree between the target expressionless reference model and the expressionless character model, in some optional embodiments of this application, determining the expressionless reference model corresponding to the minimum value as the target expressionless reference model may further include:

[0063] Determine whether the minimum value meets the set value. If so, determine the expressionless reference model corresponding to the minimum value as the target expressionless reference model.

[0064] If the minimum value does not meet the set value, a prompt message is generated to indicate that there is no target expressionless reference model in the model library that matches the facial features.

[0065] In this embodiment, the set value is used to represent the matching range. When the minimum value meets the set value, it means that the minimum value is within the set matching range, and the expressionless reference model corresponding to the minimum value is determined as the target expressionless reference model. When the minimum value is not within the set matching range, it means that the matching degree between the expressionless reference model corresponding to the minimum value and the expressionless character model is lower than the allowed matching degree, which means that there is no expressionless reference model matching the expressionless character model in the current model library. At this time, a prompt message can be generated to remind relevant personnel to intervene. For example, after receiving the prompt message, relevant personnel can decide whether to continue to determine the expressionless reference model as the target expressionless reference model and focus on checking the effect of the generated expression character model after determining the expressionless reference model as the target expressionless reference model.

[0066] In some optional embodiments of this application, determining the expressionless reference model corresponding to the minimum value as the target expressionless reference model may further include:

[0067] Determine whether the minimum value meets the set value; if not, generate a reminder message.

[0068] In this embodiment, regardless of whether the minimum value meets the set value, the expressionless reference model corresponding to the minimum value is determined as the target expressionless reference model; however, when the minimum value does not meet the set value, a reminder message is generated to remind relevant personnel to focus on checking the effect of the subsequently generated expression character model.

[0069] Optionally, to facilitate calculation, improve computational efficiency, and conserve computational resources, it is necessary to ensure that the orientation of the aforementioned expressionless character model and expressionless reference model are consistent. This can be achieved by setting the rotation data of the expressionless character model and expressionless reference model to zero. Additionally, in some optional embodiments of this application, after determining the expressionless character model of the target character, the following may also be included:

[0070] Move the expressionless reference model or expressionless character model so that the reference center of the expressionless reference model coincides with the character center of the expressionless character model.

[0071] This embodiment uses a movement operation to make the reference center of the expressionless reference model coincide with the character center of the expressionless character model, which can facilitate subsequent calculations, reduce calculation difficulty, and improve calculation efficiency.

[0072] For example, the first coordinate mean of all reference vertices of the expressionless reference model can be calculated, and the second coordinate mean of all character vertices of the expressionless character model can be calculated. Based on the difference between the first coordinate mean and the second coordinate mean, the expressionless reference model or the expressionless character model can be moved so that the first coordinate mean and the second coordinate mean are equal, even if the model centers of the two coincide.

[0073] In another optional embodiment of this application, the expressionless reference models of multiple reference characters in the model library are sorted in a preset order. The process of determining the target expressionless reference model of the target reference character that matches the facial features of the expressionless character model from the model library may include:

[0074] Determine the distance between each vertex of the expressionless character model and the center of the character model;

[0075] Determine the reference distance between each reference vertex of the expressionless reference model with the current index and the corresponding reference center with the current index;

[0076] The scaling factor of the expressionless reference model of the current number is determined based on the character distance and the current number reference distance, and the expressionless reference model of the current number is scaled according to the current number scaling factor to obtain the scaled expressionless reference model of the current number.

[0077] Determine the current value of the affine transformation matrix between the expressionless character model and the expressionless reference model scaled to the current index;

[0078] When the current value meets the set value, the expressionless reference model of the current sequence number is determined as the target expressionless reference model;

[0079] When the current value does not conform to the set value, the expressionless reference model of the current index is updated according to the sorting order of the expressionless reference model, and the step of determining the current index reference distance between each reference vertex of the expressionless reference model of the current index and the corresponding reference center is returned to continue execution.

[0080] In this embodiment, the reference model groups in the model library are sorted according to a preset order. This preset order can be a forward or reverse sort based on the order in which the reference model groups were stored in the model library, or it can be sorted according to the number of times a reference model group in the model library has been identified as a target reference model group (i.e., the reference model group corresponding to the target reference character), in descending or ascending order of the number of times. After sorting the reference model groups in the model library, we can start from the reference model group with the smallest sequence number. That is, initially, the expressionless reference model with the current sequence number is the expressionless reference model in the reference model group with the smallest sequence number. We then determine whether the expressionless reference model with the current sequence number matches the expressionless character model. If they match, the expressionless reference model with the current sequence number is identified as the target expressionless reference model. If they do not match, we then determine whether the expressionless reference model in the next sequence number of the reference model group matches the expressionless character model, that is, we update the reference model group with the current sequence number of the reference model group, until the target expressionless reference model is identified, or until all expressionless reference models in the model library have completed the matching judgment.

[0081] It should be noted that the determination and updating of the current sequence number is not limited to the above example. For example, it can also start from the reference model group with the largest sequence number. That is, initially, the expressionless model of the current sequence number is the expressionless reference model in the reference model group with the largest sequence number; when the expressionless model of the current sequence number does not match the expressionless character model, the reference model group of the previous sequence number is updated to the reference model group of the current sequence number.

[0082] The process of determining whether the expressionless reference model matches the expressionless character model can be found in the description above, and will not be repeated here.

[0083] Step 503: Based on the deformation data between each target expression reference model and the target expressionless reference model, generate multiple expression character models with different expressions corresponding to the expressionless character model.

[0084] The target facial expression reference model is obtained by deforming the target expressionless reference model. Therefore, there is deformation data between each target facial expression reference model and the target expressionless reference model. This deformation data can include the displacement data between each reference vertex of the target facial expression reference model and the corresponding reference vertex of the target expressionless reference model.

[0085] Since the facial features of the expressionless character model match those of the target expressionless reference model, after obtaining the deformation data corresponding to the target expression reference model, the character vertices of the expressionless character model can be moved according to the displacement data in the deformation data, thus automatically transforming the expressionless character model into the corresponding expression character model.

[0086] The displacement data can be the number of specific moving units or the moving ratio; this application does not specifically limit this.

[0087] This embodiment uses movement data to guide the movement of character vertices of the expressionless character model, resulting in an expression character model after deformation of the expressionless character model. Compared with other methods that use wrap deformation processing to deform the expressionless character model to obtain the corresponding deformed model, this embodiment can avoid generating additional models during the deformation process (wrap deformation processing will generate additional high-poly or low-poly models), and consumes relatively less resources.

[0088] In some examples, the process of generating multiple different expression character models corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model may include:

[0089] Based on the number of target expression reference models for the target character, the expressionless character model is copied to obtain a copy of the expressionless character model corresponding to each target expression reference model;

[0090] Based on the displacement data of each target expression reference model, the character vertices of the corresponding expressionless character model copy are moved to obtain multiple expression character models with different expressions.

[0091] In this example, by copying the expressionless character model, the same number of expressionless character model copies as the target expressionless character model are obtained, with each expressionless character model copy corresponding to a target expression reference model. By obtaining the displacement data corresponding to all target expression reference models, the character vertices of the corresponding expressionless character model copies are moved according to the displacement data of each target expression reference model, thereby deforming the expressionless character model copies. After the character vertices of the expressionless character model copies corresponding to each target expression model are moved, an expression character model with the same expression as the corresponding target expression model can be obtained. Since the target character has multiple target expression reference models with different expressions, multiple expression character models with different expressions can be obtained.

[0092] In some examples, multiple facial expression character models with different expressions can be generated sequentially. For instance, the displacement data corresponding to one of the target facial expression reference models can be obtained. After obtaining the displacement data for one target facial expression model, a copy of the expressionless character model can be made. Then, the character vertices of the expressionless character model copy can be moved according to the displacement data to transform the expressionless character model copy into an facial expression character model that belongs to the same expression as the target facial expression reference model. After obtaining one facial expression character model, the displacement data corresponding to the next target facial expression reference model can be obtained, and the above steps of copying the expressionless character model, moving the character vertices of the expressionless character model copy, and generating a facial expression model that belongs to the same expression as the target facial expression reference model can be continued. This process continues until facial expression character models that correspond one-to-one with each target facial expression reference model are generated.

[0093] In some examples, the process of generating multiple different expression character models corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model may include:

[0094] Determine the initial deformation data corresponding to the deformation of the target expressionless reference model into the expressionless character model;

[0095] Based on the initial deformation data, the deformation data corresponding to each of the target expressionless reference models are superimposed to obtain multiple target deformation data;

[0096] The target expressionless reference model is deformed based on the deformation data of each target to generate multiple expression character models with different expressions corresponding to the expressionless character model.

[0097] Since the target expressionless reference model and the expressionless character model have the same topological structure, the target expressionless reference model can be transformed into the expressionless character model, and the initial deformation data corresponding to the transformation of the target expressionless reference model into the expressionless character model can be recorded.

[0098] When the target facial expression reference model is stored in the model library in the form of facial expression name and deformation data, the deformation data corresponding to each target facial expression reference model can be obtained directly. Since the model library stores the target expressionless reference model and the deformation data of each target facial expression reference model, the deformation data corresponding to each target facial expression reference model can be obtained.

[0099] By superimposing the initial deformation data with the deformation data corresponding to each target facial expression reference model, multiple target deformation data can be obtained.

[0100] Finally, the target expressionless reference model is deformed according to the deformation data of each target, and the resulting multiple different expression models are the expression character models of the target character.

[0101] It should be noted that in some examples, the option of a target expression reference model can also be provided, allowing users to select the expression corresponding to the expression character model to be generated. That is, the required expression character model can be generated according to the actual needs of the target character, avoiding the generation of unnecessary expression character models, thereby saving resources.

[0102] For example, when the target character is a secondary character and the target reference character is a primary character, since the secondary character needs to display fewer expressions in the game than the primary character, for the target character, only an expression character model corresponding to a portion of the target reference character's expressions can be generated.

[0103] Furthermore, considering that artists may exhibit some irregularities in the process of creating expressionless character models of target characters in practical applications, such as failing to freeze the transformation of the expressionless character model, directly deforming the expressionless character model based on the deformation data of the target expression reference model would result in deviations. Therefore, in some optional embodiments of this application, before generating multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model, the method further includes:

[0104] Detect whether the expressionless character model has undergone a freeze transformation;

[0105] If not, then the expressionless character model is frozen and transformed.

[0106] In this embodiment, by detecting whether the expressionless character model has undergone a freeze transformation, and performing a freeze transformation on the expressionless character model when no freeze transformation is performed, deviations caused by non-standard production of the expressionless character model can be prevented.

[0107] For example, it can be determined whether the expressionless character model has undergone a freeze transformation by detecting the current displacement, rotation, scaling, and other parameters of the model. If the current displacement and rotation parameters are not 0, and the scaling parameter is not 1, it indicates that the expressionless character model has not undergone a freeze transformation, and therefore a freeze transformation needs to be performed on the model.

[0108] It should be noted that in other embodiments, the freeze transformation of the expressionless character model can be performed directly without making a judgment. Optionally, the historical records of the expressionless character model can also be cleared, which not only saves storage space but also avoids interference from historical records.

[0109] Furthermore, in an optional embodiment of this application, after generating multiple expression character models with different expressions corresponding to the expressionless character model, the process may further include:

[0110] Show the model of the facial expression character.

[0111] After generating the facial expression character model, this embodiment can also display the model on a corresponding display screen so that the user can judge whether the generated model meets expectations. Multiple facial expression character models with different expressions can be displayed simultaneously on one screen, or they can be displayed separately on multiple screens.

[0112] For example, when multiple different expression character models are generated sequentially, each generated expression character model can be displayed directly after it is generated; alternatively, all generated expression character models can be displayed after all of them have been generated.

[0113] Furthermore, in an optional embodiment of this application, the above method may further include:

[0114] In response to the adjustment operation for the facial expression character model, the facial expression character model is updated.

[0115] When a user views the generated emoji character models and finds that one or more models do not meet their expectations, they can adjust the corresponding models through an adjustment operation. This adjustment operation includes moving the vertices of the emoji character model. Specifically, users can move the vertices of the emoji character model using scripts or by dragging and dropping them.

[0116] After the user has finished adjusting the emoji character model, they can save the new emoji character model after the adjustment and use the new emoji character model to replace the corresponding adjusted emoji character model in order to update the adjusted emoji character model and ensure the accuracy of the target character's emoji character model.

[0117] After obtaining the facial expression model and the expressionless character model of the target character, the facial expression model of the target character can be used as the deformation target of the expressionless character model. By adjusting the blending parameters of the expressionless character model and the deformation target, the target character can be deformed between the expressionless character model and the deformation target.

[0118] This application embodiment determines a target character's expressionless character model; based on the facial features of the expressionless character model, it determines a target expressionless reference model of a target reference character matching the facial features, as well as multiple target expression reference models with different expressions, from a model library; based on the deformation data between each target expression reference model and the target expressionless reference model, it generates multiple expression character models with different expressions corresponding to the expressionless character model; this can achieve automated generation of multiple expression character models with different expressions for a target character, improving the generation efficiency of expression character models; furthermore, by determining the corresponding target expressionless reference model through the facial features of the expressionless character model, it can improve the accuracy of the generated expression character model in expressing expressions, thus improving the generation effect of the expression character model.

[0119] To facilitate understanding of this solution by those skilled in the art, the following will be combined with... Figure 6-10 The method for generating facial expression models provided in the embodiments of this application will be described and illustrated by way of example.

[0120] In this example, the facial expression model generation method provided in this application embodiment is applied in the form of a script to model-making software. This model-making software has a Blend Shape editor, such as Maya 3D animation software. When the terminal device runs the model-making software, it can receive the user's first import operation and import the reference model group selected in the first import operation into the model library. Here, the reference model group refers to a blank reference model with Blend Shape, that is, the reference model group includes a blank reference model and multiple facial expression reference models with different expressions.

[0121] To facilitate understanding of the process in this embodiment of generating an expressionless character model corresponding to a non-expressionable character model based on the deformation data between the target expression reference model and the target expressionless reference model, the imported expressionless reference model in this example is as follows: Figure 6 As shown, one of the facial expression reference models is as follows: Figure 7 As shown; that is, the target expressionless reference model is as follows: Figure 6 As shown, the target facial expression reference model is as follows: Figure 7 As shown.

[0122] Simultaneously, it can also receive a second import operation from the user, through which the expressionless model of the expression model to be generated can be determined.

[0123] For ease of understanding, in this example, the imported expressionless model of the expression model to be generated is as follows: Figure 8 As shown, the expressionless character model of the target character is as follows: Figure 8 As shown.

[0124] In response to the script execution operation, the script used to implement the expression model generation method of this application is executed, and the expression character model of the target character is automatically generated, such as... Figure 9 As shown.

[0125] like Figure 10 As shown, the script execution process includes the following steps:

[0126] Step 1001: Add the expressionless character model as a deformation target of the target expressionless reference model. That is, add the expressionless character model as a target expression reference model in the blend shape of the target expressionless reference model. Since the facial features of the target expressionless reference model match those of the expressionless character model, the expressionless character model can be used as the deformation target of the target expressionless reference model. At this point, the blend weights of all target expression reference models in the blend shape of the target expressionless reference model are adjusted to 0.

[0127] Step 1002: Adjust the blending weights of the expressionless character model to 1. This step can be understood as determining the initial deformation data when the target expressionless reference model is transformed into the expressionless character model.

[0128] Step 1003: Determine if there is an unprocessed target expression reference model. That is, determine whether the blending weights of the original target expression reference model (excluding the expressionless character model) in the Blend Shape of the target expressionless reference model have been adjusted from 0 to 1. If there is an unadjusted original target expression reference model, proceed to step 1004; otherwise, end the process.

[0129] Step 1004: Adjust the mixing weight of the first deformation target to 1. This first deformation target is one of the unprocessed target facial expression reference models. This step can be understood as superimposing the initial deformation data and the deformation data corresponding to the original target facial expression reference model.

[0130] Step 1005: Copy the current model as the expression character model for the expressionless character model. Since the mixing weights of the first deformation target and the expressionless character model are both 1, and the mixing weights of other target expression reference models are 0, the current model is obtained by mixing the first deformation target and the expressionless character model. Furthermore, since the mixing weight of the first deformation target is adjusted to 1 after the mixing weight of the expressionless character model is adjusted to 1, the current model is obtained by mixing the deformation data of the first deformation target on the basis of the expressionless character model. Therefore, the current model is the expression character model with the same expression as the expressionless character model and the first deformation target.

[0131] Assuming the first deformation target is as follows Figure 7 As shown, the expressionless character model is as follows Figure 8 As shown, the current model is Figure 9 As shown, Figure 9 yes Figure 8 The facial expression character model, and Figure 9 and Figure 7 They all have their mouths open in an expression.

[0132] Step 1006: Display the emoji character model. After obtaining the emoji character model, it can be displayed, and users can be received to adjust the displayed emoji character model in order to update the emoji character model.

[0133] Step 1007: Adjust the blending weight of the first deformation target to 0, and return to step 1003. It can be understood that after generating one facial expression character model, facial expression character models for other expressions can be generated in the same way.

[0134] After generating the expressionless character model and the corresponding expressionless character model, export both the expressionless character model and the expressionless character model for import into the engine.

[0135] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0136] Reference Figure 11 This diagram illustrates a structural block diagram of an embodiment of an expression model generation apparatus according to this application. Corresponding to the above-described expression model generation method embodiment, in this embodiment, the apparatus may include the following modules:

[0137] The target character determination module 1101 is used to determine the expressionless character model of the target character;

[0138] The reference character determination module 1102 is used to determine, from the model library, a target expressionless reference model of a target reference character that matches the facial features of the expressionless character model, and multiple target expression reference models with different expressions, based on the facial features of the expressionless character model.

[0139] The expression model generation module 1103 is used to generate multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model.

[0140] Optionally, the reference role determination module 1102 may include:

[0141] The first distance determination submodule is used to determine the character distance between each character vertex of the expressionless character model and the character center of the expressionless character model, as well as the reference distance between each reference vertex of the expressionless reference model of each reference character in the model library and the reference center of the corresponding expressionless reference model.

[0142] The first scaling submodule is used to determine the scaling factor of the corresponding expressionless reference model based on the character distance and the reference distance, and to scale the corresponding expressionless reference model according to the scaling factor to obtain the scaled expressionless reference model.

[0143] The first value determination submodule is used to determine the value of the affine transformation matrix between the expressionless character model and each scaled expressionless reference model.

[0144] The first determining submodule is used to determine the minimum value from the multiple values, and to determine the expressionless reference model corresponding to the minimum value as the target expressionless reference model.

[0145] Optionally, the first determining submodule may include:

[0146] A judgment unit is used to determine whether the minimum value meets a set value;

[0147] The first processing unit is configured to determine the expressionless reference model corresponding to the minimum value as the target expressionless reference model if the condition is met.

[0148] The second processing unit is used to generate a prompt message if the minimum value does not meet the set value, so as to indicate that there is no target expressionless reference model in the model library that matches the facial features.

[0149] Optionally, the expressionless reference models of multiple reference characters in the model library are sorted in a preset order.

[0150] Optionally, the reference role determination module 1102 may include:

[0151] The second distance determination submodule is used to determine the distance between each character vertex of the expressionless character model and the character center of the expressionless character model;

[0152] The third distance determination submodule is used to determine the reference distance between each reference vertex of the expressionless reference model of the current index and the corresponding reference center of the current index.

[0153] The second scaling submodule is used to determine the current number scaling factor of the expressionless reference model of the current number based on the character distance and the current number reference distance, and to scale the expressionless reference model of the current number according to the current number scaling factor to obtain the scaled expressionless reference model of the current number.

[0154] The second value determination submodule is used to determine the current value corresponding to the affine transformation matrix between the expressionless character model and the expressionless reference model scaled by the current index.

[0155] The second determining submodule is used to determine the expressionless reference model of the current sequence number as the target expressionless reference model when the current value meets the set value.

[0156] The first processing submodule is used to update the expressionless reference model of the current index according to the sorting order of the expressionless reference model when the current value does not conform to the set value, and return to the step of determining the reference distance of each reference vertex of the expressionless reference model of the current index and the corresponding reference center to continue execution.

[0157] Optionally, the device may further include:

[0158] The moving module is used to move the expressionless reference model or expressionless character model so that the reference center of the expressionless reference model coincides with the character center of the expressionless character model.

[0159] Optionally, the device may further include:

[0160] The detection module is used to detect whether the expressionless character model has undergone a freeze transformation;

[0161] The freeze module is used to freeze the expressionless character model if it has not been frozen.

[0162] Optionally, the deformation data includes displacement data of each reference vertex of the target facial expression reference model relative to the corresponding reference vertex of the target expressionless reference model; the facial expression model generation module 1103 may include:

[0163] The model copying submodule is used to copy the expressionless character model according to the number of target expression reference models of the target character, so as to obtain an expressionless character model copy corresponding to each target expression reference model;

[0164] The vertex movement submodule is used to move the character vertices of the corresponding expressionless character model copy according to the displacement data of each target expression reference model, so as to obtain multiple expression character models with different expressions.

[0165] Optionally, the expression model generation module 1103 may include:

[0166] The initial deformation data determination submodule is used to determine the initial deformation data corresponding to the deformation of the target expressionless reference model into the expressionless character model;

[0167] The deformation data overlay submodule is used to overlay the deformation data corresponding to each of the target expressionless reference models on the basis of the initial deformation data to obtain multiple target deformation data.

[0168] The model deformation submodule is used to deform the target expressionless reference model according to the target deformation data to generate multiple expression character models with different expressions corresponding to the expressionless character model.

[0169] Optionally, the device may further include:

[0170] The display module is used to display the facial expression character model.

[0171] Optionally, the device may further include:

[0172] An adjustment module is used to update the expression character model in response to an adjustment operation on the expression character model.

[0173] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0174] This application also discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the expression model generation method described above.

[0175] This application also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the expression model generation method described above.

[0176] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0179] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0182] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0183] The foregoing has provided a detailed description of the method, apparatus, electronic device, and storage medium for generating an expression model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for generating an expression model, characterized in that, The method includes: Determine the expressionless character model of the target character; Based on the facial features of the expressionless character model, a target expressionless reference model of a target reference character matching the facial features and multiple target expression reference models with different expressions are determined from the model library. Based on the deformation data between each target expression reference model and the target expressionless reference model, generate multiple expression character models with different expressions corresponding to the expressionless character model; The step of determining, based on the facial features of the expressionless character model, a target expressionless reference model of a target reference character matching the facial features from a model library, and multiple target expression reference models with different expressions, includes: Determine the character distance between each character vertex of the expressionless character model and the character center of the expressionless character model, and the reference distance between each reference vertex of the expressionless reference model of each reference character in the model library and the reference center of the corresponding expressionless reference model. The scaling factor of the corresponding expressionless reference model is determined based on the character distance and the reference distance, and the corresponding expressionless reference model is scaled according to the scaling factor to obtain the scaled expressionless reference model. Determine the values ​​corresponding to the affine transformation matrices between the expressionless character model and each scaled expressionless reference model; wherein, the affine transformation matrix between the expressionless character model and each scaled expressionless reference model includes: the sub-affine transformation matrix of the triangles corresponding to the expressionless character model and each scaled expressionless reference model; the value corresponding to the affine transformation matrix represents the sum or average of the sub-values ​​of all sub-affine transformation matrices; The minimum value is determined from the multiple values, and the expressionless reference model corresponding to the minimum value is determined as the target expressionless reference model.

2. The method according to claim 1, characterized in that, The step of determining the minimum value from the plurality of values ​​and determining the expressionless reference model corresponding to the minimum value as the target expressionless reference model further includes: Determine whether the minimum value meets the set value. If so, determine the expressionless reference model corresponding to the minimum value as the target expressionless reference model. If the minimum value does not meet the set value, a prompt message is generated to indicate that there is no target expressionless reference model in the model library that matches the facial features.

3. The method according to claim 1, characterized in that, The model library contains multiple expressionless reference models of reference characters, arranged in a preset order. The step of determining, based on the facial features of the expressionless character models, a target expressionless reference model of a target reference character matching the facial features, and multiple target expression reference models with different expressions, from the model library, includes: Determine the distance between each vertex of the expressionless character model and the center of the character model; Determine the reference distance between each reference vertex of the expressionless reference model with the current index and the corresponding reference center with the current index; The scaling factor of the expressionless reference model of the current number is determined based on the character distance and the current number reference distance, and the expressionless reference model of the current number is scaled according to the current number scaling factor to obtain the scaled expressionless reference model of the current number. Determine the current value of the affine transformation matrix between the expressionless character model and the expressionless reference model scaled to the current index; wherein, the affine transformation matrix between the expressionless character model and the expressionless reference model scaled to the current index includes: sub-affine transformation matrices of the triangles corresponding to the expressionless character model and the expressionless reference model scaled to the current index; the current value of the affine transformation matrix represents the sum or average of the sub-values ​​of all sub-affine transformation matrices; When the current value meets the set value, the expressionless reference model of the current sequence number is determined as the target expressionless reference model; If the current value does not conform to the set value, then the expressionless reference model of the current index is updated according to the sorting order of the expressionless reference model, and the step of determining the current index reference distance between each reference vertex of the expressionless reference model of the current index and the corresponding reference center is returned to continue execution.

4. The method according to claim 3, characterized in that, The method further includes: Move the expressionless reference model or expressionless character model so that the reference center of the expressionless reference model coincides with the character center of the expressionless character model.

5. The method according to claim 4, characterized in that, Before generating multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each target expression reference model and the target expressionless reference model, the method further includes: Detect whether the expressionless character model has undergone a freeze transformation; If not, then the expressionless character model is frozen and transformed.

6. The method according to claim 5, characterized in that, The deformation data includes displacement data of each reference vertex of the target facial expression reference model relative to the corresponding reference vertex of the target expressionless reference model; The step of generating multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model includes: Based on the number of target expression reference models for the target character, the expressionless character model is copied to obtain a copy of the expressionless character model corresponding to each target expression reference model; Based on the displacement data of each target expression reference model, the character vertices of the corresponding expressionless character model copy are moved to obtain multiple expression character models with different expressions.

7. The method according to claim 5, characterized in that, The step of generating multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model includes: Determine the initial deformation data corresponding to the deformation of the target expressionless reference model into the expressionless character model; Based on the initial deformation data, the deformation data corresponding to each of the target expressionless reference models are superimposed to obtain multiple target deformation data; The target expressionless reference model is deformed based on the deformation data of each target to generate multiple expression character models with different expressions corresponding to the expressionless character model.

8. The method according to claim 1, characterized in that, After generating multiple facial expression character models corresponding to the expressionless character model, the method further includes: Show the model of the facial expression character.

9. The method according to claim 8, characterized in that, The method further includes: In response to the adjustment operation for the facial expression character model, the facial expression character model is updated.

10. A device for generating facial expression models, characterized in that, The device includes: The target character determination module is used to determine the expressionless character model of the target character; The reference character determination module is used to determine, from the model library, a target expressionless reference model of a target reference character that matches the facial features of the expressionless character model, and multiple target expression reference models with different expressions; The expression model generation module is used to generate multiple expression character models with different expressions corresponding to the expressionless character model based on the deformation data between each of the target expression reference models and the target expressionless reference model. The reference role determination module includes: The first distance determination submodule is used to determine the character distance between each character vertex of the expressionless character model and the character center of the expressionless character model, as well as the reference distance between each reference vertex of the expressionless reference model of each reference character in the model library and the reference center of the corresponding expressionless reference model. The first scaling submodule is used to determine the scaling factor of the corresponding expressionless reference model based on the character distance and the reference distance, and to scale the corresponding expressionless reference model according to the scaling factor to obtain the scaled expressionless reference model. The first value determination submodule is used to determine the value corresponding to the affine transformation matrix between the expressionless character model and each scaled expressionless reference model; wherein, the affine transformation matrix between the expressionless character model and each scaled expressionless reference model includes: the sub-affine transformation matrix of the triangle corresponding to the expressionless character model and each scaled expressionless reference model; the value corresponding to the affine transformation matrix represents the sum or average of the sub-values ​​of all sub-affine transformation matrices; The first determining submodule is used to determine the minimum value from the multiple values, and to determine the expressionless reference model corresponding to the minimum value as the target expressionless reference model.

11. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method for generating an expression model as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method for generating an expression model as described in any one of claims 1 to 9.

Images