Methods, apparatus, and computer devices for facial expression construction based on dense facial key points

Abstract

This disclosure provides a method, apparatus, and computer device for constructing facial expressions based on dense facial keypoints. The expression construction system obtains the mapping relationship from dense facial keypoints to BFM format by aligning the model's face with the average BFM face; and obtains the conversion relationship from blendshape to BFM format by transferring the expression base of the animated face in the animation system to the average BFM face. In subsequent practical applications, the expression construction system directly solves for expression coefficients based on the above-mentioned mapping relationship from dense facial keypoints to BFM format and the conversion relationship from blendshape to BFM format for a captured frame of dense facial keypoints (the number of facial keypoints on the user's face is the same as that on the model's face). This allows the algorithm for capturing any number of facial keypoints to be connected with any expression system without requiring excessive manual annotation, making it convenient, fast, and effectively ensuring accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of facial expression driving technology, and in particular to a method, apparatus and computer device for constructing facial expressions based on dense key points of a human face. Background Technology

[0002] Real-time facial expression-driven animation has significant applications in animation and game development. The general process involves the algorithm calculating the facial expression coefficients corresponding to the BlendShape, and then driving a 3D face model to produce the corresponding facial animation. Currently, there are two main approaches to calculating these coefficients: one is to directly predict them using deep neural networks; the other is to calculate them based on key points. The latter approach typically involves fitting key points using a 3D face reconstruction method (such as 3DMM) to obtain the desired facial expression coefficients.

[0003] In the expression-driven process, the blendshape that drives the character's facial expression changes and the expression basis used by the algorithm must be strictly consistent. Therefore, the common practice is to first manually mark the key points on the blendshape, and then calculate the expression basis E based on this, which is then used by the algorithm to solve for the expression coefficients. This method is feasible for sparse key points (such as 68 points or 100+ points), but when the algorithm uses dense key points, this manual marking method becomes extremely difficult. On the one hand, the workload is enormous, and on the other hand, relying entirely on manual marking for such dense key points results in significant errors. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method, apparatus, and computer device for constructing facial expressions based on dense facial key points, which can connect algorithms for capturing any number of key points with any facial expression system without relying too much on manual labeling of key points.

[0005] To achieve the above objectives, this disclosure adopts the following technical solution: a method for constructing facial expressions based on dense facial key points, comprising:

[0006] Capture the dense facial features on the model's face;

[0007] Retrieve the Basel face model average face and the animated face, align the model face with the Basel face model average face, and obtain the index and weight of each facial key point on the Basel face model average face respectively; then transfer the expression base of the animated face to the Basel face model average face, and establish an updated expression base corresponding to the format of the Basel face model average face.

[0008] Based on the updated expression base, the indexes and weights of each facial key point on the average face of the Basel surface model, the expression coefficients are calculated.

[0009] Facial expression actions are constructed based on the expression coefficients.

[0010] This disclosure also provides an expression construction device based on dense facial key points, including:

[0011] The capture module is used to capture the dense facial key points on the model's face;

[0012] A module is established to retrieve the average face and animated face of the Basel surface model, align the model face with the average face of the Basel surface model, obtain the index and weight of each facial key point on the average face of the Basel surface model, and transfer the expression base of the animated face to the average face of the Basel surface model to establish an updated expression base corresponding to the format of the average face of the Basel surface model.

[0013] The calculation module is used to calculate the expression coefficient based on the updated expression base, the index and weight of each of the facial key points on the average face of the Basel surface model;

[0014] A construction module is used to construct facial expression actions based on the expression coefficients.

[0015] This disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0016] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0017] This disclosure provides a method, apparatus, and computer device for constructing facial expressions based on dense facial key points. The expression construction system first captures dense facial key points on a model's face, then retrieves the Basel surface model's average face and an animated face, aligning the model's face with the Basel surface model's average face to obtain the index and weight of each facial key point on the Basel surface model's average face. The expression base of the animated face is then transferred to the Basel surface model's average face to establish an updated expression base corresponding to the format of the Basel surface model's average face. In subsequent applications, the expression construction system calculates expression coefficients based on the updated expression base and the indexes and weights of each facial key point on the Basel surface model's average face, and constructs facial expression actions based on these expression coefficients. In this disclosure, the expression construction system obtains the mapping relationship from dense facial keypoints to the Basel surface model format by aligning the model's face with the average face of the Basel surface model. It also obtains the conversion relationship from blendshape to Basel surface model format by transferring the expression base of the animated face in the animation system (or animation program) to the average face of the Basel surface model. In subsequent practical applications, the expression construction system directly solves for expression coefficients based on the above mapping relationship from dense facial keypoints to the Basel surface model format and the conversion relationship from blendshape to Basel surface model format for a captured frame of dense facial keypoints (i.e., the user's face during application; the number of facial keypoints captured on the user's face during application is the same as the number of facial keypoints on the model's face). This allows the system to connect algorithms for capturing any number of facial keypoints with any expression system without requiring excessive manual annotation, making it convenient, fast, and effectively ensuring accuracy. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the steps of an expression construction method based on dense facial key points in one embodiment of this disclosure;

[0019] Figure 2 This is an overall structural block diagram of an expression construction device based on dense facial key points in one embodiment of this disclosure;

[0020] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present disclosure.

[0021] The realization of the purpose, functional features and advantages of this disclosure will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0023] Reference Figure 1 One embodiment of this disclosure provides a method for constructing facial expressions based on dense facial key points, including:

[0024] S1: Capture the dense facial key points on the model's face;

[0025] S2: Retrieve the Basel Face Model (BFM) average face and animated face, align the model face with the BFM average face, and obtain the index and weight of each facial key point on the BFM average face; then transfer the expression base of the animated face to the BFM average face and establish an updated expression base corresponding to the format of the BFM average face.

[0026] S3: Calculate the expression coefficient based on the updated expression base, the index and weight of each facial key point on the average BFM face;

[0027] S4: Construct facial expression actions based on the expression coefficients.

[0028] In this embodiment, the expression construction system first captures dense facial key points on the model's face, then retrieves the BFM average face and aligns the model's face with the BFM average face to obtain the indices and weights of the dense facial key points on the model's face on the BFM average face. Specifically, the expression construction system first performs rigid registration of a small number of facial key points on the model's face (such as key points of the eyes, nose, and mouth) with the corresponding facial key points on the BFM average face based on the user's manual selection, ensuring that the pose of the model's face is consistent with that of the BFM average face. After completing the rigid registration, the expression construction system uses an iterative nearest-point algorithm to perform non-rigid registration of the model's face and the BFM average face with the same pose, so that the dense facial key points on the model's face after non-rigid registration have a shape that is almost identical to that of the BFM average face. Finally, the expression construction system, after rigid and non-rigid registration, identifies the shortest-distance triangular patches corresponding to each facial keypoint of the model's face on the BFM average face. Based on the positional relationships between the model's facial keypoints and their corresponding triangular patches, and the distance relationships between the keypoints and the three vertices of the triangular patches, the system parses the index and weight of each facial keypoint on the BFM average face, thus obtaining the mapping relationship between the model's dense facial keypoints and the BFM format. Furthermore, the expression construction system transfers the expression base of animated faces (animated faces are built into the animation system or software and are unrelated to the model's face) to the BFM average face, establishing an updated expression base corresponding to the BFM average face format. Specifically, the expression construction system first selects a second animated face from all the first animated faces included in the animated face dataset. This second animated face has the same expression as the BFM average face (usually, the BFM average face has a natural expression, so the second animated face also has a natural expression). Then, based on this second animated face and the BFM average face, the system parses the first identity coefficient corresponding to the animated face (this first identity coefficient can be applied to first animated faces with different expressions). The expression construction system further parses the updated expression base corresponding to the format of each first animated face and the BFM average face based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face, thus obtaining the conversion relationship from the blendshape of the animation system (or the animated face) to the BFM format.

[0029] Once the expression construction system obtains the mapping relationship from dense facial keypoints to BFM format and the conversion relationship from blendshape to BFM format in the animation system, it can be applied to real-world scenarios. In real-world scenarios, the expression construction system captures the dense sixth facial keypoints on the user's face (the number of sixth facial keypoints on the user's face is the same as the number of facial keypoints on the model's face). First, based on the index and weight corresponding to each facial keypoint on the model's face in the BFM average face (i.e., the mapping relationship from dense facial keypoints to BFM format), it obtains the first BFM average face from the corresponding dimension of the BFM average face, which is the same as the dimension of the sixth facial keypoints on the user's face. It also obtains the first face shape base from the corresponding dimension of the face shape base of the BFM average face, and the first updated expression base from the corresponding dimension of the updated expression base. Then, the first BFM average face and the user's face are rigidly registered to obtain the first face scale, the first rotation matrix, and the first translation vector. Based on the first BFM average face, the first face shape basis, the first identity coefficient, the user's face, the first face scale, the first rotation matrix, and the first translation vector, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector corresponding to the user's face are calculated. Finally, the expression construction system calculates the expression coefficient corresponding to the user's face based on the first BFM average face, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficient, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector. The expression construction system can then drive the virtual image's face to perform consistent facial expression movements based on these expression coefficients.

[0030] Preferably, after steps S1 and S2 are completed, as long as the number of sixth facial key points of the user's face captured in subsequent actual applications is the same as the number of facial key points of the model's face, the expression construction system does not need to repeat steps S1 and S2. That is, when executing step S3 (i.e. in actual applications), the system can directly refer to the index and weight of each facial key point of the model's face on the BFM average face, as well as the updated expression base corresponding to the format of the animated face and the BFM average face to calculate the expression coefficient.

[0031] In this embodiment, the expression construction system obtains the mapping relationship from dense facial key points to BFM format by aligning the model's face with the average BFM face; and obtains the conversion relationship from blendshape to BFM format by transferring the expression base of the animated face in the animation system (or can be understood as the animation program) to the average BFM face. In subsequent practical applications, the expression construction system directly solves for expression coefficients based on the above-mentioned mapping relationship from dense facial key points to BFM format and the conversion relationship from blendshape to BFM format for a captured frame of dense facial key points (i.e., the user's face during application; the number of facial key points captured on the user's face during application is the same as the number of facial key points on the model's face). This allows the system to connect algorithms for capturing any number of facial key points with any expression system without requiring excessive manual annotation, making it convenient, fast, and effectively ensuring accuracy.

[0032] Furthermore, the BFM average face has multiple triangular patches distributed on it. The step of aligning the model's face with the BFM average face to obtain the index and weight of each facial key point on the BFM average face includes:

[0033] S201: Obtain selection instructions, and rigidly register multiple selected first facial key points on the model's face with multiple selected second facial key points on the BFM average face according to the selection instructions to obtain a first model face with the same posture as the BFM average face, wherein the first model face has dense third facial key points distributed on it.

[0034] S202: The iterative nearest point algorithm is used to perform non-rigid matching between the first model face and the BFM average face to obtain a second model face with a similar shape to the BFM average face. The second model face has dense fourth facial key points.

[0035] S203: Filter the triangular facets with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and parse the index and weight of each of the facial key points on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular facets.

[0036] In this embodiment, the expression construction system obtains the user's input selection command and, based on the selection command, rigidly registers multiple selected first facial key points on the model's face with multiple selected second facial key points on the BFM average face (the selection command can be understood as the user manually selecting facial key points such as eyes, nose, and mouth on the model's face, and corresponding facial key points selected on the BFM average face). After completing the rigid registration, a first model face with the same pose as the BFM average face is obtained. Specifically, the relevant calculation formulas for rigid registration are as follows: First calculation formula: Where s is the face scale, R is the rotation matrix, t is the translation vector, and K and J are the facial key points manually selected by the user on the model's face and the BFM average face, respectively; the second calculation formula is: M new =s·(RM+t), M new M represents the third facial keypoint of the model's face after rigid registration, and M represents the facial keypoint of the model's face before rigid registration. Then, the expression construction system uses an iterative nearest-point algorithm to perform non-rigid matching between the first model's face and the BFM average face, making the shape of the fourth facial keypoint of the second model's face after non-rigid registration nearly identical to that of the BFM average face. After completing both rigid and non-rigid registration, it is necessary to find which facial keypoint in the BFM average face corresponds to each facial keypoint of the model's face (i.e., the index of each facial keypoint in the BFM average face). In most cases, the facial keypoints of the model's face and the BFM average face are difficult to completely overlap; therefore, using only one facial keypoint from the BFM average face to represent a facial keypoint is inaccurate. For each facial keypoint of the model's face, it is represented using the nearest triangle in the BFM average face. Specifically, the expression construction system first calculates the centroid position of each triangular facet based on the three vertex positions of each triangular facet on the BFM average face. Then, it selects the triangular facet corresponding to the centroid position closest to the fourth facial keypoint of the second model face (i.e., the model face after rigid and non-rigid registration) as the first triangular facet corresponding to the fourth facial keypoint. Finally, it performs a weighted calculation based on the three vertex positions of the first triangular facet to obtain the keypoint, which is the facial keypoint corresponding to the fourth facial keypoint on the BFM average face. According to the above rules, the expression construction system obtains the facial keypoints corresponding to each fourth facial keypoint on the BFM average face and saves the index and weight of each facial keypoint of the model face on the BFM average face.

[0037] Furthermore, in the step of separately filtering the triangular patches with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and parsing the index and weight of each facial key point on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular patches, the calculation steps of the index and weight of a single facial key point on the average BFM face include:

[0038] S2031: Obtain the positions of the three vertices of each of the three triangular facets, and calculate the position of the centroid of each of the three triangular facets based on the positions of the three vertices.

[0039] S2032: Select the triangular facet whose centroid position is closest to the fourth facial key point as the first triangular facet corresponding to the fourth facial key point;

[0040] S2033: Based on the distance relationship between the fourth facial key point and the three first vertices of the first triangular face, calculate the weight of the facial key point on the average BFM face.

[0041] S2034: Based on the three first vertex positions and the weights corresponding to the facial key points on the average BFM face, calculate the fifth facial key point corresponding to the facial key point on the average BFM face.

[0042] S2035: Based on the correspondence between the facial key points and the fifth facial key point, construct an index corresponding to the facial key points on the average BFM face.

[0043] In this embodiment, the expression construction system first obtains the three vertex positions of each triangular facet on the BFM average face, and calculates the centroid position of each triangular facet based on the three vertex positions. The expression construction system selects the triangular facet corresponding to the centroid (i.e., the point where the centroid is located) closest to the fourth facial key point as the first triangular facet corresponding to the fourth facial key point. Then, based on the distance relationship between the fourth facial key point and the three first vertex positions of the first triangular facet (i.e., the positions of the three vertices of the first triangular facet), the system calculates the weight of the fourth facial key point (i.e., the facial key point of the registered model face) on the BFM average face. The third calculation formula corresponding to the weight is: w i For weights, d iThis represents the distance between the fourth facial keypoint and the position of the first vertex. Furthermore, the expression construction system, based on the positions of the three first vertices of the first triangular facet and their corresponding weights, substitutes them into the fourth calculation formula for weighted calculation, obtaining the facial keypoint corresponding to the fourth facial keypoint on the average BFM face, i.e., the fifth facial keypoint. The fourth calculation formula is: P represents the facial key points of the model's face, P′ represents the fifth facial key point corresponding to the facial key point P on the average BFM face, and idx represents the index; that is, the expression construction system obtains and saves the index corresponding to the facial key points on the average BFM face based on the correspondence between the facial key points and the fifth facial key point.

[0044] Furthermore, the animated face includes multiple first animated faces with different expressions. The step of transferring the expression base of the animated face to the BFM average face and establishing an updated expression base corresponding to the format of the BFM average face includes:

[0045] S204: Select a second animated face from each of the first animated faces that has the same expression as the average BFM face;

[0046] S205: Obtain the first identity coefficient corresponding to the animated face based on the second animated face and the BFM average face analysis;

[0047] S206: Based on the BFM average face, the first identity coefficient of the animated face, and the expression triangle mesh (hereinafter referred to as expression mesh) of each first animated face, the updated expression base corresponding to the format of each first animated face and the BFM average face is parsed.

[0048] In this embodiment, the expression construction system retrieves animated faces from the animation system (or animation software). These animated faces include multiple first animated faces with different expressions. Since the face shape of the animated faces differs from that of the BFM average face, this indicates that the animated faces carry identity information. If the BFM average face is fitted onto the animated faces, directly using the fitted expression mesh to calculate a new expression basis will introduce identity-related shape bases, resulting in low accuracy. Therefore, the expression construction system needs to remove the identity information from the animated faces. Specifically, the expression construction system first selects the first second animated face from each of the first animated faces whose expression is the same as that of the BFM average face (for example, the BFM average face usually has a natural expression, and the expression of the 0th first animated face in the animated faces is also a natural expression, so the 0th first animated face is selected as the second animated face). Then, the system uses a fifth calculation formula to analyze and calculate the second animated face and the BFM average face, thereby obtaining the first identity coefficient corresponding to the animated face. The fifth calculation formula is: α represents the first identity coefficient corresponding to the animated face (the identity coefficient of each first animated face is the first identity coefficient). M represents the average BFM face (i.e., the facial landmarks of the average BFM face). b The expression mesh represents the second animated face. After obtaining the first identity coefficient corresponding to the animated face, the expression construction system calculates the identity deformation by multiplying the face shape base of the BFM average face and the first identity coefficient. Then, by subtracting the BFM average face and the identity deformation from the expression mesh of each first animated face, the system obtains the new expression base corresponding to each first animated face in the BFM format after the expression is transferred to the BFM average face, i.e., the updated expression base.

[0049] Furthermore, in the step of parsing the updated expression base corresponding to the format of each first animated face and the BFM average face based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face, the parsing step of the updated expression base corresponding to a single first animated face includes:

[0050] S2061: Calculate the identity deformation by multiplying the face shape base of the BFM average face and the first identity coefficient;

[0051] S2062: Subtract the BFM average face and the identity deformation from the expression mesh of the first animated face in sequence to obtain the updated expression base corresponding to the first animated face.

[0052] In this embodiment, the parsing rules for updating the expression base corresponding to different expressions of the animated face are the same. The following explanation uses the parsing steps for updating the expression base corresponding to the expression of a single first animated face as an example. The expression construction system first calculates the identity deformation by multiplying the face shape base of the BFM average face and the first identity coefficient of the animated face. Then, it subtracts the BFM average face and the identity deformation from the expression mesh of the first animated face in sequence; the difference obtained is the updated expression base corresponding to the expression of that first animated face. Specifically, the corresponding sixth calculation formula is: E i M represents the updated expression base corresponding to the expression of the i-th first animated face. bi The expression mesh, α, representing the i-th first animated face. j F represents the first identity coefficient corresponding to the j-th column (i.e., the j-th principal component) of the animated face. j The face shape basis is represented by the j-th column (i.e. the j-th principal component) of the BFM average face.

[0053] Furthermore, the step of calculating the expression coefficient based on the updated expression base, the indexes and weights of each facial key point on the average BFM face, includes:

[0054] S301: Collect dense sixth facial key points on the user's face, wherein the number of sixth facial key points on the user's face is the same as the number of facial key points on the model's face.

[0055] S302: Based on the index and weight of each of the facial key points of the model's face corresponding to the BFM average face, obtain the first BFM average face which is the same as the sixth facial key point dimension of the user's face from the corresponding dimension of the BFM average face, obtain the first face shape base from the corresponding dimension of the face shape base of the BFM average face, and obtain the first updated expression base from the corresponding dimension of the updated expression base.

[0056] S303: Rigidly register the first BFM average face and the user's face to obtain the first face scale, the first rotation matrix, and the first translation vector;

[0057] S304: Based on the first BFM average face, the first face shape basis, the first identity coefficient, the user face, the first face scale, the first rotation matrix, and the first translation vector, calculate the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector corresponding to the user face;

[0058] S305: Calculate the expression coefficient corresponding to the user's face based on the first BFM average face, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficient, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector.

[0059] In this embodiment, after obtaining the indices and weights of dense facial keypoints on the BFM average face, and the updated expression base of the animation system transferred to the BFM average face, the expression construction system can be applied to real-world scenarios. Furthermore, in practical applications, as long as the number of sixth facial keypoints collected from the user's face is the same as the number of dense keypoints on the model's face (ensuring the mapping relationship from dense facial keypoints to the BFM format is valid), and the animation system used is the same (ensuring the conversion relationship from the blendshape of the animation system to the BFM format is valid), the expression construction system does not need to repeatedly parse the indices and weights of facial keypoints on the BFM average face, nor update the expression base. Specifically, the expression construction system collects the dense sixth facial keypoints on the user's face in a real-world scenario, and then, based on the indices and weights of each facial keypoint on the model's face obtained above on the BFM average face, it transfers the expression base from the BFM average face... (The size is m x 3, where m represents the number of vertices of the average BFM face, and 3 represents the xyz coordinate dimension) to obtain the user face K in the corresponding dimension. dense The sixth facial key point dimension is the same as the first BFM average face (Size n x 3, where n is the number of facial keypoints of the first BFM average face); similarly, the first face shape basis F′ is obtained from the corresponding dimension of the face shape basis F of the BFM average face, and the first updated expression basis E′ is obtained from the corresponding dimension of the updated expression basis E. Then, the expression construction system rigidly registers the first BFM average face and the user's face to make their poses the same, obtaining the first face scale, the first rotation matrix, and the first translation vector. The seventh calculation formula corresponding to this rigid registration is: s′ represents the first face scale, R′ represents the first rotation matrix, and t′ represents the first translation vector. The expression construction system substitutes the first BFM average face, the first face shape basis, the first identity coefficient, the user face, the first face scale, the first rotation matrix, and the first translation vector into the seventh calculation formula to calculate the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector corresponding to the user face. The eighth calculation formula is: α″ represents the second identity coefficient, s″ represents the second face scale, R″ represents the second rotation matrix, and t″ represents the second translation vector. Finally, the expression construction system calculates the expression coefficients corresponding to the user's face based on the ninth calculation formula, considering the first BFM average face, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficients, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector. The ninth calculation formula is: β * β represents the expression coefficient corresponding to the user's face, and β represents the preset expression coefficient.

[0060] Furthermore, the step of obtaining a first BFM average face that is identical to the sixth facial key point dimension of the user's face from the corresponding dimension of the BFM average face based on the index and weight of each facial key point of the model's face on the BFM average face includes:

[0061] S3021: Based on the indexes corresponding to the facial key points of the model's face on the BFM average face, each row of data of the first BFM average face is obtained by interpolation calculation of the corresponding three rows of data on the BFM average face according to weights.

[0062] In this embodiment, the acquisition rules for the first BFM average face, the first face shape base, and the first updated expression base are the same. The following explanation uses the first BFM average face as an example. The expression construction system first uses the indices corresponding to the various facial key points of the model's face on the BFM average face. Each row of data in the new first BFM average face is obtained by interpolating the corresponding three rows of data on the BFM average face according to weights. Its corresponding tenth calculation formula is: Characterization The data in the i-th row, j represents The data in the j-th column.

[0063] Reference Figure 2 This disclosure provides an expression construction device based on dense facial key points, including:

[0064] Capture module 1 is used to capture dense facial key points on the model's face;

[0065] Module 2 is established to retrieve the BFM average face and the animated face, align the model face with the BFM average face, obtain the index and weight of each facial key point on the BFM average face, and transfer the expression base of the animated face to the BFM average face to establish an updated expression base corresponding to the format of the BFM average face.

[0066] Calculation module 3 is used to calculate the expression coefficient based on the updated expression base, the index and weight of each facial key point on the average BFM face;

[0067] Module 4 is used to construct facial expression actions based on the expression coefficients.

[0068] Furthermore, the BFM has multiple triangular facets distributed on an average face, and the establishment module 2 includes:

[0069] The first rigid registration unit is used to acquire selection instructions and rigidly register multiple selected first facial key points on the model's face with multiple selected second facial key points on the BFM average face according to the selection instructions, so as to obtain a first model face with the same posture as the BFM average face, and the first model face has dense third facial key points distributed on it.

[0070] The non-rigid registration unit is used to perform non-rigid matching between the first model face and the BFM average face using the iterative nearest point algorithm to obtain a second model face with a similar shape to the BFM average face. The second model face has dense fourth facial key points distributed on it.

[0071] The first parsing unit is used to filter the triangular facets with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and to parse the index and weight of each of the facial key points on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular facets.

[0072] Furthermore, the first parsing unit includes:

[0073] The first calculation subunit is used to obtain the positions of the three vertices of each of the three triangular facets, and calculate the position of the centroid of each of the three triangular facets based on the positions of the three vertices.

[0074] The filtering subunit is used to filter the triangular facet whose centroid position is closest to the fourth facial key point as the first triangular facet corresponding to the fourth facial key point;

[0075] The second calculation subunit is used to calculate the weight of the facial key point on the average BFM face based on the distance relationship between the fourth facial key point and the three first vertices of the first triangular face.

[0076] The third calculation subunit is used to calculate the fifth facial key point corresponding to the facial key point on the average BFM face based on the weights of the three first vertex positions and the facial key points on the average BFM face.

[0077] A sub-unit is constructed to construct an index corresponding to the facial key points on the average BFM face based on the correspondence between the facial key points and the fifth facial key points.

[0078] Furthermore, the animated face includes multiple first animated faces with different expressions, and the creation module 2 further includes:

[0079] A filtering unit is used to filter a second animated face from each of the first animated faces that has the same expression as the average BFM face;

[0080] The second parsing unit is used to obtain the first identity coefficient corresponding to the animated face based on the second animated face and the BFM average face parsing.

[0081] The third parsing unit is used to parse the updated expression base corresponding to the format of each first animated face and the BFM average face based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face.

[0082] Furthermore, the third parsing unit includes:

[0083] The fourth calculation subunit is used to calculate the identity deformation by multiplying the face shape base of the BFM average face and the first identity coefficient.

[0084] The fifth calculation subunit is used to subtract the BFM average face and the identity deformation from the expression mesh of the first animated face in sequence to obtain the updated expression base corresponding to the first animated face.

[0085] Furthermore, the computing module 3 includes:

[0086] The acquisition unit is used to acquire dense sixth facial key points on the user's face, wherein the number of sixth facial key points on the user's face is the same as the number of facial key points on the model's face.

[0087] The acquisition unit is configured to obtain a first BFM average face that is the same as the sixth facial key point dimension of the user face from the corresponding dimension of the BFM average face based on the index and weight of each facial key point of the model face on the BFM average face, obtain a first face shape base from the corresponding dimension of the face shape base of the BFM average face, and obtain a first updated expression base from the corresponding dimension of the updated expression base.

[0088] The second rigid registration unit is used to rigidly register the first BFM average face and the user's face to obtain a first face scale, a first rotation matrix, and a first translation vector.

[0089] The first calculation unit is used to calculate the second identity coefficient, second face scale, second rotation matrix and second translation vector corresponding to the user face based on the first BFM average face, the first face shape basis, the first identity coefficient, the user face, the first face scale, the first rotation matrix and the first translation vector.

[0090] The second calculation unit is used to calculate the expression coefficient corresponding to the user's face based on the first BFM average face, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficient, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector.

[0091] Furthermore, the acquisition unit includes:

[0092] The sixth calculation subunit is used to calculate each row of data of the first BFM average face based on the index of each of the facial key points of the model's face corresponding to the index of the BFM average face. Each row of data of the first BFM average face is obtained by interpolation calculation of the three rows of data corresponding to the BFM average face according to the weights.

[0093] In this embodiment, each module, unit, and subunit in the facial expression construction device based on dense facial key points is used to perform the corresponding steps in the above-described facial expression construction method based on dense facial key points. The specific implementation process is not described in detail here.

[0094] This embodiment provides an expression construction device based on dense facial key points. The expression construction system first captures dense facial key points on a model's face, then retrieves the BFM average face and the animated face, aligns the model's face with the BFM average face, and obtains the index and weight of each facial key point on the BFM average face. The expression base of the animated face is then transferred to the BFM average face to establish an updated expression base corresponding to the format of the BFM average face. In subsequent applications, the expression construction system calculates expression coefficients based on the updated expression base and the indexes and weights of each facial key point on the BFM average face, and constructs facial expression actions based on these expression coefficients. In this disclosure, the expression construction system obtains the mapping relationship from dense facial key points to the BFM format through the alignment of the model's face with the BFM average face; and obtains the conversion relationship from blendshape to BFM format by transferring the expression base of the animated face in the animation system (or can be understood as an animation program) to the BFM average face. In practical applications, the subsequent expression construction system directly solves for expression coefficients based on the mapping relationship from the dense facial key points captured in a frame (i.e., the user's face during application, with the number of facial key points captured on the user's face being the same as the number of facial key points captured on the model's face). This allows the system to connect algorithms for capturing any number of facial key points with any expression system without requiring excessive manual annotation, making it convenient, fast, and effectively ensuring accuracy.

[0095] Reference Figure 3This disclosure also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores and updates data such as facial expression bases. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for constructing facial expressions based on dense facial key points.

[0096] The processor described above executes the steps of the expression construction method based on dense facial key points as follows:

[0097] S1: Capture the dense facial key points on the model's face;

[0098] S2: Retrieve the BFM average face and the animated face, align the model face with the BFM average face, and obtain the index and weight of each facial key point on the BFM average face; then transfer the expression base of the animated face to the BFM average face and establish an updated expression base corresponding to the format of the BFM average face.

[0099] S3: Calculate the expression coefficient based on the updated expression base, the index and weight of each facial key point on the average BFM face;

[0100] S4: Construct facial expression actions based on the expression coefficients.

[0101] Furthermore, the BFM average face has multiple triangular patches distributed on it. The step of aligning the model's face with the BFM average face to obtain the index and weight of each facial key point on the BFM average face includes:

[0102] S201: Obtain selection instructions, and rigidly register multiple selected first facial key points on the model's face with multiple selected second facial key points on the BFM average face according to the selection instructions to obtain a first model face with the same posture as the BFM average face, wherein the first model face has dense third facial key points distributed on it.

[0103] S202: The iterative nearest point algorithm is used to perform non-rigid matching between the first model face and the BFM average face to obtain a second model face with a similar shape to the BFM average face. The second model face has dense fourth facial key points.

[0104] S203: Filter the triangular facets with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and parse the index and weight of each of the facial key points on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular facets.

[0105] Furthermore, in the step of separately filtering the triangular patches with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and parsing the index and weight of each facial key point on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular patches, the calculation steps of the index and weight of a single facial key point on the average BFM face include:

[0106] S2031: Obtain the positions of the three vertices of each of the three triangular facets, and calculate the position of the centroid of each of the three triangular facets based on the positions of the three vertices.

[0107] S2032: Select the triangular facet whose centroid position is closest to the fourth facial key point as the first triangular facet corresponding to the fourth facial key point;

[0108] S2033: Based on the distance relationship between the fourth facial key point and the three first vertices of the first triangular face, calculate the weight of the facial key point on the average BFM face.

[0109] S2034: Based on the three first vertex positions and the weights corresponding to the facial key points on the average BFM face, calculate the fifth facial key point corresponding to the facial key point on the average BFM face.

[0110] S2035: Based on the correspondence between the facial key points and the fifth facial key point, construct an index corresponding to the facial key points on the average BFM face.

[0111] Furthermore, the animated face includes multiple first animated faces with different expressions. The step of transferring the expression base of the animated face to the BFM average face and establishing an updated expression base corresponding to the format of the BFM average face includes:

[0112] S204: Select a second animated face from each of the first animated faces that has the same expression as the average BFM face;

[0113] S205: Obtain the first identity coefficient corresponding to the animated face based on the second animated face and the BFM average face analysis;

[0114] S206: Based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face, the updated expression base corresponding to the format of each first animated face and the BFM average face is parsed.

[0115] Furthermore, in the step of parsing the updated expression base corresponding to the format of each first animated face and the BFM average face based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face, the parsing step of the updated expression base corresponding to a single first animated face includes:

[0116] S2061: Calculate the identity deformation by multiplying the face shape base of the BFM average face and the first identity coefficient;

[0117] S2062: Subtract the BFM average face and the identity deformation from the expression mesh of the first animated face in sequence to obtain the updated expression base corresponding to the first animated face.

[0118] Furthermore, the step of calculating the expression coefficient based on the updated expression base, the indexes and weights of each facial key point on the average BFM face, includes:

[0119] S301: Collect dense sixth facial key points on the user's face, wherein the number of sixth facial key points on the user's face is the same as the number of facial key points on the model's face.

[0120] S302: Based on the index and weight of each of the facial key points of the model's face corresponding to the BFM average face, obtain the first BFM average face which is the same as the sixth facial key point dimension of the user's face from the corresponding dimension of the BFM average face, obtain the first face shape base from the corresponding dimension of the face shape base of the BFM average face, and obtain the first updated expression base from the corresponding dimension of the updated expression base.

[0121] S303: Rigidly register the first BFM average face and the user's face to obtain the first face scale, the first rotation matrix, and the first translation vector;

[0122] S304: Based on the first BFM average face, the first face shape basis, the first identity coefficient, the user face, the first face scale, the first rotation matrix, and the first translation vector, calculate the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector corresponding to the user face;

[0123] S305: Calculate the expression coefficient corresponding to the user's face based on the first BFM average face, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficient, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector.

[0124] Furthermore, the step of obtaining a first BFM average face that is identical to the sixth facial key point dimension of the user's face from the corresponding dimension of the BFM average face based on the index and weight of each facial key point of the model's face on the BFM average face includes:

[0125] S3021: Based on the indexes corresponding to the facial key points of the model's face on the BFM average face, each row of data of the first BFM average face is obtained by interpolation calculation of the corresponding three rows of data on the BFM average face according to weights.

[0126] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for constructing facial expressions based on dense facial key points. Specifically, the method for constructing facial expressions based on dense facial key points includes:

[0127] S1: Capture the dense facial key points on the model's face;

[0128] S2: Retrieve the BFM average face and the animated face, align the model face with the BFM average face, and obtain the index and weight of each facial key point on the BFM average face; then transfer the expression base of the animated face to the BFM average face and establish an updated expression base corresponding to the format of the BFM average face.

[0129] S3: Calculate the expression coefficient based on the updated expression base, the index and weight of each facial key point on the average BFM face;

[0130] S4: Construct facial expression actions based on the expression coefficients.

[0131] Furthermore, the BFM average face has multiple triangular patches distributed on it. The step of aligning the model's face with the BFM average face to obtain the index and weight of each facial key point on the BFM average face includes:

[0132] S201: Obtain selection instructions, and rigidly register multiple selected first facial key points on the model's face with multiple selected second facial key points on the BFM average face according to the selection instructions to obtain a first model face with the same posture as the BFM average face, wherein the first model face has dense third facial key points distributed on it.

[0133] S202: The iterative nearest point algorithm is used to perform non-rigid matching between the first model face and the BFM average face to obtain a second model face with a similar shape to the BFM average face. The second model face has dense fourth facial key points.

[0134] S203: Filter the triangular facets with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and parse the index and weight of each of the facial key points on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular facets.

[0135] Furthermore, in the step of separately filtering the triangular patches with the shortest distance corresponding to each of the fourth facial key points on the average BFM face, and parsing the index and weight of each facial key point on the average BFM face according to the pose relationship between each of the fourth facial key points and their corresponding triangular patches, the calculation steps of the index and weight of a single facial key point on the average BFM face include:

[0136] S2031: Obtain the positions of the three vertices of each of the three triangular facets, and calculate the position of the centroid of each of the three triangular facets based on the positions of the three vertices.

[0137] S2032: Select the triangular facet whose centroid position is closest to the fourth facial key point as the first triangular facet corresponding to the fourth facial key point;

[0138] S2033: Based on the distance relationship between the fourth facial key point and the three first vertices of the first triangular face, calculate the weight of the facial key point on the average BFM face.

[0139] S2034: Based on the three first vertex positions and the weights corresponding to the facial key points on the average BFM face, calculate the fifth facial key point corresponding to the facial key point on the average BFM face.

[0140] S2035: Based on the correspondence between the facial key points and the fifth facial key point, construct an index corresponding to the facial key points on the average BFM face.

[0141] Furthermore, the animated face includes multiple first animated faces with different expressions. The step of transferring the expression base of the animated face to the BFM average face and establishing an updated expression base corresponding to the format of the BFM average face includes:

[0142] S204: Select a second animated face from each of the first animated faces that has the same expression as the average BFM face;

[0143] S205: Obtain the first identity coefficient corresponding to the animated face based on the second animated face and the BFM average face analysis;

[0144] S206: Based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face, the updated expression base corresponding to the format of each first animated face and the BFM average face is parsed.

[0145] Furthermore, in the step of parsing the updated expression base corresponding to the format of each first animated face and the BFM average face based on the BFM average face, the first identity coefficient of the animated face, and the expression mesh of each first animated face, the parsing step of the updated expression base corresponding to a single first animated face includes:

[0146] S2061: Calculate the identity deformation by multiplying the face shape base of the BFM average face and the first identity coefficient;

[0147] S2062: Subtract the BFM average face and the identity deformation from the expression mesh of the first animated face in sequence to obtain the updated expression base corresponding to the first animated face.

[0148] Furthermore, the step of calculating the expression coefficient based on the updated expression base, the indexes and weights of each facial key point on the average BFM face, includes:

[0149] S301: Collect dense sixth facial key points on the user's face, wherein the number of sixth facial key points on the user's face is the same as the number of facial key points on the model's face.

[0150] S302: Based on the index and weight of each of the facial key points of the model's face corresponding to the BFM average face, obtain the first BFM average face which is the same as the sixth facial key point dimension of the user's face from the corresponding dimension of the BFM average face, obtain the first face shape base from the corresponding dimension of the face shape base of the BFM average face, and obtain the first updated expression base from the corresponding dimension of the updated expression base.

[0151] S303: Rigidly register the first BFM average face and the user's face to obtain the first face scale, the first rotation matrix, and the first translation vector;

[0152] S304: Based on the first BFM average face, the first face shape basis, the first identity coefficient, the user face, the first face scale, the first rotation matrix, and the first translation vector, calculate the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector corresponding to the user face;

[0153] S305: Calculate the expression coefficient corresponding to the user's face based on the first BFM average face, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficient, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector.

[0154] Furthermore, the step of obtaining a first BFM average face that is identical to the sixth facial key point dimension of the user's face from the corresponding dimension of the BFM average face based on the index and weight of each facial key point of the model's face on the BFM average face includes:

[0155] S3021: Based on the indexes corresponding to the facial key points of the model's face on the BFM average face, each row of data of the first BFM average face is obtained by interpolation calculation of the corresponding three rows of data on the BFM average face according to weights.

[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this disclosure and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0157] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, first object, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, first object, or method. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, first object, or method that includes that element.

[0158] The above description is only a preferred embodiment of this disclosure and does not limit the patent scope of this disclosure. Any equivalent structural or procedural changes made based on the content of this disclosure and its drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A method for constructing facial expressions based on dense facial key points, wherein, include: Capture the dense facial features on the model's face; Retrieve the average face of the Basel surface model and the animated face, align the model's face with the average face of the Basel surface model, and obtain the index and weight of each facial key point on the average face of the Basel surface model. The expression base of the animated face is then transferred to the average face of the Basel surface model, and an updated expression base corresponding to the format of the average face of the Basel surface model is established. Based on the updated expression base, the indexes and weights of each facial key point on the average face of the Basel surface model, the expression coefficients are calculated. Facial expression actions are constructed based on the expression coefficients; The Basel surface model has multiple triangular facets distributed on its average face. The step of aligning the model's face with the Basel surface model's average face to obtain the index and weight of each facial key point on the Basel surface model's average face includes: Obtain a selection instruction, and rigidly register multiple selected first facial key points on the model's face with multiple selected second facial key points on the average face of the Basel surface model according to the selection instruction, to obtain a first model face with the same posture as the average face of the Basel surface model, and the first model face has densely distributed third facial key points. The iterative nearest point algorithm is used to perform non-rigid matching between the first model face and the average face of the Basel surface model to obtain a second model face with a similar shape to the average face of the Basel surface model. The second model face has dense fourth facial key points. Each of the fourth facial key points is selected from the triangular facets with the shortest distance corresponding to the average face of the Basel facet model. Based on the pose relationship between each of the fourth facial key points and their corresponding triangular facets, the index and weight of each of the facial key points on the average face of the Basel facet model are obtained.

2. The expression construction method based on dense facial key points according to claim 1, wherein, In the step of separately filtering the triangular facets with the shortest distance corresponding to each of the fourth facial key points on the average face of the Basel facet model, and parsing the index and weight of each facial key point on the average face of the Basel facet model based on the pose relationship between each of the fourth facial key points and their corresponding triangular facets, the calculation steps of the index and weight of each facial key point on the average face of the Basel facet model include: Obtain the positions of the three vertices of each of the three triangular facets, and calculate the position of the centroid of each of the three triangular facets based on the positions of the three vertices; The triangular facet whose centroid is closest to the fourth facial key point is selected as the first triangular facet corresponding to the fourth facial key point; Based on the distance relationship between the fourth facial key point and the three first vertices of the first triangular facet, the weight of the facial key point corresponding to the average face of the Basel facet model is calculated. Based on the weights of the three first vertex positions and the facial key points on the average face of the Basel surface model, the fifth facial key point corresponding to the facial key point on the average face of the Basel surface model is calculated. Based on the correspondence between the facial key points and the fifth facial key point, an index is constructed corresponding to the facial key points on the average human face in the Basel surface model.

3. The expression construction method based on dense facial key points according to claim 1, wherein, The animated face includes multiple first animated faces with different expressions. The step of transferring the expression base of the animated face to the Basel surface model average face and establishing an updated expression base corresponding to the format of the Basel surface model average face includes: Select a second animated face from each of the first animated faces that has the same expression as the average face of the Basel face model; The first identity coefficient corresponding to the animated face is obtained based on the second animated face and the average face analysis of the Basel surface model; Based on the average face of the Basel face model, the first identity coefficient of the animated face, and the expression triangle mesh of each of the first animated faces, the updated expression base corresponding to the format of each of the first animated faces and the average face of the Basel face model is obtained by parsing.

4. The expression construction method based on dense facial key points according to claim 3, wherein, In the step of parsing the updated expression base corresponding to the format of each first animated face and the average face of the Basel surface model based on the average face of the Basel surface model, the first identity coefficient of the animated face, and the expression triangle mesh of each first animated face, the parsing step for the updated expression base corresponding to a single first animated face includes: The identity deformation is calculated by multiplying the face shape basis of the average face in the Basel surface model with the first identity coefficient. The updated expression base corresponding to the first animated face is obtained by subtracting the Basel face model's average face and the identity deformation from the expression triangle mesh of the first animated face in sequence.

5. The expression construction method based on dense facial key points according to claim 3, wherein, The step of calculating the expression coefficient based on the updated expression base, the indexes and weights of each facial key point on the average face of the Basel surface model, includes: Collect dense sixth facial key points on the user's face, wherein the number of sixth facial key points on the user's face is the same as the number of facial key points on the model's face; Based on the indexes and weights of the facial key points of the model's face corresponding to the average face of the Basel surface model, a first Basel surface model average face is obtained from the corresponding dimension of the average face of the Basel surface model, which is the same as the sixth facial key point dimension of the user's face. A first face shape basis is obtained from the corresponding dimension of the face shape basis of the average face of the Basel surface model, and a first updated expression basis is obtained from the corresponding dimension of the updated expression basis. The average face of the first Basel surface model and the user's face are rigidly registered to obtain the first face scale, the first rotation matrix, and the first translation vector. Based on the average face of the first Basel surface model, the first face shape basis, the first identity coefficient, the user face, the first face scale, the first rotation matrix, and the first translation vector, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector corresponding to the user face are calculated. The expression coefficients corresponding to the user's face are calculated based on the average face of the first Basel surface model, the first face shape basis, the first identity coefficient, the first updated expression basis, the user's face, the preset expression coefficient, the second identity coefficient, the second face scale, the second rotation matrix, and the second translation vector.

6. The expression construction method based on dense facial key points according to claim 5, wherein, The step of obtaining a first Basel model average face with the same sixth facial key point dimension as the user's face from the corresponding dimension of the Basel model average face based on the index and weight of each facial key point of the model's face in the Basel model average face includes: Based on the indices corresponding to the facial key points of the model's face on the average face of the Basel surface model, each row of data in the first Basel surface model average face is obtained by interpolation calculation based on the weights of the three corresponding rows of data on the average face of the Basel surface model.

7. An expression construction device based on dense facial key points, wherein, include: The capture module is used to capture the dense facial key points on the model's face; A module is established to retrieve the average face of the Basel surface model and the animated face, align the model face with the average face of the Basel surface model, and obtain the index and weight of each facial key point on the average face of the Basel surface model. The expression base of the animated face is then transferred to the average face of the Basel surface model, and an updated expression base corresponding to the format of the average face of the Basel surface model is established. The calculation module is used to calculate the expression coefficient based on the updated expression base, the index and weight of each of the facial key points on the average face of the Basel surface model; A construction module is used to construct facial expression actions based on the expression coefficients; The Basel face model has multiple triangular facets distributed on an average human face. The creation module includes: The first rigid registration unit is used to acquire selection instructions and rigidly register multiple selected first facial key points on the model face with multiple selected second facial key points on the average face of the Basel surface model according to the selection instructions, so as to obtain a first model face with the same posture as the average face of the Basel surface model, and the first model face has dense third facial key points distributed on it. The non-rigid registration unit is used to perform non-rigid matching between the first model face and the average face of the Basel surface model using the iterative nearest point algorithm, to obtain a second model face with a similar shape to the average face of the Basel surface model, wherein the second model face has densely distributed fourth facial key points. The first parsing unit is used to filter the triangular facets with the shortest distance corresponding to each of the fourth facial key points on the average face of the Basel facet model, and to parse the index and weight of each of the facial key points on the average face of the Basel facet model according to the pose relationship between each of the fourth facial key points and their corresponding triangular facets.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images