Face continuous expression migration method and system based on shape space

Abstract

The invention relates to the technical field of data processing, and discloses a shape space-based face continuous expression migration method and system. The method comprises the following steps: obtaining shape space representation data through Kendall shape space transformation, generating a source face continuous expression curve by using a spherical B-spline interpolation algorithm, constructing a two-parameter geodesic mapping function, calculating a target face interpolation point by using a De Boor recursive algorithm, obtaining a deformation sequence through Riemannian manifold parallel transmission, and obtaining the shape space representation data through the deformation sequence. And finally, performing Kendall shape space inverse transformation to obtain expression migration data. According to the method, continuous and natural expression migration is realized, and meanwhile, the workload of manual adjustment is reduced.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a method and system for continuous facial expression transfer based on shape space. Background Art

[0002] In the prior art, facial expression transfer technology is mainly applied to the fields of automatic expression generation of digital humans, facial animation, and digital entertainment. Current facial expression transfer methods can be divided into three categories: learning-based, geometry-based, and parameterized methods. Learning-based expression transfer methods, such as Generative Adversarial Networks (GANs) and conditional GANs (cGANs) derived from GANs, double-encoder cGANs, etc., mainly solve the problem of two-dimensional facial expression transfer. For three-dimensional facial expressions, a combined method of parameterized methods and geometry-based methods, such as blendshape technology, is widely used. This technology transfers facial expressions through parameterized methods and combines geometric constraints to maintain the consistency of facial features.

[0003] However, there are the following deficiencies in the prior art: Existing expression transfer methods fail to fully consider the morphological characteristics of a specific person, resulting in insufficient accuracy of the expression transfer results. Although methods such as convolutional neural networks have achieved good results in the two-dimensional field, there is still a lack of a representation method that can directly express three-dimensional shapes in the three-dimensional field, and it is difficult to achieve continuous expression transfer. In addition, existing methods often require a large amount of manual annotation and adjustment work, increasing the implementation difficulty and time cost. Summary of the Invention

[0004] This application provides a method and system for continuous facial expression transfer based on shape space, which is used to accurately express the morphological characteristics of a human face in three-dimensional facial expression transfer, achieve continuous and natural expression transfer, and reduce the workload of manual adjustment.

[0005] In the first aspect, this application provides a method for continuous facial expression transfer based on shape space. The method for continuous facial expression transfer based on shape space includes: According to the three-dimensional mesh data of the source facial expression model and the target facial expression model, obtain the source facial shape space representation data and the target facial shape space representation data through Kendall shape space transformation; Use the spherical B-spline interpolation algorithm to perform interpolation calculation on the source facial shape space representation data to obtain the source facial continuous expression curve and the source facial control vertex sequence; Construct a two-parameter geodesic mapping function based on the geodesic distance between the source facial control vertex sequence and the source facial continuous expression curve; According to the two-parameter geodesic mapping function, the De Boor recurrence algorithm is used to calculate interpolation points for the target face shape space representation data, and a target face continuous expression curve is generated; Based on the target face continuous expression curve and the parallel transport operation of the Riemannian manifold, a target face shape space deformation sequence is obtained; The target face shape space deformation sequence is converted into three-dimensional mesh data through the Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

[0006] In a second aspect, the present application provides a face continuous expression migration system based on a shape space. The face continuous expression migration system based on a shape space includes: An acquisition module, configured to obtain source face shape space representation data and target face shape space representation data through Kendall shape space transformation according to the three-dimensional mesh data of the source face expression model and the target face expression model; An interpolation module, configured to perform interpolation calculation on the source face shape space representation data by using a spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence; A construction module, configured to construct a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; A calculation module, configured to calculate interpolation points for the target face shape space representation data by using the De Boor recurrence algorithm according to the two-parameter geodesic mapping function, and generate a target face continuous expression curve; An operation module, configured to obtain a target face shape space deformation sequence based on the target face continuous expression curve and the parallel transport operation of the Riemannian manifold; A conversion module, configured to convert the target face shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

[0007] In the technical solution provided by this application, shape space representation data is obtained through Kendall shape space transformation, eliminating the scale, rotation, and translation differences between the source face and the target face, and improving the stability of expression transfer. The spherical B-spline interpolation algorithm's processing of the shape space representation data ensures the continuity and smoothness of expression changes, avoiding the jumps and discontinuities that occur in traditional methods. The construction of the two-parameter geodesic mapping function provides an accurate description of the deformation path, achieving an accurate deformation mapping from the source face to the target face. The De Boor recurrence algorithm maintains numerical stability during the calculation of interpolation points, and the generated continuous expression curve of the target face has good geometric properties. The Riemannian manifold parallel transport operation maintains the local geometric structure during the shape deformation process, making the deformation result more natural. Finally, the process of converting the shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transformation ensures the accuracy and authenticity of the final expression transfer result. The entire solution not only reduces the computational complexity but also does not require a large amount of training data, is applicable to small-sample scenarios, and at the same time maintains a high expression transfer accuracy and naturalness. Brief Description of the Drawings

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0009] Figure 1 It is a schematic diagram of an embodiment of the continuous facial expression transfer method based on shape space in the embodiments of this application; Figure 2 It is a schematic diagram of the result after continuous facial expression transfer in the embodiments of this application; Figure 3 It is a schematic diagram of an embodiment of the continuous facial expression transfer system based on shape space in the embodiments of this application. Detailed Embodiment

[0010] The embodiments of the present application provide a method and system for continuous facial expression transfer based on a shape space. Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the term "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0011] For ease of understanding, the specific process of the embodiments of the present application will be described below. Please refer to Figure 1 , an embodiment of the method for continuous facial expression transfer based on a shape space in the embodiments of the present application includes: Step S101: Obtain source face shape space representation data and target face shape space representation data through Kendall shape space transformation according to the three-dimensional mesh data of the source face expression model and the target face expression model; Step S102: Use the spherical B-spline interpolation algorithm to perform interpolation calculation on the source face shape space representation data to obtain the source face continuous expression curve and the source face control vertex sequence; Step S103: Construct a two-parameter geodesic mapping function based on the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; Step S104: According to the two-parameter geodesic mapping function, use the De Boor recurrence algorithm to calculate interpolation points for the target face shape space representation data to generate the target face continuous expression curve; Step S105: Based on the target face continuous expression curve and the Riemannian manifold parallel transport operation, obtain the target face shape space deformation sequence; Step S106: Convert the target face shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transformation to obtain the target face continuous expression transfer data.

[0012] It can be understood that the execution subject of the present application can be a system for continuous facial expression transfer based on a shape space, or a terminal or a server. Specifically, it is not limited here. The embodiments of the present application will be described by taking the server as the execution subject as an example.

[0013] Specifically, by processing the 3D mesh data of the source facial expression model and the target facial expression model, the 3D mesh data contains spatial geometric information such as the vertex coordinates and topological connection relationships of the facial model. The Kendall shape space transformation maps this geometric information into the shape space, which is a mathematical expression space that can eliminate the influence of factors such as scale, rotation, and translation of the facial model. During the data processing, first, noise reduction is performed on the 3D mesh data. The high-frequency noise in the data is eliminated through discrete cosine transform. For the 3D facial mesh data with 1024 sampling points, the sampling noise can be effectively filtered after discrete cosine transform, and the main feature information of the face is retained. Subsequently, the source facial shape space representation data is processed through the spherical B-spline interpolation algorithm. The spherical B-spline interpolation algorithm is a mathematical method for data interpolation on the spherical space, which can maintain the continuity and smoothness of the data on the sphere. The algorithm first calculates the geodesic distance between adjacent data points, constructs the knot vector space, obtains the initial control points through progressive iterative approximation, and then obtains the final control point sequence through error minimization iterative optimization. For the source facial data, the initial sampling points are 512, and 32 control vertices are obtained after spherical B-spline interpolation. These control vertices can accurately describe the change characteristics of facial expressions.

[0014] Based on the obtained source facial control vertex sequence and the continuous expression curve, a two-parameter geodesic mapping function is constructed by calculating the geodesic distance. The geodesic distance reflects the shortest distance between two points on the surface, and the two-parameter geodesic mapping function describes the deformation path from the source face to the target face. The continuous expression curve is sampled through the curve closest point algorithm to obtain a dense sampling point sequence. The spherical geodesic distances between these sampling points and the control vertices are calculated to construct a two-parameter geodesic distance field. Next, the De Boor recurrence algorithm is used to perform interpolation calculations on the target facial shape space representation data. The De Boor recurrence algorithm is a numerically stable spline curve calculation method that can ensure that the generated curve has good continuity. The algorithm first uniformly samples the target facial data to obtain the initial interpolation point set, then constructs the spherical knot vector space in combination with the two-parameter geodesic mapping function, generates the control point data through spherical linear interpolation, and finally obtains the continuous expression curve of the target face.

[0015] After obtaining the continuous expression curve of the target face, the shape space deformation sequence is calculated through the Riemannian manifold parallel transport operation. The Riemannian manifold is a class of smooth manifolds with local Euclidean metrics, and the parallel transport operation can maintain the parallel properties of the vector field on the surface. In the specific operation, first, the continuous expression curve is decomposed into the tangent space to obtain the basis vector group, then the Riemannian connection coefficients are calculated, the geodesic covariant derivative is obtained by solving the parallel transport equation, and finally, the shape deformation field and the deformation matrix are constructed.

[0016] Restore the target face shape space deformation sequence to 3D mesh data through the inverse Kendall shape space transformation. This process first maps the shape space data to the Euclidean space through spherical projection, then restores the vertex coordinates in the 3D space through rigid transformation and affine transformation, and finally reconstructs the mesh connection relationship to obtain the final expression transfer result.

[0017] For example: For a high-precision 3D face model with 1024 vertices, noise reduction is first performed through discrete cosine transform, and 256 main frequency components are retained. During the spherical B-spline interpolation process, 32 key control vertices are determined by calculating the geodesic distance. The selection of these control vertices is based on key regions of expression changes, such as the eyebrows, eyes, mouth, etc. During the construction of the two-parameter geodesic mapping function, dense sampling of 128 sampling points is performed on the expression curve, and the complete geodesic distance field is calculated through these sampling points. Finally, the expression transfer result reconstructed on the target face retains the characteristics of the source face expression and adapts to the morphological characteristics of the target face. As Figure 2 shown, it is a schematic diagram of the result after continuous face expression transfer, where the blue represents the original curve and the red represents the expression of the corresponding face after transfer.

[0018] In the embodiment of the present application, the shape space representation data is obtained through the Kendall shape space transformation, eliminating the scale, rotation, and translation differences between the source face and the target face, and improving the stability of expression transfer. The spherical B-spline interpolation algorithm's processing of the shape space representation data ensures the continuity and smoothness of expression changes, avoiding the jumps and discontinuities that occur in traditional methods. The construction of the two-parameter geodesic mapping function provides an accurate description of the deformation path, realizing an accurate deformation mapping from the source face to the target face. The De Boor recurrence algorithm maintains numerical stability during the calculation of interpolation points, and the generated continuous expression curve of the target face has good geometric properties. The Riemannian manifold parallel transport operation maintains the local geometric structure during the shape deformation process, making the deformation result more natural. Finally, the process of converting the shape space deformation sequence to 3D mesh data through the inverse Kendall shape space transformation ensures the accuracy and authenticity of the final expression transfer result. The entire solution not only reduces the computational complexity but also does not require a large amount of training data, is applicable to small-sample scenarios, and at the same time maintains a high expression transfer accuracy and naturalness.

[0019] In a specific embodiment, the process of executing step S101 may specifically include the following steps: (1) Perform noise reduction processing on the 3D mesh data of the source face expression model and the target face expression model through discrete cosine transform to obtain the filtered 3D mesh data; (2) Calculate the geodesic distance for the filtered three-dimensional mesh data to generate a point-to-point geodesic distance matrix between the source face and the target face; (3) Based on the point-to-point geodesic distance matrix, perform rigid transformation alignment on the three-dimensional mesh data of the source face and the target face through least squares optimization to obtain the aligned three-dimensional mesh data; (4) Based on the aligned three-dimensional mesh data and the spherical projection algorithm, construct the unit spherical coordinate representations of the source face and the target face; (5) According to the unit spherical coordinate representations and the spherical geodesic distance calculation, generate the shape space distance function between the source face and the target face; (6) Through the shape space distance function and the Kendall shape space transformation mapping, obtain the shape space representation data of the source face and the shape space representation data of the target face.

[0020] Specifically, perform noise reduction processing on the three-dimensional mesh data of the source face and the target face. The discrete cosine transform converts the three-dimensional mesh data from the spatial domain to the frequency domain, and realizes data noise reduction by filtering out high-frequency components, retaining the main morphological features of the face, and removing the noise generated during the sampling and measurement processes. For the noise-reduced three-dimensional mesh data, it is necessary to calculate the geodesic distance.

[0021]

[0022] Wherein: represents the geodesic distance between adjacent vertices; and represent the coordinates of adjacent mesh vertices; and represent the magnitudes of the vertex coordinate vectors.

[0023] By calculating the geodesic distances between all vertex pairs, a complete point-to-point geodesic distance matrix is generated. This matrix reflects the shortest path length between any two points on the three-dimensional mesh. Then, the least squares optimization method is used to perform rigid transformation alignment on the source face and the target face. Based on the geodesic distance matrix, the optimal rotation matrix and translation vector are found through an iterative method to minimize the sum of the squared distances between corresponding points. The rigid transformation ensures that the face shape does not deform during the alignment process.

[0024] The aligned three-dimensional mesh data is mapped onto the unit sphere through the spherical projection algorithm. The parametric expression is used:

[0025] Wherein: is the parametric value of the vertex; and are the geodesic distances; n is the total number of vertices.

[0026] The unit spherical coordinate representation obtained by projection eliminates the influence of scale differences and provides a standardized basis for subsequent shape space distance calculation. Based on the unit spherical coordinates, the spherical geodesic distance between the source face and the target face is calculated to generate the shape space distance function. Spherical linear interpolation is used, and the construction of the knot vector during the interpolation process follows the formula:

[0027] where the meanings of the respective parameters are: represents the degree of the B-spline curve, which controls the smoothness of the curve; represents the parameter value of the i-th vertex, which is calculated from the geodesic distance; represents the current calculated knot index, and the range is from 1 to n - p; n represents the total number of control points; represents the total length of the knot vector, satisfying the relationship m = n + p + 1; to are all set to 0, indicating the starting boundary condition of the curve; to are all set to 1, indicating the ending boundary condition of the curve.

[0028] It is used to construct the knot vector during the spherical B-spline interpolation process, ensuring the continuity and numerical stability of the interpolation curve. Through the shape space distance function and the Kendall shape space transformation mapping, the representation data of the source face and the target face in the shape space is obtained. The normalized representation of the face shape is realized, laying a foundation for subsequent expression transfer.

[0029] For example: For a 3D face model with 2048 vertices, 512 main frequency components are retained after noise reduction by discrete cosine transform. The geodesic distance calculation generates a 2048×2048 distance matrix, which records the distance values between all vertex pairs. The rigid transformation alignment process achieves the exact matching of the source face and the target face through 50 iterations. After spherical projection, all vertices are mapped onto the unit sphere, and a shape space distance function is constructed with 128 control points.

[0030] In a specific embodiment, the process of executing step S102 may specifically include the following steps: (1) Measure the distances between adjacent data points of the source face shape space representation data through geodesic distance calculation to obtain the source face shape space geodesic distance sequence; (2) Based on the source face shape space geodesic distance sequence and the knot vector generation algorithm, construct the spherical B-spline knot vector space; (3) Obtain the initial control point data of the source face shape space according to the spherical B-spline knot vector space and the progressive iterative approximation operation; (4) Based on the initial control point data of the source face shape space, perform curve fitting on the source face shape space representation data through spherical error minimization iteration to generate optimized control points for the source face shape space; (5) Use the optimized control points of the source face shape space and B-spline basis functions to calculate and construct a continuous curve expression for the source face shape space; (6) Through De Boor recurrence decomposition of the continuous curve expression of the source face shape space, obtain the continuous expression curve of the source face and the sequence of source face control vertices.

[0031] Specifically, in the process of face expression transfer based on the shape space, the processing of the source face shape space representation data starts with geodesic distance calculation. The geodesic distance represents the shortest path length between two points on the surface. Based on the calculated geodesic distance sequence, it is necessary to construct a spherical B-spline knot vector space. The construction of the knot vector space uses a parameterization method. First, calculate the parameter values, and then construct the knot vector. Obtain the initial control point data through a progressive iterative approximation algorithm. The control point calculation formula for the spherical B-spline curve is:

[0032] Where: is the i-th control point in the r-th iteration; is the iteration weight coefficient; is the spherical angle parameter; and are the control points of the previous iteration.

[0033] Perform spherical error minimization iteration optimization on the initial control points to generate optimized control points. The optimization process uses the least squares criterion, and adjusts the control point positions through repeated iterations until the curve fitting error is less than a preset threshold. Use the optimized control points and B-spline basis functions to construct a continuous curve expression for the source face shape space. Finally, perform De Boor recurrence decomposition on the continuous curve expression to obtain the continuous expression curve and the sequence of control vertices. The De Boor recurrence formula is as follows:

[0034] Where: represents the curve point coordinates at parameter t; represents the position of the i-th control point after p times of recurrence; represents the parameter value of the current calculation point; represents the parameter value of the i-th knot; represents the degree of the B-spline curve; r represents the current recurrence times; i represents the index of the control point; represents the interpolation weight coefficient at the r-th recurrence.

[0035] For example, assume that a 3D face model with 1024 vertices is processed. First, the geodesic distance between adjacent data points is calculated to generate a geodesic distance sequence consisting of 1024 distance values. A parameter space is constructed based on these distance values, and 32 control points are selected as the initial control point set. After 50 progressive iterative approximations, the weight coefficient in each iteration gradually decreases from 1.0 to 0.1, and finally an optimized control point set is obtained. The optimized control points are used to construct a B-spline curve with a degree of 3, and 100 sampling points are calculated through the De Boor recurrence algorithm to generate an expression curve.

[0036] In a specific embodiment, the process of executing step S103 may specifically include the following steps: (1) Sampling and calculating the source face continuous expression curve through the curve closest point algorithm to obtain a sequence of source face expression curve sampling points; (2) Performing spherical geodesic distance operations on the sequence of source face expression curve sampling points to obtain a source face expression curve distance matrix; (3) Constructing a source face bi-parameter geodesic distance field based on the source face expression curve distance matrix and the source face control vertex sequence through the Laplace operator; (4) Generating a geodesic direction vector group based on the source face bi-parameter geodesic distance field and the manifold gradient descent algorithm; (5) Constructing a source face geodesic transformation matrix using the geodesic direction vector group and the spherical exponential map; (6) Decomposing the source face geodesic transformation matrix through the Riemannian metric tensor to obtain a bi-parameter geodesic mapping function.

[0037] Specifically, in the process of face expression transfer based on the shape space, constructing a bi-parameter geodesic mapping function is a key step. First, the source face continuous expression curve is sampled through the curve closest point algorithm, which calculates the shortest distance between any point on the curve and a given point. The selection formula for sampling points is as follows:

[0038] Where: represents the distance from the curve point to the given point; represents the point on the curve; represents the given point.

[0039] Perform spherical geodesic distance operations on the sampling point sequence, and the calculation process uses the derivative equation:

[0040] Where: represents the derivative of the distance with respect to the parameter; Represents the tangent vector of the curve.

[0041] Based on the expression curve distance matrix and the control vertex sequence, a two-parameter geodesic distance field is constructed using the Laplace operator. The Laplace operator describes the local geometric features on the surface, and the distribution of the distance field is obtained by solving the Laplace equation. Then, using the manifold gradient descent algorithm, the geodesic direction vector is calculated based on the geodesic distance field. This algorithm iteratively optimizes on the Riemannian manifold, and the vector is updated according to the gradient direction in each iteration. The spherical exponential map maps the vector in the tangent space to the manifold to construct the geodesic transformation matrix.

[0042] Finally, the two-parameter geodesic mapping function is obtained through the decomposition of the Riemannian metric tensor. This process involves the eigenvalue decomposition of the tensor, thereby obtaining the local geometric structure of the mapping space.

[0043] For example: For a face model composed of 1024 vertices, first, 256 sampling points are selected on the continuous expression curve. By calculating the spherical geodesic distance between these sampling points and the control vertices, a 256×256 distance matrix is generated. When constructing the distance field using the Laplace operator, a 16×16 neighborhood window is selected for local geometric feature extraction. The learning rate of the manifold gradient descent algorithm is set to 0.01, and a stable set of direction vectors is obtained after 100 iterations. The finally generated two-parameter geodesic mapping function can accurately describe the deformation path from the source face to the target face, providing a mathematical basis for subsequent expression transfer. The entire process strictly follows the differential geometry theory, ensuring the accuracy and continuity of expression transfer. In a specific calculation example, the threshold of the nearest point distance is set to 0.001, and the iteration stops when the distance change obtained by the iterative calculation is less than this threshold. The Gaussian weight function is used for the construction of the distance field, and the weight coefficient decays exponentially with the increase of the distance. During the calculation of the geodesic direction vector, the step size of the gradient descent is dynamically adjusted according to the curvature at the current position, ensuring the convergence of the algorithm. The entire processing process is guaranteed the stability and reliability of expression transfer through strict mathematical derivation and experimental verification.

[0044] In a specific embodiment, the process of executing step S104 may specifically include the following steps: (1) Uniformly sample the target face shape space representation data through the curve parameterization algorithm to obtain the initial interpolation point set of the target face; (2) Based on the initial interpolation point set of the target face and the two-parameter geodesic mapping function, construct the spherical node vector space; (3) According to the spherical node vector space and the initial interpolation point set of the target face, generate the target face curve control point data through spherical linear interpolation operation; (4) Use the target face curve control point data and the De Boor recurrence algorithm to construct the target face spherical B-spline basis function; (5) Obtain the optimized interpolation point sequence of the target face according to the spherical B-spline basis function of the target face and the geodesic distance calculation. (6) Perform spherical curve fitting operation on the optimized interpolation point sequence of the target face to obtain the continuous expression curve of the target face.

[0045] Specifically, during the target face expression transfer process, it is first necessary to uniformly sample the data representing the target face shape space. Based on the initial interpolation point set and the two-parameter geodesic mapping function, a spherical knot vector space is constructed. When performing linear interpolation operation on the sphere, the spherical linear interpolation formula is used:

[0046] Where: represents the interpolation point; represents the interpolation parameter; represents the spherical angle; , represent the interpolation endpoints. Construct the spherical B-spline basis function according to the De Boor recurrence algorithm. For example: for a target face model with 2048 vertices, first perform uniform sampling and select 256 initial interpolation points. These points are evenly distributed in the parameter space, and the parameter values range from 0 to 1. When constructing the spherical knot vector space, select the degree of the B-spline curve to be 3 and generate a knot vector with a length of 260. For the spherical linear interpolation operation, select 32 control points, and each control point is obtained by weighted averaging of adjacent interpolation points.

[0047] The B-spline basis function constructed using the De Boor recurrence algorithm has the number of iterations set to 50, and the weight coefficient in each iteration gradually decreases from 1.0 to 0.1. The geodesic distance calculation uses a numerical method with a precision of 0.001, ensuring the precise positioning of the interpolation points. Finally, through spherical curve fitting, a continuous curve describing the expression change of the target face is generated. In the specific implementation, the curve parameterization uses the cumulative chord length parameterization method to ensure the uniform distribution of the sampling points. The construction of the spherical knot vector space takes into account the local support property, making each basis function non-zero only in a finite interval. During the generation process of the control points, spherical distance weighted averaging is used to ensure that the interpolation result is on the sphere.

[0048] In a specific embodiment, the process of executing step S105 may specifically include the following steps: (1) Perform tangent space decomposition on the continuous expression curve of the target face through the Riemann metric tensor to obtain the tangent space basis vector group of the target face; (2) Based on the tangent space basis vector group of the target face and the Christoffel symbol calculation, construct the Riemann connection coefficients of the target face; (3) Generate the geodesic covariant derivative of the target face according to the Riemannian connection coefficient of the target face and the manifold parallel transport equation; (4) Construct the shape deformation field of the target face by using the geodesic covariant derivative of the target face and the spherical exponential map; (5) Calculate the shape space deformation matrix of the target face based on the shape deformation field of the target face and the Riemannian manifold parallel transport operator; (6) Obtain the shape space deformation sequence of the target face through the homeomorphic transformation of the shape space of the target face shape space deformation matrix.

[0049] Specifically, in the key steps of target face expression transfer, the Riemannian geometry theory provides important mathematical tools. First, use the Riemannian metric tensor to decompose the continuous expression curve of the target face into the tangent space. Based on the tangent space basis vector group, calculate the Riemannian connection coefficient through the Christoffel symbol. The calculation of the connection coefficient uses the derivative equation. According to the Riemannian connection coefficient and the manifold parallel transport equation, calculate the geodesic covariant derivative. This process follows the parallel transport condition:

[0050] Combine the geodesic covariant derivative with the spherical exponential map to construct the shape deformation field. The exponential map maps the vector in the tangent space to the manifold, maintaining the local characteristics of the geometric structure.

[0051] For example: For a target face expression curve containing 1024 vertices, 16 basis vectors are selected for the tangent space decomposition, and each basis vector corresponds to a main deformation direction. The Christoffel symbol calculation generates a 16×16 connection coefficient matrix, which describes the local geometric characteristics of the surface. The calculation of the geodesic covariant derivative uses the numerical integration method, and the step size is set to 0.01 to ensure the calculation accuracy. During the construction of the shape deformation field, the local deformation vector is calculated for each vertex, and the intensity of the deformation field decays exponentially with the distance to the deformation center. Through the Riemannian manifold parallel transport operator, the local deformation is propagated to the entire face model. The finally obtained shape space deformation sequence contains 32 key frames, and each key frame records the gradual change process of the face shape. The entire processing process strictly follows the differential geometry theory, ensuring the continuity and naturalness of the expression transfer. In a specific experiment, the selection of the tangent space basis vectors considered the main deformation modes of the face expression, including the motion characteristics of areas such as eyebrows, eyes, and mouth. The calculation accuracy of the Riemannian connection coefficient was set to 0.001 to ensure the accurate description of the local geometric characteristics. The construction of the shape deformation field adopted an adaptive weight scheme, and the weight coefficient was dynamically adjusted according to the local curvature. The entire algorithm process has been strictly verified, achieving a stable and accurate expression transfer effect.

[0052] In a specific embodiment, the process of executing step S106 may specifically include the following steps: (1) Perform coordinate transformation on the target face shape space deformation sequence through the spherical projection algorithm to obtain the target face Euclidean space deformation data; (2) Use the target face Euclidean space deformation data and the rigid transformation matrix to construct the target face shape alignment relationship; (3) Generate the target face shape parameter sequence according to the target face shape alignment relationship and the spatial scale factor; (4) Calculate the target face vertex mapping coordinates based on the target face shape parameter sequence and the affine transformation matrix; (5) Construct the target face mesh connection relationship based on the target face vertex mapping coordinates and the topology-preserving algorithm; (6) Obtain the target face continuous expression transfer data through mesh reconstruction operation on the target face mesh connection relationship.

[0053] Specifically, in the process of completing the conversion from the shape space to the three-dimensional mesh data, first process the target face shape space deformation sequence through the spherical projection algorithm. According to the projection formula:

[0054] Where: Represents the projected parameter value, Represents the geodesic distance. n represents the total number of vertices.

[0055] Align the obtained Euclidean space deformation data through the rigid transformation matrix. The rigid transformation maintains the shape invariance, including rotation and translation transformations. According to the shape alignment relationship, combined with the spatial scale factor, generate the shape parameter sequence. The shape parameter sequence describes the geometric features of the model at different scales. The affine transformation matrix maps the shape parameters to the three-dimensional space coordinate system to calculate the final position of each vertex.

[0056] The topology-preserving algorithm ensures that the connection relationship of the mesh remains unchanged during the deformation process. Finally, generate the complete expression transfer data through mesh reconstruction operation.

[0057] For example: For a target face model with 2048 vertices, first, the shape space data is mapped to the Euclidean space through spherical projection. The relative position relationships of 256 key vertices are maintained during the projection process. The rigid transformation matrix is represented in homogeneous coordinates of 4×4 and includes rotation and translation components. The scale factor is calculated based on the size of the bounding box of the model, with an initial value set to 1.0 and then dynamically adjusted according to the degree of deformation. The affine transformation uses a 3×4 transformation matrix to map the coordinates of each vertex. The mesh connection relationships are represented by triangular patches, with a total of 4096 patches. The finally reconstructed expression transfer data includes 32 key frames, and each frame maintains the topological structure of the original mesh. The entire processing process ensures the accuracy of the geometry and topology, achieving a natural and smooth expression transfer effect. In the specific implementation, the numerical precision of the spherical projection is set to 0.001 to ensure the accuracy of the coordinate transformation. The number of optimization iterations for the rigid transformation is 50 times, and the residual of each iteration is less than 0.0001. The topological structure preservation algorithm maintains the local features of the mesh through the Laplace operator.

[0058] The above describes the method for continuous facial expression transfer based on shape space in the embodiments of the present application. Next, the system for continuous facial expression transfer based on shape space in the embodiments of the present application will be described. Please refer to Figure 3 , an embodiment of the system for continuous facial expression transfer based on shape space in the embodiments of the present application includes: An acquisition module, configured to obtain source face shape space representation data and target face shape space representation data through Kendall shape space transformation according to the three-dimensional mesh data of the source face expression model and the target face expression model; An interpolation module, configured to perform interpolation calculation on the source face shape space representation data by using the spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence; A construction module, configured to construct a two-parameter geodesic mapping function based on the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; A calculation module, configured to calculate interpolation points for the target face shape space representation data by using the De Boor recurrence algorithm according to the two-parameter geodesic mapping function to generate a target face continuous expression curve; An operation module, configured to obtain a target face shape space deformation sequence based on the target face continuous expression curve and the Riemannian manifold parallel transport operation; A conversion module, configured to convert the target face shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transformation to obtain target face continuous expression transfer data.

[0059] Through the collaborative cooperation of the above-mentioned various components, the shape space representation data is obtained through the Kendall shape space transformation, eliminating the scale, rotation, and translation differences between the source face and the target face, and improving the stability of expression transfer. The spherical B-spline interpolation algorithm's processing of the shape space representation data ensures the continuity and smoothness of expression changes, avoiding the jumps and discontinuities that occur in traditional methods. The construction of the two-parameter geodesic mapping function provides an accurate description of the deformation path, achieving an accurate deformation mapping from the source face to the target face. The De Boor recurrence algorithm maintains numerical stability during the calculation of interpolation points, and the generated continuous expression curve of the target face has good geometric properties. The Riemannian manifold parallel transport operation maintains the local geometric structure during the shape deformation process, making the deformation result more natural. Finally, the process of converting the shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transformation ensures the accuracy and authenticity of the final expression transfer result. The entire scheme not only reduces the computational complexity but also does not require a large amount of training data, is applicable to small-sample scenarios, and at the same time maintains a high expression transfer accuracy and naturalness.

[0060] As described above, the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for transferring continuous facial expressions based on shape space, characterized in that: The method for transferring continuous facial expressions based on shape space comprises: According to the three-dimensional mesh data of the source facial expression model and the target facial expression model, the source facial shape space representation data and the target facial shape space representation data are obtained by Kendall shape space transformation; Using a spherical B-spline interpolation algorithm to perform interpolation calculation on the source face shape space representation data, to obtain a source face continuous expression curve and a source face control vertex sequence; Constructing a dual-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; According to the dual-parameter geodesic mapping function, the De Boor recursive algorithm is used to calculate interpolation points for the target face shape space representation data to generate a target face continuous expression curve; Based on the target face continuous expression curve and the Riemann manifold parallel transmission operation, a target face shape space deformation sequence is obtained; The target face shape space deformation sequence is converted into three-dimensional mesh data through Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

2. The method for transferring continuous facial expressions based on shape space according to claim 1, characterized in that: The method of obtaining source face shape space representation data and target face shape space representation data by Kendall shape space transformation according to the three-dimensional mesh data of the source face expression model and the target face expression model comprises: Performing noise reduction processing on the three-dimensional mesh data of the source facial expression model and the target facial expression model by discrete cosine transform to obtain filtered three-dimensional mesh data; Performing geodesic distance calculation on the filtered three-dimensional grid data to generate a point-to-point geodesic distance matrix between the source face and the target face; According to the point-to-point geodesic distance matrix, the three-dimensional mesh data of the source face and the target face are rigidly transformed and aligned by least squares optimization to obtain aligned three-dimensional mesh data; Based on the aligned three-dimensional mesh data and the spherical projection algorithm, constructing unit spherical coordinate representations of the source face and the target face; Generate a shape space distance function between a source face and a target face according to the unit spherical coordinate representation and the spherical geodesic distance calculation; The source face shape space representation data and the target face shape space representation data are obtained through the shape space distance function and the Kendall shape space transformation mapping.

3. The method for transferring continuous facial expressions based on shape space according to claim 1, characterized in that: The method of using a spherical B-spline interpolation algorithm to interpolate the source face shape space representation data to obtain a source face continuous expression curve and a source face control vertex sequence includes: Measuring the distances of adjacent data points of the source face shape space representation data by geodesic distance calculation to obtain a source face shape space geodesic distance sequence; Based on the source face shape space geodesic distance sequence and the node vector generation algorithm, a spherical B-spline node vector space is constructed; According to the spherical B-spline node vector space and the progressive iterative approximation operation, initial control point data of the source face shape space is obtained; According to the initial control point data of the source face shape space, curve fitting is performed on the source face shape space representation data through spherical error minimization iteration to generate the source face shape space optimization control points; Utilizing the source face shape space optimization control points and B-spline basis function calculation, a source face shape space continuous curve expression is constructed; The source face continuous expression curve and the source face control vertex sequence are obtained by performing De Boor recursive decomposition on the source face shape space continuous curve expression.

4. The method for transferring continuous facial expressions based on shape space according to claim 1, characterized in that: The step of constructing a dual-parameter geodesic mapping function based on the geodesic distance between the source face control vertex sequence and the source face continuous expression curve comprises: Sampling and calculating the source face continuous expression curve by using a curve nearest point algorithm to obtain a source face expression curve sampling point sequence; Obtaining a source facial expression curve distance matrix according to the source facial expression curve sampling point sequence and the spherical geodesic distance operation; According to the source face expression curve distance matrix and the source face control vertex sequence, a source face dual-parameter geodesic distance field is constructed by Laplacian operator; Generate a geodesic direction vector group based on the source face dual-parameter geodesic distance field and manifold gradient descent algorithm; Constructing a source face geodesic transformation matrix using the geodesic direction vector group and spherical index mapping; The source face geodesic transformation matrix is ​​decomposed through the Riemann metric tensor to obtain the two-parameter geodesic mapping function.

5. The method for transferring continuous facial expressions based on shape space according to claim 4, characterized in that: The method of calculating interpolation points of the target face shape space representation data using the De Boor recursive algorithm according to the dual-parameter geodesic mapping function to generate a target face continuous expression curve includes: Uniformly sampling the target face shape space representation data by using a curve parameterization algorithm to obtain an initial interpolation point set of the target face; Constructing a spherical node vector space based on the target face initial interpolation point set and the dual-parameter geodesic mapping function; Generating target face curve control point data through spherical linear interpolation operation according to the spherical node vector space and the target face initial interpolation point set; Using the target face curve control point data and the De Boor recursive algorithm, constructing the target face spherical B-spline basis function; Obtaining a sequence of optimized interpolation points of the target face according to the spherical B-spline basis function of the target face and the geodesic distance calculation; The optimized interpolation point sequence of the target face is subjected to a spherical curve fitting operation to obtain the continuous expression curve of the target face.

6. The method for transferring continuous facial expressions based on shape space according to claim 5, characterized in that: The step of obtaining a target face shape space deformation sequence based on the target face continuous expression curve and the Riemann manifold parallel transmission operation includes: Performing tangent space decomposition on the continuous expression curve of the target face by using the Riemann metric tensor to obtain a tangent space basis vector group of the target face; Constructing the Riemann connection coefficient of the target face based on the target face tangent space basis vector group and Kristofer symbol calculation; Generate a geodesic covariant derivative of the target face according to the Riemann connection coefficient of the target face and the manifold parallel transmission equation; Constructing a target face shape deformation field by using the target face geodesic covariant derivative and spherical exponential mapping; Calculating a target face shape spatial deformation matrix according to the target face shape deformation field and the Riemann manifold parallel transmission operator; The target face shape space deformation matrix is ​​transformed through shape space homeomorphism to obtain the target face shape space deformation sequence.

7. The method for transferring continuous facial expressions based on shape space according to claim 6, characterized in that: The step of converting the target face shape space deformation sequence into three-dimensional mesh data by Kendall shape space inverse transformation to obtain the target face continuous expression migration data includes: Performing coordinate transformation on the target face shape space deformation sequence by using a spherical projection algorithm to obtain target face Euclidean space deformation data; Using the target face Euclidean space deformation data and the rigid transformation matrix, constructing a target face shape alignment relationship; Generating a target face shape parameter sequence according to the target face shape alignment relationship and the spatial scale factor; Calculating the target face vertex mapping coordinates according to the target face shape parameter sequence and the affine transformation matrix; Based on the target face vertex mapping coordinates and the topology structure preservation algorithm, constructing the target face mesh connection relationship; The target face mesh connection relationship is subjected to mesh reconstruction operation to obtain the target face continuous expression migration data.

8. A system for continuous facial expression migration based on shape space, used to implement the method for continuous facial expression migration based on shape space as described in any one of claims 1 to 7, characterized in that: The face continuous expression migration system based on shape space includes: An acquisition module, for obtaining source face shape space representation data and target face shape space representation data through Kendall shape space transformation according to the three-dimensional mesh data of the source face expression model and the target face expression model; An interpolation module, used for performing interpolation calculation on the source face shape space representation data using a spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence; A construction module, used for constructing a dual-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; A calculation module, for calculating interpolation points of the target face shape space representation data using the De Boor recursive algorithm according to the dual-parameter geodesic mapping function to generate a continuous expression curve of the target face; A calculation module, used for obtaining a target face shape space deformation sequence based on the target face continuous expression curve and Riemann manifold parallel transmission calculation; The conversion module is used to convert the target face shape space deformation sequence into three-dimensional grid data through Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

Images