Method and system for continuous facial expression transfer based on shape space

Abstract

This application relates to the field of data processing technology and discloses a method and system for continuous facial expression transfer based on shape space. The method comprises: obtaining shape space representation data through Kendall shape space transformation; generating a continuous expression curve for the source face using a spherical B-spline interpolation algorithm; constructing a two-parameter geodesic mapping function; calculating the target face interpolation points using the De Boor recursive algorithm; obtaining a deformation sequence through Riemannian manifold parallel transmission; and finally obtaining expression transfer data through an inverse Kendall shape space transformation. This application achieves continuous and natural expression transfer while reducing the workload of manual adjustment.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and system for continuously migrating facial expressions based on shape space. Background Art

[0002] In the existing technology, facial expression transfer technology is mainly used in the fields of automatic expression generation of digital humans, facial animation and digital entertainment. The current facial expression transfer methods can be divided into three categories: learning-based, geometry-based and parameterized methods. Learning-based expression transfer methods, such as generative adversarial networks (GANs) and conditional GANs (cGANs) derived from GANs, dual-encoder cGAN, etc., mainly solve the problem of two-dimensional facial expression transfer. For three-dimensional facial expressions, the combination of parametric methods and geometric-based methods, such as blendshape technology, is widely used. This technology transfers facial expressions through a parametric method and combines geometric constraints to maintain the consistency of facial features.

[0003] However, existing technologies suffer from the following shortcomings: existing expression transfer methods fail to fully consider the morphological characteristics of specific individuals, resulting in inaccurate expression transfer results. While methods such as convolutional neural networks have achieved promising results in two-dimensional domains, a representation method that can directly represent three-dimensional shapes is still lacking in three-dimensional domains, making continuous expression transfer difficult. Furthermore, existing methods often require extensive manual annotation and adjustment, increasing implementation complexity and time costs. Summary of the Invention

[0004] The present application provides a method and system for continuous facial expression migration based on shape space, which is used to accurately express facial morphological features in three-dimensional facial expression migration, and to achieve continuous and natural expression migration, while reducing the workload of manual adjustment.

[0005] In a first aspect, the present application provides a method for transferring continuous facial expressions based on a shape space, the method comprising:

[0006] According to the three-dimensional mesh data of the source facial expression model and the target facial expression model, the source facial shape space representation data and the target facial shape space representation data are obtained by Kendall shape space transformation;

[0007] Using a spherical B-spline interpolation algorithm to perform interpolation calculation on the source face shape space representation data to obtain a source face continuous expression curve and a source face control vertex sequence;

[0008] Constructing a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve;

[0009] According to the dual-parameter geodesic mapping function, the De Boor recursive algorithm is used to calculate interpolation points for the target face shape space representation data to generate a continuous expression curve for the target face;

[0010] Based on the parallel transmission operation of the continuous expression curve of the target face and the Riemann manifold, a spatial deformation sequence of the target face shape is obtained;

[0011] The target face shape space deformation sequence is converted into three-dimensional grid data through Kendall shape space inverse transformation to obtain target face continuous expression migration data.

[0012] In a second aspect, the present application provides a system for continuously migrating facial expressions based on a shape space, the system comprising:

[0013] An acquisition module is used to obtain source face shape space representation data and target face shape space representation data through Kendall shape space transformation based on the three-dimensional mesh data of the source face expression model and the target face expression model;

[0014] An interpolation module, configured to perform interpolation calculation on the source face shape space representation data using a spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence;

[0015] A construction module, configured to construct a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve;

[0016] a calculation module, configured to calculate interpolation points for the target face shape space representation data using the De Boor recursive algorithm according to the dual-parameter geodesic mapping function, and generate a continuous expression curve for the target face;

[0017] A calculation module, configured to obtain a target face shape spatial deformation sequence based on a parallel transmission operation between the target face continuous expression curve and the Riemann manifold;

[0018] The conversion module is used to convert the target face shape space deformation sequence into three-dimensional grid data through Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

[0019] The technical solution provided in this application obtains shape space representation data through Kendall shape space transformation, eliminating scale, rotation, and translation differences between the source and target faces and improving the stability of expression transfer. The spherical B-spline interpolation algorithm processes the shape space representation data, ensuring the continuity and smoothness of expression changes, avoiding the jumps and discontinuities seen in traditional methods. The construction of a two-parameter geodesic mapping function provides an accurate description of the deformation path, achieving precise deformation mapping from the source face to the target face. The De Boor recursive algorithm maintains numerical stability during the calculation of interpolation points, and the generated continuous expression curve for the target face has good geometric properties. The Riemannian manifold parallel transport operation preserves the local geometric structure during the shape deformation process, making the deformation result more natural. Finally, the process of converting the shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transformation ensures the accuracy and authenticity of the final expression transfer result. This entire solution not only reduces computational complexity but also eliminates the need for extensive training data, making it suitable for small sample scenarios while maintaining high expression transfer accuracy and naturalness. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a schematic diagram of an embodiment of a method for continuous facial expression migration based on shape space in an embodiment of the present application;

[0022] Figure 2 This is a schematic diagram of the result after continuous facial expression migration in an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of an embodiment of a continuous facial expression migration system based on shape space in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application provide a method and system for continuous facial expression migration based on shape space. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In one embodiment of the present application, a method for continuously migrating facial expressions based on shape space includes:

[0026] Step S101, obtaining source face shape space representation data and target face shape space representation data through Kendall shape space transformation according to the three-dimensional mesh data of the source face expression model and the target face expression model;

[0027] Step S102: using a spherical B-spline interpolation algorithm to perform interpolation calculation on the source face shape space representation data to obtain a source face continuous expression curve and a source face control vertex sequence;

[0028] Step S103: constructing a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve;

[0029] Step S104: using the De Boor recursive algorithm to calculate interpolation points for the target face shape space representation data based on the dual-parameter geodesic mapping function, and generating a continuous expression curve for the target face;

[0030] Step S105: obtaining a target face shape spatial deformation sequence based on the target face continuous expression curve and the Riemann manifold parallel transmission operation;

[0031] Step S106: convert the target face shape space deformation sequence into three-dimensional mesh data through Kendall shape space inverse transformation to obtain target face continuous expression migration data.

[0032] It is understandable that the execution subject of this application can be a face continuous expression migration system based on shape space, or a terminal or a server, which is not limited here. The embodiment of this application is described by taking the server as the execution subject as an example.

[0033] Specifically, the Kendall shape space transform processes the 3D mesh data of the source and target facial expression models. The 3D mesh data contains spatial geometric information, such as the facial model's vertex coordinates and topological connectivity. This geometric information is then mapped into shape space using the Kendall shape space transform, a mathematical representation that eliminates the effects of scale, rotation, and translation on the facial model. During data processing, the 3D mesh data is first subjected to denoising, using a discrete cosine transform to remove high-frequency noise. For 3D facial mesh data with 1024 sampling points, the discrete cosine transform effectively removes sampling noise, preserving the key facial features. The source facial shape space representation data is then processed using the spherical B-spline interpolation algorithm, a mathematical method for interpolating data on a spherical surface that maintains continuity and smoothness. The algorithm first calculates the geodesic distances between adjacent data points, constructs a node vector space, and obtains initial control points through progressive iterative approximation. The final control point sequence is then obtained through iterative optimization using error minimization. For the source face data, the initial sampling points are 512, and 32 control vertices are obtained after spherical B-spline interpolation. These control vertices can accurately describe the changing characteristics of facial expressions.

[0034] Based on the obtained source face control vertex sequence and continuous expression curve, a two-parameter geodesic mapping function is constructed by calculating geodesic distances. Geodesic distance reflects the shortest distance between two points on a surface, while the two-parameter geodesic mapping function describes the deformation path from the source face to the target face. The continuous expression curve is sampled using the curve nearest point algorithm to obtain a dense sequence of sampling points. The spherical geodesic distances between these sampling points and the control vertices are calculated to construct a two-parameter geodesic distance field. Next, the De Boor recursion algorithm is used to interpolate the target face shape space representation data. The De Boor recursion algorithm is a numerically stable spline curve calculation method that ensures the generated curve has good continuity. The algorithm first uniformly samples the target face data to obtain an initial interpolation point set. Then, combined with the two-parameter geodesic mapping function, a spherical node vector space is constructed. Spherical linear interpolation is used to generate control point data, ultimately resulting in a continuous expression curve for the target face.

[0035] After obtaining the target face's continuous expression curve, the shape space deformation sequence is calculated using Riemannian manifold parallel transport operations. Riemannian manifolds are a type of smooth manifold with a local Euclidean metric. Parallel transport operations preserve the parallel nature of vector fields on the surface. Specifically, the continuous expression curve is first decomposed into tangent space to obtain a set of basis vectors. The Riemannian connection coefficients are then calculated. The geodesic covariant derivatives are obtained by solving the parallel transport equations, ultimately constructing the shape deformation field and deformation matrix.

[0036] The target face's shape space deformation sequence is restored to 3D mesh data through the inverse Kendall shape space transform. This process first maps the shape space data to Euclidean space through spherical projection, then recovers the vertex coordinates in 3D space through rigid and affine transformations. Finally, the mesh connectivity is reconstructed to obtain the final expression transfer result.

[0037] For example: For a high-precision 3D facial model containing 1024 vertices, noise reduction is first performed through discrete cosine transform, and 256 main frequency components are retained. In the spherical B-spline interpolation process, 32 key control vertices are determined by calculating the geodesic distance. The selection of these control vertices is based on the key areas of expression changes, such as eyebrows, eyes, mouth and other parts. In the process of constructing the two-parameter geodesic mapping function, the expression curve is densely sampled with 128 sampling points, and the complete geodesic distance field is calculated through these sampling points. The expression migration result finally reconstructed on the target face maintains the characteristics of the source face expression, while adapting to the morphological characteristics of the target face. Figure 2 The figure shows the result of continuous facial expression migration, where blue represents the original curve and red represents the facial expression of the corresponding face after migration.

[0038] In the embodiments of the present application, shape space representation data is obtained through the Kendall shape space transform, eliminating scale, rotation, and translation differences between the source and target faces and improving the stability of expression transfer. The spherical B-spline interpolation algorithm processes the shape space representation data to ensure the continuity and smoothness of expression changes, avoiding the jumps and discontinuities found in traditional methods. The construction of a two-parameter geodesic mapping function provides an accurate description of the deformation path, achieving precise deformation mapping from the source face to the target face. The De Boor recursive algorithm maintains numerical stability during the calculation of interpolation points, and the generated continuous expression curve of the target face has good geometric properties. The Riemannian manifold parallel transport operation preserves the local geometric structure during the shape deformation process, making the deformation result more natural. Finally, the process of converting the shape space deformation sequence into three-dimensional mesh data through the Kendall shape space inverse transform ensures the accuracy and authenticity of the final expression transfer result. The entire solution not only reduces computational complexity but also does not require a large amount of training data, making it suitable for small sample scenarios, while maintaining high expression transfer accuracy and naturalness.

[0039] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0040] (1) Denoising the three-dimensional mesh data of the source and target facial expression models by discrete cosine transform to obtain filtered three-dimensional mesh data;

[0041] (2) Perform geodesic distance calculation on the filtered 3D grid data to generate a point-to-point geodesic distance matrix between the source and target faces;

[0042] (3) Based on the point-to-point geodesic distance matrix, the 3D mesh data of the source and target faces are rigidly aligned by least squares optimization to obtain the aligned 3D mesh data;

[0043] (4) Based on the aligned 3D mesh data and spherical projection algorithm, construct the unit spherical coordinate representation of the source and target faces;

[0044] (5) Generate the shape space distance function between the source face and the target face based on the unit spherical coordinate representation and the spherical geodesic distance calculation;

[0045] (6) The source face shape space representation data and the target face shape space representation data are obtained through the shape space distance function and the Kendall shape space transformation mapping.

[0046] Specifically, noise reduction is performed on the 3D mesh data of the source and target faces. The discrete cosine transform converts the 3D mesh data from the spatial domain to the frequency domain, filtering out high-frequency components to achieve data noise reduction. This preserves the primary morphological features of the face while removing noise generated during the sampling and measurement processes. The geodesic distance is then calculated for the denoised 3D mesh data.

[0047]

[0048] in: Represents the geodesic distance between adjacent vertices; and Represents the coordinates of adjacent mesh vertices; and Indicates the magnitude of the vertex coordinate vector.

[0049] By calculating the geodesic distances between all pairs of vertices, a complete point-to-point geodesic distance matrix is ​​generated. This matrix reflects the shortest path length between any two points on the 3D grid. A least-squares optimization method is then used to perform a rigid transformation alignment of the source and target faces. Based on the geodesic distance matrix, an optimal rotation matrix and translation vector are iteratively found to minimize the sum of squared distances between corresponding points. This rigid transformation ensures that the facial shape does not deform during the alignment process.

[0050] The aligned 3D mesh data is projected onto the unit sphere using a spherical projection algorithm. The parametric expression is:

[0051]

[0052] in: is the parameterized value of the vertex; and is the geodesic distance; n is the total number of vertices.

[0053] The projected unit spherical coordinate representation eliminates the effects of scale differences and provides a standardized basis for subsequent shape space distance calculations. Based on the unit spherical coordinates, the spherical geodesic distance between the source and target faces is calculated to generate the shape space distance function. Spherical linear interpolation is used, and the construction of the node vector during the interpolation process follows the formula:

[0054]

[0055] The meaning of each parameter is: Indicates the degree of the B-spline curve, which controls the smoothness of the curve; represents the parameter value of the i-th vertex, which is calculated by the geodesic distance; Indicates the node index currently being calculated, ranging from 1 to np; n represents the total number of control points; Represents the total length of the node vector, satisfying the relationship m=n+p+1; arrive The values ​​of are all set to 0, indicating the initial boundary conditions of the curve; arrive The values ​​of are all set to 1, indicating the termination boundary condition of the curve.

[0056] This is used to construct knot vectors during spherical B-spline interpolation, ensuring the continuity and numerical stability of the interpolation curve. By mapping the shape space distance function with the Kendall shape space transform, the source and target face representations in shape space are obtained. This achieves a standardized representation of facial shape, laying the foundation for subsequent expression transfer.

[0057] For example, for a 3D face model consisting of 2048 vertices, discrete cosine transform (DCT) denoising retained 512 primary frequency components. Geodesic distance calculation generated a 2048×2048 distance matrix, recording the distance values ​​between all vertex pairs. The rigid transformation alignment process achieved an exact match between the source and target faces through 50 iterations. After spherical projection, all vertices were mapped onto the unit sphere, and a shape space distance function was constructed using 128 control points.

[0058] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0059] (1) The distance between adjacent data points of the source face shape space representation data is measured by geodesic distance calculation to obtain the source face shape space geodesic distance sequence;

[0060] (2) Based on the geodesic distance sequence of the source face shape space and the node vector generation algorithm, a spherical B-spline node vector space is constructed;

[0061] (3) Obtaining the initial control point data of the source face shape space based on the spherical B-spline node vector space and the progressive iterative approximation operation;

[0062] (4) Based on the initial control point data of the source face shape space, curve fitting is performed on the source face shape space representation data through spherical error minimization iteration to generate the optimized control points of the source face shape space;

[0063] (5) Using the optimized control points and B-spline basis function calculations in the source face shape space, a continuous curve expression of the source face shape space is constructed;

[0064] (6) By performing De Boor recursive decomposition on the continuous curve expression of the source face shape space, the source face continuous expression curve and the source face control vertex sequence are obtained.

[0065] Specifically, in the process of facial expression transfer based on shape space, the processing of the source face shape space representation data begins with the calculation of geodesic distance. Geodesic distance represents the shortest path length between two points on a surface. Based on the calculated geodesic distance sequence, it is necessary to construct a spherical B-spline node vector space. The node vector space is constructed using a parameterized method. First, the parameter values ​​are calculated, and then the node vectors are constructed.

[0066] The initial control point data is obtained through the progressive iterative approximation algorithm. The control point calculation formula of the spherical B-spline curve is:

[0067]

[0068] in: is the i-th control point of the r-th iteration; is the iteration weight coefficient; is the spherical angle parameter; and is the control point of the previous iteration.

[0069] The initial control points are iteratively optimized to minimize the spherical error, generating optimized control points. The optimization process uses the least squares criterion, repeatedly adjusting the control point positions until the curve fitting error is less than a preset threshold. Using the optimized control points and B-spline basis functions, a continuous curve expression in the source face shape space is constructed. Finally, the continuous curve expression is subjected to De Boor recursive decomposition to obtain a continuous expression curve and a sequence of control vertices. The De Boor recursive formula is as follows:

[0070]

[0071] in: represents the coordinates of the curve point at parameter t; Indicates the position of the i-th control point after p recursions; Indicates the parameter value of the current calculation point; Represents the parameter value of the i-th node; Indicates the degree of the B-spline curve; r indicates the number of current recursions; i indicates the index of the control point; Represents the interpolation weight coefficient at the rth recursion.

[0072] For example, suppose a 3D facial model with 1024 vertices is processed. First, the geodesic distances between adjacent data points are calculated, generating a sequence of 1024 distance values. Based on these distance values, a parameter space is constructed, and 32 control points are selected as the initial control point set. After 50 progressive approximations, with the weight coefficient gradually reduced from 1.0 to 0.1 with each iteration, an optimized set of control points is obtained. Using these optimized control points, a B-spline curve with a degree of 3 is constructed. The expression curve is then generated by calculating 100 sampling points using the De Boor recursive algorithm.

[0073] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0074] (1) Sampling and calculating the continuous expression curve of the source face through the curve nearest point algorithm to obtain the sampling point sequence of the source face expression curve;

[0075] (2) Obtain the source facial expression curve distance matrix based on the source facial expression curve sampling point sequence and spherical geodesic distance calculation;

[0076] (3) Based on the source face expression curve distance matrix and the source face control vertex sequence, the source face dual-parameter geodesic distance field is constructed through the Laplacian operator;

[0077] (4) Generate a set of geodesic direction vectors based on the dual-parameter geodesic distance field of the source face and the manifold gradient descent algorithm;

[0078] (5) Using the geodesic direction vector group and spherical index mapping, construct the source face geodesic transformation matrix;

[0079] (6) The source face geodesic transformation matrix is ​​decomposed through the Riemann metric tensor to obtain a two-parameter geodesic mapping function.

[0080] Specifically, in the process of facial expression transfer based on shape space, constructing a two-parameter geodesic mapping function is a key step. First, the continuous expression curve of the source face is sampled using the curve closest point algorithm. This algorithm calculates the shortest distance between any point on the curve and a given point. The formula for selecting the sampling point is as follows:

[0081]

[0082] in: Indicates the distance from the curve point to the given point; represents a point on the curve; Represents a given point.

[0083] Perform spherical geodesic distance calculation on the sampling point sequence, and the calculation process uses the derivative equation:

[0084]

[0085] in: represents the derivative of the distance with respect to the parameter; Represents the tangent vector of the curve.

[0086] Based on the expression curve distance matrix and the control vertex sequence, a two-parameter geodesic distance field is constructed using the Laplace operator. The Laplace operator describes the local geometric features on the surface, and the distribution of the distance field is obtained by solving the Laplace equation. Next, the geodesic direction vector is calculated based on the geodesic distance field using the manifold gradient descent algorithm. This algorithm iteratively optimizes on a Riemannian manifold, updating the vector at each iteration based on the gradient direction. The spherical exponential map maps the vector in tangent space onto the manifold, constructing the geodesic transformation matrix.

[0087] Finally, the two-parameter geodesic mapping function is obtained by Riemannian metric tensor decomposition. This process involves the eigenvalue decomposition of the tensor, thereby obtaining the local geometric structure of the mapping space.

[0088] For example, for a face model consisting of 1024 vertices, 256 sampling points are first selected on the continuous expression curve. The spherical geodesic distances between these sampling points and the control vertices are calculated to generate a 256×256 distance matrix. When constructing the distance field using the Laplacian operator, a 16×16 neighborhood window is selected for local geometric feature extraction. The manifold gradient descent algorithm is set with a learning rate of 0.01, and after 100 iterations, a stable set of direction vectors is obtained. The resulting two-parameter geodesic mapping function accurately describes the deformation path from the source face to the target face, providing a mathematical foundation for subsequent expression transfer. The entire process strictly adheres to differential geometry theory, ensuring the accuracy and continuity of expression transfer. In the specific calculation example, the threshold for the closest point distance is set to 0.001, and iterations are terminated when the distance change from the iterative calculation is less than this threshold. The distance field is constructed using a Gaussian weighting function, with the weight coefficient exponentially decaying with increasing distance. During the calculation of the geodesic direction vector, the gradient descent step size is dynamically adjusted based on the curvature of the current position, ensuring the algorithm's convergence. The entire process is rigorously mathematically derived and experimentally verified, ensuring the stability and reliability of expression transfer.

[0089] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0090] (1) Uniformly sample the target face shape space representation data through the curve parameterization algorithm to obtain the initial interpolation point set of the target face;

[0091] (2) Based on the initial interpolation point set of the target face and the two-parameter geodesic mapping function, a spherical node vector space is constructed;

[0092] (3) Based on the spherical node vector space and the initial interpolation point set of the target face, the target face curve control point data is generated through spherical linear interpolation operation;

[0093] (4) Using the target face curve control point data and De Boor recursive algorithm, construct the target face spherical B-spline basis function;

[0094] (5) Obtain the optimized interpolation point sequence of the target face based on the spherical B-spline basis function of the target face and the geodesic distance calculation;

[0095] (6) The optimized interpolation point sequence of the target face is subjected to spherical curve fitting operation to obtain the continuous expression curve of the target face.

[0096] Specifically, during the target facial expression transfer process, the target facial shape space representation data must first be uniformly sampled. Based on the initial interpolation point set and the two-parameter geodesic mapping function, a spherical node vector space is constructed. When performing linear interpolation on the sphere, the spherical linear interpolation formula is used:

[0097]

[0098] in: represents the interpolation point; represents the interpolation parameter; represents the spherical angle; 、 Denotes the interpolation endpoints. The spherical B-spline basis functions are constructed using the De Boor recursive algorithm. For example, for a target face model with 2048 vertices, uniform sampling is first performed, selecting 256 initial interpolation points. These points are evenly distributed in the parameter space, with parameter values ​​ranging from 0 to 1. When constructing the spherical knot vector space, a B-spline curve degree of 3 is selected, generating knot vectors of length 260. The spherical linear interpolation operation selects 32 control points, each of which is obtained by taking a weighted average of adjacent interpolation points.

[0099] The B-spline basis functions were constructed using the De Boor recursive algorithm, with the number of iterations set to 50 and the weight coefficient gradually decreasing from 1.0 to 0.1 for each iteration. Geodesic distance calculations were performed using a numerical method with an accuracy of 0.001, ensuring the precise positioning of interpolation points. Finally, a continuous curve describing the target facial expression was generated through spherical curve fitting. In the implementation, the curve was parameterized using cumulative chord length parameterization to ensure a uniform distribution of sampling points. The construction of the spherical node vector space took into account the local support property, ensuring that each basis function was nonzero only within a finite interval. A weighted spherical distance average was used to generate control points, ensuring that the interpolation result lay on the sphere.

[0100] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0101] (1) Decompose the continuous expression curve of the target face into tangent space using the Riemann metric tensor to obtain the tangent space basis vector group of the target face;

[0102] (2) Based on the target face tangent space basis vector group and Kristoffel symbol calculation, the target face Riemann connection coefficient is constructed;

[0103] (3) Generate the geodesic covariant derivative of the target face based on the Riemann connection coefficient of the target face and the manifold parallel transmission equation;

[0104] (4) Using the geodesic covariant derivative of the target face and the spherical index map, the target face shape deformation field is constructed;

[0105] (5) Calculate the target face shape spatial deformation matrix based on the target face shape deformation field and the Riemannian manifold parallel transmission operator;

[0106] (6) The target face shape space deformation matrix is ​​transformed through shape space homeomorphism to obtain the target face shape space deformation sequence.

[0107] Specifically, Riemannian geometry theory provides an important mathematical tool in the key steps of transferring the target face's expression. First, the Riemannian metric tensor is used to decompose the target face's continuous expression curve into tangent space. Based on the tangent space basis vectors, the Riemannian connection coefficient is calculated using the Kristoffer symbol. The connection coefficient is calculated using a derivative equation. Based on the Riemannian connection coefficient and the manifold parallel transport equation, the geodesic covariant derivative is calculated. This process follows the parallel transport condition:

[0108]

[0109] Geodesic covariant derivatives are combined with spherical exponential mappings to construct shape deformation fields. The exponential mapping maps vectors in tangent space onto the manifold, preserving the local properties of the geometric structure.

[0110] For example, for a target facial expression curve consisting of 1024 vertices, 16 basis vectors were selected from the tangent space decomposition, each corresponding to a primary deformation direction. Kristoffer symbolic computation generated a 16×16 connection coefficient matrix, describing the local geometric properties of the surface. Geodesic covariant derivatives were calculated using numerical integration with a step size of 0.01 to ensure accuracy. During the construction of the shape deformation field, a local deformation vector was calculated for each vertex, with the field strength decaying exponentially with distance from the deformation center. The local deformation was propagated to the entire face model using the Riemannian manifold parallel transport operator. The resulting shape space deformation sequence consisted of 32 keyframes, each capturing the gradual transformation of the facial shape. The entire process strictly adhered to differential geometry theory, ensuring the continuity and naturalness of expression transfer. In the specific experiments, the selection of the tangent space basis vectors took into account the primary deformation patterns of facial expressions, including the motion characteristics of areas such as the eyebrows, eyes, and mouth. The Riemann connection coefficient is calculated with an accuracy of 0.001, ensuring accurate description of local geometric properties. The shape deformation field is constructed using an adaptive weighting scheme, with the weight coefficients dynamically adjusted based on local curvature. The entire algorithmic process has been rigorously validated, achieving stable and accurate expression transfer results.

[0111] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0112] (1) Using the spherical projection algorithm, coordinate transformation is performed on the target face shape space deformation sequence to obtain the target face Euclidean space deformation data;

[0113] (2) Using the target face Euclidean space deformation data and rigid transformation matrix, the target face shape alignment relationship is constructed;

[0114] (3) Generate a target face shape parameter sequence based on the target face shape alignment relationship and spatial scale factor;

[0115] (4) Calculate the target face vertex mapping coordinates based on the target face shape parameter sequence and the affine transformation matrix;

[0116] (5) Based on the target face vertex mapping coordinates and topology structure preservation algorithm, the target face mesh connection relationship is constructed;

[0117] (6) The target face mesh connection relationship is reconstructed through mesh operation to obtain the target face continuous expression migration data.

[0118] Specifically, in the process of converting shape space to 3D mesh data, the target face shape space deformation sequence is first processed by the spherical projection algorithm. According to the projection formula:

[0119]

[0120] in: represents the parameter value after projection, represents the geodesic distance. n represents the total number of vertices.

[0121] The resulting Euclidean space deformation data is aligned using a rigid transformation matrix. Rigid transformations maintain shape invariance, including rotation and translation. Based on the shape alignment relationship and the spatial scale factor, a shape parameter sequence is generated. This shape parameter sequence describes the geometric characteristics of the model at different scales. The affine transformation matrix maps the shape parameters into a three-dimensional coordinate system, calculating the final position of each vertex.

[0122] The topology preservation algorithm ensures that the connection relationship of the mesh remains unchanged during the deformation process. Finally, the mesh reconstruction operation is performed to generate the complete expression migration data.

[0123] For example, for a target face model consisting of 2048 vertices, the shape space data is first projected into Euclidean space using spherical projection. This projection process maintains the relative positions of the 256 key vertices. The rigid transformation matrix is ​​represented using 4×4 homogeneous coordinates, including rotation and translation components. The scale factor is calculated based on the model's bounding box size, initially set to 1.0 and then dynamically adjusted based on the degree of deformation. The affine transformation uses a 3×4 transformation matrix to map the coordinates of each vertex. The mesh connectivity is represented using triangular facets, totaling 4096 facets. The final reconstructed expression transfer data consists of 32 keyframes, each of which maintains the topology of the original mesh. The entire process ensures the accuracy of the geometric shape and topology, achieving a natural and smooth expression transfer effect. In the specific implementation, the numerical precision of the spherical projection is set to 0.001 to ensure the accuracy of the coordinate transformation. The rigid transformation optimization is performed over 50 iterations, with the residual error for each iteration being less than 0.0001. The topology-preserving algorithm maintains the local characteristics of the mesh through the Laplace operator.

[0124] The above describes the face continuous expression migration method based on shape space in the embodiment of the present application. The following describes the face continuous expression migration system based on shape space in the embodiment of the present application. Figure 3 In one embodiment of the present application, a system for continuous facial expression migration based on shape space includes:

[0125] An acquisition module is used to obtain source face shape space representation data and target face shape space representation data through Kendall shape space transformation based on the three-dimensional mesh data of the source face expression model and the target face expression model;

[0126] An interpolation module, configured to perform interpolation calculation on the source face shape space representation data using a spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence;

[0127] A construction module, configured to construct a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve;

[0128] a calculation module, configured to calculate interpolation points for the target face shape space representation data using the De Boor recursive algorithm according to the dual-parameter geodesic mapping function, and generate a continuous expression curve for the target face;

[0129] A calculation module, configured to obtain a target face shape spatial deformation sequence based on a parallel transmission operation between the target face continuous expression curve and the Riemann manifold;

[0130] The conversion module is used to convert the target face shape space deformation sequence into three-dimensional grid data through Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

[0131] Through the collaborative efforts of the aforementioned components, shape space representation data is obtained through the Kendall shape space transform, eliminating scale, rotation, and translation differences between the source and target faces and improving the stability of expression transfer. The spherical B-spline interpolation algorithm processes the shape space representation data, ensuring the continuity and smoothness of expression changes, avoiding the jumps and discontinuities often seen in traditional methods. The construction of a two-parameter geodesic mapping function provides an accurate description of the deformation path, enabling precise deformation mapping from the source to the target face. The De Boor recursive algorithm maintains numerical stability during the calculation of interpolation points, generating continuous expression curves for the target face with good geometric properties. The Riemannian manifold parallel transport operation preserves local geometric structure during shape deformation, resulting in a more natural-looking deformation result. Finally, the inverse Kendall shape space transform converts the shape space deformation sequence into 3D mesh data, ensuring the accuracy and authenticity of the final expression transfer results. This entire solution not only reduces computational complexity but also requires no extensive training data, making it suitable for small sample sizes, while maintaining high expression transfer accuracy and naturalness.

[0132] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for continuous facial expression migration based on shape space, characterized in that: The method for transferring continuous facial expressions based on shape space includes: According to the three-dimensional mesh data of the source facial expression model and the target facial expression model, the source facial shape space representation data and the target facial shape space representation data are obtained by Kendall shape space transformation; Using a spherical B-spline interpolation algorithm to perform interpolation calculation on the source face shape space representation data to obtain a source face continuous expression curve and a source face control vertex sequence; Constructing a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; According to the dual-parameter geodesic mapping function, the De Boor recursive algorithm is used to calculate interpolation points for the target face shape space representation data to generate a continuous expression curve for the target face; Based on the parallel transmission operation of the continuous expression curve of the target face and the Riemann manifold, a spatial deformation sequence of the target face shape is obtained; The target face shape space deformation sequence is converted into three-dimensional grid data through Kendall shape space inverse transformation to obtain target face continuous expression migration data.

2. The method for transferring continuous facial expressions based on shape space according to claim 1, characterized in that: The method of obtaining source face shape space representation data and target face shape space representation data by Kendall shape space transformation based on the three-dimensional mesh data of the source face expression model and the target face expression model comprises: Performing noise reduction processing on the three-dimensional mesh data of the source facial expression model and the target facial expression model by discrete cosine transform to obtain filtered three-dimensional mesh data; Performing geodesic distance calculation on the filtered three-dimensional grid data to generate a point-to-point geodesic distance matrix between the source face and the target face; performing rigid transformation alignment on the three-dimensional mesh data of the source face and the target face by least squares optimization according to the point-to-point geodesic distance matrix to obtain aligned three-dimensional mesh data; constructing unit spherical coordinate representations of the source face and the target face based on the aligned three-dimensional mesh data and the spherical projection algorithm; Generate a shape space distance function between the source face and the target face based on the unit spherical coordinate representation and the spherical geodesic distance calculation; The source face shape space representation data and the target face shape space representation data are obtained through the shape space distance function and the Kendall shape space transformation mapping.

3. The method for transferring continuous facial expressions based on shape space according to claim 1, characterized in that: The method of performing interpolation calculation on the source face shape space representation data using a spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence includes: Measuring the distances between adjacent data points of the source face shape space representation data by geodesic distance calculation to obtain a source face shape space geodesic distance sequence; Constructing a spherical B-spline node vector space based on the source face shape space geodesic distance sequence and the node vector generation algorithm; Obtaining initial control point data of the source face shape space based on the spherical B-spline node vector space and progressive iterative approximation operation; According to the initial control point data of the source face shape space, curve fitting is performed on the source face shape space representation data through spherical error minimization iteration to generate optimized control points of the source face shape space; Utilizing the optimized control points and B-spline basis function calculations in the source face shape space to construct a continuous curve expression in the source face shape space; The source human face continuous expression curve and the source human face control vertex sequence are obtained by performing De Boor recursive decomposition on the source human face shape space continuous curve expression.

4. The method for transferring continuous facial expressions based on shape space according to claim 1, wherein: The step of constructing a two-parameter geodesic mapping function based on the geodesic distance between the source face control vertex sequence and the source face continuous expression curve comprises: Sampling and calculating the source face continuous expression curve by using a curve nearest point algorithm to obtain a source face expression curve sampling point sequence; Obtaining a source facial expression curve distance matrix based on the source facial expression curve sampling point sequence and spherical geodesic distance calculation; Constructing a source face dual-parameter geodesic distance field using a Laplacian operator according to the source face expression curve distance matrix and the source face control vertex sequence; Generate a geodesic direction vector group based on the dual-parameter geodesic distance field of the source face and the manifold gradient descent algorithm; Constructing a source face geodesic transformation matrix using the geodesic direction vector group and spherical index mapping; The source face geodesic transformation matrix is ​​decomposed through the Riemann metric tensor to obtain the two-parameter geodesic mapping function.

5. The method for transferring continuous facial expressions based on shape space according to claim 4, characterized in that: The method of calculating interpolation points on the target face shape space representation data using the De Boor recursive algorithm according to the dual-parameter geodesic mapping function to generate a target face continuous expression curve includes: uniformly sampling the target face shape space representation data using a curve parameterization algorithm to obtain an initial interpolation point set for the target face; Constructing a spherical node vector space based on the target face initial interpolation point set and the dual-parameter geodesic mapping function; Generating target face curve control point data by spherical linear interpolation operation based on the spherical node vector space and the target face initial interpolation point set; Constructing a spherical B-spline basis function of the target face using the target face curve control point data and the De Boor recursive algorithm; Obtaining a target face optimized interpolation point sequence based on the target face spherical B-spline basis function and geodesic distance calculation; The optimized interpolation point sequence of the target face is subjected to a spherical curve fitting operation to obtain the continuous expression curve of the target face.

6. The method for transferring continuous facial expressions based on shape space according to claim 5, characterized in that: The step of obtaining a target face shape spatial deformation sequence based on the target face continuous expression curve and the Riemann manifold parallel transmission operation includes: Performing tangent space decomposition on the continuous expression curve of the target face using the Riemann metric tensor to obtain a tangent space basis vector group of the target face; Constructing the Riemann connection coefficient of the target face based on the target face tangent space basis vector group and Kristoffer symbol calculation; generating a geodesic covariant derivative of the target face according to the Riemann connection coefficient of the target face and the manifold parallel transmission equation; constructing a target face shape deformation field by using the target face geodesic covariant derivative and spherical exponential mapping; Calculating a target face shape spatial deformation matrix based on the target face shape deformation field and the Riemannian manifold parallel transmission operator; The target face shape space deformation matrix is ​​transformed through shape space homeomorphism to obtain the target face shape space deformation sequence.

7. The method for transferring continuous facial expressions based on shape space according to claim 6, characterized in that: The step of converting the target face shape space deformation sequence into three-dimensional mesh data by using an inverse Kendall shape space transform to obtain the target face continuous expression migration data includes: Performing coordinate transformation on the target face shape space deformation sequence by a spherical projection algorithm to obtain target face Euclidean space deformation data; Using the target face Euclidean space deformation data and the rigid transformation matrix, constructing a target face shape alignment relationship; generating a target face shape parameter sequence according to the target face shape alignment relationship and the spatial scale factor; Calculating the target face vertex mapping coordinates based on the target face shape parameter sequence and the affine transformation matrix; Constructing a target face mesh connection relationship based on the target face vertex mapping coordinates and topology structure preservation algorithm; The target face mesh connection relationship is subjected to a mesh reconstruction operation to obtain the target face continuous expression migration data.

8. A system for continuously migrating facial expressions based on shape space, for implementing the method for continuously migrating facial expressions based on shape space as claimed in any one of claims 1 to 7, characterized in that: The face continuous expression migration system based on shape space includes: An acquisition module is used to obtain source face shape space representation data and target face shape space representation data through Kendall shape space transformation based on the three-dimensional mesh data of the source face expression model and the target face expression model; An interpolation module, configured to perform interpolation calculation on the source face shape space representation data using a spherical B-spline interpolation algorithm to obtain a source face continuous expression curve and a source face control vertex sequence; A construction module, configured to construct a two-parameter geodesic mapping function according to the geodesic distance between the source face control vertex sequence and the source face continuous expression curve; a calculation module, configured to calculate interpolation points for the target face shape space representation data using the De Boor recursive algorithm according to the dual-parameter geodesic mapping function, and generate a continuous expression curve for the target face; A calculation module, configured to obtain a target face shape spatial deformation sequence based on a parallel transmission operation between the target face continuous expression curve and the Riemann manifold; The conversion module is used to convert the target face shape space deformation sequence into three-dimensional grid data through Kendall shape space inverse transformation to obtain the target face continuous expression migration data.

Images