A method and system for constructing a matching point pair of a lip region for three-dimensional face homogenization non-rigid registration

Abstract

The application relates to the field of oral digitization, and provides a method and system for constructing a lip area matching point pair for three-dimensional face homologization non-rigid registration. The method comprises the following steps: performing spatial pose alignment on a source three-dimensional face mesh model and a target three-dimensional face mesh model to obtain an initial alignment model pair; extracting a loop of a lip area boundary of a plurality of mesh models based on topological features to obtain a lip topological boundary loop; performing segmentation processing on the lip topological boundary loop to obtain a lip boundary segmentation set, and establishing a segmented corresponding relationship between the source three-dimensional face mesh model and the target three-dimensional face mesh model; adaptively sampling and quota distributing a plurality of lip boundary segments of the target three-dimensional face mesh model in the segmented corresponding relationship according to the arc length proportion of the plurality of segments in the lip boundary segmentation set, and performing proportional linear step sampling to obtain a lip area matching point pair with consistent logical indexes. The application improves the alignment accuracy of the lip boundary.

Description

Technical Field

[0001] This invention relates to the field of digital oral cavity technology, and in particular to a method and system for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology. Background Technology

[0002] Three-dimensional facial morphology analysis and dynamic sequence construction are core technical aspects of anterior aesthetic restoration design, orthodontic and orthognathic surgery efficacy prediction, and doctor-patient communication in oral medicine. With the widespread adoption of high-precision three-dimensional scanning technology, acquiring facial mesh models has become a fundamental clinical procedure. To achieve quantitative comparison and morphological transfer of facial models from different individuals and under different facial expressions, non-rigid registration techniques are typically employed. This involves mapping a template model with a fixed topological structure onto the surface of a target model, establishing a stable point-to-point mapping relationship, thereby achieving homogenization of facial data.

[0003] Among existing non-rigid registration techniques, the non-rigid iterative nearest neighbor algorithm (Non-rigid ICP) is the mainstream scheme for establishing the correspondence of 3D facial point clouds. This algorithm is highly dependent on the guiding quality of the initial feature points. However, the lip region lacks bony support and the soft tissue exhibits significant nonlinear deformation during functional movements such as smiling. This results in smooth changes in the geometric curvature of the region and insignificant local features. The algorithm is prone to getting stuck in local optima near the vermilion border, causing anatomical drift of the template vertices and even topological adhesion of the upper and lower lip meshes when the lips are open and closed. Summary of the Invention

[0004] This invention provides a method and system for constructing matching point pairs in the lip region for non-rigid registration of three-dimensional facial homology, in order to overcome the shortcomings of the prior art.

[0005] This invention provides a method for constructing lip region matching point pairs for non-rigid registration of 3D facial homology, including: S1: Align the source 3D face mesh model with the target 3D face mesh model in spatial pose to obtain an initial aligned model pair; S2: Based on topological features, loop extraction is performed on the lip region boundaries of multiple mesh models in the initial alignment model pair to obtain the lip topological boundary loop; S3: Mark the anatomical anchor points on the lip topological boundary loop, perform segmentation processing on the lip topological boundary loop to obtain the lip boundary segment set, and establish the segment correspondence between the source 3D facial mesh model and the target 3D facial mesh model; S4: Based on the arc length ratio of multiple segments in the lip boundary segment set, adaptive sampling quota allocation is performed on multiple lip boundary segments of the target three-dimensional face mesh model in the segment correspondence relationship, and proportional linear step sampling is performed to obtain lip region matching point pairs with consistent logical indexes.

[0006] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S1 further includes: S11: Input the source 3D facial mesh model and the target 3D facial mesh model, and locate multiple anatomical landmarks on the two models respectively using the anatomical landmark detection method to obtain the source model landmark set and the target model landmark set; S12: Calculate the spatial transformation matrix between the source model marker point set and the target model marker point set using a rigid transformation estimation algorithm. Based on the spatial transformation matrix, map the source 3D facial mesh model to the coordinate system of the target 3D facial mesh model to obtain an initial aligned model pair.

[0007] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S2 further includes: S21: Traverse all mesh edges of each mesh model in the initial alignment model pair, determine the edges that are only referenced by a single facet as boundary edges, and form a boundary vertex set by the vertices corresponding to the boundary edges to obtain the original lip boundary vertex set. S22: By using a graph connectivity algorithm, multiple boundary vertices in the original lip boundary vertex set are traversed by adjacency relationship, and the unordered boundary vertices are reorganized into closed paths with strict vertex order, thus obtaining the lip topological boundary loop.

[0008] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S22 further includes: S221: Starting from any boundary vertex, continuously search for the next adjacent node in the original lip boundary vertex set based on the depth-first traversal strategy until the path is closed at both ends, thus obtaining a closed path. S222: Verify the vertex order of the obtained closed path, rearrange the vertex index sequence according to the unified wrapping direction, and obtain a lip topological boundary loop with topological consistency.

[0009] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S3 further includes: S311: For the lip topological boundary loop of each mesh model, obtain the target position coordinates of the two anchor points selected by the interaction point, calculate the distance between the target position coordinates of each anchor point and all boundary vertices in the lip topological boundary loop, and determine the boundary vertex with the smallest distance corresponding to the target position coordinates of each anchor point as the left corner point and the right corner point, and obtain the corner anchor point pair. S312: For the lip topological boundary loop of each mesh model, the lip topological boundary loop is divided into an upper lip path segment and a lower lip path segment, with the vertex index of the corner anchor point pair as the boundary; on the upper lip path segment, the target position coordinates of the first feature point selected by the interaction point are obtained, the distance between the target position coordinates of the first feature point and all boundary vertices in the upper lip path segment is calculated, and the boundary vertex with the smallest distance is determined as the upper lip peak point; on the lower lip path segment, the target position coordinates of the second feature point selected by the interaction point are obtained, the distance between the target position coordinates of the second feature point and all boundary vertices in the lower lip path segment is calculated, and the boundary vertex with the smallest distance is determined as the lower lip midpoint; S33: Based on the semantic correspondence of the anatomical anchor points, perform a segment-by-segment hard matching between the lip boundary segment set of the source 3D facial mesh model and the lip boundary segment set of the target 3D facial mesh model to obtain the segment correspondence.

[0010] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S31 further includes: S311: For the lip topological boundary loop of each mesh model, calculate the local geometric curvature of each vertex, and determine the two vertices with the largest change in curvature along the horizontal direction as the left corner point and the right corner point, respectively, to obtain the corner anchor point pair; S312: For the lip topological boundary loop of each mesh model, on the upper lip path segment located between the corner anchor point pairs, the local extreme vertex is determined by curvature extreme value detection and marked as the upper lip peak point; on the lower lip path segment, the vertex at the midpoint of the arc length is marked as the lower lip midpoint. S313: For each mesh model, the upper lip peak point, the lower lip midpoint, and the corner anchor point pair are arranged in ascending order by vertex index to obtain the source model anchor point sequence and the target model anchor point sequence.

[0011] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S4 further includes: S41: Calculate the cumulative arc length of the vertex sequence of multiple lip boundary segments of the source 3D face mesh model in the segmented correspondence relationship, and use the ratio of the arc length of each segment to the total arc length of the lip topological boundary loop as the geometric contribution weight of the corresponding segment to obtain the source model segment weight set. S42: Multiply the preset total number of sampling points by the weights of each segment in the source model segment weight set, round down and allocate the remaining quotas to each segment, and compensate the remaining quotas to the corresponding segments in descending order of weight to obtain the target model segment sampling quota set. S43: Based on the target model segmented sampling quota set, perform equal arc length linear step sampling on each lip boundary segment of the target three-dimensional facial mesh model to obtain lip region matching point pairs with consistent logical indexes.

[0012] According to the present invention, a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology is provided, wherein step S43 further includes: S431: Divide the lip boundary of each segment of the target three-dimensional facial mesh model into equal parts according to the segment sampling quota, calculate the interpolation position of multiple equal parts nodes on the segment vertex sequence, and obtain the coordinate sequence of sampling points of each segment of the target model. S432: According to the vertex index arrangement order of each segment of the source model in the segmented correspondence relationship, the sampling point coordinate sequence of each segment of the target model is paired point by point with the sampling point coordinate sequence of the corresponding segment of the source model to obtain a lip region matching point pair with a consistent logical index.

[0013] According to the present invention, a method for constructing matching point pairs of the lip region for non-rigid registration of facial homogeneity is provided. In the iterative process of non-rigid registration, the matching point pairs of the lip region are used as constraint point sets and input into the non-rigid iterative nearest neighbor algorithm. The corresponding vertices in the matching point pairs of the lip region are used as hard constraints to guide the template model to converge to the lip topological boundary of the target three-dimensional facial mesh model, so as to constrain the template vertices at the vermilion border and force the independence of the sampling index to be maintained at the segment boundary of the upper and lower lips, so as to prevent the upper and lower lip meshes from generating topological adhesion in the opening and closing regions.

[0014] This invention also provides a system for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology, used to execute a method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology as described in any of the above claims, comprising: Alignment module: used to align the spatial pose of the source 3D facial mesh model and the target 3D facial mesh model to obtain an initial aligned model pair; Extraction module: used to extract the lip region boundary of multiple mesh models in the initial alignment model pair based on topological features, and obtain the lip topological boundary loop; Segmentation module: used to mark anatomical anchor points on the lip topological boundary loop, segment the lip topological boundary loop to obtain a set of lip boundary segments, and establish the segmentation correspondence between the source 3D facial mesh model and the target 3D facial mesh model; Output module: Based on the arc length ratio of multiple segments in the lip boundary segment set, adaptively allocate sampling quotas for multiple lip boundary segments of the target 3D face mesh model in the segment correspondence relationship, and perform proportional linear step sampling to obtain lip region matching point pairs with consistent logical indexes.

[0015] This invention provides a method and system for constructing matching point pairs in the lip region for non-rigid registration of 3D facial homology. By introducing a lip boundary loop extraction mechanism based on topological features, the registration search domain is strictly limited to closed topological paths with vertex order. This fundamentally avoids the template vertex anatomical drift problem caused by the insignificant local geometric features in the lip region in traditional non-rigid iterative nearest neighbor algorithms, ensuring that template vertices are stably constrained near the vermilion border, significantly improving the alignment reliability of the lip boundary. Secondly, this invention performs semantic segmentation of the lip topological boundary loop through anatomical anchor points and establishes segment correspondences, ensuring that each segment of the upper and lower lips is matched. The invention establishes clear index isolation beforehand, effectively preventing mesh topology adhesion caused by confusion between the upper and lower lip vertex indices during the opening and closing of the lips, thus ensuring the physical correctness of the dynamic simulation sequence. Secondly, based on the geometric contribution weight calculated by the proportion of each segment's arc length to the total boundary arc length, the invention drives the dynamic allocation of sampling quotas for each segment of the target model and equal-arc-length linear step sampling. This allows the final generated lip region matching point pairs to adaptively compensate for the local nonlinear stretching of the lips during smiling movements in terms of spatial distribution. Compared to the equal-interval sampling scheme, the resulting matching point pairs are more biomechanically logical, and the naturalness of the simulated motion trajectory is significantly improved. Furthermore, all steps of this invention are completed automatically, without relying on extensive and meticulous manual annotation, providing an efficient and robust technical approach for large-scale clinical construction of high-precision homogeneous three-dimensional facial dynamic models. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for constructing matching point pairs in the lip region for non-rigid registration of three-dimensional facial homology provided by the present invention; Figure 2 This is a schematic diagram of a lip region matching point pair construction system for non-rigid registration of three-dimensional facial homology provided by the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0019] The embodiments of the present invention are described below with reference to the figures.

[0020] like Figure 1 As shown, this invention provides a method for constructing lip region matching point pairs for non-rigid registration of 3D facial homology, comprising: S1: Align the source 3D face mesh model with the target 3D face mesh model in spatial pose to obtain an initial aligned model pair.

[0021] Furthermore, the present invention first receives two input data, including a source 3D facial mesh model (Template) and a target 3D facial mesh model (Target), both stored in the form of 3D meshes, each containing a set of vertex coordinates and a set of facet connections. Since the two models are in independent coordinate systems during acquisition, their spatial positions and poses differ. Therefore, the present invention first needs to complete spatial pose alignment through two sub-steps, S11 to S12.

[0022] Step S1 further includes: S11: Input the source 3D facial mesh model and the target 3D facial mesh model, and locate multiple anatomical landmarks on the two models respectively using the anatomical landmark detection method to obtain the source model landmark set and the target model landmark set.

[0023] In step S11, this invention locates anatomical landmarks on both the source and target 3D facial mesh models. These anatomical landmarks are specific points on the face with clear anatomical significance, such as the tip of the nose, the corner of the eye, and the corner of the mouth. These points exhibit stable anatomical correspondence across different individual models. Specifically, this invention processes the vertex coordinate data of the two mesh models using an anatomical landmark detection method, outputting a source model landmark set and a target model landmark set. Each landmark set is a list of 3D coordinate points, with points in the source and target model landmark sets corresponding one-to-one according to anatomical semantics.

[0024] S12: Calculate the spatial transformation matrix between the source model marker point set and the target model marker point set using a rigid transformation estimation algorithm. Based on the spatial transformation matrix, map the source 3D facial mesh model to the coordinate system of the target 3D facial mesh model to obtain an initial aligned model pair.

[0025] Further, in step S12, the present invention uses the source model marker point set and the target model marker point set as inputs, and calculates the spatial transformation matrix between the two sets of point sets using a rigid transformation estimation algorithm. Specifically, the rigid transformation estimation algorithm works as follows: First, using the coordinate data of the two corresponding point sets as inputs, it calculates the rotation matrix and translation vector that minimize the sum of the overall Euclidean distances between the source model marker point set and the target model marker point set after transformation, and combines the two to form the spatial transformation matrix. Subsequently, the present invention applies this spatial transformation matrix to the coordinates of all vertices of the source 3D facial mesh model, and applies rotation and translation operations to each vertex coordinate in sequence to complete the unified mapping of the coordinate system and output the initial aligned model pair. In the obtained initial aligned model pair, the source 3D facial mesh model has been transformed to the same coordinate system as the target 3D facial mesh model, and the two have achieved preliminary consistency in spatial position and orientation.

[0026] S2: Based on topological features, loop extraction is performed on the lip region boundaries of multiple mesh models in the initial alignment model pair to obtain the lip topological boundary loop.

[0027] Step S2 further includes: S21: Traverse all mesh edges of each mesh model in the initial alignment model pair, determine the edges that are only referenced by a single facet as boundary edges, and form a boundary vertex set by the vertices corresponding to the boundary edges to obtain the original lip boundary vertex set.

[0028] In step S21, the present invention traverses all mesh edges of each mesh model in the initial alignment model pair. Each mesh edge is a line segment connecting two adjacent vertices, and each edge is shared by two faces in a normal closed mesh; however, at the open lip boundary, the boundary edge is referenced by only a single face. Based on this, the present invention counts the number of times each mesh edge is referenced by a face, identifies edges with a reference count of one as boundary edges, and extracts the vertices corresponding to all boundary edges to form the original lip boundary vertex set. This vertex set contains the three-dimensional coordinates and vertex indices of all boundary vertices on the lip contour, but there is no ordered sequence relationship between the vertices; it is an unordered set of points.

[0029] S22: By using a graph connectivity algorithm, multiple boundary vertices in the original lip boundary vertex set are traversed by adjacency relationship, and the unordered boundary vertices are reorganized into closed paths with strict vertex order, thus obtaining the lip topological boundary loop.

[0030] In step S22, the present invention takes the original set of lip boundary vertices as input and uses a graph connectivity algorithm to reorganize the unordered boundary vertices into ordered closed paths. During execution, the graph connectivity algorithm treats each boundary vertex as a node in the graph and each boundary edge as an undirected edge in the graph, thereby transforming the original set of lip boundary vertices into a graph structure.

[0031] Step S22 further includes: S221: Starting from any boundary vertex, continuously search for the next adjacent node in the original lip boundary vertex set based on the depth-first traversal strategy until the path is closed at both ends, thus obtaining a closed path.

[0032] In S221, the present invention arbitrarily selects a boundary vertex from the graph structure as the starting node, and visits the boundary vertices adjacent to the current node that have not yet been visited in turn based on the depth-first traversal strategy. The visiting order is recorded as a vertex index sequence. The optimization is continued until the path backtracks from the termination node to the starting node, and the path is closed at both ends, thus obtaining a closed path. The obtained closed path is stored in the form of an ordered list of vertex indices, which expresses the connection order of each vertex on the lip contour.

[0033] S222: Verify the vertex order of the obtained closed path, rearrange the vertex index sequence according to the unified wrapping direction, and obtain a lip topological boundary loop with topological consistency.

[0034] In step S222, the present invention verifies the surrounding direction of the vertex index sequence of the closed path. It determines whether the current sequence is clockwise or counterclockwise by calculating the directed area of ​​the path. If it does not conform to the preset unified direction, a flip operation is performed on the vertex index sequence, ultimately outputting a lip topological boundary loop with topological consistency. The obtained loop is represented by an ordered sequence of vertex indices with a unified surrounding direction. The source 3D face mesh model and the target 3D face mesh model each correspond to a lip topological boundary loop. The vertex arrangement directions of the two loops are consistent, providing a unified topological path basis for subsequent segmentation processing.

[0035] S3: Mark the anatomical anchor points on the lip topological boundary loop, segment the lip topological boundary loop to obtain a set of lip boundary segments, and establish the segment correspondence between the source 3D facial mesh model and the target 3D facial mesh model.

[0036] Step S3 further includes: S31: For the lip topological boundary loop of each mesh model, mark a number of anatomical anchor points, including at least the left corner of the mouth and the right corner of the mouth, to obtain the source model anchor point sequence and the target model anchor point sequence.

[0037] Step S31 further includes: S311: For the lip topological boundary loop of each mesh model, obtain the target position coordinates of the two anchor points selected by the interaction point, calculate the distance between the target position coordinates of each anchor point and all boundary vertices in the lip topological boundary loop, and determine the boundary vertex with the smallest distance corresponding to the target position coordinates of each anchor point as the left corner point and the right corner point, respectively, to obtain the corner anchor point pair.

[0038] In step S311, the present invention first renders and displays the lip topological boundary loop in a visual interface. Then, the operator selects the target positions corresponding to the left and right corners of the mouth on the interface, and the system records the three-dimensional coordinates of the target positions for each selection operation. Subsequently, the present invention traverses all boundary vertices on the lip topological boundary loop, calculates the Euclidean distance between each boundary vertex and the coordinates of the left target position, and the Euclidean distance between each boundary vertex and the coordinates of the right target position. From all boundary vertices, the boundary vertex with the smallest Euclidean distance to the left target position is identified and designated as the left corner point. The vertex index and three-dimensional coordinates of this vertex are recorded. Similarly, the boundary vertex with the smallest Euclidean distance to the right target position is identified and designated as the right corner point. The vertex index and three-dimensional coordinates of this vertex are recorded. Together, they form a corner anchor point pair.

[0039] It should be noted that the target location coordinates selected by the operator can be any coordinate point in three-dimensional space, and are not required to fall precisely on the boundary vertex. The system will automatically align it to the nearest boundary vertex on the lip topological boundary loop through nearest neighbor Euclidean distance matching, ensuring that the anchor point is strictly located in the vertex sequence of the lip topological boundary loop, thereby ensuring the topological correctness of subsequent segmentation operations.

[0040] S312: For the lip topological boundary loop of each mesh model, the lip topological boundary loop is divided into an upper lip path segment and a lower lip path segment, with the vertex index of the corner anchor point pair as the boundary. On the upper lip path segment, the target position coordinates of the first feature point selected by the interaction point are obtained, and the distance between the target position coordinates of the first feature point and all boundary vertices in the upper lip path segment is calculated. The boundary vertex with the smallest distance is determined as the upper lip peak point. On the lower lip path segment, the target position coordinates of the second feature point selected by the interaction point are obtained, and the distance between the target position coordinates of the second feature point and all boundary vertices in the lower lip path segment is calculated. The boundary vertex with the smallest distance is determined as the lower lip midpoint.

[0041] In step S312, the present invention uses the vertex indices of the left and right corner points of the mouth as boundary nodes to divide the vertex sequence of the lip topological boundary loop into two segments: the vertex subsequence located above the face and connecting the left and right corner points is the upper lip path segment; the vertex subsequence located below the face and connecting the left and right corner points is the lower lip path segment. Each of the two vertex subsequences contains the vertex indices and three-dimensional coordinates of all ordered boundary vertices within the corresponding segment.

[0042] For the upper lip path segment, this invention highlights the segment in the visualization interface. The operator selects the target position at the top of the upper lip arch structure. This invention records the three-dimensional coordinates of the target position selected by the operation. Then, it traverses all boundary vertices within the upper lip path segment, calculates the Euclidean distance between each boundary vertex and the target position coordinates, and determines the boundary vertex with the smallest Euclidean distance as the upper lip peak point, recording its vertex index and three-dimensional coordinates.

[0043] For the lower lip path segment, the operator selects the target position in the middle of the lower lip contour. The system also traverses all the boundary vertices in the lower lip path segment, calculates the Euclidean distance between each boundary vertex and the coordinates of the target position, and determines the boundary vertex with the smallest Euclidean distance as the midpoint of the lower lip, and records its vertex index and three-dimensional coordinates.

[0044] In addition, the positioning logic of the upper lip peak and the lower lip midpoint is consistent with the positioning logic of the corner of the mouth in S311. Both use the operator's selected coordinates as a reference and automatically align the anchor points to the boundary vertex sequence through nearest neighbor Euclidean distance matching to ensure that all anchor points strictly fall on the vertices of the lip topological boundary loop.

[0045] S313: For each mesh model, the upper lip peak point, the lower lip midpoint, and the corner anchor point pair are arranged in ascending order by vertex index to obtain the source model anchor point sequence and the target model anchor point sequence.

[0046] In step S313, the present invention arranges the four anatomical anchor points—the upper lip peak point, the lower lip midpoint, and the corner of the mouth anchor point pair—in ascending order according to their vertex indices in the lip topological boundary loop, and outputs the source model anchor point sequence and the target model anchor point sequence. Each sequence is an ordered list containing vertex indices and three-dimensional coordinates.

[0047] In steps S311 and S312, the method of locating anatomical anchor points is not limited to a specific implementation. The operator can mark anchor points by directly selecting vertices on the visualization interface of the lip topological boundary loop, with the system recording the vertex index and three-dimensional coordinates of the selected vertex. Alternatively, the local geometric curvature of each vertex on the lip topological boundary loop can be calculated, automatically identifying the two vertices with the largest horizontal curvature changes as the left and right corner points of the mouth. The upper lip peak point is automatically located on the upper lip path segment through curvature extremum detection, and the lower lip midpoint is automatically located on the lower lip path segment through cumulative arc length midpoint calculation. Both methods aim to output anatomical anchor point records containing vertex indices and three-dimensional coordinates, and the results are completely consistent in data structure and subsequent processing logic. Other methods of selecting feature points also fall within the scope of this invention.

[0048] S32: Using the vertex indices of the corresponding anchor points in the source model anchor point sequence and the target model anchor point sequence as dividing nodes, the lip topological boundary loop of each mesh model is divided into multiple continuous geometric path segments to obtain a lip boundary segment set.

[0049] Furthermore, in step S32, the present invention uses the vertex indices of each anchor point in the source model anchor point sequence and the target model anchor point sequence as dividing nodes to cut the respective lip topological boundary loops. The cutting method is as follows: the position of each anchor point is located sequentially from the vertex index sequence, and the vertex subsequence between two adjacent anchor points is extracted as an independent geometric path segment. Taking four anchor points as an example, the lip topological boundary loop is divided into four continuous geometric path segments. Each segment takes the vertex indices of the starting and ending anchor points as endpoints and contains the coordinates and index information of all ordered vertices within the segment. The four segments together constitute the lip boundary segment set. Finally, the source 3D face mesh model and the target 3D face mesh model each output a lip boundary segment set.

[0050] S33: Based on the semantic correspondence of the anatomical anchor points, perform a segment-by-segment hard matching between the lip boundary segment set of the source 3D facial mesh model and the lip boundary segment set of the target 3D facial mesh model to obtain the segment correspondence.

[0051] In step S33, based on the semantic correspondence of anatomical anchor points, this invention performs a hard match between each segment in the source 3D facial mesh model's lip boundary segment set and the corresponding segment in the target 3D facial mesh model's lip boundary segment set that has the same anatomical semantics. That is, they are bound one-to-one according to the segment number to form a segment correspondence. The obtained segment correspondence is stored in the form of a key-value pair list, where each record contains the vertex index sequence of a segment in the source model and the vertex index sequence of the corresponding segment in the target model.

[0052] S4: Based on the arc length ratio of multiple segments in the lip boundary segment set, adaptive sampling quota allocation is performed on multiple lip boundary segments of the target three-dimensional face mesh model in the segment correspondence relationship, and proportional linear step sampling is performed to obtain lip region matching point pairs with consistent logical indexes.

[0053] Step S4 further includes: S41: Calculate the cumulative arc length of each vertex sequence of multiple lip boundary segments of the source 3D face mesh model in the segmented correspondence relationship, and use the ratio of each segment arc length to the total arc length of the lip topological boundary loop as the geometric contribution weight of the corresponding segment to obtain the source model segment weight set.

[0054] In step S41, the present invention calculates the cumulative arc length of the vertex sequence of each segment in the lip boundary segment set of the source 3D face mesh model. The cumulative arc length is the total length value obtained by successively adding the Euclidean distance between adjacent vertices along the vertex sequence of the segment, reflecting the actual spatial length of the contour path of the segment. Subsequently, the present invention divides the cumulative arc length of each segment by the total arc length of the lip topological boundary loop (i.e., the sum of the cumulative arc lengths of all segments) to obtain the geometric contribution weight of each segment, which is stored in decimal form. The sum of the weights of all segments is one, forming the source model segment weight set.

[0055] S42: Multiply the preset total number of sampling points by the weights of each segment in the source model segment weight set, round down and allocate the remaining quotas to each segment, and compensate the remaining quotas to the corresponding segments in descending order of weight to obtain the target model segment sampling quota set.

[0056] In step S42, the present invention multiplies the preset total number of sampling points (i.e., the total number of matching points expected to be generated on the entire lip contour) with the weight value of each segment in the source model segment weight set to obtain the initial sampling quota floating-point number for each segment. Then, the floating-point number of each segment is rounded down to obtain the integer quota. Since rounding down will cause the sum of the integer quotas of each segment to be less than the preset total number of sampling points, the present invention calculates the difference as the remainder quota. The remainder quota is then allocated to the corresponding segments in descending order of the segment weights, with one additional point added, until all the remainder quotas are allocated. Finally, the target model segment sampling quota set is output, where each segment corresponds to a specific integer number of sampling points.

[0057] S43: Based on the target model segmented sampling quota set, perform equal arc length linear step sampling on each lip boundary segment of the target three-dimensional facial mesh model to obtain lip region matching point pairs with consistent logical indexes.

[0058] S431: For each segment of the lip boundary of the target 3D facial mesh model, divide the cumulative arc length of the corresponding segment equally according to the segment sampling quota, calculate the interpolation position of multiple equally divided nodes on the segment vertex sequence, and obtain the coordinate sequence of sampling points of each segment of the target model.

[0059] In step S431, for each segment of the vertex sequence of the target model, the present invention first calculates the cumulative arc length of the segment, then divides the cumulative arc length into a corresponding number of intervals according to the number of sampling quotas allocated to the segment, calculates the target arc length value at each equally divided node, locates the adjacent vertex interval in the vertex sequence into which the target arc length value falls, and performs linear interpolation on the three-dimensional coordinates of the two vertices at both ends of the interval according to the proportion to obtain the three-dimensional coordinates of the equally divided node. All the coordinates of the equally divided node are arranged in order to form the sampling point coordinate sequence of each segment of the target model.

[0060] S432: According to the vertex index arrangement order of each segment of the source model in the segmented correspondence relationship, the sampling point coordinate sequence of each segment of the target model is paired point by point with the sampling point coordinate sequence of the corresponding segment of the source model to obtain a lip region matching point pair with a consistent logical index.

[0061] In step S432, the present invention performs the same equal-arc-length linear step sampling as in S431 on each segment of the source 3D facial mesh model according to the vertex index arrangement order of each segment in the segmented correspondence relationship, to obtain the coordinate sequence of sampling points of each segment of the source model. Then, the coordinate sequence of sampling points of each segment of the target model is paired point by point with the corresponding coordinate sequence of sampling points of the source model in point position order. Each pair of points is composed of the coordinates of the sampling points of the source model and the target model. All pairing results are spliced ​​together in segmented order to output a pair of matching points for the lip region with a consistent logical index. The index position of the Nth point pair in the lip region matching point pair strictly corresponds on the source model and the target model, and both belong to the same lip contour position in anatomical semantics.

[0062] In the iterative process of non-rigid registration, the matching point pairs of the lip region are used as constraint point sets and input into the non-rigid iterative nearest neighbor algorithm. The corresponding vertices in the matching point pairs of the lip region are used as hard constraints to guide the template model to converge to the lip topological boundary of the target three-dimensional facial mesh model, so as to constrain the template vertices to the vermilion border and force the independence of the sampling index to be maintained at the segment boundary of the upper and lower lips, so as to prevent the upper and lower lip meshes from generating topological adhesion in the opening and closing regions.

[0063] After obtaining the matching point pairs in the lip region, this invention inputs them as a constraint point set into a non-rigid iterative nearest neighbor algorithm. In each iteration, the non-rigid iterative nearest neighbor algorithm receives the current vertex coordinates of the template model and the vertex coordinates of the target model, calculates the nearest neighbor correspondence, and updates the deformation field. In this process, this invention uses the corresponding vertices in the matching point pairs in the lip region as hard constraints, forcing the specified lip contour vertices of the template model to converge to the corresponding sampling points on the topological boundary loop of the lip of the target 3D facial mesh model. Simultaneously, the sampling point indices at each segment boundary are maintained independently, and the vertex index sequences of each segment of the upper lip and each segment of the lower lip do not intersect, thereby preventing vertex index aliasing and mesh topological adhesion in the lip opening and closing region during iterative deformation.

[0064] like Figure 2 As shown, the present invention also provides a system for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology, comprising: Alignment module 100: used to align the spatial pose of the source 3D facial mesh model and the target 3D facial mesh model to obtain an initial aligned model pair; Extraction module 200: used to extract loops from the lip region boundaries of multiple mesh models in the initial alignment model pair based on topological features, and obtain lip topological boundary loops; Segmentation module 300: used to mark anatomical anchor points on the lip topological boundary loop, segment the lip topological boundary loop to obtain a set of lip boundary segments, and establish a segmentation correspondence between the source 3D facial mesh model and the target 3D facial mesh model; Output module 400: Based on the arc length ratio of multiple segments in the lip boundary segment set, it performs adaptive sampling quota allocation on multiple lip boundary segments of the target three-dimensional face mesh model in the segment correspondence relationship, and performs proportional linear step sampling to obtain lip region matching point pairs with consistent logical indexes.

[0065] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0066] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology as described in the various embodiments or some parts of the embodiments.

[0067] This invention proposes a method for constructing matching point pairs for the lip region in 3D facial homology registration. First, based on manifold features, the topological loops of the lip edges are automatically extracted, restricting the registration search domain to ordered topological paths. Second, anatomical anchor points are used to semantically segment the loops, establishing a local correspondence between the source and target models. Finally, geometric contribution weights are introduced for adaptive sampling, constructing homology matching point pairs with strictly consistent logical indices. This point set can serve as an initial constraint for non-rigid registration, supporting the smooth construction of smile simulation data.

[0068] This invention introduces topological loop constraints, providing clear anatomical path guidance for non-rigid registration. This helps to stabilize the template points near the actual vermilion border, effectively suppressing the anatomical drift phenomenon common in traditional algorithms and significantly improving the alignment accuracy of the lip boundary.

[0069] Secondly, the adaptive sampling mechanism based on geometric contribution of this invention can better compensate for the nonlinear stretching of the lips in the smiling motion. Compared with equal-interval sampling, the matching point pairs generated by this invention are more reasonable in terms of biomechanical logic, significantly improving the naturalness of the motion trajectory of the simulation sequence and enhancing the robustness to non-uniform and large-amplitude deformation.

[0070] In addition, this invention pre-establishes the relationship of homologous points based on segments before registration, forcibly dividing the sampling boundaries of the upper and lower lips. Furthermore, the explicit index topology constraints help avoid erroneous mesh adhesion in the NICP algorithm in the lip opening and closing area, ensuring the physical correctness of dynamic simulation and effectively avoiding topological adhesion in complex opening and closing states.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing matching point pairs in the lip region for non-rigid registration of 3D facial homology, characterized in that, include: S1: Align the source 3D face mesh model with the target 3D face mesh model in spatial pose to obtain an initial aligned model pair; S2: Based on topological features, loop extraction is performed on the lip region boundaries of multiple mesh models in the initial alignment model pair to obtain the lip topological boundary loop; S3: Mark the anatomical anchor points on the lip topological boundary loop, perform segmentation processing on the lip topological boundary loop to obtain the lip boundary segment set, and establish the segment correspondence between the source 3D facial mesh model and the target 3D facial mesh model; S4: Based on the arc length ratio of multiple segments in the lip boundary segment set, adaptive sampling quota allocation is performed on multiple lip boundary segments of the target three-dimensional face mesh model in the segment correspondence relationship, and proportional linear step sampling is performed to obtain lip region matching point pairs with consistent logical indexes.

2. The method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology as described in claim 1, characterized in that, Step S1 further includes: S11: Input the source 3D facial mesh model and the target 3D facial mesh model, and locate multiple anatomical landmarks on the two models respectively using the anatomical landmark detection method to obtain the source model landmark set and the target model landmark set; S12: Calculate the spatial transformation matrix between the source model marker point set and the target model marker point set using a rigid transformation estimation algorithm. Based on the spatial transformation matrix, map the source 3D facial mesh model to the coordinate system of the target 3D facial mesh model to obtain an initial aligned model pair.

3. The method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology as described in claim 1, characterized in that, Step S2 further includes: S21: Traverse all mesh edges of each mesh model in the initial alignment model pair, determine the edges that are only referenced by a single facet as boundary edges, and form a boundary vertex set by the vertices corresponding to the boundary edges to obtain the original lip boundary vertex set. S22: By using a graph connectivity algorithm, multiple boundary vertices in the original lip boundary vertex set are traversed by adjacency relationship, and the unordered boundary vertices are reorganized into closed paths with strict vertex order, thus obtaining the lip topological boundary loop.

4. The method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology as described in claim 3, characterized in that, Step S22 further includes: S221: Starting from any boundary vertex, continuously search for the next adjacent node in the original lip boundary vertex set based on the depth-first traversal strategy until the path is closed at both ends, thus obtaining a closed path. S222: Verify the vertex order of the obtained closed path, rearrange the vertex index sequence according to the unified wrapping direction, and obtain a lip topological boundary loop with topological consistency.

5. The method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology as described in claim 1, characterized in that, Step S3 further includes: S31: For the lip topological boundary loop of each mesh model, mark a number of anatomical anchor points, the anatomical anchor points include at least the left corner of the mouth and the right corner of the mouth, and obtain the source model anchor point sequence and the target model anchor point sequence; S32: Using the vertex indices of the corresponding anchor points in the source model anchor point sequence and the target model anchor point sequence as dividing nodes, the lip topological boundary loop of each mesh model is divided into multiple continuous geometric path segments to obtain a lip boundary segment set. S33: Based on the semantic correspondence of the anatomical anchor points, perform a segment-by-segment hard matching between the lip boundary segment set of the source 3D facial mesh model and the lip boundary segment set of the target 3D facial mesh model to obtain the segment correspondence.

6. The method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology as described in claim 5, characterized in that, Step S31 further includes: S311: For the lip topological boundary loop of each mesh model, obtain the target position coordinates of the two anchor points selected by the interaction point, calculate the distance between the target position coordinates of each anchor point and all boundary vertices in the lip topological boundary loop, and determine the boundary vertex with the smallest distance corresponding to the target position coordinates of each anchor point as the left corner point and the right corner point, and obtain the corner anchor point pair. S312: For the lip topological boundary loop of each mesh model, the lip topological boundary loop is divided into an upper lip path segment and a lower lip path segment, with the vertex index of the corner anchor point pair as the boundary; on the upper lip path segment, the target position coordinates of the first feature point selected by the interaction point are obtained, the distance between the target position coordinates of the first feature point and all boundary vertices in the upper lip path segment is calculated, and the boundary vertex with the smallest distance is determined as the upper lip peak point; on the lower lip path segment, the target position coordinates of the second feature point selected by the interaction point are obtained, the distance between the target position coordinates of the second feature point and all boundary vertices in the lower lip path segment is calculated, and the boundary vertex with the smallest distance is determined as the lower lip midpoint; S313: For each mesh model, the upper lip peak point, the lower lip midpoint, and the corner anchor point pair are arranged in ascending order by vertex index to obtain the source model anchor point sequence and the target model anchor point sequence.

7. The method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology according to claim 1, characterized in that, Step S4 further includes: S41: Calculate the cumulative arc length of the vertex sequence of multiple lip boundary segments of the source 3D face mesh model in the segmented correspondence relationship, and use the ratio of the arc length of each segment to the total arc length of the lip topological boundary loop as the geometric contribution weight of the corresponding segment to obtain the source model segment weight set. S42: Multiply the preset total number of sampling points by the weights of each segment in the source model segment weight set, round down and allocate the remaining quotas to each segment, and compensate the remaining quotas to the corresponding segments in descending order of weight to obtain the target model segment sampling quota set. S43: Based on the target model segmented sampling quota set, perform equal arc length linear step sampling on each lip boundary segment of the target three-dimensional facial mesh model to obtain lip region matching point pairs with consistent logical indexes.

8. A method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology according to claim 7, characterized in that, Step S43 further includes: S431: Divide the lip boundary of each segment of the target three-dimensional facial mesh model into equal parts according to the segment sampling quota, calculate the interpolation position of multiple equally divided nodes on the segment vertex sequence, and obtain the coordinate sequence of sampling points of each segment of the target model. S432: According to the vertex index arrangement order of each segment of the source model in the segmented correspondence relationship, the sampling point coordinate sequence of each segment of the target model is paired point by point with the sampling point coordinate sequence of the corresponding segment of the source model to obtain a lip region matching point pair with a consistent logical index.

9. A method for constructing matching point pairs of the lip region for non-rigid registration of three-dimensional facial homology according to claim 1, characterized in that, During the iterative process of non-rigid registration, the matching point pairs of the lip region are used as constraint point sets and input into the non-rigid iterative nearest neighbor algorithm. The corresponding vertices in the matching point pairs of the lip region are used as hard constraints to guide the template model to converge to the lip topological boundary of the target three-dimensional facial mesh model, so as to constrain the template vertices at the vermilion border and force the independence of the sampling index to be maintained at the segment boundary of the upper and lower lips, preventing the upper and lower lip meshes from generating topological adhesion in the opening and closing regions.

10. A system for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology, used to execute the method for constructing lip region matching point pairs for non-rigid registration of three-dimensional facial homology as described in any one of claims 1 to 9, characterized in that, include: Alignment module: used to align the spatial pose of the source 3D facial mesh model and the target 3D facial mesh model to obtain an initial aligned model pair; Extraction module: used to extract the lip region boundary of multiple mesh models in the initial alignment model pair based on topological features, and obtain the lip topological boundary loop; Segmentation module: used to mark anatomical anchor points on the lip topological boundary loop, segment the lip topological boundary loop to obtain a set of lip boundary segments, and establish the segmentation correspondence between the source 3D facial mesh model and the target 3D facial mesh model; Output module: Based on the arc length ratio of multiple segments in the lip boundary segment set, adaptively allocate sampling quotas for multiple lip boundary segments of the target 3D face mesh model in the segment correspondence relationship, and perform proportional linear step sampling to obtain lip region matching point pairs with consistent logical indexes.

Images