Heterogeneous face point cloud registration method with adaptive feature matching

Abstract

A kind of self-adapting feature matching heterogeneous face point cloud registration method, first, the source face point cloud and target face point cloud are preprocessed, the initial pose deviation of both is quickly reduced through rough registration;Then the ISS key points of the point cloud after rough registration are extracted, the normal vector and curvature value of each key point are calculated, and the registration primitive covering multidimensional geometric features is constructed;Then, combined with the inherent geometric law of face, an adaptive weight distribution mechanism is designed, by real-time analysis of the curvature difference of key points, normal angle and point cloud data characteristics, the weight proportion of Euclidean distance, normal consistency and curvature similarity is dynamically adjusted;Then, based on the adaptive weight, a weighted error function is constructed, and the optimization process of weighted point-to-plane ICP is iteratively executed;When the iteration termination condition is reached, the final transformation matrix is output, and the high-precision fine registration of heterogeneous face point cloud is completed.The present application realizes accurate and robust alignment of heterogeneous face point cloud.

Description

Technical Field

[0001] This invention relates to the fields of medical imaging and neurosurgical assistance in computer graphics, and in particular to an adaptive feature matching method for heterogeneous facial point cloud registration. Background Technology

[0002] With the development of 3D sensing and medical imaging technologies, point cloud registration has become one of the core technologies in fields such as 3D face reconstruction and neurosurgical navigation. Among these, heterogeneous face point cloud registration (such as the alignment of face point clouds extracted from medical images with face point clouds acquired through structured light / laser scanning) is a crucial step in achieving accurate face pose estimation and surgical navigation positioning. However, current technologies still have significant shortcomings in this scenario: In face-related 3D applications, differences in the sources of heterogeneous face point clouds (such as the density uniformity of medical image point clouds, noise and local missing parts of scanned point clouds) can lead to inconsistent distribution of geometric features (normal, curvature) of the point clouds. Traditional ICP registration algorithms rely solely on Euclidean distance to measure correspondence, without considering the inherent geometric attributes of the face, which can easily lead to mismatches in areas with similar facial structures (such as cheeks and foreheads). At the same time, traditional ICP is highly sensitive to initial pose, and when the initial deviation of heterogeneous face point clouds is large, it is very easy to get stuck in local optima, which cannot meet the requirements of high-precision scenarios.

[0003] To compensate for the shortcomings of traditional ICP, existing technologies have developed weighted ICP improvement methods that integrate geometric features such as normal and curvature. However, these methods are mostly designed for general point cloud scenarios and do not adapt to the special geometric features of human faces (such as left-right facial symmetry and differences in curvature / normal distribution of local feature points). Furthermore, they adopt fixed weight allocation strategies and cannot dynamically adjust the weight ratio of features such as distance, normal, and curvature based on the local features of heterogeneous face point clouds (such as high curvature areas of the bridge of the nose and low curvature areas of the cheeks). As a result, the registration accuracy is greatly affected by the density and noise differences of heterogeneous data.

[0004] In high-precision scenarios such as neurosurgical navigation, heterogeneous face point clouds also suffer from anisotropic deformation (inconsistent scale changes in different spatial directions). Existing registration methods (such as SAC-IA coarse registration + traditional ICP fine registration) have not designed adaptation strategies for this deformation, and the registration error is difficult to control within a clinically acceptable range. On the other hand, deep learning-based registration methods (such as PointNetLK and DCP) rely on large-scale labeled data and high-performance hardware, and have poor interpretability, which cannot meet the deployment requirements of lightweight and highly reliable scenarios such as surgical navigation.

[0005] Furthermore, among existing surgical assistance techniques related to the skull base and face, research on registration methods for heterogeneous facial point clouds is extremely scarce. Most solutions follow general point cloud registration logic and do not utilize the inherent geometric attributes of the face to optimize the registration process, making it difficult to balance the robustness and accuracy of registration.

[0006] Therefore, in order to address the pain points of registration of heterogeneous face point clouds, there is an urgent need for an improved ICP method that combines facial geometric features and supports adaptive weight adjustment, so as to improve the accuracy, robustness and scene adaptability of heterogeneous face point cloud registration, and meet the high-precision requirements of scenarios such as face pose estimation and neurosurgical navigation. This research is of great significance. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides an adaptive feature matching method for heterogeneous facial point cloud registration, achieving accurate and robust alignment of heterogeneous facial point clouds. The method first extracts the ISS keypoints of the heterogeneous facial point cloud and calculates the normal and curvature features of the keypoints; then, based on facial geometry, it constructs an adaptive weight model, dynamically allocating the weight proportions of distance, normal, and curvature features; finally, through weighted ICP iterative optimization, it completes the precise registration of the heterogeneous facial point cloud.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: An adaptive feature matching method for heterogeneous facial point cloud registration includes the following steps: 1) Key point and geometric feature extraction: Extract ISS key points from the source (heterogeneous source scan) face point cloud and the target (medical image) face point cloud respectively, and calculate the normal vector and curvature value of each key point; 2) Adaptive weight model construction: Combining the geometric rules of the face (such as the symmetrical areas of the face and the curvature distribution of local feature points), the weight coefficients of Euclidean distance, normal consistency and curvature similarity are dynamically allocated based on the curvature difference and normal angle of key points. 3) Weighted ICP fine registration: Based on the adaptive weight model, a weighted error function is constructed, and the optimization process of weighted point-to-surface ICP is iteratively executed. When the iteration termination condition is reached, the final transformation matrix is ​​output to complete the high-precision fine registration of heterogeneous facial point clouds.

[0009] Furthermore, in step 1), the fine registration acceleration strategy based on ISS keypoint detection involves introducing the ISS (Intrinsic Shape Signatures) keypoint detection method during the fine registration stage to extract keypoints from both the source and target point clouds, and then performing weighted point-to-surface ICP optimization based on a subset of keypoints. ISS keypoints can significantly reduce the number of points involved in the optimization while maintaining the representativeness of the geometric structure, thereby effectively improving the computational efficiency of the fine registration stage.

[0010] The technical concept of this invention is as follows: Based on the good initial pose provided by coarse registration, the core revolves around constructing a fine registration framework with adaptive optimization. First, the key points of the ISS and the geometric features of normal and curvature of the point cloud after coarse registration are extracted. Combined with the inherent geometric rules of the face (such as the symmetrical regions of the face and the differences in curvature / normal distribution of local feature points), a dynamic weight allocation mechanism is designed. By analyzing the local feature similarity (curvature difference, normal angle) and data characteristics (density, noise distribution) of heterogeneous point clouds in real time, the weight ratio of Euclidean distance, normal consistency, and curvature similarity is adaptively adjusted. This adaptive weight is integrated into the error function of ICP registration, and the accurate matching of corresponding point pairs is achieved through iterative optimization. At the same time, it adapts to the anisotropic deformation of heterogeneous point clouds, and finally achieves high-precision and high-robust fine registration of heterogeneous face point clouds.

[0011] The beneficial effects of this invention are mainly reflected in the following aspects: This invention fully considers multiple features, and its adaptive weighting mechanism can respond in real time to the density difference, noise distribution, and anisotropic deformation of medical images and scanned point clouds. According to the inherent geometric rules of the human face (such as high curvature areas like the bridge of the nose and flat areas like the cheeks), it dynamically adjusts the weight ratio of Euclidean distance, normal consistency, and curvature similarity, strengthens the discriminative power of local features to avoid mismatches in structurally similar regions, significantly improves registration accuracy, and meets the stringent requirements of high-precision scenarios such as neurosurgical navigation. At the same time, this adaptive mechanism can dynamically offset the negative impacts of missing data and environmental interference. Combined with the stability of ISS key points, it greatly enhances the robustness and reliability of the registration results.

[0012] This invention provides an automated technical solution for heterogeneous face point cloud registration using "adaptive fine registration optimization." The entire registration process is based on geometric feature optimization design, requiring no large-scale labeled data or high-performance hardware support. Its lightweight nature allows for direct deployment in various scenarios such as surgical navigation devices, effectively lowering the application threshold. Through the collaborative adaptive mechanism, it overcomes the limitations of traditional methods, reducing reliance on manual intervention. It not only meets the accuracy requirements of multiple scenarios such as medical assistance but also helps promote the widespread adoption and large-scale application of 3D face-related technologies in multi-device, multi-data source scenarios. Attached Figure Description

[0013] Figure 1 This is a flowchart of a heterogeneous facial point cloud registration method based on adaptive feature matching.

[0014] Figure 2 The diagram shows the visualization results, where (a) represents the initial pose, (b) represents ICP, (c) represents PICP, (d) represents GICP, (e) represents TICP, (f) represents SIPC, (g) represents TEASER++, (h) represents KISS-Matcher, and (i) represents the method of the present invention. Detailed Implementation

[0015] The present invention will be further described below.

[0016] Reference Figure 1 and Figure 2 An adaptive feature matching method for heterogeneous facial point cloud registration is proposed. First, the source face point cloud (acquired by scanning) and the target face point cloud (extracted from medical images) are preprocessed to quickly reduce the initial pose deviation between the two through coarse registration, providing stable initial conditions for fine registration. Next, key points (ISS) are extracted from the coarsely registered point cloud, and the normal vector and curvature value of each key point are calculated to construct registration primitives covering multi-dimensional geometric features. Then, combined with the inherent geometric rules of the face, an adaptive weight allocation mechanism is designed. By analyzing the curvature difference, normal angle, and point cloud data characteristics (density, noise distribution) of key points in real time, the weight ratios of Euclidean distance, normal consistency, and curvature similarity are dynamically adjusted. Based on this adaptive weight, a weighted error function is constructed, and the weighted point-to-surface ICP optimization process is iteratively executed. Finally, when the iteration termination condition is met, the final transformation matrix is ​​output, completing the high-precision fine registration of heterogeneous face point clouds.

[0017] The method in this embodiment includes the following steps: 1) Key point and geometric feature extraction: Let the source point cloud be The target point cloud is The initial rigidity change was obtained through the coarse registration stage. The goal of fine registration is to further solve for the optimal rigid transformation based on the initial pose: (1); Based on point-to-surface constraints, the fundamental residual for each pair of correspondences is defined as: (2); in, It is the normal vector. For source point cloud, For the target point cloud.

[0018] 2) A weighted strategy based on distance, normal, and curvature, the process of which is as follows: a) Distance weight Distance weighting is used to reduce the impact of corresponding point pairs that are spatially distant and have low confidence. It is defined as follows: (3); (4); in, Source Point Cloud To the target point cloud distance, This is the distance attenuation parameter; b) Normal consistency weight To ensure that the correct corresponding points should maintain consistency in the local surface direction, a normal consistency weight is introduced, which is defined as: (5); (6); in, It is the normal vector. The angle between the normal vectors of the source point cloud and the target point cloud is denoted as . This is the normal uniformity decay parameter; c) Curvature weight To reduce the interference of high curvature regions and noise points on the registration results, a weighting term based on the curvature of the target point is introduced, defined as follows: (7); in, For curvature, This refers to the curvature attenuation parameter; d) Comprehensive weight model : It adopts a linear combination form, which is defined as: (8); in, These are experimental parameters.

[0019] 3) Construct the weighted point-to-surface ICP optimization model, the process is as follows: a) Weighted point-to-surface residual: (9); in, It is the normal vector. For source point cloud, For the target point cloud, As weight; b) Constructing the overall nonlinear least squares objective function: (10); The objective function described above is iteratively optimized using the Levenberg-Marquard algorithm. c) LM nonlinear optimization solution: In the k-th iteration, the residuals are linearized to the first order: (11); Where J is the Jacobian matrix of the weighted residuals with respect to the pose parameters; The LM update equation is: (12); After solving for the pose increment, the rigid body transformation is updated through exponential mapping: (13); d) Iteration termination condition: The iteration terminates when any of the following conditions are met: pose increment The value is less than the threshold, the residual change is less than the set threshold, or the maximum number of iterations has been reached.

[0020] This embodiment is based on an accelerated fine registration strategy using ISS keypoint detection: In the fine registration stage, the ISS (Intrinsic Shape Signatures) keypoint detection method is introduced to extract keypoints from the source point cloud and the target point cloud, and weighted point-to-surface ICP optimization is performed based on the keypoint subset.

[0021] In this embodiment, classic ICP-type improved algorithms such as ICP, PICP, GICP, TICP, and SICP are selected for comparison with mainstream global registration methods in recent years, such as TEASER++ and KISS-Matcher. ICP is the most fundamental iterative algorithm in the field of fine point cloud registration. PICP, GICP, TICP, and SICP improve upon traditional ICP from the perspectives of probabilistic models, geometric constraints, transform estimation, and robust loss, respectively, and are widely representative. TEASER++ and KISS-Matcher are currently global registration methods with excellent accuracy and efficiency, often used as benchmarks in high-noise scenes. By comparing with algorithms of different types and improvement approaches, the performance of the method in this embodiment in terms of registration accuracy can be evaluated more comprehensively and objectively. (Refer to...) Figure 2 The visualization results of this invention fully capture the key geometric features of heterogeneous point clouds, ensuring registration accuracy.

[0022] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A heterogeneous facial point cloud registration method based on adaptive feature matching, characterized in that, First, the source face point cloud obtained from scanning and the target face point cloud extracted from medical images are preprocessed. Coarse registration is used to quickly reduce the initial pose deviation between the two, providing stable initial conditions for fine registration. Next, ISS keypoints are extracted from the coarsely registered point cloud, and the normal vector and curvature value of each keypoint are calculated to construct registration primitives covering multi-dimensional geometric features. Then, combined with the inherent geometric rules of the face, an adaptive weight allocation mechanism is designed. By analyzing the curvature difference, normal angle, and point cloud data characteristics of keypoints in real time, the weight ratio of Euclidean distance, normal consistency, and curvature similarity is dynamically adjusted. Finally, a weighted error function is constructed based on the adaptive weights, and the weighted point-to-surface ICP optimization process is iteratively executed. Finally, when the iteration termination condition is met, the final transformation matrix is ​​output, completing the high-precision registration of heterogeneous facial point clouds.

2. The heterogeneous facial point cloud registration method with adaptive feature matching as described in claim 1, characterized in that, The method includes the following steps: 1) Let the source point cloud be... The target point cloud is The initial rigidity change was obtained through the coarse registration stage. The goal of fine registration is to further solve for the optimal rigid transformation based on the initial pose: (1)； Based on point-to-surface constraints, the fundamental residual for each pair of correspondences is defined as: (2)； in, It is the normal vector. For source point cloud, For the target point cloud.

3. The heterogeneous facial point cloud registration method with adaptive feature matching as described in claim 2, characterized in that, The method further includes the following steps: 2) A weighted strategy based on distance, normal, and curvature, the process of which is as follows: a) Distance weight Distance weighting is used to reduce the impact of corresponding point pairs that are spatially distant and have low confidence. It is defined as follows: (3)； (4)； in, Source Point Cloud To the target point cloud distance, This is the distance attenuation parameter; b) Normal consistency weight To ensure that the correct corresponding points should maintain consistency in the local surface direction, a normal consistency weight is introduced, which is defined as: (5)； (6)； in, It is the normal vector. The angle between the normal vectors of the source point cloud and the target point cloud is denoted as . This is the normal uniformity decay parameter; c) Curvature weight To reduce the interference of high curvature regions and noise points on the registration results, a weighting term based on the curvature of the target point is introduced, defined as follows: (7)； in, For curvature, This refers to the curvature attenuation parameter; d) Comprehensive weight model : It adopts a linear combination form, which is defined as: (8)； in, These are experimental parameters.

4. The heterogeneous facial point cloud registration method with adaptive feature matching as described in claim 3, characterized in that, The method also Includes the following steps: 3) Construct the weighted point-to-surface ICP optimization model, the process is as follows: a) Weighted point-to-surface residual: (9)； in, It is the normal vector. For source point cloud, For the target point cloud, As weight; b) Constructing the overall nonlinear least squares objective function: (10)； The objective function described above is iteratively optimized using the Levenberg-Marquard algorithm. c) LM nonlinear optimization solution: In the k-th iteration, the residuals are linearized to the first order: (11)； Where J is the Jacobian matrix of the weighted residuals with respect to the pose parameters; The LM update equation is: (12)； After solving for the pose increment, the rigid body transformation is updated through exponential mapping: (13)； d) Iteration termination condition: The iteration terminates when any of the following conditions are met: pose increment The value is less than the threshold, the residual change is less than the set threshold, or the maximum number of iterations has been reached.

5. The heterogeneous facial point cloud registration method with adaptive feature matching as described in claim 4, characterized in that, In the first part, a fine registration acceleration strategy based on ISS key point detection is integrated: the ISS key point detection method is introduced in the fine registration stage to extract key points from the source point cloud and the target point cloud, and weighted point-to-surface ICP optimization is performed based on the key point subset.

Images