Registration method and related device based on multi-scale hierarchical voxel fusion architecture

Abstract

The application discloses a registration method based on a multi-scale hierarchical voxel fusion architecture and related equipment, and relates to the technical field of computer vision and digital oral medical treatment. Macrostructure and microtexture are extracted in parallel by using a coarse and fine double-layer voxel architecture, false features caused by scanning noise points are effectively removed through cross-level consistency verification, and a semantic perception weighting mechanism is combined to intelligently identify and inhibit the interference of soft tissues such as gums which are easy to deform, so that the registration process focuses on the rigid structure of the teeth which is high in stability, and further cooperates with a two-stage rigid transformation strategy with a search radius from large to small, which not only guarantees the global convergence ability under the initial large displacement, but also ensures the sub-voxel level accuracy of the final pose. A series of cooperative mechanisms effectively solve the problems of large soft tissue interference and easily confused features in a complex oral environment, and improve the robustness of registration of key functional areas of the oral cavity and the geometric accuracy of the final reconstruction model.

Description

Technical Field

[0001] This application relates to the fields of computer vision and digital oral healthcare technology, and in particular to a registration method and related equipment based on a multi-scale hierarchical voxel fusion architecture. Background Technology

[0002] In digital oral diagnosis and treatment technology, obtaining accurate three-dimensional models of the dentition and jawbone through intraoral scanning or model scanning is fundamental to subsequent steps such as precise diagnosis, prosthesis design, and orthodontic treatment. This process typically involves stitching and registering multi-view three-dimensional point cloud data obtained from multiple scans to construct a complete and high-precision digital oral model.

[0003] In related technologies, a registration strategy based on the Iterative Closest Point (ICP) algorithm is commonly used. This method assumes that the two sets of point clouds to be registered have a certain initial overlap. Then, in three-dimensional space, for each data point in the source point cloud, the closest point in the target point cloud with the lowest Euclidean distance is found as the corresponding point pair. Based on these corresponding point pairs, the algorithm constructs an error objective function and solves for the rotation and translation matrix that minimizes the average distance between the corresponding point pairs using the least squares method. Subsequently, the position of the source point cloud is updated using the calculated transformation matrix, and the process of finding the nearest point and solving the transformation matrix is ​​repeated until the error converges, thus completing the alignment of the point clouds.

[0004] However, the processing performance of oral scan data is highly dependent on the initial pose and data quality of the point cloud. In real-world scanning scenarios, the oral environment includes various tissues with different shapes and reflective properties, such as teeth, gums, tongue, and buccal soft tissue. When using global point clouds for iterative optimization, the algorithm is easily affected by smooth soft tissue regions such as gums that lack significant geometric features, or by artifact noise points generated during the scanning process. This can cause the transformation solution process to get stuck in a local optimum dominated by non-critical regions, resulting in registration deviations in key functional areas such as teeth, and thus reducing the accuracy of 3D model reconstruction. Summary of the Invention

[0005] This application provides a registration method and related equipment based on a multi-scale hierarchical voxel fusion architecture to improve the registration accuracy of key functional areas of the oral cavity and the accuracy of the reconstructed model in complex oral environments.

[0006] Firstly, this application provides a registration method based on a multi-scale hierarchical voxel fusion architecture, applied to a 3D reconstruction device. This multi-scale hierarchical voxel fusion architecture includes a coarse-scale voxel layer and a fine-scale voxel layer. The method includes: after determining the coarse-scale geometric features of each activated voxel in the coarse-scale voxel layer and the fine-scale geometric features of each activated voxel in the fine-scale voxel layer, based on the spatial mapping relationship between the coarse-scale voxel layer and the fine-scale voxel layer, performing cross-level consistency verification on the coarse-scale geometric features and the fine-scale geometric features to determine candidate feature voxels that meet a preset consistency threshold; performing geometric attribute analysis on the local region where the candidate feature voxels are located to determine the oral tissue semantic category corresponding to the candidate feature voxels; and filtering out voxels belonging to a preset interference language based on the oral tissue semantic category. After identifying the feature voxels of the semantic category, among the candidate feature voxels that do not belong to the preset interference semantic category, the confidence is weighted based on the enhancement coefficient of the corresponding oral tissue semantic category to obtain a preferred feature voxel set. Based on the preferred feature voxels in the coarse-scale voxel layer of the preferred feature voxel set, a first-stage rigid body transformation is performed in the coarse-scale voxel layer to obtain the solution result. The search radius of the first-stage rigid body transformation is a preset first multiple of the coarse-scale voxel resolution, and a first maximum number of iterations is set. Using the solution result as the initial value, a second-stage rigid body transformation is performed in the fine-scale voxel layer through the preferred feature voxels to obtain the final point cloud pose. The search radius of the second-stage rigid body transformation is a preset second multiple of the fine-scale voxel resolution, and a second maximum number of iterations is set.

[0007] By employing the aforementioned technical solution, a coarse-to-fine dual-layer voxel architecture is used to extract macroscopic structures and microscopic textures in parallel. Cross-level consistency verification effectively eliminates spurious features caused by scanning noise. Combined with a semantically aware weighted mechanism, interference from easily deformable soft tissue areas such as the gingiva can be intelligently identified and suppressed, allowing the registration process to focus on the highly stable rigid structure of the teeth. Furthermore, a two-stage rigid body transformation strategy with a search radius decreasing from large to small ensures both global convergence under initial large displacements and sub-voxel-level accuracy of the final pose. This series of collaborative mechanisms effectively solves the problems of large soft tissue interference and easily confused features in complex oral environments, improving the robustness of registration of key oral functional areas and the geometric accuracy of the final reconstructed model.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of determining the coarse-scale geometric features of each active voxel in the coarse-scale voxel layer specifically includes: determining the curvature extreme response value, the intrinsic shape feature response value, and the Harris3D response value based on the covariance matrix of the point cloud distribution inside each active voxel in the coarse-scale voxel layer; when it is determined that a specific feature response value exceeds a preset significance threshold, multiplying the corresponding weight coefficient in the preset weight fusion strategy with a preset enhancement factor to obtain a target weight fusion strategy, wherein the specific feature response value is one or more of the curvature extreme response value, the intrinsic shape feature response value, and the Harris3D response value; and performing a weighted summation of the curvature extreme response value, the intrinsic shape feature response value, and the Harris3D response value based on the target weight fusion strategy to generate a confidence score for the corresponding active voxel.

[0009] By employing the above technical solution, multimodal geometric features (curvature, intrinsic shape, Harris3D) can be fused and analyzed, and a dynamic weight enhancement strategy can be used to specifically amplify the most significant geometric attribute response values ​​of the current voxel. This mechanism avoids the problem of single features being scarce in certain regions and feature matching failing, ensuring that the generated confidence score accurately reflects the physical uniqueness and matching reliability of the feature point. Guided by high-weight features, subsequent registration algorithms can prioritize locking high-quality anchor points, thereby effectively improving the saliency and discriminative power of feature extraction in weakly textured regions.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of performing cross-level consistency verification on the coarse-scale geometric features and the fine-scale geometric features corresponding to spatial locations based on the spatial mapping relationship between the coarse-scale voxel layer and the fine-scale voxel layer, and determining candidate feature voxels that meet a preset consistency threshold, specifically includes: determining the corresponding parent coarse-scale voxel in the coarse-scale voxel layer based on the parent voxel reference pointer of the potential candidate feature voxel in the fine-scale voxel layer; and determining that at the spatial projection position of the potential candidate feature voxel in the corresponding parent coarse-scale voxel, there exists a value exceeding a preset significance threshold. If the potential candidate feature voxel satisfies the positional consistency verification when the spatial positional deviation between the potential candidate feature voxel and the parent coarse-scale voxel is not higher than a preset positional deviation threshold, then the potential candidate feature voxel is determined to satisfy the positional consistency verification. The feature response correlation coefficient between the fine-scale fusion feature descriptor of the potential candidate feature voxel and the coarse-scale fusion feature descriptor of the parent coarse-scale voxel is calculated. If the potential candidate feature voxel satisfies the positional consistency verification and the feature response correlation coefficient is greater than a preset correlation threshold, then the potential candidate feature voxel is determined to be a candidate feature voxel that satisfies the preset consistency threshold.

[0011] By employing the aforementioned technical solution and a verification mechanism based on parent-child voxel spatial constraints, this approach verifies whether fine-scale features have corresponding coarse-scale parent structural support (dual verification of position and feature response). This allows for the identification and elimination of isolated noise points or transient artifacts that, while exhibiting high local responses, lack structural support at the macroscopic scale. This cross-scale dual verification logic ensures that only voxels possessing spatial stability and feature continuity are selected for the candidate set, reducing the false matching rate and guaranteeing the purity and spatial distribution of the feature point set.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the step of performing geometric attribute analysis on the local region where the candidate feature voxel is located to determine the oral tissue semantic category corresponding to the candidate feature voxel specifically includes: calculating the average curvature and normal vector consistency of all point clouds within the fine-scale voxel where each candidate feature voxel is located; and determining the oral tissue semantic category corresponding to the candidate feature voxel based on the average curvature, the normal vector consistency, and the preset soft tissue classification rules.

[0013] In conjunction with some embodiments of the first aspect, in some embodiments, the step of filtering out feature voxels belonging to a preset interference semantic category from candidate feature voxels according to the oral tissue semantic category, and then weighting the confidence level based on the enhancement coefficient of the corresponding oral tissue semantic category among the candidate feature voxels that do not belong to the preset interference semantic category to obtain an optimal feature voxel set, specifically includes: determining whether the spatial position of the candidate feature voxel conforms to the anatomical distribution map of the corresponding oral tissue semantic category based on prior knowledge of oral anatomy, and identifying candidate feature voxels that violate the anatomical distribution map as preset interference semantic categories; filtering out feature voxels belonging to the preset interference semantic category from candidate feature voxels, and then weighting the confidence level based on the enhancement coefficient of the corresponding oral tissue semantic category to obtain an optimal feature voxel set.

[0014] By employing the above technical solution, anatomical prior knowledge is introduced as a secondary error correction mechanism for semantic classification, which can correct classification errors that violate physiological structures (such as misjudging "gingiva" in the center of the occlusal surface of teeth). Based on this, by filtering out high-risk soft tissue interference points and weighting and enhancing rigid tooth features, the energy function of rigid body transformation can be dominated by highly stable anatomical structures. This achieves improved registration accuracy in a dynamic oral environment.

[0015] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of obtaining the final point cloud pose, the method further includes: if the quality integrity check of the point cloud pose is determined to be passed, then the point cloud pose is output and the global three-dimensional model is updated based on the point cloud pose; if the quality integrity check is determined to be failed, then the point cloud pose is replaced by inertial measurement data and / or historical pose data and the global three-dimensional model is updated based on the replaced point cloud pose.

[0016] By adopting the above technical solution, a closed-loop access control mechanism for data quality is constructed, prohibiting poses that fail quality inspection from participating in global model updates, thereby preventing the accumulation of errors from causing irreversible damage to the 3D model (such as layering or ghosting). Simultaneously, a multi-source sensor backoff strategy is integrated, enabling seamless switching to inertial or historical data-driven modes the instant the visual algorithm fails. This mechanism not only protects the geometric integrity and surface smoothness of the globally reconstructed model but also ensures scanning continuity in visual blind spots or under extreme motion, achieving high reliability and a smooth user experience.

[0017] In conjunction with some embodiments of the first aspect, in some embodiments, the step of replacing the point cloud pose with inertial measurement data and / or historical pose data and updating the global 3D model based on the replaced point cloud pose specifically includes: when valid data input is detected from the inertial measurement unit bound to the scanning device, performing time integration calculation based on the acceleration and angular velocity data corresponding to the current frame timestamp to determine the relative pose change, and superimposing the relative pose change onto the point cloud pose of the previous frame to generate an inertial prediction pose; when no valid data input is detected, reusing the point cloud pose of the previous frame, performing smooth extrapolation based on historical motion trajectories to generate a motion prediction pose; determining the inertial prediction pose or the motion prediction pose as the replaced point cloud pose, and using the replaced point cloud pose to update the global 3D model with the point cloud data of the current frame.

[0018] By employing the aforementioned technical solution, short-time integral calculations are performed using the high-frequency physical characteristics of the inertial measurement unit (IMU), or smooth extrapolation is performed based on historical motion trajectories, thereby generating usable predicted poses within millisecond-level gaps where effective visual features are lacking. This updated pose is used to maintain state refresh, effectively filling the time gaps in visual blind spots and preventing forced resets due to momentary tracking loss. This not only solves the problem of scanning interruptions in fast-moving or featureless areas but also reduces the scanning difficulty for operators and improves the continuous tracking capability of handheld scanners under extreme conditions.

[0019] In a second aspect, this application provides a three-dimensional reconstruction device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the three-dimensional reconstruction device to perform the method described in the first aspect and any possible implementation thereof.

[0020] Thirdly, this application provides a computer program product containing instructions that, when run on a 3D reconstruction device, cause the 3D reconstruction device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a 3D reconstruction device, cause the 3D reconstruction device to perform the method described in the first aspect and any possible implementation thereof.

[0022] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0023] 1. By employing a feature collaborative extraction technology in coarse and fine voxel spaces, combined with a feature optimization mechanism involving cross-level consistency verification and semantic weighted filtering, and supplemented by a two-stage rigid body registration strategy from coarse to fine, the technical problem of registration drift in complex oral environments is effectively solved through in-depth verification and intelligent filtering of multimodal geometric information. This leads to an improvement in the robustness of registration in key oral functional areas and the geometric accuracy of the final 3D reconstruction model.

[0024] 2. Due to the adoption of cross-level dual verification technology based on parent-child voxel spatial mapping, that is, by forcibly detecting whether fine-scale features have macroscopic positional support and feature response correlation of coarse-scale parent structure, it can identify and eliminate those isolated pseudo-feature noise points that, even if the local response is high, lack macroscopic structural stability. This effectively solves the technical problem of high feature mismatch rate caused by noise or transient artifacts in existing high-resolution scanning, thereby enhancing the purity and spatial distribution rationality of the feature point set used for registration calculation.

[0025] 3. By employing a multi-level backoff compensation technique that integrates inertial measurement unit (IMU) pre-integration and pure kinematic model extrapolation, high-frequency predicted pose replacement can be generated using physical sensor data when visual registration fails. This prevents resets or stuttering caused by feature tracking loss, effectively solving the technical problem of scanning interruption and trajectory breakage that easily occurs when handheld scanning devices move quickly or scan textureless areas. This, in turn, optimizes the continuous tracking capability of scanning operations and the smoothness of user experience under extreme conditions. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a registration method based on a multi-scale hierarchical voxel fusion architecture in an embodiment of this application.

[0027] Figure 2 This is another flowchart illustrating the registration method based on a multi-scale hierarchical voxel fusion architecture in the embodiments of this application;

[0028] Figure 3 This is an exemplary hardware structure diagram of a three-dimensional reconstruction device in the embodiments of this application. Detailed Implementation

[0029] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0030] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0031] Please see Figure 1 This is a flowchart illustrating a registration method based on a multi-scale hierarchical voxel fusion architecture in an embodiment of this application.

[0032] S101. After determining the coarse-scale geometric features of each active voxel in the coarse-scale voxel layer and the fine-scale geometric features of each active voxel in the fine-scale voxel layer, based on the spatial mapping relationship between the coarse-scale voxel layer and the fine-scale voxel layer, cross-level consistency verification is performed on the coarse-scale geometric features and the fine-scale geometric features to determine candidate feature voxels that meet the preset consistency threshold.

[0033] Among them, the coarse-scale voxel layer and the fine-scale voxel layer refer to two layers with different physical resolutions in the multi-scale hierarchical voxel fusion architecture. The resolution of the coarse-scale voxel layer is lower than that of the fine-scale voxel layer. For example, the coarse-scale voxel layer uses voxels of 320μm, and the fine-scale voxel layer uses voxels of 80μm. The active voxel refers to a voxel in the voxel layer that contains point cloud data and meets specific conditions and is marked as "valid". It usually corresponds to the actual tissue surface in the oral cavity. Geometric features refer to mathematical quantities that describe the spatial distribution characteristics of voxels and their internal point cloud, including curvature, normal vector, ISS intrinsic shape features, Harris 3D corner points, etc. Parent-child index refers to a data structure link relationship used to represent the spatial enclosure and mapping relationship between the coarse-scale voxel (parent) and the fine-scale voxels (child) it contains. Each coarse-scale voxel indexes multiple fine-scale voxels located within its spatial range in logical or physical storage.

[0034] This step is typically performed after the 3D reconstruction device completes the voxelization mapping of the current frame's point cloud data and constructs coarse and fine voxel networks. At the coarse-scale voxel layer, the 3D reconstruction device traverses all effective coarse voxels within the main field of view of the current frame. For each coarse voxel, the device calculates the average normal vector and aggregate curvature based on its internally integrated point cloud data, and uses these as inputs to run the curvature analysis algorithm, ISS eigenvalue analysis algorithm, and Harris3D corner detection algorithm in parallel, obtaining three independent feature response scalars. Simultaneously, the 3D reconstruction device performs the same operation logic at the fine-scale voxel layer, calculating the feature response values ​​of the above three modalities for each fine voxel using its denser internal point cloud distribution. Subsequently, the device executes adaptive fusion logic, weighting and summing the three features at both the coarse and fine levels according to a preset initial weight ratio (e.g., curvature:ISS:Harris3D=40%:35%:25%) and the real-time calculated response amplitude, generating a coarse-grained fusion feature descriptor and a fine-grained fusion feature descriptor corresponding to each voxel. Finally, the 3D reconstruction device performs cross-level verification. For voxel points marked as candidate features in fine layers, the device directly indexes the coarse voxel to which it belongs through the parent node pointer of the hash table, calculates the spatial Euclidean distance between the center of the fine voxel and the feature center of the coarse voxel, and calculates the mathematical correlation coefficient between the fine-grained descriptor and the coarse-grained descriptor vector. Only when the spatial distance is less than the preset voxel tolerance and the descriptor correlation is higher than the preset threshold, the point is confirmed as a stable feature point in the final output.

[0035] Specifically, the feature calculation process is initiated simultaneously for each active voxel in both the coarse-scale and fine-scale voxel layers. For the coarse-scale layer, the 3D reconstruction device reads the covariance matrix of the point cloud inside each active coarse voxel (this matrix has been pre-calculated and stored during voxel activation), and extracts three eigenvalues ​​by performing eigenvalue decomposition on the covariance matrix. And their corresponding eigenvectors. Based on these eigenvalues, the characteristic responses of three modes are calculated in parallel: the curvature extreme response value is obtained through the formula... The calculation shows that this value reflects the degree of curvature of the local surface, with a larger response in high-curvature areas such as cusps and fissures; the ISS intrinsic shape characteristic response value is obtained through the eigenvalue ratio relationship. The calculation shows that this value is sensitive to anisotropic geometry in three-dimensional space; the Harris3D corner response value is obtained through... The value is calculated (where Σ is the covariance matrix and k is an empirical constant), and it peaks at 3D corners and edge transitions. After obtaining these three raw response values, an adaptive weight fusion strategy is used for integration: first, it is detected whether a specific feature response value exceeds its corresponding preset significance threshold (e.g., curvature threshold 0.05, ISS threshold). Harris3D threshold If an out-of-threshold condition is detected, the weight coefficient corresponding to that modality in the preset weight fusion strategy is multiplied by a preset enhancement factor, thereby dynamically increasing the proportion of that modality in the fusion. This yields a target weight fusion strategy optimized for the current voxel. Based on this target weight fusion strategy, the three response values ​​are weighted and summed to generate the final confidence score and coarse-scale fusion feature descriptor for the activated coarse voxel. For the fine-scale layer, the same feature extraction and fusion process is performed. However, due to the higher point cloud density and finer spatial resolution within the fine voxel, the calculated fine-scale fusion feature descriptor can capture more subtle surface geometric changes, such as the micro-texture of enamel or the fine contours of restoration edges.

[0036] After completing feature extraction at two levels, for each potential candidate voxel exhibiting a high confidence score in the fine-scale voxel layer, the 3D reconstruction device locates its corresponding parent coarse-scale voxel in the coarse-scale voxel layer by reading the parent voxel reference pointer stored in the fine-scale voxel layer. After localization, positional consistency verification is performed: It checks whether there is a coarse-scale feature response exceeding a preset saliency threshold at the spatial projection position of the potential candidate voxel on its parent coarse-scale voxel. Simultaneously, it calculates the Euclidean distance deviation between the spatial center coordinates of the potential candidate voxel and the center coordinates of the parent coarse-scale voxel, and determines whether this deviation is not higher than a preset positional deviation threshold (e.g., ±0.5 times the fine voxel size, approximately ±40 μm). Only when the parent voxel also exhibits significant features at the corresponding position and the spatial positional deviation is within a tolerable range can the potential candidate voxel pass the positional consistency verification. Next, response consistency verification is performed: the feature response correlation coefficient between the fine-scale fused feature descriptor of the potential candidate voxel and the coarse-scale fused feature descriptor of its parent coarse-scale voxel is calculated. This coefficient can be calculated using methods such as vector dot product normalization or Pearson correlation coefficient, and is used to quantify the similarity between the feature descriptors at two different scales. Only when this correlation coefficient is greater than a preset correlation threshold (e.g., >0.7) can the voxel pass the positional consistency verification. Only then is it considered that the feature has a consistent geometric expression on both coarse and fine scales.

[0037] Based on the results of comprehensive positional consistency verification and response consistency verification, a potential candidate feature voxel is only determined to meet the preset consistency threshold if it simultaneously satisfies all of the following conditions: the voxel has a high confidence score at the fine-scale layer; its parent coarse-scale voxel has a significant feature response at the corresponding spatial location; the spatial positional deviation between the two is within the tolerance range; and the correlation coefficient of their feature descriptors exceeds the threshold. This rigorous dual verification mechanism effectively filters out three types of unstable features: high-frequency noise points visible only at the fine scale (such as scanning artifacts and illumination reflection spots), which may exhibit high local responses at the fine scale but will be smoothed out at the coarse scale and fail the significance test of the parent voxel; low-frequency pseudostructures visible only at the coarse scale (such as random undulations in large smooth areas), which may be misclassified as features at the coarse scale but lack stable geometric support at the fine scale; and drifting features with inconsistent spatial positions, which may cause the feature response centers of the coarse and fine layers to shift due to uneven data sampling or boundary effects, and can be eliminated through positional deviation testing. The final output set of candidate feature voxels has extremely high scale invariance and noise resistance, providing a reliable data foundation for subsequent semantic analysis and registration calculations.

[0038] It is understandable that other methods can be used to achieve cross-level verification, such as introducing intermediate scale layers (e.g., 160μm). A three-level progressive verification mechanism can be constructed to further improve feature stability through step-by-step verification; or a feature consistency prediction model based on machine learning can be adopted, using historical scan data to train a classifier to determine the consistency probability of coarse and fine features, which is not limited here.

[0039] S102. Perform geometric attribute analysis on the local region where the candidate feature voxels are located to determine the semantic category of oral tissue corresponding to the candidate feature voxels.

[0040] Among them, geometric attribute analysis refers to performing statistical calculations on the point cloud data in the aforementioned local area to extract parameters that reflect the essence of the surface morphology, mainly including average curvature (reflecting the degree of curvature of the surface). Normal vector consistency (reflects the dispersion of surface normals); mean curvature refers to the arithmetic mean of the principal curvatures of all point clouds in a local area, used to quantify the overall bending characteristics of the surface. High curvature usually corresponds to sharp edges or pits and fissures of teeth, while low curvature corresponds to smooth crown surfaces or gingiva; normal vector consistency refers to the concentration of the distribution of point cloud normal vector directions in a local area, usually quantified by calculating the variance, standard deviation, or eigenvalue ratio of the covariance matrix of the normal vectors. High consistency indicates a regular and smooth surface (such as tooth enamel), while low consistency indicates a rough or easily deformable surface (such as soft tissue of the gingiva); oral tissue semantic category refers to the classification of scanned objects into different types of biological tissues or artificial restorations based on geometric morphology, such as "crown surface" (usually relatively smooth, hard, and with high normal vector consistency). "Gingival soft tissue" (usually smooth but easily deformable, with low normal vector consistency) "Cerebroidered fissures" (large curvature, dramatic changes in normal vector) "Resuscitation edge" (distinct curvature features and clear geometric transition) wait.

[0041] The 3D reconstruction device first traverses each validated fine-scale candidate feature voxel. For the currently processed voxel, the device reads its stored geometric attributes (such as normal vectors and curvature) and uses the architecture's neighborhood indexing function to quickly obtain all neighboring fine voxels within a preset radius (e.g., 0.5 mm). Subsequently, the device calculates the variance of normal vectors within this local neighborhood to assess surface smoothness; it also calculates a statistical histogram of curvature within the neighborhood to determine whether it is flat, convex, or concave; the device may further calculate surface roughness indices. Finally, the 3D reconstruction device inputs these calculated local geometric descriptors into a predefined decision tree or logistic classifier. For example, if a local region is detected with highly consistent normal vectors and a surface curvature close to zero or a constant positive value, the device classifies that region as a rigid "crown surface"; if a local surface is detected with roughness, disordered normal vectors, and gentle curvature changes, the device classifies it as easily deformable "gingival soft tissue"; if extremely high curvature abrupt changes or specific high reflectivity features (if combined with intensity information) are detected, the device labels it as a "restoration / crown margin". Finally, the device assigns a corresponding semantic label to each feature voxel and may adjust its confidence weight accordingly (e.g., enhancing the crown, suppressing the gingiva), outputting a feature point set with semantic information for registration.

[0042] In some embodiments, semantic category determination can be achieved in multiple ways: Optionally, the 3D reconstruction device can achieve this through rule-based judgment based on a threshold of the angle between normal vectors. First, the dot product of the candidate voxel normal vector and the normal vectors of all neighboring voxels is calculated. The proportion of neighboring points whose dot product is greater than a preset threshold (e.g., 0.95, indicating a very small angle) is counted. If the proportion is extremely high (90%), it is determined to be a smooth tooth surface or restoration; if the proportion is low, it is determined to be rough gingiva. Then, curvature symbols are used to distinguish between tooth surfaces (mostly convex) and tooth necks (mostly concave). Optionally, the 3D reconstruction device can achieve this through lightweight machine learning inference. First, the calculated local geometric attributes (mean curvature, curvature variance, ratio of eigenvalues ​​of the normal vector covariance matrix, etc.) are combined into a feature vector, which is then input into a pre-trained Random Forest or Support Vector Machine (SVM) classifier. The classifier directly outputs the probability distribution of the voxel belonging to "tooth", "gingiva" or "noise", and the class with the highest probability is taken as the final semantic label. Understandably, other methods can also be used, such as combining texture and color information to assist classification, which is not limited here. In addition, after classification, the device usually performs a spatial smoothing filter operation (such as a voting mechanism). If a point identified as "gum" is surrounded by a large number of "teeth" points, the device will correct its label to "teeth" to eliminate isolated classification noise.

[0043] In other embodiments, color information may also be incorporated (if the scanner supports color scanning). Auxiliary classification, such as by analyzing the RGB mean and variance of local regions, can help determine the final semantic category. Gum typically appears pink (high R channel value), while teeth appear white or pale yellow (close and high RGB channel values). This can be achieved through weighted voting or Bayesian fusion of geometric and color attributes, thus addressing areas where geometric features are not obvious (such as the junction between flat tooth surfaces and flat gingiva). Improving classification accuracy is not a specific objective here.

[0044] S103. Based on the semantic category of oral tissue, after filtering out feature voxels belonging to the preset interference semantic category from the candidate feature voxels, the confidence is weighted based on the enhancement coefficient of the corresponding oral tissue semantic category among the candidate feature voxels that do not belong to the preset interference semantic category, so as to obtain the preferred feature voxel set.

[0045] Among them, the preset interference semantic category represents unstable tissue types in the oral clinical scenario that have a negative impact on or contribute very little to the rigid registration process. It refers to the set of labels pre-stored in the 3D reconstruction equipment that have been marked as "high deformation risk" or "low geometric stability", such as "gingival soft tissue" which is easily deformed by instrument compression or moves with breathing, and "tongue / buccal mucosa region" which may contain saliva reflective noise. Candidate feature voxels that do not belong to the preset interference semantic category represent feature points that are judged to have high rigidity and geometric stability, usually including "crown surface", "restoration surface" and key "anatomical margin (such as cervical margin)".

[0046] The 3D reconstruction device traverses all classified candidate feature voxels, checking whether their semantic labels belong to a pre-defined blacklist (i.e., interference categories). Once the device detects a feature voxel labeled "gingival soft tissue" or "buccal mucosa," it directly removes the voxel's index from the processing list or invalidates its valid flag to prevent it from introducing registration errors. For the remaining feature voxels labeled "crown surface," "restoration," or "margin," the device consults its internally stored weighted mapping table. For example, for voxels labeled "crown surface," the device reads its original confidence score and multiplies it by a base enhancement coefficient (e.g., 1.3) to obtain a new score; for voxels marked as highly significant areas such as "restoration margin" or "pit and fissure," the device may apply a higher coefficient (e.g., 1.5). Through this differentiated weighting, the 3D reconstruction device artificially amplifies the influence of clinical core region features in the registration energy function. Finally, the device packages these filtered and weighted voxels and their corresponding updated descriptors and confidence scores into a final "preferred feature voxel set," and sorts them according to their confidence scores so that the registration algorithm can prioritize the use of high-scoring features.

[0047] In some embodiments, weight adjustment and set generation can be achieved in multiple ways:

[0048] Optionally, the 3D reconstruction device can be implemented by combining a look-up table (LUT) with scalar multiplication. First, a hash table (Map) is built in memory with semantic ID as the key and floating-point coefficient as the value, for example {ID_Gingiva:0.0,ID_Tooth:1.2,ID_Restoration:1.5}. When traversing feature points, the table is looked up directly according to the semantic ID and the confidence is updated by one floating-point multiplication. If the coefficient is 0, the point is discarded directly.

[0049] Optionally, the 3D reconstruction device can be implemented using a smooth weighting method based on the Gaussian distribution function. Considering that there may be boundary errors in semantic classification, it does not directly perform hard 0 or 1 cut-off. Instead, it assigns a gradual suppression coefficient (such as 0.3) between 0 and 1 to the feature points of the transition area between the "gingiva" edge and the "teeth". It assigns a full enhancement coefficient to the completely defined tooth center area, thereby retaining some environmental constraint information but significantly reducing its weight.

[0050] It is understandable that other methods can also be used, such as dynamically adjusting the coefficients based on the reliability of the scanning perspective; this is not limited here. Furthermore, when generating the preferred set, the device can also implement a spatial homogenization sampling strategy. If high-resolution features are too densely packed in a certain tooth surface region after weighting, the device will suppress some redundant points according to spatial distance to ensure the rationality of the spatial distribution of the preferred set.

[0051] S104. Based on the preferred feature voxels in the coarse-scale voxel layer of the preferred feature voxel set, perform the first-stage rigid body transformation solution in the coarse-scale voxel layer to obtain the solution result.

[0052] The first stage of rigid body transformation solution represents the initial pose estimation process with coarse-grained features as input and large search radius and fast convergence as objectives. It usually refers to the Iterative Closest Point (ICP) algorithm or feature matching algorithm based on coarse voxels. The solution result is the rigid body transformation matrix obtained by iterative optimization calculation, which includes the rotation matrix R and the translation vector t, and is used to describe the spatial pose relationship between the current frame point cloud and the target point cloud.

[0053] The 3D reconstruction device first extracts all feature points belonging to the coarse-scale voxel layer from the preferred feature voxel set as the source point cloud, and extracts the corresponding coarse-scale surfaces from the global TSDF map or the previous frame data as the target point cloud. Next, the device initializes the transformation matrix (usually based on motion trend prediction from the previous frame or set as the identity matrix). It then enters an iterative loop for coarse registration: in each iteration, the device uses the spatial hash index of the coarse voxel architecture to find the nearest neighbor corresponding point of each source feature point in the target point cloud with a relatively wide search radius (e.g., 6 times the side length of the coarse voxel, approximately 2 mm). When searching for corresponding points, the device considers not only Euclidean distance but also the similarity of feature descriptors (calculated in S101) to eliminate mismatches. After finding corresponding point pairs, the device constructs an objective function (usually the sum of squared distance errors between point-to-point or point-to-surface points). At this point, the "semantic confidence weight" calculated in S103 is directly substituted into the objective function, making the contribution of high-weighted tooth / edge points to pose calculation much greater than that of low-weighted soft tissue points. Finally, the device minimizes the objective function using singular value decomposition (SVD) or nonlinear least squares (Levenberg-Marquardt) to calculate the current pose increment and update the transformation matrix until the number of iterations reaches the upper limit (e.g., 50 times) or the error change is less than the convergence threshold.

[0054] In some embodiments, the first stage of the solution can be implemented in several ways: Optionally, the 3D reconstruction device can be implemented using a point-to-plane weighted ICP algorithm, first calculating the average normal vector of the target coarse voxels, then projecting the source feature points onto the tangent plane defined by the target normal vector, and constructing the error equation. ,in For semantic weights, the rigid body transformation is solved by minimizing the perpendicular distance from the point to the plane. This method converges faster when dealing with flat or smooth surfaces. Optionally, the 3D reconstruction device can be implemented using a RANSAC (Random Sample Consensus)-like algorithm based on feature descriptor histogram matching. When the displacement is too large and the nearest neighbor search fails, the device directly uses the coarse-grained feature descriptors generated by S101 to perform global feature matching, selects three pairs of feature points with high matching degree to calculate candidate transformation matrices, and selects the best transformation by counting the number of interior points, which serves as the initial value of ICP. Understandably, a tightly coupled optimization method combining inertial measurement unit (IMU) pre-integrated data as a constraint term can also be used, which is not limited here. In addition, at this stage, the device usually sets a relatively lenient normal vector consistency threshold (e.g., 45 degrees) to allow potential correspondences to be found even with poor initial alignment.

[0055] S105. Using the solution result as the initial value, perform the second-stage rigid body transformation within the fine-scale voxel layer by selecting the preferred feature voxels to obtain the final point cloud pose.

[0056] After the initial coarse registration is completed, the 3D reconstruction equipment uses the coarse registration result as the initial pose for the current frame's point cloud. The equipment then extracts the data from the preferred feature voxel set belonging to the "fine-scale voxel layer" (e.g., voxels with 80 μm resolution). This data contains fine geometric information such as the microscopic texture of the tooth surface, pit and fissure margins, and restoration seams. The equipment then tightens the search radius to the micrometer level (e.g., 0.24 mm). At this stage, the initial pose is already very close to the true solution (the error is typically less than 0.5 mm). The algorithm can safely perform high-frequency iterative optimization within a very small range, gradually eliminating residual rotation and translation errors. The device allows for more iterations (e.g., 100 times) to ensure that the optimization process fully converges to the global optimum, and the final output pose accuracy can reach within 10 micrometers, fully meeting the clinical precision requirements for dental prosthesis fabrication.

[0057] In some embodiments, the second-stage transformation can be implemented in several ways: optionally, the 3D reconstruction device can be implemented using a point-to-surface ICP algorithm with a robust kernel function, in constructing the error function. When this is the case, introduce the Tukey or Huber kernel function. When the distance between a pair of fine feature points is within the search radius but the directional deviation is large (possibly due to local noise), the kernel function will automatically truncate or reduce the influence of this error term to prevent individual noise points from skewing the high-precision result. Optionally, the 3D reconstruction device can be implemented using a 4D-ICP algorithm that combines photometric / texture information. In flat tooth surface areas where pure geometric constraints are insufficient, the device calculates the grayscale or color gradient of the feature voxels and adds the color difference term to the geometric distance error term. This method utilizes stains, cracks, or color textures on the tooth surface to lock the "sliding" degree of freedom, achieving sub-voxel-level alignment. Understandably, it can also be achieved using a direct photometric method based on the Gauss-Newton method; this is not limited here. Furthermore, before outputting the final result, the device typically performs a jitter detection. If the fine registration result undergoes a physically impossible drastic change compared to the coarse registration result, the device will discard the fine registration result and trigger a rollback.

[0058] In this embodiment, the 3D reconstruction device employs a core technology based on feature collaboration and semantic perception using a multi-scale hierarchical voxel architecture. During processing, the device utilizes a cross-level consistency verification mechanism, leveraging the macroscopic structural stability of coarse voxels to verify the microscopic features of fine voxels, thereby eliminating a large number of false features caused by scanning noise at the source. Furthermore, it introduces oral semantic perception logic to intelligently classify and weight the verified feature points, actively filtering out easily deformable soft tissue (gingiva) interference and directionally enhancing the weight of rigid structures (teeth / restorations). This effectively solves the technical problems of poor robustness due to a single feature scale and registration drift or failure caused by misleading non-rigid soft tissues due to a lack of semantic understanding in related technologies. This results in improved feature point extraction purity and stability in complex and variable oral scanning environments (such as saliva interference and large-area soft tissue peristalsis).

[0059] In the above embodiments, the 3D reconstruction device can achieve high-precision real-time registration in complex oral environments by fusing multimodal geometric response, cross-level consistency verification, semantic weighted filtering, and hierarchical registration. In practical applications, when executing the above methods, extreme situations may arise (such as severe scanner vibration, sudden swallowing by the patient causing large-area obstruction, or extremely limited features within the scanned field of view, such as edentulous regions), where even the optimized feature point set may not provide sufficient geometric constraints. This technical problem can be addressed by fusing quality integrity closed-loop verification with an IMU / kinematic multi-level backoff mechanism, thereby improving the technical performance.

[0060] Please see Figure 2 This is another flowchart illustrating the registration method based on a multi-scale hierarchical voxel fusion architecture in the embodiments of this application.

[0061] S201. After determining the coarse-scale geometric features of each activated voxel in the coarse-scale voxel layer and the fine-scale geometric features of each activated voxel in the fine-scale voxel layer, based on the spatial mapping relationship between the coarse-scale voxel layer and the fine-scale voxel layer, cross-level consistency verification is performed on the coarse-scale geometric features and the fine-scale geometric features to determine the candidate feature voxels that meet the preset consistency threshold.

[0062] S202. Perform geometric attribute analysis on the local region where the candidate feature voxels are located to determine the semantic category of oral tissue corresponding to the candidate feature voxels.

[0063] S203. Based on the semantic category of oral tissue, after filtering out feature voxels belonging to the preset interference semantic category from the candidate feature voxels, the confidence is weighted based on the enhancement coefficient of the corresponding oral tissue semantic category among the candidate feature voxels that do not belong to the preset interference semantic category, so as to obtain the preferred feature voxel set.

[0064] S204. Based on the preferred feature voxels in the coarse-scale voxel layer of the preferred feature voxel set, perform the first-stage rigid body transformation solution in the coarse-scale voxel layer to obtain the solution result.

[0065] S205. Using the solution result as the initial value, perform the second-stage rigid body transformation within the fine-scale voxel layer by selecting the preferred feature voxels, and obtain the final point cloud pose.

[0066] Steps S201~S205 and Figure 1 In the illustrated embodiment, steps S101 to S105 are similar, and can be found in the descriptions of steps S101 to S105, which will not be repeated here.

[0067] S206. If the quality integrity check of the point cloud pose is passed, output the point cloud pose and update the global 3D model based on the point cloud pose.

[0068] After the S205 completes the fine ICP iteration and outputs the convergence result, the 3D reconstruction device first extracts the pose transformation matrix output by the S205. And read the pose of the previous frame. Calculate the relative pose increment between the two. The equipment performs a kinematic rationality check on this increment: determining the translation amount. Whether it is less than a preset threshold (e.g., 20mm), and the rotation angle. The system checks whether the angle is less than a preset threshold (e.g., 15°) to exclude unnatural, instantaneous jumps. Simultaneously, the system checks the sufficiency of feature constraints, counting the number of valid feature point pairs involved in the calculation during the last iteration of S205, requiring it to be greater than a minimum lower limit (e.g., 100 pairs) to prevent spurious convergence due to feature sparsity. Finally, the system calculates the root mean square error of the registration, requiring it to be less than a precision threshold (e.g., 50 μm). The system determines "verification passed" and immediately proceeds only if all the above conditions are met simultaneously. Marked as valid, it drives the TSDF fusion engine, which uses the pose to "sculpt" the point cloud geometric details of the current frame into the global model through ray casting or voxel update algorithms, thereby achieving real-time growth of the model.

[0069] In some embodiments, quality integrity checks can be implemented in multiple ways: Optionally, the 3D reconstruction device can implement a check based on feature distribution entropy, which calculates not only the number of matching points but also the uniformity of the matching points' distribution in space (entropy value). If all matching points are concentrated on one side of the tooth while the other side is missing, even if the error is small, it is judged as "under-constrained" and thus rejected to prevent the model from bending and deforming. Optionally, the 3D reconstruction device can implement a check based on closed-loop residual detection, which quickly selects key frames before the current frame (such as the 10th frame before) in the background for a rapid verification registration. If the direct registration pose of the current frame relative to the 10th frame before exceeds a threshold compared with the pose calculated through cumulative estimation, the current frame is judged to have a cumulative drift risk, and the check fails.

[0070] Understandably, this can also be achieved by combining a deep learning confidence scoring network; this is not a limitation here. Furthermore, when updating the global model, the device typically dynamically adjusts the fusion weights based on the RMSE (Real-Time Error). The smaller the RMSE (higher quality), the greater the weights during fusion, and vice versa.

[0071] S207. If the quality integrity check fails, then if valid data input is detected in the inertial measurement unit bound to the scanning device, the relative pose change is determined by time integration based on the acceleration and angular velocity data corresponding to the current frame timestamp, and the relative pose change is superimposed on the point cloud pose of the previous frame to generate the inertial prediction pose.

[0072] After confirming geometric registration failure, the 3D reconstruction device captures the timestamp of the current frame image. and the timestamp of the previous successful frame Next, the device retrieves data from the circular buffer that falls within the time interval. All raw IMU data packets within (typically containing dozens of accelerometer data packets) and angular velocity (Data). The device processes this data using a pre-integration algorithm: first, it eliminates the bias of the gyroscope and accelerometer, then it integrates the angular velocity using the Rodriguez formula to calculate the rotational increment during this tiny time interval. Acceleration is processed using double integration (first integrating to obtain velocity, then integrating to obtain displacement), and gravitational acceleration is deducted when transforming to the world coordinate system. The effect of translation increment is calculated. Finally, the 3D reconstruction device reads the reliable pose from the previous frame. The calculated relative change is then superimposed on it. This generates an inertial prediction pose containing 6 degrees of freedom information.

[0073] S208. If no valid data input is detected, reuse the point cloud pose of the previous frame, perform smooth extrapolation based on the historical motion trajectory, and generate the motion prediction pose.

[0074] The 3D reconstruction device executes this step after determining geometric registration failure in S206 and determining that valid IMU data cannot be acquired in S207. The device queries the hardware status register or data buffer flags to confirm that the IMU is unavailable. Subsequently, the device reads the final pose matrix of the previous frame, which was verified in S206, stored in memory. In the simplest "zero-speed model," the device directly sets the pose of the current frame. This assumes the scanner remains stationary. In the slightly more complex "constant velocity model," the device reads the pose from the previous two frames. Calculate the linear velocity at the previous moment. and angular velocity Assuming that the motion continues at the current speed, the result is calculated through linear extrapolation. The device will calculate this. As a motion prediction pose, its confidence level is set to the global minimum (e.g., 0.1).

[0075] In some embodiments, motion prediction can be achieved in several ways: Optionally, the 3D reconstruction device can achieve this through trajectory extrapolation based on a smoothing filter. The device maintains a historical queue containing the poses of the most recent N frames (e.g., 5 frames), fits a locally smooth curve (e.g., a spline curve) using the least squares method, and then extrapolates the position of the next frame along the tangent direction of this curve. This method simulates the smooth inertia of the operator's hand movement better than simple linear extrapolation. Optionally, the 3D reconstruction device can achieve this through a region of interest (ROI) locking mechanism. When data is interrupted, the device does not calculate global coordinates but only locks the pixel region at the center of the current field of view, assuming that the 3D spatial position corresponding to this region remains unchanged, thereby generating a relative virtual pose until the features reappear. It is understood that a purely visual-assisted prediction method combined with image optical flow can also be used, which is not limited here.

[0076] S209. Determine the inertial predicted pose or motion predicted pose as the changed point cloud pose, and use the changed point cloud pose to update the global 3D model with the point cloud data of the current frame.

[0077] The 3D reconstruction device performs this step after generating the backup pose (S207 or S208). Specifically, the 3D reconstruction device first makes a selection based on priority logic: if an inertial prediction pose generated in S207 exists, it is marked as the "replaced point cloud pose"; if not, the motion prediction pose from S208 is used. Next, the device reads the pose matrix and executes dual-channel update logic: in the display channel, the device immediately uses the pose to render the currently scanned point cloud onto the screen, ensuring that the scan image seen by the doctor closely follows the probe movement without any stuttering or delay; in the model fusion channel, the device checks the confidence level of the pose. Since this is a predicted pose, the device automatically adjusts the TSDF fusion weight of the current frame to an extremely low level (e.g., 0.01) or even zero. This means that although the data of the current frame is placed in the global map in terms of spatial location, it hardly modifies the distance values ​​(TSDF values) already stored in the voxels, or simply acts as a transparent "phantom" covering the model surface. This ensures that the operator can see the scan coverage area while avoiding "double eyelids" or layering artifacts on the model surface caused by prediction bias.

[0078] In this embodiment, when the device detects visual tracking failure (such as registration divergence caused by rapid movement, oral saliva reflection, or non-featured areas), the device can switch the erroneous pose output to IMU inertial pre-integration or historical motion trend extrapolation mode, and use high-frequency physical sensor data or kinematic constraints to generate an alternative pose to maintain the operation of the scanning path. This effectively solves the technical problem that related technologies are prone to scanning path interruption when faced with rapid handheld scanner shaking, unconscious patient swallowing, or scanning to non-featured smooth areas (such as the buccal mucosa), thereby achieving a smooth and continuous scanning experience, and improving robustness to unstable clinical operating environments while ensuring model geometric accuracy.

[0079] The following describes an exemplary three-dimensional reconstruction device 300 provided in an embodiment of this application. Figure 3 This is an exemplary hardware structure diagram of the three-dimensional reconstruction device 300 provided in this application embodiment.

[0080] In some embodiments, the 3D reconstruction device 300 is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0081] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0082] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0083] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0084] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0085] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A registration method based on a multi-scale hierarchical voxel fusion architecture, characterized in that, Applied to 3D reconstruction equipment, the multi-scale hierarchical voxel fusion architecture includes coarse-scale voxel layers and fine-scale voxel layers, and the method includes: After determining the coarse-scale geometric features of each active voxel in the coarse-scale voxel layer and the fine-scale geometric features of each active voxel in the fine-scale voxel layer, based on the spatial mapping relationship between the coarse-scale voxel layer and the fine-scale voxel layer, cross-level consistency verification is performed on the coarse-scale geometric features and the fine-scale geometric features to determine candidate feature voxels that meet the preset consistency threshold. Geometric attribute analysis is performed on the local region where the candidate feature voxels are located to determine the oral tissue semantic category corresponding to the candidate feature voxels; Based on the oral tissue semantic category, after filtering out feature voxels belonging to the preset interference semantic category from the candidate feature voxels, the confidence is weighted based on the enhancement coefficient of the corresponding oral tissue semantic category among the candidate feature voxels that do not belong to the preset interference semantic category, to obtain the preferred feature voxel set. Based on the preferred feature voxels in the coarse-scale voxel layer of the preferred feature voxel set, a first-stage rigid body transformation solution is performed in the coarse-scale voxel layer to obtain the solution result. The search radius configured for the first-stage rigid body transformation is a preset first multiple of the coarse-scale voxel resolution, and a first maximum number of iterations is set. Using the solution result as the initial value, a second-stage rigid body transformation is performed within the fine-scale voxel layer using the preferred feature voxels to obtain the final point cloud pose. The search radius configured for the second-stage rigid body transformation is a preset second multiple of the fine-scale voxel resolution, and a second maximum number of iterations is set.

2. The method according to claim 1, characterized in that, The step of determining the coarse-scale geometric features of each activated voxel in the coarse-scale voxel layer specifically includes: Based on the covariance matrix of the point cloud distribution inside each active voxel in the coarse-scale voxel layer, the curvature extreme value response value, the inherent shape feature response value, and the Harris3D response value are determined. When it is determined that the response value of a specific feature exceeds a preset significance threshold, the corresponding weight coefficient in the preset weight fusion strategy is multiplied by the preset enhancement factor to obtain the target weight fusion strategy. The specific feature response value is one or more of the curvature extreme value response value, the inherent shape feature response value and the Harris3D response value. Based on the target weight fusion strategy, the curvature extreme value response value, the intrinsic shape feature response value, and the Harris3D response value are weighted and summed to generate a confidence score for the corresponding activated voxel.

3. The method according to claim 1, characterized in that, The step of performing cross-level consistency verification on the coarse-scale geometric features and fine-scale geometric features corresponding to spatial locations based on the spatial mapping relationship between the coarse-scale voxel layer and the fine-scale voxel layer, and determining candidate feature voxels that meet the preset consistency threshold, specifically includes: Based on the parent voxel reference pointer of the potential candidate feature voxels in the fine-scale voxel layer, the corresponding parent coarse-scale voxel is determined in the coarse-scale voxel layer. If a potential candidate feature voxel has a coarse-scale feature response exceeding a preset significance threshold at the spatial projection position of the corresponding parent coarse-scale voxel, and the spatial position deviation between the potential candidate feature voxel and the parent coarse-scale voxel is not higher than a preset position deviation threshold, then the potential candidate feature voxel satisfies the position consistency verification. Calculate the feature response correlation coefficient between the fine-scale fused feature descriptor of the potential candidate feature voxel and the coarse-scale fused feature descriptor of the parent coarse-scale voxel; If a potential candidate feature voxel satisfies the positional consistency verification and the feature response correlation coefficient is greater than a preset correlation threshold, then the potential candidate feature voxel is determined to be a candidate feature voxel that satisfies the preset consistency threshold.

4. The method according to claim 3, characterized in that, The step of performing geometric attribute analysis on the local region where the candidate feature voxel is located to determine the semantic category of the oral tissue corresponding to the candidate feature voxel specifically includes: Calculate the average curvature and normal vector consistency of all point clouds within the fine-scale voxel where each candidate feature voxel is located; Based on the average curvature, the consistency of the normal vector, and the preset soft tissue classification rules, the semantic category of the oral tissue corresponding to the candidate feature voxel is determined.

5. The method according to claim 4, characterized in that, The step of filtering out feature voxels belonging to a preset interference semantic category from candidate feature voxels according to the semantic category of oral tissue, and then weighting the confidence scores of candidate feature voxels that do not belong to the preset interference semantic category based on the enhancement coefficient of the corresponding oral tissue semantic category to obtain an optimal feature voxel set, specifically includes: Based on prior knowledge of oral anatomy, it is determined whether the spatial location of the candidate feature voxels conforms to the anatomical distribution map of the corresponding oral tissue semantic category, and candidate feature voxels that violate the anatomical distribution map are identified as preset interference semantic categories. After filtering out feature voxels belonging to the preset interference semantic category from the candidate feature voxels, the confidence is weighted based on the enhancement coefficient of the corresponding oral tissue semantic category to obtain the preferred feature voxel set.

6. The method according to claim 1, characterized in that, After obtaining the final point cloud pose, the method further includes: If the quality integrity check of the point cloud pose is passed, the point cloud pose is output and the global 3D model is updated based on the point cloud pose. If the quality integrity check is deemed to have failed, the point cloud pose is replaced using inertial measurement data and / or historical pose data, and the global 3D model is updated based on the replaced point cloud pose.

7. The method according to claim 6, characterized in that, The step of changing the point cloud pose using inertial measurement data and / or historical pose data and updating the global 3D model based on the changed point cloud pose specifically includes: When valid data input is detected in the inertial measurement unit bound to the scanning device, the relative pose change is determined by time integration based on the acceleration and angular velocity data corresponding to the current frame timestamp, and the relative pose change is superimposed on the point cloud pose of the previous frame to generate the inertial predicted pose. If no valid data input is detected, the point cloud pose of the previous frame is reused, and smooth extrapolation is performed based on the historical motion trajectory to generate a motion prediction pose. The inertial predicted pose or the motion predicted pose is determined as the changed point cloud pose, and the point cloud data of the current frame is updated using the changed point cloud pose to update the global 3D model.

8. A three-dimensional reconstruction device, characterized in that, The 3D reconstruction device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the 3D reconstruction device to perform the method as described in any one of claims 1-7.

9. A computer program product containing instructions, characterized in that, When the computer program product is run on a 3D reconstruction device, the 3D reconstruction device performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on a 3D reconstruction device, the 3D reconstruction device performs the method as described in any one of claims 1-7.

Images