Three-dimensional point cloud tooth segmentation method based on geometric guidance and instance perception

Abstract

The invention relates to the technical field of image processing, in particular to a three-dimensional point cloud tooth segmentation method based on geometric guidance and instance perception, and the method comprises the steps: obtaining three-dimensional point cloud data of tooth feature information; processing the point cloud and the candidate centroid position by using a BAS module to obtain a tooth boundary sampling point cloud; processing the point cloud by using a CM module, and outputting a predicted candidate centroid position; inputting the tooth boundary sampling point cloud into a plurality of residual error connected MHAG modules, and outputting local graph features; performing global maximum pooling and average pooling operation on the local graph features in sequence to obtain global graph features; carrying out weighted fusion on the global graph features and the local graph features; inputting the fused features into an MLP; and a centroid constraint mechanism based on point-level distance prediction is introduced to correct label distribution deviation. The problem that the calculation cost or the training difficulty is increased only through data enhancement or complex model architecture in an existing method is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a three-dimensional point cloud tooth segmentation method based on geometric guidance and instance perception. BACKGROUND

[0002] With the improvement of residents' oral health awareness, the demand for fine diagnosis and treatment such as orthodontic correction and implant repair has increased rapidly, and high-precision tooth structure data has become the core prerequisite for scheme development; the point cloud model data obtained by 3D oral scanning is large in amount, and in clinical practice, patients often have complex conditions such as tooth crowding and misplacement, which makes the operation of manually segmenting three-dimensional point cloud models face significant challenges.

[0003] Traditional segmentation methods rely on manual design of features and geometric rules, and achieve segmentation by extracting tooth coordinates, normal vectors, surface curvature and other parameters, and combining region growing algorithm, dental arch shape gap judgment and other logic; but such methods have strong dependence on domain knowledge, and are prone to failure in complex scenarios such as tooth crowding and abnormal morphology, and require manual interaction for adjustment, making it difficult to achieve complete automation; Direct processing of three-dimensional point cloud (such as PointNet, graph network), although it can retain more original geometric features, has obvious technical limitations: uniform or distance-dependent strategies are often used for downsampling, which easily ignores high-curvature points in boundary regions such as adjacent tooth gaps and gum lines, resulting in loss of key boundary features; centroid positioning relies on simple regression or manual labeling, which is difficult to adapt to complex shapes such as tooth deformity and crowding, resulting in poor clustering of similar points and blurred segmentation boundaries; the feature fusion level does not fully utilize multi-dimensional geometric features such as coordinates, curvature and normal vectors, and it is difficult to capture long-range dependencies between point cloud nodes, affecting the robustness of segmentation, in addition, most methods require time-consuming post-processing to refine the boundaries, which cannot meet the efficiency requirements of clinical practice.

[0004] In addition, existing point cloud tooth segmentation methods mostly rely on coordinates or local features, and perform poorly in cases of tooth arrangement changes, tooth crowding or boundary details; traditional uniform downsampling will lose boundary information; relying solely on data augmentation or complex model architecture will increase computational cost or training difficulty. SUMMARY

[0005] To address the shortcomings of existing methods, the present application combines boundary perception sampling and centroid guidance with adaptive kernel convolution to simultaneously enhance boundary and instance positions, thereby improving tooth segmentation accuracy and robustness.

[0006] The technical solution adopted by the present application is: a three-dimensional point cloud tooth segmentation method based on geometric guidance and instance perception includes the following steps: Step 1: Obtain three-dimensional point cloud data of tooth feature information; Step two, processing the point cloud and candidate centroid position by using the BAS module to obtain the tooth boundary sampling point cloud; As a preferred embodiment of the present application, step two specifically includes: Step 21, calculating the centroid of each tooth in the upper and lower jaw point cloud subsets respectively The coordinate mean value; Step 22, taking the upper and lower jaw tooth centroid As the center, the radius R Carry out the ball cutting circle, and combine the Z-axis threshold to filter the upper and lower jaws, and obtain the sub-point cloud of each tooth; As a preferred embodiment of the present application, if the sample belongs to the upper jaw, only the points of Are retained; otherwise, the points of Are retained, z is the specific coordinate of Z, Is the Z-axis threshold; and XY plane limiting is introduced, and the out-of-circle area is taken as the reserved area according to the tooth position.

[0007] Step 23, calculating the curvature of the point cloud by using K-nearest neighbor and covariance matrix eigenvalue; Step 24, screening Top curvature points; Step 25, boundary clustering of the screened curvature points; Step 26, using the combination strategy of inner pool weighted sampling and outer pool uniform sampling to distribute the Top curvature point pool and the non-boundary point pool; Step 27, calculating the probability Of the candidate point by using the adaptive weight function ; Step 28, randomly sampling the original point cloud data to output the down-sampling point cloud data.

[0008] As a preferred embodiment of the present application, the random sampling includes: Let the total sampling budget be Generate Independent uniform random numbers , construct the probability cumulative distribution function , and obtain the probability Of the first Sampling point.

[0009] Step three, processing the point cloud by using the CM module to output the predicted candidate centroid position; As a preferred embodiment of the present application, the CM module includes: A downsampling combination block is formed by a DownSampling Transition block and a Point Transformer block, and an upsampling combination block is formed by an Upsampling Transition block and a Point Transformer block. The three downsampling combination blocks are cascaded with the three upsampling combination blocks and the MLP in sequence. The first downsampling combination block is residually connected to the first upsampling combination block. The second downsampling combination block is residually connected to the second upsampling combination block.

[0010] As a preferred embodiment of the present invention, the total loss function is utilized. Training the CM model, λ 1 to λ 4 represents the weighting coefficient. For the loss of the center of mass, For directional loss, For Chamfer distance loss, Cross-entropy loss for category segmentation.

[0011] In a preferred embodiment of the present invention, the formula for centroid loss is:

[0012] in, Indicates the number of valid categories (centroid count). Indicates category The number of point clouds; Represents the original point cloud coordinates; This represents the corresponding predicted offset vector; Indicate category The center of mass.

[0013] In a preferred embodiment of the present invention, the formula for the direction loss is:

[0014] in, Indicates the number of valid categories; Indicates category c The effective number of point clouds; This represents the cosine similarity between the predicted offset direction and the direction from the point to the centroid.

[0015] Step 4: Input the tooth boundary sampling point cloud into several residual-connected MHAG modules and output local map features; In a preferred embodiment of the present invention, the plurality of residual-connected MHAG modules include: three cascaded MHAG modules, wherein the outputs of the first MHAG module and the second MHAG module are concat-joined with the output of the third MHAG module.

[0016] As a preferred embodiment of the application, the MHAG module comprises: a self-attention branch and a geometric feature enhancement branch; wherein the self-attention branch comprises: The node feature matrix is linearly transformed into query Q , key K and value V ; The Q , K , V is split into value sub-matrix according to the number of attention heads; The attention weight matrix is calculated, and the features of a single head are output ; The self-attention features , h =1,2,3; The geometric feature enhancement branch comprises: The curvature feature matrix and the normal vector feature matrix of the input point cloud are inputted; The curvature global feature and the normal vector global feature are calculated; The and are mapped by MLP to obtain mapped features and ; The features are outputted.

[0017] Step five, sequentially performing global maximum pooling and average pooling operations on the local graph features to obtain global graph features; and weighting and fusing the global graph features and the local graph features; Step six, inputting the fused features into MLP; and introducing a centroid constraint mechanism based on point-level distance prediction to correct label assignment bias.

[0018] The beneficial effects of the application are: 1. The BAS module is constructed, the boundary point sampling weight is enhanced by curvature calculation, and the boundary feature loss problem in the down-sampling stage is solved; 2. The centroid prediction module is constructed, the accurate centroid regression is realized based on the PointTransformer architecture, and the same kind of point aggregation and boundary clarity are optimized by fusing multiple loss function constraints; 3. The multi-head graph attention (MHAG) module is constructed, the long-distance dependence is captured by multi-head self-attention, and the segmentation representation ability is enhanced by fusing curvature, normal vector and other geometric features. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a three-dimensional point cloud tooth segmentation logic flowchart based on geometric guidance and instance perception of the present application; Figure 2 is a BAS module flowchart of the present application; Figure 3 is a CM model structure diagram of the present application; Figure 4 is a comparison chart of the present application and existing methods; Figure 5 is a schematic diagram of the MHAG module of the present application; Figure 6 is a schematic diagram of a tooth model.

[0020] Figure 7 is a BAS ablation experiment result chart; Figure 8 is a Centroid Mode ablation experiment result chart; Figure 9 is a tooth centroid prediction result chart; Figure 10 is a MHAG ablation experiment result chart. DETAILED DESCRIPTION

[0021] The present application will be further described below in conjunction with the accompanying drawings and examples, which are simplified schematic diagrams and only schematically illustrate the basic structure of the present application, and thus only show the configurations related to the present application.

[0022] As shown in Figure 1 , a three-dimensional point cloud tooth segmentation method based on geometric guidance and instance perception includes the following steps: Step one, obtain three-dimensional point cloud data of tooth feature information; First, input the original oral cavity scan three-dimensional point cloud data; Obtain several original point cloud data of oral cavity scans containing known tooth feature information, pre-process and normalize the point cloud data, then establish a mapping relationship between the processed point cloud data and the corresponding tooth feature information, and construct an oral scan data set; wherein the tooth feature information includes tooth coordinates and normal vectors.

[0023] The original point cloud data contains three-dimensional geometric information of oral tissues such as teeth and gums; before subsequent processing, the original point cloud is preliminarily pre-processed such as denoising to reduce the influence of noise on subsequent segmentation.

[0024] Step two, construct a boundary perception sampling (BAS, Boundary Aware Sampling) module to process the point cloud and obtain tooth boundary sampling point cloud; like Figure 2 and 6 As shown, the coordinate distribution characteristics of the point cloud on the Z-axis (usually set to the direction perpendicular to the occlusal plane) are used to distinguish between the maxilla and mandible. Generally speaking, the coordinate values ​​of the maxillary point cloud in the positive Z-axis direction are generally greater than those of the mandibular point cloud in the positive Z-axis direction. By setting an appropriate Z-axis coordinate threshold, the original point cloud can be divided into a maxillary point cloud subset and a mandibular point cloud subset. For the divided maxillary and mandibular point cloud subsets, the centroid of each tooth is calculated separately. The centroid is calculated using the three-dimensional centroid formula, i.e., for the point cloud subset... , center of mass The calculation formula is as follows:

[0025] in, Indicates the category of each tooth t The centroid (predicted centroid point output by the CM module); The coordinate components representing points in the point cloud; This indicates the category of the tooth at the current point. Indicates the type of teeth.

[0026] calculate The average value is calculated using the following formula:

[0027] in, n The number of points for different teeth.

[0028] Calculate the centroid of the maxilla or mandible as the center of the sphere, set an appropriate radius of the sphere (this radius needs to be determined based on oral anatomy knowledge and scanning range, and generally can cover all teeth and part of the gingival tissue of the corresponding jaw), retain the point cloud inside the sphere, filter out irrelevant point cloud outside the sphere, and initially reduce the point cloud range for subsequent processing. like Figure 6 As shown, based on the calculated tooth centroid , radius R Range of ball cutting combined with Z-axis threshold (depending on tooth position) Implement maxillary / maxillary filtering: If the sample belongs to the maxilla, only the maxilla is retained. The point, and conversely, the point. The point is z, which is the specific coordinate of Z; finally, the XY plane is introduced for amplitude limiting, and the outer zone of the circular ring is adaptively retained according to the tooth position to suppress interference between adjacent teeth and soft tissue, and finally the sub-point cloud of each tooth is obtained.

[0029] On the basis of spherical filtering, combined with the distribution difference of the upper and lower jaws on the Z-axis, the Z-axis coordinate range is further set, and the point cloud within the range is retained, so as to more accurately cut out the effective point cloud area of the target jaw.

[0030] The curvature of the point cloud is calculated by the K-neighbor combined with the covariance eigenvalue method.

[0031] For each point in the point cloud , search for K nearest neighbor points K around it (the value is usually determined according to the point cloud density and the required accuracy, generally 10-30), to form a local point set ; On each cropped sub-point cloud, a geometric sensitivity measure is established by curvature calculation, and a higher sampling weight is given to the area with a sharp change in point cloud surface; at the tooth-gum junction, due to the geometric discontinuity of the anatomical structure, the local curvature will show significant changes, which can be quantified by eigenvalue decomposition: for a fixed point on the sub-point cloud and its neighbor point set ; For the local point set , calculate its covariance matrix , the formula is:

[0032] wherein, is any one of the neighborhood points in , and is the neighborhood center point; that is, the closest points to selected by the Euclidean distance.

[0033] Solve the eigenvalue of the covariance matrix, calculate the curvature , the formula is:

[0034] The ratio can effectively distinguish between flat areas and boundary areas significantly increased.

[0035] Outlier filtering is performed, and the top high curvature points are extracted, including: After calculating the curvature values of all points, the statistical outlier removal (SOR) algorithm is used to filter outliers; the algorithm calculates the distance of each point to its KAverage distance of the neighboring points, points with average distance exceeding a certain threshold are determined as outliers and removed to reduce the impact of noise on subsequent processing; After outlier filtering, the point cloud is sorted by curvature value from large to small, and the top curvature value points are extracted as high curvature candidate points, which usually correspond to key regions such as tooth boundaries; Subsequently, a curvature threshold is introduced to preliminarily filter the oral scan point cloud data to obtain a high curvature candidate point set , The th i candidate point in the candidate point set is clustered using the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm to achieve effective denoising and boundary feature extraction; The specific clustering process includes: By setting two key parameters, neighborhood radius and minimum point threshold , high curvature candidate points with adjacent spatial positions are aggregated into connected clusters; The resulting clusters are sorted in descending order of size, and the largest cluster and all clusters with a size no less than 10% of the largest cluster are retained to improve the stability of boundary extraction and avoid small clusters and noise dominating the sampling distribution, and finally only the points belonging to the effective clusters are retained.

[0036] For boundary points obtained by boundary clustering, different weights are assigned according to their curvature values and other features, and the probability of sampling is higher for points with larger curvature values to ensure that key information in the boundary region is retained; For non-boundary points, random sampling is used to select a certain proportion of point clouds to ensure that point cloud information in the non-boundary region is also retained to some extent, maintaining the global features of the point cloud.

[0037] Considering that the tooth boundary region accounts for a very small proportion of the entire point cloud data, the index set of the global effective high curvature points is defined as , and the index set of the remaining non-boundary region points is Under the constraint of a pre-set total sampling budget M , a combined sampling strategy of "two-pool hierarchical allocation - weighted sampling in the inner pool - uniform sampling in the outer pool " is designed, and the specific implementation steps are as follows: (5) The total budget is allocated between the effective high curvature point pool (index set ) and the non-boundary point pool (index set ), where the parameter The role is to explicitly increase the sample quota of the boundary point pool, ensure that the boundary region points are significantly oversampled under the premise of constant total sampling amount; and the non-boundary point pool maintains the coverage of the global point cloud by uniform sampling, avoiding local feature loss.

[0038] Considering that the sample distribution of the boundary region and the non-boundary region in the tooth point cloud is significantly unbalanced, a large number of redundant samples in the non-boundary region are easy to dominate the sampling process, leading to the dilution of the sampling weight of the boundary key feature region, and further affecting the capture accuracy of the model on the geometric features of the tooth edge; therefore, the adaptive weight function is designed, and the formula is:

[0039] For and Definition: for any point in the point cloud , the curvature value is , the value rule of the indicator function is: when the condition in the parentheses is established, the function value is 1; when the condition is not established, the function value is 0; that is: , (7) The definition of the index set itself depends on the indicator function represents the set of all high-curvature feature points; and corresponds to the set of low-curvature non-feature points; therefore, the indicator function is the core basis for dividing the two point pools (inner pool and outer pool ), which directly determines the size of (the total number of inner pool points) and (the total number of outer pool points).

[0040] The threshold is used to distinguish the feature region, ensure the basic sampling coverage, strengthen the feature region reservation; the probability distribution is obtained by normalization, and the formula is:

[0041] Make the sampling process meet ; this probability sampling based on geometric importance can automatically adapt to the boundary features of different tooth shapes. Randomly sample the original point cloud data to output down-sampled point cloud data; random sampling is used ; let the total sampling budget be , generate independent uniform random numbers ), construct the probability cumulative distribution function , then the first a sampling point The selection formula is: (9) Step three, using a tooth centroid prediction module (Centroid Model, CM) to process the point cloud, output a predicted candidate centroid position, and input the predicted candidate centroid into the BAS module; In the boundary-aware sampling process, the CM module first outputs the candidate centroid position of each tooth, and the BAS module preferentially retains the points near the centroid and the tooth boundary, that is, by increasing the sampling weight of these area points, it ensures that the subsequent feature extraction focuses on the core structure and edge features of the tooth; As Figure 3 is the CM module, the existing method relies on manually designed features or simple regression models for centroid prediction, which often has two limitations: one is the lack of global shape perception ability, which makes it difficult to extract common geometric features in the case of large individual differences; the second is the lack of local structure direction constraint, which leads to the predicted result deviating from the true centroid, and then causes problems such as poor aggregation of similar point clouds and fuzzy boundaries in subsequent segmentation; Therefore, the present application proposes a CM module based on PointTransformer architecture to realize accurate regression of the tooth centroid in the tooth point cloud data; First, the original three-dimensional point cloud data is sampled by the farthest point to obtain the initial feature ; wherein, is the number of point cloud nodes, is the feature dimension; usually taking 3-dimensional coordinate information as the input of the CM model, gradually aggregating and refining the point cloud features through the feature processing flow composed of DownSamplingTransition (downsampling module), Point Transformer Block, and Upsampling Transition (upsampling module); 3 DownSampling Transition are used to realize the compression of the feature from , which reduces the number of nodes while improving the feature dimension to capture the global shape; Upsampling Transition uses 3 interpolation-based and feature concatenation-based to realize the dimension recovery of the feature from , which combines the local details of the adjacent down-sampling layer features; 6 Point Transformer Blocks (the feature dimensions of each layer are adapted to 32, 64, 128, 128, 64, and 32, respectively) are connected in the middle, which uses the self-attention mechanism to capture the long-distance dependency relationship between nodes and strengthens the feature correlation; finally, after the processing of the multi-layer perceptron (MLP) containing 3 fully connected layers (the hidden layer dimensions are 64, 32, and 3, respectively), the offset prediction result , the offset vector mapping of the corresponding node coordinates to the target centroid; adding the offset matrix to the original coordinate matrix Point by point, a new point cloud after offset is obtained, and finally the new point cloud is clustered to obtain the centroid coordinates of each tooth.

[0042] The working principle of the CM module is as follows: The CM module aims to provide a "tooth core position constraint" for subsequent segmentation, solving the limitations of centroid positioning relying on manual annotation or simple regression in traditional methods. The input data is the original point cloud data.

[0043] The preprocessed oral point cloud data is input into the CM module, and the input point cloud is a three-dimensional point set containing N points , where the three-dimensional coordinate information of each point is represented as:

[0044] In order to adapt to the input dimension requirements of the subsequent module, the original coordinates are expanded in feature dimension, and 3D coordinates are converted to 32D initial features through 1-layer linear transformation, the formula is:

[0045] wherein, is the weight matrix, is the bias vector, and the final input feature matrix is formed.

[0046] represents the feature of one of the points, is the feature matrix, representing the features of N points.

[0047] DownSampling Transition adopts a "Farthest Point Sampling (FPS) + feature concatenation aggregation" downsampling strategy, a total of 3 cascaded downsampling transition modules are set, and the calculation logic is as follows: Farthest Point Sampling (FPS): select sampling points from the input point set , the sampling rule is: 1. Randomly select the initial sampling point ; 2. For the remaining points , calculate the minimum distance ( t is the current number of selected sampling points) from the selected sampling points; 3. Select The maximum point as the next sampling point Repeat steps 2-3 until the selected points are full ; For each sampling point , search for the nearest neighbor in the original point set , the formula is:

[0048] Concatenate the neighborhood point features and the sampling point features, and aggregate them through linear transformation, the formula is:

[0049] wherein , The final output is the first down-sampling module feature matrix .

[0050] The Point Transformer block can capture long-distance dependencies between points in the point cloud, dynamically allocate feature weights through self-attention mechanisms, and extract the context features of the point cloud, providing rich local details and global structure information for centroid prediction.

[0051] Each Point Transformer block contains three steps: "position encoding -> attention weight calculation -> feature update", the specific formula is as follows: Calculate the relative position vector of each point and its neighborhood points (neighborhood search uses k nearest neighbor), and encode it into a position feature through linear transformation, the formula is:

[0052] wherein is the relative position vector, , , is the encoded position feature.

[0053] Calculate the attention weight combining the point feature and the position feature, the formula is:

[0054] wherein , are the input features of points , respectively; , , is the attention weight, satisfying . ​

[0055] The current point feature is updated by aggregating the neighborhood features through attention weights, and the formula is:

[0056] wherein, D is the input feature dimension, such as 128; is a layer normalization operation to ensure training stability. This module is repeatedly used at each down-sampling and up-sampling stage to form multi-scale feature extraction capability.

[0057] UpSampling Transition is used to restore the resolution of the point cloud processed by the DownSampling Transition block and the PointTransformer block, while fusing the features of the corresponding down-sampling stage, realizing the complementarity of "global abstract features + local detailed features", and providing high-precision feature support for subsequent centroid prediction.

[0058] UpSampling Transition adopts an up-sampling strategy of "inverse distance weighted interpolation (IDW) + feature fusion", and a total of 3 UpSampling Transitions are arranged in series; The above sampling target point set is taken as the benchmark, and for each target point , search =3 nearest neighbors in the input point set (the output point set of the second down-sampling module, a total of points) to calculate the interpolated feature through inverse distance weighting, and the formula is: k

[0059] wherein, is the distance attenuation coefficient, is the feature of the input point , is the interpolated feature, and D is the feature dimension.

[0060] The interpolated feature and the original feature output by the first down-sampling module are fused through linear transformation to reduce to the target dimension, and the formula is:

[0061] wherein, , , and finally output the first up-sampling module feature matrix .

[0062] ​​The CM model is trained using a total loss function; wherein, the centroid loss formula is as follows:

[0063] wherein, ' represents the number of effective categories (centroid count), represents the point cloud number of the category ' . represents the original point cloud coordinates; represents the corresponding predicted offset vector; represents the centroid of the category .

[0064] By calculating the average of the square distance between the position of each point after moving to the centroid and the centroid, the same type of tooth point cloud is forced to gather to its centroid, so that the model can effectively capture the geometric center trend of the tooth, enhance the learning ability of the tooth morphology commonality, reduce the segmentation error caused by the individual morphology difference of the tooth, and improve the fitting precision of the model to the overall contour of the tooth.

[0065] The direction loss formula is as follows:

[0066] wherein, represents the number of effective categories (direction count); represents the effective point cloud number of the category c’ (offset vector norm greater than the threshold value); represents the cosine similarity between the predicted offset direction and the direction of the point to the centroid.

[0067] Formula (20) first calculates the “1 minus the square of the cosine similarity” (quantifies the direction deviation, the greater the deviation, the greater the value) of each effective point in the class, then averages it according to the number of effective points in the class ( ) to eliminate the influence of the number of points, and finally averages it according to the number of categories ( ) to balance the loss contribution of each category; this process forces the predicted offset direction of the same type of tooth point to align with the real direction; this process can make the model accurately learn the direction features of a single tooth, avoid the mutual offset of the direction errors of different teeth, and at the same time exclude the interference of invalid points, enhance the model's ability to distinguish direction deviation, reduce the centroid positioning deviation caused by direction prediction error, and finally improve the boundary accuracy of tooth instance separation, and meet the strict requirements of clinical tooth contour differentiation.

[0068] The Chamfer distance loss formula is as follows:

[0069] to the predicted point set Each point in Find the nearest real point (minimize Then sum the squares of these minimum distances and finally average them; for the true point set Each point in Find the nearest prediction point Similarly, calculate the square of the minimum distance and sum them, then average them. This means measuring the similarity between the predicted point set and the real point set from a global perspective, and ensuring that the predicted results closely match the actual tooth morphology by calculating the mean square distance between the nearest points in both directions.

[0070] The category-segmentation cross-entropy loss is as follows: (twenty two) in, This represents the total number of points in the tooth point cloud, i.e., the total number of all valid points involved in the segmentation prediction. This represents the total number of valid categories in the tooth segmentation task, corresponding to the number of instances of all teeth in the mouth; Representative point i Corresponding category c The true label (one-hot encoded form), if the point Actually belongs to the first One tooth, then The remaining categories corresponding , used to provide real supervision signals for model training; Indicates the prediction point Category The probability value of is in the range [0,1] and satisfies . (Obtained through normalization using the Softmax function), reflecting the model's confidence in classifying the point; The overall loss function expression is:

[0071] in, λ 1 to λ 4 represents the weighting coefficient.

[0072] Step 4: Input the tooth boundary sampling point cloud output by the BAS module into a multi-head graph attention module (MHAG) with several residual connections to extract features and obtain local graph features. The number of MHAG modules can be customized. In this embodiment, it is preferred to cascade three MHAG modules, and the first MHAG module, the second MHAG module, and the third MHAG module are residually connected; that is, the outputs of the first MHAG module, the second MHAG module, and the third MHAG module are concat-joined. like Figure 5 As shown, after using KNN (k-nearest neighbor) features, the MHAG module aims to capture long-distance dependencies between point cloud nodes through a multi-head attention mechanism and fuse multi-dimensional geometric features to enhance segmentation representation capabilities. The point cloud features after multi-scale feature extraction and fusion are mapped to the three-dimensional coordinates of the tooth centroid, realizing the transformation from feature space to physical space.

[0073] The MHAG module sequentially extracts features from the point cloud data after boundary-aware sampling and centroid constraint processing. The multi-head graph attention mechanism can capture the complex relationships between points in the point cloud data and obtain multi-dimensional feature information from different attention heads, thereby extracting local features of the tooth point cloud more comprehensively and obtaining local graph features. After processing by multiple MHAG modules, the features can be further integrated to obtain global graph features.

[0074] The core of the MHAG module adopts a parallel architecture of "multi-head self-attention branch + geometric feature enhancement branch", with the two types of branches working together to extract the semantic associations and geometric attributes of the point cloud.

[0075] Feature mapping, the node feature matrix of the graph structure output by the BAS module. ( For the number of nodes, =10 is the node feature dimension), which is mapped to query, key, and value matrices through three independent linear transformations, respectively, as shown in the formula:

[0076] in, The learnable parameter matrix; Represents the feature dimensions of each attention head; The number of attention heads.

[0077] Will Q , K , V By number of attention points Split into Submatrix, the nth submatrix The submatrix corresponding to each head is , , ; Each attention head independently calculates an attention weight matrix , which is normalized by a Softmax function, with the formula:

[0078] wherein, is a scaling factor used to alleviate the problem of excessively large weight values caused by dimension growth, ensuring the stability of the Softmax function output; represents the attention weight of the th node to the th node, reflecting the semantic association strength between the two nodes.

[0079] Each attention head is weighted and summed by a weight matrix to the value matrix to obtain the output feature of a single head : .

[0080] The output features of H attention heads are concatenated by column to obtain a feature matrix with a dimension of , which is then fused through linear transformation to finally output the multi-head self-attention feature : ; wherein, is a feature concatenation operation, used to integrate semantic information of different attention heads and strengthen feature representativeness.

[0081] A geometric feature enhancement branch is set up, which supplements the geometric attribute information of the point cloud through global statistics and high-dimensional mapping of key geometric features (curvature, normal vector), and forms a complement with semantic features. The specific steps are as follows: Geometric feature extraction and global statistics: The curvature feature matrix (curvature value of each node) and the normal vector feature matrix (normalized normal vector of each node) of the input point cloud are subjected to a joint operation of maximum pooling and average pooling to extract global statistical information: Curvature global feature: wherein, is the maximum curvature value of all nodes, reflecting the most prominent boundary region feature in the point cloud; is the average curvature value, reflecting the overall geometric bending degree; Normal vector global feature: wherein, , are the maximum and average values of each dimension of the normal vector, respectively, reflecting the overall orientation distribution of the point cloud surface; The low-dimensional global statistical features are mapped to a high-dimensional space by sharing a multi-layer perception (MLP) with a ReLU activation function, and the dimension D of the node features is kept consistent, as follows:

[0082] The broadcasting mechanism (Broadcasting) is used to expand the matrix , to , ensuring that the dimensions match the node feature matrix , , facilitating subsequent feature fusion; The final output feature of the MHAG module is obtained by fusing the multi-head self-attention feature and the geometry-enhanced feature, and introducing a residual connection to alleviate the gradient vanishing problem, as follows: (27) where the residual connection is used to alleviate the gradient vanishing problem of deep networks, and the output feature retains the semantic association and geometric properties between nodes, providing more robust representations for subsequent class prediction by the segmentation head.

[0083] Step five, perform global maximum pooling and average pooling operations on the local graph features in sequence to obtain global graph features; and perform weighted fusion of the global graph features and the local graph features; On the basis of extracting local graph features, global maximum pooling and average pooling operations are performed on the local features of all points to compress the local features of the point cloud into a high-dimensional vector, which is used to represent the global graph features of the entire dentition (such as the arrangement direction of the dentition and the overall morphological symmetry), i.e., GlobalFeature; then a geometry attribute-driven weighted fusion strategy is adopted: The weighted fusion strategy includes: First, the concatenated features are decomposed into coordinate-related features, normal vector-related features, and curvature-related features according to the core geometric properties of the tooth point cloud, and then the weight proportions of the three types of subspaces are calculated by an attribute importance quantification model (denoted as , , respectively, and the sum of the weights is 1), followed by element-wise weighting of each feature subspace (i.e., multiplying the subspace feature by the corresponding weight), and finally the weighted three types of subspace features are stacked dimension by dimension to generate a fusion feature that considers position information, surface attitude, and local structure complexity, effectively enhancing the feature's ability to represent key structures such as tooth edges and occlusal surfaces.

[0084] After processing by the 3 MHAG modules, the output of each MHAG module is Meanwhile, the semantic association (from the multi-head self-attention branch) and the geometric property (from the geometry feature enhancement branch) are combined to generate the local graph feature (LocalFeature) and the global graph feature (GlobalFeature): 1、LocalFeature: for The node-preserved feature directly reflects the semantic-geometric fusion information of a single node; 2、GlobalFeature: for When performing attention pooling, the geometric feature is used as an additional constraint (positive correlation between weight and ) for the pooling weight, ensuring that the global feature pays more attention to the geometric key area and provides more accurate feature support for the segmentation head.

[0085] Step six, input the fused features into a multi-layer perceptron (MLP) and introduce a centroid constraint mechanism based on point-level distance prediction to correct label assignment bias: The output of the MLP is Mxp, which is a probability matrix; where M represents the number of points in the point cloud, and p represents the type of teeth, i.e., the predicted probability of each point belonging to different classes of teeth; the predicted result is obtained by taking the class with the highest predicted probability as the tooth class; The MLP classification head maps high-dimensional features to the probability of each point belonging to different tooth classes and determines the final prediction label based on the maximum probability to achieve accurate segmentation of tooth point clouds.

[0086] The MLP classification head adopts a 3-layer fully connected (FC) + activation function architecture to adapt to the tooth class prediction requirements. The number of human permanent teeth is usually 32, so the output class number = 32, including the wisdom tooth class; The Softmax function is used to map high-dimensional features to the probability of 32 tooth classes, ensuring that the sum of the class probabilities of each point is 1, and the formula is:

[0087] where Pc represents the probability of the cth tooth belonging to the cth tooth, is the 64-dimensional feature output by the second layer, and are the weight matrix and bias vector of the output layer, respectively, is the exponential function, and the denominator is the sum of the exponential values of the 32 tooth classes, ensuring that and .

[0088] For each point n , the probability matrix​P The maximum value of the row corresponds to the category of the prediction label of the point, and the formula is: n

[0089] wherein, is the predicted tooth label of the nth point, The function returns the class index that maximizes the probability. This operation can directly output discrete segmentation labels, meeting the practical application requirements of clinical tooth segmentation.

[0090] Based on the centroid coordinates of each tooth corresponding to the CM module output, the Euclidean distance (i.e., point-level distance prediction value) of each point in the preliminary segmentation result to the centroid corresponding to its current assigned label is first predicted; then a distance threshold is set, such as 2mm determined according to the tooth anatomical size, to ensure covering the maximum radial range of a single tooth, and if the predicted distance of a point exceeds the threshold, it is determined that the label assignment of the point is wrong; at this time, the predicted distance of the point to all tooth preset centroids is recalculated, and the label of the point is reassigned to the category corresponding to the centroid of the target tooth with the smallest predicted distance; that is, through the process of "point-level distance prediction- abnormal point determination-label reassignment", the segmentation result is forced to align with the centroid geometric prior from the point-by-point level, avoiding the label confusion of the points in a single tooth caused by local feature ambiguity, and further improving the fineness and accuracy of the segmentation result.

[0091] Experimental process: All training and experiments of the present application were performed under the PyTorch framework on a NVIDIA GeForce RTX 4090 GPU for 200 rounds of training, with a batch size of 1; an Adam optimizer with an initial learning rate of 0.001 was used, and the learning rate was reduced by 50% every 20 rounds.

[0092] To evaluate the performance of the model of the present application, it was comprehensively compared with seven advanced 3D point cloud segmentation networks: PointNet, PointNet++, DGCNN, TSGCNet, SGTNet, TSegFormer and PointTransformer; Table 1 shows the quantitative comparison results of overall accuracy (OA) and mean intersection over union (mIoU), and the IoU index includes the segmentation results of incisors, canines, premolars, molars and gums.

[0093] Table 1 Comparison of the present application with existing models

[0094] ​As shown in Table 1, the model of the application achieves the highest performance in OA (97.2947%) and mIoU (92.609%), surpassing all baseline models; compared with the suboptimal model PointTransformer, the model of the application improves by 0.9467 percentage points in OA and 1.803 percentage points in mIoU, proving the superiority of the boundary-aware sampling module and the centroid prediction module in capturing global and local geometric features.

[0095] Compared with the point-based feature extraction models PointNet++ and DGCNN, the model of the application shows significant performance improvement; for example, compared with PointNet++, the model of the application improves by 3.7517 percentage points in OA and 7.599 percentage points in mIoU; this improvement is due to the boundary-aware sampling module, which enhances the ability to capture geometric differences between neighboring points through adaptive sampling, while PointNet++ and DGCNN mainly rely on coordinate features, making it difficult to handle complex tooth structures.

[0096] The model of the application achieves higher IoU in all tooth structure segmentation, specifically incisors (90.825%), canines (91.05%), premolars (94.225%), molars (92.775%), and gums (95.8%); compared with TSGCNet and SGTNet, which use graph-based feature enhancement, the model of the application improves by about 4 to 12 percentage points in IoU for each type of tooth structure, highlighting the robustness of the centroid prediction module in handling complex tooth anatomical structure boundaries, which optimizes segmentation accuracy through accurate centroid positioning.

[0097] As Figure 4 , the visualization results further verify the quantitative analysis; in the malpositioned tooth area, PointNet and PointNet++ have low segmentation accuracy due to their inability to effectively model local geometric changes; similarly, DGCNN and TSGCNet exhibit over-segmentation at the tooth-gum boundary due to their rough feature fusion strategy; in contrast, the model of the application exhibits excellent segmentation performance at the tooth boundary, especially in areas with subtle geometric transitions; this is due to the boundary-aware sampling module, which makes the network more sensitive to subtle geometric feature changes in the grid at the boundary, and the centroid prediction module, which accurately locates the tooth area.

[0098] Compared with TSegFormer and PointTransformer using advanced Transformer architecture, the model of the application shows better generalization ability; its continuous improvement in the IoU index shows that the boundary perception sampling module and the centroid prediction module effectively promote feature complementarity, integrate local and global context information, and thus achieve robust segmentation; especially in the segmentation of molars, the IoU of TSegFormer is only 86.615%, while the model of the application reaches 92.775%, highlighting its ability to handle complex occlusal surfaces; in summary, the experimental results verify the superiority of the model based on the boundary perception sampling module and the centroid prediction module of the application in the 3D tooth segmentation task; by combining boundary-sensitive adaptive sampling and precise centroid positioning strategies, the model of the application outperforms existing methods in overall accuracy and fine-grained tooth structure segmentation; its excellent performance in boundary regions and complex geometric structures further proves its potential application value in tooth analysis and treatment planning.

[0099] Table 2 ablation experiment:

[0100] To verify the effectiveness of the boundary-aware sampling (BAS), a comparative experiment was conducted between BAS and the traditional farthest point sampling (FPS), and the quantitative results are shown in the table. As can be seen from Table 2, the model using BAS is significantly better than FPS in OA (overall accuracy) and mIoU (mean intersection over union): OA is improved from 94.213 to 97.2947, and mIoU is improved from 86.658 to 92.609, with a more obvious improvement in mIoU.

[0101] This result shows that BAS can more accurately preserve the geometric features of the tooth edge by preferentially enhancing the sampling weight of the tooth boundary region (i.e., assigning a higher sampling probability to high-curvature boundary points). In contrast, FPS only performs uniform sampling based on spatial distance, which is prone to lose key boundary information in crowded or morphologically variant scenarios, leading to confusion of internal features in similar teeth or blurring of boundaries between adjacent teeth. By focusing on the details of boundary region sampling, BAS makes it easier for the model to capture morphological differences between teeth during the learning process, thereby effectively improving the instance discrimination and boundary accuracy of segmentation.

[0102] As Figure 7Further visualization of the segmentation results shows that the tooth edges are clearer under BAS sampling, especially in areas where multiple teeth are closely adjacent, and the interdental boundary is more accurate. The results of FPS sampling are prone to edge adhesion in complex morphologies, making it difficult to distinguish the individual boundaries of adjacent teeth. This confirms the effectiveness of BAS in optimizing the sampling strategy by perceiving boundary features, providing better geometric prior information for subsequent feature learning.

[0103] To verify the effectiveness of the Centroid branch, the baseline model without this module and the model with the Centroid prediction branch were compared. The quantitative results are shown in the table. The data shows that after introducing the Centroid offset branch, the model has significantly improved in OA (overall accuracy) and mIoU (mean intersection over union): OA from 95.887 to 97.294, mIoU from 91.035 to 92.609, indicating the optimization effect of the Centroid prediction mechanism on segmentation accuracy.

[0104] As Figure 8 and Figure 9 This improvement is due to the Centroid offset branch providing global position priors for tooth instances. Baseline relies only on local geometric features for segmentation, and in crowded or morphologically variant scenarios, it is prone to instance confusion due to local feature similarity (e.g., misjudgment of adjacent tooth boundaries). The Centroid branch injects a global clue of "individual attribution" into the model by predicting the offset vector of each point to the corresponding tooth centroid - the point set of each tooth forms a compact distribution around its centroid, and the consistency of the offset vector can help distinguish adjacent or morphologically similar tooth instances. Further analysis of the segmentation results shows that the Centroid module performs better in areas where multiple teeth are closely arranged: the interdental boundary is clearer, avoiding the occasional "adhesion" phenomenon in Baseline. This is because the Centroid offset gives each point a clear instance-level label guide. When local geometric features are not sufficient to distinguish adjacent teeth, the difference in global centroid position can be used as a supplementary criterion, thereby improving the model's ability to distinguish tooth instances and the robustness of segmentation.

[0105] To verify the effectiveness of the multi-head graph attention module (MHAG) in feature fusion and long-distance dependency modeling, three groups of control models are set up: the first group is the basic graph model, which only uses normal vectors and coordinates as input features; the second group is the enhanced graph model, which additionally introduces curvature features based on normal vectors and coordinates; the third group is the optimized model, which further embeds MHAG based on the feature input of the enhanced graph model. The quantitative results are shown in the table. The data shows that the segmentation performance of the three groups of models presents a clear step-by-step improvement: compared with the basic graph model using only normal vectors and coordinates, the OA is 96.075 and the mIoU is 91.254. From the visualization results, it can be seen that the enhanced graph model that supplements the curvature, a local geometric feature, performs better, clearly verifying the complementary value of multi-source geometric features. The performance of the optimized model embedded with MHAG is further improved, with OA reaching 97.2947 and mIoU reaching 92.609, which produces significant gains in the two core indicators compared to the enhanced graph model, fully demonstrating the optimization effect of MHAG on the feature representation ability of the model.

[0106] The core logic of this performance improvement is that the utilization of multi-source features by ordinary graph models (including the basic graph model and the enhanced graph model) is generally limited to the shallow fusion mode of "feature concatenation - local convolution". This mode cannot deeply excavate the associated semantics among normal vectors, coordinates, and curvature - normal vectors reflect surface orientation, coordinates mark spatial position, and curvature represents bending degree, and the synergistic effect of the three cannot be fully activated. It is also difficult to model the long-distance dependencies between tooth point cloud nodes, such as the global morphological correlation between the crown and the root, which cannot be effectively captured, ultimately leading to the problem of feature fragmentation in the context of tooth crowding and morphological variation. MHAG, relying on the multi-head self-attention mechanism, constructs a more efficient feature processing path: on the one hand, it can dynamically allocate weights to multi-source features, strengthening the feature contribution of curvature and normal vectors in tooth boundary areas and focusing on the spatial correlation of coordinate features in tooth interior areas, achieving adaptive focusing of features; on the other hand, it can cross the local neighborhood constraint to capture the morphological correlation of non-adjacent nodes, such as the distal cusp and the proximal root of the same tooth, thereby constructing a globally consistent feature representation. Taking the area where multiple teeth are closely adjacent as an example, ordinary graph models are prone to misjudging the boundary points of adjacent teeth as the same instance due to local feature similarity, while MHAG can distinguish the global morphological differences of different teeth through long-distance dependency modeling, achieving precise division of interdental boundaries.

[0107] As Figure 10The visualization results verify this advantage: the tooth segmentation results output by the optimization model embedded with the MHAG not only have sharper edges, more complete details at the tooth-gum junction, and better aggregation of similar point clouds, but also have higher point set annotation consistency in the root area; while the segmentation results of the ordinary graph model are prone to feature confusion in complex morphologies, such as tooth proximal surface contact areas, and even have local point cloud mislabeling. This phenomenon shows that the MHAG provides more discriminative feature representations for the model by optimizing the multi-source feature fusion efficiency and the long-distance dependency modeling capability, thereby effectively improving the instance discrimination and boundary accuracy of tooth point cloud segmentation.

[0108] With the above ideal embodiments according to the present application as the inspiration, through the above description, relevant personnel can certainly make various changes and modifications without deviating from the technical idea of the present application. The technical scope of the present application is not limited to the contents in the specification, and must be determined according to the scope of claims.

Claims

1. A 3D point cloud tooth segmentation method based on geometry guidance and instance awareness, characterized in that, Includes the following steps: Step 1: Obtain 3D point cloud data containing tooth feature information; Step 2: Use the BAS module to process the point cloud and candidate centroid positions to obtain the tooth boundary sampling point cloud; Step 3: Use the CM module to process the point cloud and output the predicted candidate centroid positions; Step 4: Input the tooth boundary sampling point cloud into several residual-connected MHAG modules and output local map features; Step 5: Perform global max pooling and average pooling operations on the local graph features sequentially to obtain global graph features; then perform weighted fusion of global graph features and local graph features. Step 6: Input the weighted fusion features into the MLP; A centroid constraint mechanism based on point-level distance prediction is introduced to correct label assignment bias.

2. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 1, characterized in that, Step two specifically includes: Step 21: Calculate the centroid of each tooth in the maxillary and mandibular point cloud sets respectively. The mean of the coordinates; Step 22, above, centroid of mandibular teeth Center radius R Perform sphere-cut circle cutting and combine Z-axis thresholding for maxillary and mandibular filtering to obtain sub-point clouds for each tooth; Step 23: Calculate the curvature of the point cloud using the K-nearest neighbors and the eigenvalues ​​of the covariance matrix; Step 24: Filter Top Curvature Points; Step 25: Perform boundary clustering on the selected curvature points; Step 26: Use a combination strategy of inner pool weighted sampling and outer pool uniform sampling to allocate the TOP curvature point pool and non-boundary point pool; Step 27: Utilize the adaptive weight function Calculate the probability of candidate points ; Step 28: Randomly sample the original point cloud data and output the downsampled point cloud data.

3. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 2, characterized in that, If the sample belongs to the maxilla, then only retain [the sample]. Otherwise, retain the point; The point is z, where z is the specific coordinate of Z. The Z-axis threshold is set; and XY plane limiting is introduced, with the outer circle as the reserved area according to the tooth position.

4. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 2, characterized in that, Random sampling includes: Let the total sampling budget be ,generate A number of independent uniformly random numbers Construct the cumulative probability distribution function , obtained the The probability of each sampling point .

5. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 1, characterized in that, The CM module includes: A downsampling combination block is formed by a DownSampling Transition block and a Point Transformer block, and an upsampling combination block is formed by an Upsampling Transition block and a Point Transformer block. The three downsampling combination blocks are cascaded with the three upsampling combination blocks and the MLP in sequence. The first downsampling combination block is residually connected to the first upsampling combination block. The second downsampling combination block is residually connected to the second upsampling combination block.

6. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 5, characterized in that, Using the total loss function Training the CM model, λ 1 to λ 4 represents the weighting coefficient. For the loss of the center of mass, For directional loss, For Chamfer distance loss, Cross-entropy loss for category segmentation.

7. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 6, characterized in that, The formula for centroid loss is: in, Indicates the number of valid categories (centroid count). Indicates category The number of point clouds; Represents the original point cloud coordinates; This represents the corresponding predicted offset vector; Indicates category The center of mass.

8. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 6, characterized in that, The formula for direction loss is: in, Indicates the number of valid categories; Indicates category c The effective number of point clouds; This represents the cosine similarity between the predicted offset direction and the direction from the point to the centroid.

9. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 1, characterized in that, The residual connection of the MHAG module includes: three cascaded MHAG modules, and the outputs of the first and second MHAG modules are concatted with the output of the third MHAG module.

10. The 3D point cloud tooth segmentation method based on geometry guidance and instance awareness according to claim 9, characterized in that, The MHAG module includes: a self-attention branch and a geometric feature enhancement branch; the self-attention branch includes: Node feature matrix Linear transformation mapping to query Q ,key K Sum V ; Will Q , K , V Split into sub-matrices based on the number of attention points ; Calculate the attention weight matrix Output the features of a single head ; Output self-attention features , h =1,2,3; The geometric feature enhancement branch includes: The curvature feature matrix of the input point cloud With normal vector characteristic matrix ; Calculate global curvature features and normal vector global features ; right and Perform MLP mapping to obtain the mapped features. and ; Output features .

Images