Point cloud tooth segmentation method based on rotation invariant feature

Abstract

The invention discloses a point cloud tooth segmentation method based on rotation invariant features, and relates to the technical field of oral medical treatment. Comprising the following steps: firstly, acquiring a plurality of oral scanning point cloud data with known tooth feature information, performing normalization processing, and performing one-to-one mapping on the oral scanning point cloud data, a tooth mass center position and a segmentation contour to generate an oral scanning data set; secondly, extracting local and global rotation invariant features of the point cloud, training a centroid prediction model based on global features, and taking the point cloud data as input and the tooth centroid position as a label; then, the rotation invariant features and the predicted centroid position are input into a segmentation encoder, global and local consistent features are generated, comprehensive representative features are obtained through feature fusion, a tooth segmentation prediction model is trained based on the comprehensive representative features, and a tooth segmentation contour is labeled. And feature extraction and prediction are carried out on the cloud data of the points to be segmented, a tooth segmentation contour is output, accurate segmentation is realized, and the method has the characteristics of high robustness and rotation invariance.

Description

Technical Field

[0001] The present invention relates to the field of oral medical technology, and in particular to a point cloud tooth segmentation method based on rotation invariant features. Background Art

[0002] Computer-aided design (CAD) has been widely used in modern stomatology (dental medicine) with a wide range of applications, from orthodontic diagnosis to preoperative simulation, where the patient's 3D tooth model is usually obtained by scanning and processed through task-specific operations. All these applications share a ubiquitous procedure, namely tooth segmentation, which aims to accurately segment each tooth in the 3D tooth model. Accurate tooth segmentation results can provide orthodontists with detailed tooth position and morphology information, and assist in formulating more scientific and reasonable correction plans; therefore, exploring an efficient, fully automatic, and accurate point cloud tooth segmentation method can help doctors clearly understand the anatomical structure of teeth, plan surgical paths in advance, effectively reduce surgical risks, and improve the success rate of treatment and patient satisfaction.

[0003] Most previous tooth segmentation methods are either experimented on datasets with well-aligned data; the actual collected tooth point cloud data inevitably rotates due to factors such as differences in the patient's posture during the scanning process, the position and angle changes of the scanning equipment, etc. At this time, if a large amount of data needs to be aligned, it is obviously cumbersome and time-consuming; or through data enhancement methods, the network has seen data from various angles during training, but this will not only increase the demand for data storage, but also significantly prolong the training time of the model. Multi-angle rotation enhancement of large-scale point cloud data will take up a lot of memory and CPU and GPU resources, making the training process slower and the effect is also poor.

[0004] To address this problem, researchers have tried to enhance the robustness of the model by extracting rotation-invariant representation (RIR) in recent years. Rotation-invariant features can effectively eliminate the interference of rotation on segmentation results by encoding stable geometric and topological relationships in point clouds. However, most of the current point cloud tooth segmentation methods based on rotation-invariant features rely only on local features, ignoring the influence of global geometric structure on tooth segmentation. In addition, existing methods lack the ability to accurately predict the center of mass position of teeth, which further limits the accuracy and usability of segmentation.

[0005] In the prior art, the publication number CN113344950A discloses a CBCT image tooth segmentation method combining deep learning with point cloud semantics, including the following steps: step 1, based on a deep learning segmentation model, such as a 3D segmentation network or a 2D segmentation network, tooth region segmentation is performed to extract the tooth region; step 2, the extracted tooth region is three-dimensionally reconstructed into dentition mesh data using a surface drawing method; step 3, point cloud feature data of the mesh data is extracted, and instance segmentation based on point cloud semantics is performed using a point cloud instance segmentation deep learning network to obtain a tooth instance of the mesh data; step 4, according to the coordinate correspondence information, the tooth corresponding area of ​​the mesh data is mapped to the CBCT to obtain a CBCT tooth instance. However, the scheme does not involve a rotation-invariant feature extraction mechanism. If the point cloud generated by the CBCT is rotated or the dentition posture in the data set changes greatly, the semantic segmentation network (such as based on PointNet++ or other point cloud segmentation networks) may find it difficult to effectively learn features across the rotation domain. This makes the segmentation performance of the scheme susceptible to serious changes in the directionality of the input data, thereby reducing the generalization ability of the model, and thus resulting in reduced accuracy and effectiveness of the tooth segmentation results.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a point cloud tooth segmentation method based on rotation invariant features to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A point cloud tooth segmentation method based on rotation invariant features, the specific steps include:

[0010] Acquire oral scan point cloud data with known tooth feature information, perform normalization preprocessing on the point cloud data, and map the normalized preprocessed oral scan point cloud data with the corresponding tooth feature information one by one to generate an oral scan data set, wherein the tooth feature information includes the centroid position of each tooth and the tooth segmentation contour;

[0011] Based on the oral scan point cloud data preprocessed in the oral scan data set, the rotation invariant features of the point cloud data are extracted, and the rotation invariant features include local rotation invariant features and global rotation invariant features. A centroid prediction model is established, and the global rotation invariant features are used as input of the centroid prediction model. The corresponding tooth centroid position is used as a label to train the centroid prediction model.

[0012] The extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model are input into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, and the obtained global consistent features and local consistent features are subjected to feature fusion to obtain comprehensive representative features. Based on the obtained comprehensive representative features, a tooth segmentation prediction model is established, and the comprehensive representative features are used as input, and the corresponding tooth segmentation contours in the oral scan dataset are used as labels to train the tooth segmentation prediction model;

[0013] The oral scan point cloud data to be segmented is obtained, and after preprocessing, the target comprehensive representative features are obtained through tooth centroid prediction, rotation invariant feature extraction and feature fusion. The target comprehensive representative features are input into the trained tooth segmentation prediction model to obtain the tooth segmentation contour corresponding to the oral scan point cloud data to be segmented, and the tooth segmentation is completed.

[0014] Furthermore, the specific method for extracting rotation-invariant features from the normalized preprocessed point cloud data is as follows: using the coordinates of the points in the input point cloud in 3D space, calculating the inner product of each point with the coordinates of other points, and obtaining the global rotation-invariant feature F gri , where the global rotation invariant feature F gri The specific formula on which the calculation is based is:

[0015]

[0016] Where, X gri is the global rotation invariant feature of the i-th point in the point cloud data, X represents the point cloud block, and x i Represents the coordinates of the i-th point in the point cloud data in 3D space, where i is the index of the point in the point cloud data, i = 1, 2, ..., n, n is the total number of points in the point cloud data, X∈R 3×K , R represents a real number set, K is the number of columns of the point cloud block;

[0017] The KNN algorithm is used to find U nearest neighbor points for each point in the point cloud, and a local region is constructed for each point. In each local region, the inner product between each point and other points is calculated to obtain the local rotation invariant feature F. lri The specific calculation method is similar to the global rotation invariant feature F gri The calculation method is consistent, and U is a positive integer.

[0018] Furthermore, the loss function expression of the centroid prediction model is set as:

[0019]

[0020] In the formula, is the loss function of the centroid prediction model, is a smooth loss function; where the smooth loss function The specific expression is:

[0021]

[0022] In the formula, p j is the true label value of the jth sample data, is the label value predicted by the centroid prediction model for the jth sample data, j is the index of the sample data, and G is the total number of sample data.

[0023] Further, the extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model are input into a point cloud tooth segmentation encoder to obtain a global consistent feature and a local consistent feature, wherein the point cloud tooth segmentation encoder includes a local branch and a global branch;

[0024] The point cloud data is subjected to different data enhancements, and the point cloud tooth segmentation encoder is pre-trained based on the enhanced data. The pre-training includes: inputting the enhanced point cloud data into the encoder, extracting the feature vectors of multiple views, and mapping the features to a specific space through a projection head. The data enhancement includes rotation and translation, and the contrast loss is calculated based on the feature vectors of multiple views. The point cloud tooth segmentation encoder is adjusted by the contrast loss to minimize the contrast loss. The contrast loss function of the point cloud tooth segmentation encoder is expressed as:

[0025]

[0026] Where, L cvq is the contrast loss function of the point cloud tooth segmentation encoder, s(.) represents the cosine similarity function, τ represents the temperature coefficient, B represents the batch size, and They respectively represent the projection vectors of the qth point cloud data in the same batch obtained through the point cloud tooth segmentation encoder and the projection head, T1 and T2 respectively represent the data enhancement for translation and rotation of the data, q is the index of the point cloud data in the same batch, a is the index of the point cloud data in all batches of point cloud data after the two data enhancements, a=1,2,…,2B-1, q=1,2,…,B-1.

[0027] Furthermore, the specific method for obtaining the global consistent features and the local consistent features is:

[0028] The point cloud tooth segmentation encoder consists of two branches, including a local branch and a global branch. The input received by the local branch is the concatenation data of the local rotation invariant feature and the predicted tooth centroid. Then, by cascading multiple edge convolutions, the local consistency feature is extracted. Finally, the outputs of multiple edge convolutions are concatenated in the feature dimension to obtain the local consistency feature F.L , where the edge convolution formula is as follows:

[0029] e il =h Θ (x i ,x l -x i )

[0030]

[0031] In the formula, e il It represents the feature obtained by combining the local rotation invariant feature corresponding to the lth connected point on the connected edge of the i-th point cloud data point and the tooth centroid feature data, x i represents the feature after the local rotation invariant feature of the i-th point is spliced ​​with the tooth centroid feature data, x l represents the feature obtained by combining the local rotation invariant feature corresponding to the lth connected point with the tooth centroid feature data, h Θ represents a nonlinear function with a set of learnable parameters Θ, represents the local features of the ith point after updating through edge convolution, represents the neighbor point set of point i, including all points connected to point i by edges, l is the index of the point connected to point i by edges, e il It represents the feature obtained by concatenating the local rotation invariant feature corresponding to the lth connected point of the i-th point cloud data point and the tooth centroid feature data;

[0032] Global rotation invariant feature F gri Input to the global branch, model the global information through multiple mamba modules, use the space filling curve to convert the unstructured point cloud into a regular sequence, and then extract the global information of the point cloud through multiple cascaded mamba modules to obtain the global consistent feature F G .

[0033] Furthermore, the obtained global consistent features and local consistent features are fused to obtain comprehensive representative features, wherein the formula for calculating the comprehensive representative features is:

[0034] F=w G F G +w L F L

[0035] In the formula, F is the comprehensive representative feature, w G and w L are the weight coefficients of the global consistent features and the local consistent features, F G and F L are the global consistent features and the local consistent features, respectively, where the weight coefficient w of the global consistent featureG The calculation is based on the formula:

[0036]

[0037] In the formula, F gri and F lri represents the global rotation invariant features and local rotation invariant features of the i-th point in the point cloud data, where w L With w G The calculation method is the same.

[0038] Furthermore, the global information is modeled through multiple mamba modules, the space filling curve is used to convert the unstructured point cloud into a regular sequence, and the global information of the point cloud is extracted through multiple cascaded mamba modules, where the calculation formula of the mamba module is:

[0039] F′ L-1 =LN(F L-1 )

[0040] F′ L =σ(DW(Linear(F′ L-1 )))

[0041] F″ L =σ(Linear(F′ L-1 ))

[0042] F L =Linear(SelectiveSSM(F′ L )⊙F″ L )+F L-1

[0043] In the formula, F L-1 represents the global rotation invariant feature input of the L-1th layer of the mamba module, F′ L-1 Represents the input feature F of the l-1 layer L-1 The result after layer normalization, LN means layer normalization operation, DW means depth separable convolution, Linear means linear transformation, σ is SiLU activation function, F′ L Indicates F′ L-1 The global feature after depthwise separable convolution and linear transformation, F″ L Indicates F′ L-1 The global features after direct linear transformation and activation operations, SelectiveSSM is a selective state space model, and ⊙ represents element-by-element multiplication.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] First, by normalizing the point cloud data for preprocessing, it is ensured that the data has a uniform scale and geometric center in the preliminary processing stage, thus providing a reliable basis for subsequent feature extraction and modeling. Normalization can effectively reduce the scale and position differences in the point cloud data caused by the scanning device or patient posture, and provide support for the standardized training of the model. Secondly, this method introduces rotation invariant features in the tooth segmentation task, including the extraction of global rotation invariant features and local rotation invariant features. Compared with the method that only relies on local geometric features, this scheme explicitly encodes the overall geometric structure of the point cloud through global feature extraction, thereby enhancing the model's ability to handle complex tooth arrangements. In addition, local rotation invariant features can capture the detailed structure and neighborhood information of the teeth. The combination of the two forms global consistent features and local consistent features, and the segmentation accuracy and model robustness are further improved through feature fusion. In addition, by establishing a centroid prediction model, this method can accurately predict the centroid position of each tooth in the oral point cloud. As the key geometric information for segmentation, the centroid position of the tooth can effectively guide the regional division of the tooth and reduce the confusion problem of the model at the tooth boundary. This process significantly improves the contextual consistency of the point cloud segmentation results, allowing the model to more accurately separate adjacent teeth. Finally, through feature fusion and the construction of comprehensive representative features, a full combination of global and local information is achieved, and a tooth segmentation prediction model based on deep learning is established. The model uses comprehensive representative features as input and can efficiently generate tooth segmentation contours. This method shows stronger robustness and higher segmentation accuracy when dealing with challenges such as rotation, point cloud noise and sparsity. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the overall method flow of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0048] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] Example:

[0050] See also Figure 1 , the present invention provides a technical solution:

[0051] A point cloud tooth segmentation method based on rotation invariant features, the specific steps include:

[0052] Step 1: Obtain some oral scan point cloud data with known tooth feature information, perform normalization preprocessing on the point cloud data, and map the normalized preprocessed oral scan point cloud data with the corresponding tooth feature information one by one to generate an oral scan data set, wherein the tooth feature information includes the centroid position of each tooth and the tooth segmentation contour.

[0053] An oral scanner or similar equipment (such as an intraoral scanner, CBCT reconstruction) is used to obtain three-dimensional point cloud data of the patient's oral cavity. The point cloud data is usually an unstructured three-dimensional coordinate representation and contains surface geometric information of the teeth, gums, and surrounding tissues.

[0054] Use professional dental software (such as MeshLab or Geomagic) to manually mark the segmentation contours of the teeth (i.e. the boundaries of each tooth) and accurately locate the centroid of each tooth. The centroid can be obtained by geometric calculation or determined based on prior knowledge of tooth morphology.

[0055] The point cloud data is normalized to unify the scale, position and direction of the data to ensure the stability of feature extraction and model training. The specific steps of normalization include: removing abnormal points, using outlier detection technology (such as statistical filtering, radius filtering, etc.) to clean up noise points or abnormal points in the point cloud; point cloud alignment, translating the geometric center of the point cloud to the global coordinate origin to eliminate the position difference of the point cloud, and using principal component analysis (PCA) to align the point cloud (that is, unify the main axis direction of the point cloud); scale normalization, scaling the point cloud data and normalizing it to the unit sphere (or unit cube). The specific operation is to calculate the maximum boundary size of the point cloud and divide the coordinates of all points by this size.

[0056] The normalized preprocessed point cloud data are mapped one by one with the corresponding tooth feature information (center of mass position and segmentation contour) to generate a standardized oral scan data set. The specific process includes: establishing a corresponding relationship to match the points in each point cloud sample with its corresponding tooth feature information (center of mass and segmentation contour); using the annotated segmentation contour, each point in the point cloud is classified into the corresponding tooth area, and according to the center of mass position of each tooth, the adjacent points of the tooth are classified into the tooth instance corresponding to the center of mass.

[0057] Step 2: Based on the preprocessed oral scan point cloud data in the oral scan dataset, extract the rotation invariant features of the point cloud data, which include local rotation invariant features and global rotation invariant features, establish a center of mass prediction model, use the global rotation invariant features as the input of the center of mass prediction model, and use the corresponding tooth center of mass position as the label to train the center of mass prediction model.

[0058] The specific method for extracting rotation-invariant features from normalized preprocessed point cloud data is as follows: using the coordinates of the points in the input point cloud in 3D space, calculate the inner product of each point with the coordinates of other points, and obtain the global rotation-invariant feature F gri , where the global rotation invariant feature F gri The specific formula on which the calculation is based is:

[0059]

[0060] In the formula, F gri is the global rotation invariant feature of the i-th point in the point cloud data, X represents the point cloud block, and x i Represents the coordinates of the i-th point in the point cloud data in 3D space, where i is the index of the point in the point cloud data, i = 1, 2, ..., n, n is the total number of points in the point cloud data, X∈R 3×K , R represents a real number set, K is the number of columns of the point cloud block;

[0061] The KNN algorithm is used to find U nearest neighbor points for each point in the point cloud, and a local region is constructed for each point. In each local region, the inner product between each point and other points is calculated to obtain the local rotation invariant feature F. lri The specific calculation method is similar to the global rotation invariant feature F gri The calculation method is consistent, and U is a positive integer.

[0062] Based on the global rotation invariant features, a centroid prediction model is established, in which the specific steps of obtaining the final predicted centroid position include:

[0063] First, the extracted global rotation invariant features are downsampled and feature dimension reduced through several self-attention mechanism modules. The self-attention mechanism module will give priority to sampling the farthest features. The specific method is as follows: randomly select an initial point s1 as the first sampling point, add it to the sampling set S, and initialize a distance array D, in which each element D[O] records point s O The minimum distance to the nearest sampling point, the initial value is infinite; iterate M times, and select sampling points according to the following steps: for each unsampled point s O , calculate the Euclidean distance d from the nearest point in the current sampling point set S, update the distance array D[O]=min(D[o],d); among the unsampled points, select the point with the largest value in the distance array D as the next sampling point, add the point to the sampling set S, and add the new sampling point s next Add the sampling set S and update the distances from all points to the sampling set. When the sampling set contains M points, the algorithm terminates and M representative points are obtained, which are recorded as representative points.

[0064] Secondly, the KNN algorithm is used to find the nearest k points in the feature space to form a neighborhood with the same number as the representative point. Then, downsampling is performed through the multi-layer perceptron layer and the maximum pooling layer to obtain deep features. Finally, the final predicted centroid position is obtained through two convolution + normalization + ReLu activation function modules.

[0065] Among them, the loss function expression of the centroid prediction model is set as:

[0066]

[0067] In the formula, is the loss function of the centroid prediction model, is a smooth loss function; where the smooth loss function The specific expression is:

[0068]

[0069] In the formula, p jis the true label value of the jth sample data, is the label value predicted by the centroid prediction model for the jth sample data, j is the index of the sample data, and G is the total number of sample data.

[0070] Step 3: Input the extracted rotation invariant features and the tooth center of mass position obtained by the center of mass prediction model into the point cloud tooth segmentation encoder to obtain globally consistent features and locally consistent features, perform feature fusion on the globally consistent features and locally consistent features, obtain comprehensive representative features, and establish a tooth segmentation prediction model based on the comprehensive representative features. Take the comprehensive representative features as input and the corresponding tooth segmentation contours in the oral scan dataset as labels to train the tooth segmentation prediction model.

[0071] Input the extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model into a point cloud tooth segmentation encoder to obtain a global consistent feature and a local consistent feature, wherein the point cloud tooth segmentation encoder includes a local branch and a global branch;

[0072] The point cloud data is subjected to different data enhancements, and the point cloud tooth segmentation encoder is pre-trained based on the enhanced data. The pre-training includes: inputting the enhanced point cloud data into the encoder, extracting the feature vectors of multiple views, and mapping the features to a specific space through a projection head. The data enhancement includes rotation and translation, and the contrast loss is calculated based on the feature vectors of multiple views. The point cloud tooth segmentation encoder is adjusted by the contrast loss to minimize the contrast loss. The contrast loss function of the point cloud tooth segmentation encoder is expressed as:

[0073]

[0074] Where, L cvq is the contrast loss function of the point cloud tooth segmentation encoder, s(.) represents the cosine similarity function, τ represents the temperature coefficient, B represents the batch size, and They respectively represent the projection vectors of the qth point cloud data in the same batch obtained through the point cloud tooth segmentation encoder and the projection head, T1 and T2 respectively represent the data enhancement for translation and rotation of the data, q is the index of the point cloud data in the same batch, a is the index of the point cloud data in all batches of point cloud data after the two data enhancements, a=1,2,…,2B-1, q=1,2,…,B-1.

[0075] The point cloud tooth segmentation encoder consists of two branches, including a local branch and a global branch. The input received by the local branch is the concatenation data of the local rotation invariant feature and the predicted tooth centroid. Then, by cascading multiple edge convolutions, the local consistency feature is extracted. Finally, the outputs of multiple edge convolutions are concatenated in the feature dimension to obtain the local consistency feature F. L , use softmax to replace the last max operation of edge convolution. Compared with the max operation, using softmax for feature aggregation can process the features in the neighborhood more smoothly, avoiding the problem of focusing on only the most significant feature and ignoring other valuable information. Then the outputs of multiple edge convolutions are concatenated in the feature dimension through concatenation to obtain local consistency features.

[0076] Among them, the formula of edge convolution is expressed as follows:

[0077] e il =h Θ (x i ,x l -x i )

[0078]

[0079] In the formula, e il It represents the feature obtained by combining the local rotation invariant feature corresponding to the lth connected point on the connected edge of the i-th point cloud data point and the tooth centroid feature data, x i represents the feature after the local rotation invariant feature of the i-th point is spliced ​​with the tooth centroid feature data, x l represents the feature obtained by combining the local rotation invariant feature corresponding to the lth connected point with the tooth centroid feature data, h Θ represents a nonlinear function with a set of learnable parameters Θ, represents the local features of the ith point after updating through edge convolution, represents the neighbor point set of point i, including all points connected to point i by edges, l is the index of the point connected to point i by edges, e il It represents the feature obtained by concatenating the local rotation invariant feature corresponding to the lth connected point of the i-th point cloud data point and the tooth centroid feature data;

[0080] Global rotation invariant feature F gri Input to the global branch, model the global information through multiple mamba modules, use the space filling curve to convert the unstructured point cloud into a regular sequence, and then extract the global information of the point cloud through multiple cascaded mamba modules to obtain the global consistent feature F G .

[0081] The obtained global consistent features and local consistent features are fused to obtain comprehensive representative features, where the formula for calculating the comprehensive representative features is:

[0082] F=w G F G +w L F L

[0083] In the formula, F is the comprehensive representative feature, w G and w L are the weight coefficients of the global consistent features and the local consistent features, F G and F L are the global consistent features and the local consistent features, respectively, where the weight coefficient w of the global consistent feature G The calculation is based on the formula:

[0084]

[0085] In the formula, F gri and F lri represents the global rotation invariant features and local rotation invariant features of the i-th point in the point cloud data, where w L With w G The calculation method is the same.

[0086] The global information is modeled through multiple mamba modules, and the space filling curve is used to convert the unstructured point cloud into a regular sequence. The global information of the point cloud is extracted through multiple cascaded mamba modules, where the calculation formula of the mamba module is:

[0087] F′ L-1 =LN(F L-1 )

[0088] F′ L =σ(DW(Linear(F′ L-1 )))

[0089] F″ L =σ(Linear(F′ L-1 ))

[0090] F L =Linear(SelectiveSSM(F′ L )⊙F″ L )+F L-1

[0091] In the formula, F L-1 represents the global rotation invariant feature input of the L-1th layer of the mamba module, F′ L-1 Indicates the input feature F of the L-1 layerL-1 The result after layer normalization, LN means layer normalization operation, DW means depth separable convolution, Linear means linear transformation, σ is SiLU activation function, F′ L Indicates F′ L-1 The global feature after depthwise separable convolution and linear transformation, F″ L Indicates F′ L-1 The global features after direct linear transformation and activation operations, SelectiveSSM is a selective state space model, and ⊙ represents element-by-element multiplication.

[0092] Step 4: Obtain the oral scan point cloud data to be segmented, preprocess it, and obtain the target comprehensive representative features through tooth centroid prediction, rotation invariant feature extraction and feature fusion. Input the target comprehensive representative features into the trained tooth segmentation prediction model to obtain the tooth segmentation contour corresponding to the oral scan point cloud data to be segmented, and complete the tooth segmentation.

[0093] In the same manner as above, the target comprehensive representative features are obtained, and teeth segmentation is performed on the oral scan point cloud data to be segmented based on the target comprehensive representative features.

[0094] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0095] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A point cloud tooth segmentation method based on rotation invariant features, characterized in that: The specific steps include: Acquire oral scan point cloud data with known tooth feature information, perform normalization preprocessing on the point cloud data, and map the normalized preprocessed oral scan point cloud data with the corresponding tooth feature information one by one to generate an oral scan data set, wherein the tooth feature information includes the centroid position of each tooth and the tooth segmentation contour; Based on the oral scan point cloud data preprocessed in the oral scan data set, the rotation invariant features of the point cloud data are extracted, and the rotation invariant features include local rotation invariant features and global rotation invariant features. A centroid prediction model is established, and the global rotation invariant features are used as input of the centroid prediction model. The corresponding tooth centroid position is used as a label to train the centroid prediction model. The extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model are input into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, and the obtained global consistent features and local consistent features are subjected to feature fusion to obtain comprehensive representative features. Based on the obtained comprehensive representative features, a tooth segmentation prediction model is established, and the comprehensive representative features are used as input, and the corresponding tooth segmentation contours in the oral scan dataset are used as labels to train the tooth segmentation prediction model; The oral scan point cloud data to be segmented is obtained, and after preprocessing, the target comprehensive representative features are obtained through tooth centroid prediction, rotation invariant feature extraction and feature fusion. The target comprehensive representative features are input into the trained tooth segmentation prediction model to obtain the tooth segmentation contour corresponding to the oral scan point cloud data to be segmented, and the tooth segmentation is completed.

2. The method for tooth segmentation based on point cloud based on rotation invariant features according to claim 1, characterized in that: The specific method for extracting rotation-invariant features from normalized preprocessed point cloud data is as follows: using the coordinates of the points in the input point cloud in 3D space, calculate the inner product of each point with the coordinates of other points, and obtain the global rotation-invariant feature F gri , where the global rotation invariant feature F gri The specific formula on which the calculation is based is: In the formula, F gri is the global rotation invariant feature of the i-th point in the point cloud data, X represents the point cloud block, and x i Represents the coordinates of the i-th point in the point cloud data in 3D space, where i is the index of the point in the point cloud data, i = 1, 2, ..., n, n is the total number of points in the point cloud data, X∈R 3×K , R represents a real number set, K is the number of columns of the point cloud block; Use the KNN algorithm to find U nearest neighbor points for each point in the point cloud, build a local region for each point, and calculate the inner product between each point and other points in each local region to obtain the local rotation invariant feature F. lri The specific calculation method is similar to the global rotation invariant feature F gri The calculation method is consistent, and U is a positive integer.

3. The method for tooth segmentation based on point cloud based on rotation invariant features according to claim 2, characterized in that: The loss function expression of the centroid prediction model is set as: In the formula, is the loss function of the centroid prediction model, is a smooth loss function; where the smooth loss function The specific expression is: In the formula, p j is the true label value of the jth sample data, is the label value predicted by the centroid prediction model for the jth sample data, j is the index of the sample data, and G is the total number of sample data.

4. The method for tooth segmentation based on point cloud based on rotation invariant features according to claim 3, characterized in that: Input the extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model into a point cloud tooth segmentation encoder to obtain a global consistent feature and a local consistent feature, wherein the point cloud tooth segmentation encoder includes a local branch and a global branch; The point cloud data is subjected to different data enhancements, and the point cloud tooth segmentation encoder is pre-trained based on the enhanced data. The pre-training includes: inputting the enhanced point cloud data into the encoder, extracting the feature vectors of multiple views, and mapping the features to a specific space through a projection head. The data enhancement includes rotation and translation, and the contrast loss is calculated based on the feature vectors of multiple views. The point cloud tooth segmentation encoder is adjusted by the contrast loss to minimize the contrast loss. The contrast loss function of the point cloud tooth segmentation encoder is expressed as: Where, L cvq is the contrast loss function of the point cloud tooth segmentation encoder, s(.) represents the cosine similarity function, τ represents the temperature coefficient, B represents the batch size, and They respectively represent the projection vectors of the qth point cloud data in the same batch obtained through the point cloud tooth segmentation encoder and the projection head, T1 and T2 respectively represent the data enhancement for translation and rotation of the data, q is the index of the point cloud data in the same batch, a is the index of the point cloud data in all batches of point cloud data after the two data enhancements, a=1,2,…,2B-1, q=1,2,…,B-1.

5. The method for tooth segmentation based on point cloud based on rotation invariant features according to claim 2, characterized in that: The specific method for obtaining global consistent features and local consistent features is: The point cloud tooth segmentation encoder consists of two branches, including a local branch and a global branch. The input received by the local branch is the concatenation data of the local rotation invariant feature and the predicted tooth centroid. Then, by cascading multiple edge convolutions, the local consistency feature is extracted. Finally, the outputs of multiple edge convolutions are concatenated in the feature dimension to obtain the local consistency feature F. L , where the edge convolution formula is as follows: e il =h Θ (x i ,x l -x i ) In the formula, e il It represents the feature obtained by combining the local rotation invariant feature corresponding to the lth connected point on the connected edge of the i-th point cloud data point and the tooth centroid feature data, x i represents the feature after the local rotation invariant feature of the i-th point is spliced ​​with the tooth centroid feature data, x l represents the feature obtained by combining the local rotation invariant feature corresponding to the lth connected point with the tooth centroid feature data, h Θ represents a nonlinear function with a set of learnable parameters Θ, represents the local features of the ith point after updating through edge convolution, represents the neighbor point set of point i, including all points connected to point i by edges, l is the index of the point connected to point i by edges, e il It represents the feature obtained by concatenating the local rotation invariant feature corresponding to the lth connected point of the i-th point cloud data point and the tooth centroid feature data; Global rotation invariant feature F gri Input to the global branch, model the global information through multiple mamba modules, use the space filling curve to convert the unstructured point cloud into a regular sequence, and then extract the global information of the point cloud through multiple cascaded mamba modules to obtain the global consistent feature F G .

6. The method for tooth segmentation based on point cloud based on rotation invariant features according to claim 5, characterized in that: The obtained global consistent features and local consistent features are fused to obtain comprehensive representative features, where the formula for calculating the comprehensive representative features is: F=w G F G +w L F L In the formula, F is the comprehensive representative feature, w G and w L are the weight coefficients of the global consistent features and the local consistent features, F G and F L are the global consistent features and the local consistent features, respectively, where the weight coefficient w of the global consistent feature G The calculation is based on the formula: In the formula, F gri and F lri represents the global rotation invariant features and local rotation invariant features of the i-th point in the point cloud data, where w L With w G The calculation method is the same.

7. The method for tooth segmentation based on point cloud based on rotation invariant features according to claim 6, characterized in that: The global information is modeled through multiple mamba modules, and the space filling curve is used to convert the unstructured point cloud into a regular sequence. The global information of the point cloud is extracted through multiple cascaded mamba modules, where the calculation formula of the mamba module is: F′ L-1 =LN(F L-1 ) F′ L =σ(DW(Linear(F′ L-1 ))) F′ L =σ(Linear(F′ L-1 )) F L =Linear(SelectiveSSM(F′ L )⊙F″ L )+F L-1 In the formula, F L-1 represents the global rotation invariant feature input of the L-1th layer of the mamba module, F′ L-1 Indicates the input feature F of the L-1 layer L-1 The result after layer normalization, LN means layer normalization operation, DW means depth separable convolution, Linear means linear transformation, σ is SiLU activation function, F′ L Indicates F′ L-1 The global feature after depth-wise separable convolution and linear transformation, F′ L Indicates F′ L-1 The global features after direct linear transformation and activation operations, SelectiveSSM is a selective state space model, and ⊙ represents element-by-element multiplication.

Images