A point cloud tooth segmentation method based on rotation invariant features

Abstract

The application discloses a point cloud tooth segmentation method based on a rotation invariant feature, and relates to the technical field of oral medical treatment. The method comprises the following steps: firstly, obtaining a plurality of oral scanning point cloud data with known tooth feature information and performing normalization processing, mapping the data one by one with tooth centroid positions and segmentation contours to generate an oral scanning data set; secondly, extracting local and global rotation invariant features of the point cloud, training a centroid prediction model based on the global features, taking the point cloud data as input and the tooth centroid position as a label; then, inputting the rotation invariant features and the predicted centroid position into a segmentation encoder to generate global and local consistent features, obtaining comprehensive representative features through feature fusion, training a tooth segmentation prediction model based on the comprehensive representative features, and taking the tooth segmentation contour as a label; and finally, performing feature extraction and prediction on the point cloud data to be segmented, outputting the tooth segmentation contour, realizing accurate segmentation, and having the characteristics of strong robustness and rotation invariance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of oral medical treatment, in particular to a point cloud tooth segmentation method based on rotation-invariant features. BACKGROUND

[0002] Computer-aided design (CAD) has been widely applied in modern oral medicine (dental medicine) with a wide range of applications, from orthodontic diagnosis to preoperative simulation, in which the 3D tooth model of the patient is usually obtained by scanning and processed through the operation of specific tasks. All these applications share a ubiquitous procedure, namely tooth segmentation, which aims to accurately segment each tooth in the 3D tooth model. Precise tooth segmentation results can provide orthodontists with detailed tooth position and morphology information, assisting in developing more scientific and reasonable treatment plans; therefore, exploring an efficient, fully automatic, and accurate point cloud tooth segmentation method can help doctors clearly understand the anatomical structure of the teeth, plan the surgical path in advance, effectively reduce the risk of surgery, and improve the success rate of treatment and patient satisfaction.

[0003] Most previous tooth segmentation methods either experiment on datasets with well-aligned data; while the actual collected tooth point cloud data inevitably has a rotation phenomenon due to factors such as the posture difference of the patient during the scanning process, the position and angle change of the scanning device, etc., at this time, if a large amount of data still needs to be aligned, it is obviously tedious and time-consuming; or through data enhancement methods, so that the network sees various angle data during training, but this not only increases the demand for data storage, but also significantly prolongs the training time of the model. Rotating and enhancing multiple angles of large-scale point cloud data will occupy a large amount of memory and CPU, GPU resources, making the training process become more slow, and the effect is also poor.

[0004] In view of this problem, in recent years, researchers have tried to enhance the robustness of the model by extracting rotation-invariant features (Rotation-Invariant Representation, RIR). Rotation-invariant features can effectively eliminate the interference of rotation on the segmentation result by encoding the stable geometric and topological relationships in the point cloud. However, most current point cloud tooth segmentation methods based on rotation-invariant features only rely on local features, ignoring the influence of global geometric structure on tooth segmentation. In addition, existing methods lack accurate prediction ability for tooth centroid position, which further limits the accuracy and usability of segmentation.

[0005] In the prior art, the disclosure number CN113344950A discloses a CBCT image tooth segmentation method combining deep learning and point cloud semantics, including the following steps: step 1, based on a deep learning segmentation model, such as a 3D segmentation network or a 2D segmentation network, tooth region segmentation is performed to extract the tooth region; step 2, the extracted tooth region is three-dimensionally reconstructed into a dentition grid data using a surface rendering method; step 3, point cloud feature data of the grid data is extracted, and a point cloud instance segmentation deep learning network is used for instance segmentation based on point cloud semantics to obtain tooth instances of the grid data; step 4, according to the coordinate correspondence information, the tooth corresponding region of the grid data is mapped to the CBCT to obtain the CBCT tooth instance. However, the scheme does not involve a rotation invariant feature extraction mechanism. If the point cloud generated by the CBCT is rotated or the dentition posture in the data set is greatly transformed, the semantic segmentation network (such as based on PointNet++ or other point cloud segmentation networks) may be difficult to effectively learn the features across the rotation domain. This makes the segmentation performance of the scheme easily affected by the directionality change of the input data, thereby reducing the generalization ability of the model, and thus reducing the accuracy and effectiveness of the tooth segmentation result.

[0006] The above information disclosed in the background section is only intended to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute the prior art known to those of ordinary skill in the art. SUMMARY

[0007] The purpose of the present application is to provide a point cloud tooth segmentation method based on rotation invariant features to solve the problems raised in the background.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0009] A point cloud tooth segmentation method based on rotation invariant features, the specific steps comprising:

[0010] Obtain a plurality of oral scan point cloud data of known tooth feature information, normalize the point cloud data, and map the normalized oral scan point cloud data to the corresponding tooth feature information one by one to generate an oral scan data set, wherein the tooth feature information includes the centroid position and tooth segmentation contour of each tooth;

[0011] Based on the pre-processed oral scan point cloud data in the oral scan data set, extract the rotation invariant features of the point cloud data, including local rotation invariant features and global rotation invariant features, establish a centroid prediction model, use the global rotation invariant features as the input of the centroid prediction model, and use the corresponding tooth centroid position as the label to train the centroid prediction model;

[0012] The extracted rotation-invariant features and tooth centroid position obtained by the centroid prediction model are input into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, the global consistent features and the local consistent features are fused to obtain comprehensive representative features, and based on the comprehensive representative features, a tooth segmentation prediction model is established, the comprehensive representative features are taken as input, and the tooth segmentation contour corresponding to the mouth scan data set is taken as a label, and the tooth segmentation prediction model is trained.

[0013] The mouth scan point cloud data to be segmented is obtained, preprocessed, and after tooth centroid prediction, rotation-invariant feature extraction and feature fusion, target comprehensive representative features are obtained, the target comprehensive representative features are input into the tooth segmentation prediction model which is completed training, and the tooth segmentation contour corresponding to the mouth scan point cloud data to be segmented is obtained, and the tooth segmentation is completed.

[0014] Further, the specific method for extracting rotation-invariant features from the normalized and preprocessed point cloud data is as follows: the coordinates of the points in the input point cloud in the 3D space are used to calculate the inner product of each point with other points, and global rotation-invariant features F gri are obtained. gri The specific formula for calculation is as follows:

[0015]

[0016] In the formula, X gri is the global rotation-invariant feature of the i-th point in the point cloud data, X represents a point cloud block, x i represents the coordinates of the i-th point in the point cloud data in the 3D space, wherein i is the index of the point in the point cloud data, i=1, 2, …, n, n is the total number of points in the point cloud data, X∈R 3×K , R represents a set of real numbers, and K is the number of columns of the point cloud block.

[0017] U nearest neighbor points are found for each point in the point cloud by using the KNN algorithm, and a local region is constructed for each point. In each local region, the inner product between each point and other points is calculated to obtain local rotation-invariant features F lri , and the specific calculation method is consistent with that of the global rotation-invariant features F gri , and U is a positive integer.

[0018] Further, the loss function expression of the centroid prediction model is set as:

[0019]

[0020] In the formula, L is the loss function of the centroid prediction model, a smoothing loss function; wherein the smoothing loss function The specific expression of the smoothing loss function is as follows:

[0021]

[0022] In the formula, p j is a true label value of the jth sample data, is a label value predicted by the centroid prediction model for the jth sample data, j is an index of the sample data, and G is a total number of sample data.

[0023] Further, the extracted rotation-invariant features and the tooth centroid positions obtained by the centroid prediction model are input into a point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, wherein the point cloud tooth segmentation encoder includes a local branch and a global branch.

[0024] Different data augmentations are performed on the point cloud data, and the point cloud tooth segmentation encoder is pre-trained based on the augmented data. The pre-training includes: inputting the augmented point cloud data into the encoder, extracting a multi-view feature vector, and mapping the features to a specific space through a projection head. The data augmentations include rotation and translation. A contrastive loss is calculated based on the multi-view feature vector, and the point cloud tooth segmentation encoder is adjusted through the contrastive loss to minimize the contrastive loss. The expression of the contrastive loss function of the point cloud tooth segmentation encoder is as follows:

[0025]

[0026] In the formula, L cvq is the contrastive loss function of the point cloud tooth segmentation encoder, s(.) represents a cosine similarity function, τ represents a temperature coefficient, B represents a batch size, and respectively represent projection vectors obtained by the point cloud tooth segmentation encoder and the projection head for the qth point cloud data in the same batch, T1 and T2 respectively represent data augmentations of translation and rotation, q is an index of the point cloud data in the same batch, a is an index of the point cloud data in all batches of the two kinds of data augmentations, a = 1, 2, …, 2B-1, and q = 1, 2, …, B-1.

[0027] Further, the specific method for obtaining the global consistent features and the local consistent features is as follows:

[0028] The point cloud tooth segmentation encoder is specifically composed of two branches, including a local branch and a global branch. The local branch receives input of spliced data of the local rotation-invariant features and the predicted tooth centroids, then extracts local consistent features through cascading multiple edge convolutions, and finally splices outputs of the multiple edge convolutions in the feature dimension to obtain the local consistent features FL where the formula of edge convolution is as follows:

[0029] e il = h Θ (x i , x l - x i )

[0030]

[0031] wherein e il represents the feature of the i-th point cloud data point after splicing the local rotation invariant feature corresponding to the l-th connected point on the connected edge and the tooth centroid feature data, x i represents the feature of the i-th point after splicing the local rotation invariant feature and the tooth centroid feature data, x l represents the feature of the l-th connected point after splicing the local rotation invariant feature and the tooth centroid feature data, h Θ represents a nonlinear function with a set of learnable parameters Θ, represents the local feature of the i-th point after updating by edge convolution, represents the neighbor point set of point i, containing all points connected to point i by edges, and l is the index of the point connected to point i by edges, e il represents the feature of the l-th connected point corresponding to the i-th point cloud data point after splicing the local rotation invariant feature and the tooth centroid feature data;

[0032] Global rotation invariant feature F gri is input into the global branch, and global information is modeled through multiple mamba modules. The unstructured point cloud is converted into a regular sequence by using a space-filling curve, and the global information of the point cloud is extracted through multiple cascaded mamba modules to obtain the global consistent feature F G .

[0033] Further, the global consistent feature and the local consistent feature obtained are fused to obtain a comprehensive representative feature, wherein the formula according to which the comprehensive representative feature is calculated is as follows:

[0034] F = w G F G +w L F L

[0035] wherein F is the comprehensive representative feature, w G and w L are weight coefficients of the global consistent feature and the local consistent feature respectively, F G and F L are the global consistent feature and the local consistent feature respectively, and the weight coefficient of the global consistent feature is wG The formula for calculation is:

[0036]

[0037] In the formula, F gri and F lri represent the global rotation invariant feature and the local rotation invariant feature of the i-th point in the point cloud data, wherein w L is calculated in the same way as w G .

[0038] Further, the global information is modeled by multiple mamba modules, the unstructured point cloud is converted into a regular sequence by using a space-filling curve, and the global information of the point cloud is extracted by multiple cascaded mamba modules, wherein the calculation formula of the mamba module is:

[0039] F′ L-1 = LN(F L-1 )

[0040] F′ L = sigma(DW(Linear(F′ L-1 )))

[0041] F″ L = sigma(Linear(F′ L-1 ))

[0042] F L = Linear(SelectiveSSM(F′ L ) circled dot F″ L ) + F L-1

[0043] In the formula, F L-1 represents the global rotation invariant feature input of the L-1 layer of the mamba module, F′ L-1 represents the result of layer normalization on the input feature F L-1 of the l-1 layer, LN represents the layer normalization operation, DW represents the depth separable convolution, Linear represents the linear transformation, sigma is the SiLU activation function, F′ L represents the global feature after the depth separable convolution and the linear transformation on F′ L-1 , F″ L represents the global feature after the linear transformation and the activation operation on F′ L-1 , SelectiveSSM is the selective state space model, and circled dot represents element-wise multiplication.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] Firstly, by normalizing the point cloud data, the data is ensured to have a uniform scale and geometric center in the preliminary processing stage, thus providing a reliable foundation for subsequent feature extraction and modeling. Normalization processing can effectively reduce the scale and position differences in point cloud data caused by scanning equipment or patient posture, providing support for standardized training of the model. Secondly, the method introduces rotation-invariant features in the tooth segmentation task, including the extraction of global rotation-invariant features and local rotation-invariant features. Compared with methods that rely only on local geometric features, this scheme extracts global features to explicitly encode the overall geometric structure of the point cloud, thereby enhancing the model's ability to handle complex tooth arrangements. In addition, local rotation-invariant features can capture the detailed structure and neighborhood information of teeth, and the combination of global consistent features and local consistent features through feature fusion further improves the segmentation accuracy and model robustness. Furthermore, by establishing a centroid prediction model, the method can accurately predict the centroid position of each tooth in the oral point cloud. The tooth centroid position, as key geometric information for segmentation, can effectively guide the regional division of teeth and reduce the confusion problem of the model at the tooth boundary. This process significantly improves the contextual consistency of the point cloud segmentation result, enabling the model to more accurately separate adjacent teeth. Finally, through feature fusion and the construction of comprehensive representative features, the global information and local information are fully combined, and a deep learning-based tooth segmentation prediction model is established. The model takes comprehensive representative features as input and can efficiently generate tooth segmentation contours. The method shows stronger robustness and higher segmentation accuracy in dealing with challenges such as rotation, point cloud noise, and sparsity. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The overall method flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0047] To make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with specific embodiments.

[0048] It should be noted that the technical terms or scientific terms used in the present application should be understood as the general meaning understood by those skilled in the art to which the present application belongs, unless otherwise defined. The "first", "second" and similar words used in the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected" or "connected" and similar words are not limited to physical or mechanical connection, but can include electrical connection, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent the relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] Embodiments:

[0050] Please refer to Figure 1 The present application provides a technical solution:

[0051] A point cloud tooth segmentation method based on rotation invariant features, the specific steps comprising:

[0052] Step 1: Obtain a plurality of oral scanning point cloud data of known tooth feature information, pre-process the point cloud data, map the pre-processed oral scanning point cloud data to the corresponding tooth feature information one by one, generate an oral scanning data set, and the tooth feature information includes the centroid position of each tooth and the tooth segmentation contour.

[0053] Use an oral scanner or similar device (such as an intraoral scanner, CBCT reconstruction) to obtain three-dimensional point cloud data of the patient's oral cavity. The point cloud data is usually in an unstructured three-dimensional coordinate representation form, and contains the surface geometric information of teeth, gums and surrounding tissues.

[0054] Use professional dental software (such as MeshLab or Geomagic) to manually label the segmentation contour of the tooth (i.e. the boundary of each tooth), and accurately position the centroid position of each tooth. The centroid can be obtained by geometric calculation, or can be determined based on prior knowledge of tooth morphology.

[0055] The point cloud data is normalized to unify the scale, position and direction of the data, and to ensure the stability of feature extraction and model training. The specific steps of normalization processing include: removing abnormal points, using outlier detection technology (such as statistical filtering, radius filtering, etc.) to clean up noise points or abnormal points in the point cloud; point cloud alignment, moving the geometric center of the point cloud to the global coordinate origin to eliminate the position difference of the point cloud, using principal component analysis (PCA) to align the direction of the point cloud (i.e. unify the main axis direction of the point cloud); scale normalization, scaling the point cloud data to normalize it into a unit sphere (or unit cube), and the specific operation is to calculate the maximum boundary size of the point cloud, and divide the coordinates of all points by the size.

[0056] The normalized and preprocessed point cloud data is mapped one-to-one with the corresponding tooth feature information (centroid position and segmentation contour) to generate a standardized oral scan data set. The specific process includes: establishing a correspondence relationship, matching the points in each point cloud sample with their corresponding tooth feature information (centroid and segmentation contour); using the labeled segmentation contour, classifying each point in the point cloud into the corresponding tooth region, and according to the centroid position of each tooth, classifying the adjacent points of the tooth into the tooth instance corresponding to the centroid.

[0057] Step 2: Based on the preprocessed oral scan point cloud data in the oral scan data set, extract the rotation invariant features of the point cloud data, including local rotation invariant features and global rotation invariant features, establish a centroid prediction model, use the global rotation invariant features as the input of the centroid prediction model, and use the corresponding tooth centroid position as the label, train the centroid prediction model.

[0058] The specific method for extracting rotation invariant features from normalized and preprocessed point cloud data is: using the coordinates of points in the input point cloud in 3-dimensional space, calculating the inner product of each point with other point coordinates to obtain global rotation invariant features F gri , where the global rotation invariant features F gri The specific formula for calculation is:

[0059]

[0060] In the formula, F gri is the global rotation invariant feature of the i-th point in the point cloud data, X represents the point cloud block, x i represents the coordinates of the i-th point in the point cloud data in 3-dimensional space, where i is the index of the point in the point cloud data, i = 1, 2, …, n, n is the total number of points in the point cloud data, X ∈ R 3×K , R represents the set of real numbers, and K is the column number of the point cloud block;

[0061] Use the KNN algorithm to find U nearest neighbor points for each point in the point cloud, build a local region for each point, and calculate the inner product between each point and other points in each local region to obtain the local rotation invariant feature F. lri The specific calculation method is the same as the global rotation invariant feature F gri The calculation method is consistent, and U is a positive integer.

[0062] Based on the global rotation invariant feature, a centroid prediction model is established. The specific steps to obtain the final predicted centroid position include:

[0063] First, the extracted global rotation invariant features are downsampled and feature dimension reduced through several self-attention mechanism modules. The self-attention mechanism module will give priority to sampling the farthest features. The specific method is as follows: randomly select an initial point s1 as the first sampling point, add it to the sampling set S, and initialize a distance array D, where each element D[O] records the point s O The minimum distance to the nearest sampling point, the initial value is infinite; iterate M times, and select the sampling point according to the following steps: for each unsampled point s O , calculate the Euclidean distance d from it to the nearest point in the current sampling point set S, update the distance array D[O]=min(D[o],d); among the unsampled points, select the point with the largest value in the distance array D as the next sampling point, add the point to the sampling set S, and add the new sampling point s next Add the sampling set S and update the distances of all points to the sampling set. When the sampling set contains M points, the algorithm terminates and M representative points are obtained, which are recorded as representative points.

[0064] Secondly, the KNN algorithm is used to find the k nearest points in the feature space to form a neighborhood with the same number of representative points. Then, downsampling is performed through a multi-layer perceptron layer and a maximum pooling layer to obtain deep features. Finally, two convolution + normalization + ReLu activation function modules are used to obtain the final predicted centroid position.

[0065] Among them, the loss function expression of the centroid prediction model is set as:

[0066]

[0067] Where, is the loss function of the centroid prediction model, is a smooth loss function; where the smooth loss function The specific expression is:

[0068]

[0069] Where p jis a true label value of the jth sample data, is a label value predicted by the centroid prediction model for the jth sample data, j is an index of the sample data, and G is a total number of sample data.

[0070] Step 3: input the extracted rotation-invariant features and the tooth centroid positions obtained by the centroid prediction model into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, perform feature fusion on the obtained global consistent features and local consistent features to obtain comprehensive representative features, and based on the obtained comprehensive representative features, establish a tooth segmentation prediction model, take the comprehensive representative features as input, and take the corresponding tooth segmentation contours in the mouth scan data set as labels to train the tooth segmentation prediction model.

[0071] input the extracted rotation-invariant features and the tooth centroid positions obtained by the centroid prediction model into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, wherein the point cloud tooth segmentation encoder includes a local branch and a global branch;

[0072] perform different data augmentations on the point cloud data, and pre-train the point cloud tooth segmentation encoder based on the augmented data, the pre-training including: inputting the augmented point cloud data into the encoder, extracting a feature vector of a multi-view, mapping the feature to a specific space through a projection head, the data augmentation including rotation and translation, calculating a contrastive loss based on the feature vector of the multi-view, and adjusting the point cloud tooth segmentation encoder through the contrastive loss to minimize the contrastive loss, wherein an expression of the contrastive loss function of the point cloud tooth segmentation encoder is:

[0073]

[0074] In the formula, L cvq is a contrastive loss function of the point cloud tooth segmentation encoder, s(.) represents a cosine similarity function, τ represents a temperature coefficient, B represents a batch size, and respectively represent projection vectors of the qth point cloud data in the same batch through the point cloud tooth segmentation encoder and the projection head, T1 and T2 respectively represent data augmentations of translation and rotation on the data, q is an index of the point cloud data in the same batch, a is an index of the point cloud data in all batches of the point cloud data after the two data augmentations, a = 1, 2, …, 2B-1, and q = 1, 2, …, B-1.

[0075] The point cloud tooth segmentation encoder is specifically composed of two branches, including a local branch and a global branch. The input received by the local branch is the spliced data of the local rotation invariant feature and the predicted tooth centroid, and then a plurality of edge convolutions are cascaded to extract the local consistent feature. The outputs of the plurality of edge convolutions are spliced in the feature dimension to obtain the local consistent feature F L The last max operation of the edge convolution is replaced by a softmax. Compared with the max operation, the use of the softmax for feature aggregation can more smoothly process the features in the neighborhood, avoiding the problem of only focusing on a single most significant feature and ignoring other valuable information. Then the outputs of the plurality of edge convolutions are spliced in the feature dimension by concatenate to obtain the local consistent feature.

[0076] The formula of the edge convolution is as follows:

[0077] e il =h Θ (x i ,x l -x i )

[0078]

[0079] In the formula, e il represents the local rotation invariant feature corresponding to the lth connected point on the connected edge of the ith point cloud data point spliced with the tooth centroid feature data, x i represents the feature after splicing the local rotation invariant feature of the ith point with the tooth centroid feature data, x l represents the feature after splicing the local rotation invariant feature corresponding to the lth connected point with the tooth centroid feature data, h Θ represents a nonlinear function with a set of learnable parameters Θ, represents the local feature of the ith point after updating by the edge convolution, represents the neighbor point set of point i, including all points connected to point i by an edge, and l is the index of the point connected to point i by an edge, e il represents the feature after splicing the local rotation invariant feature corresponding to the lth connected point of the ith point cloud data point with the tooth centroid feature data;

[0080] The global rotation invariant feature F gri is input into the global branch, and a plurality of mamba modules are used to model the global information. The unstructured point cloud is converted into a regular sequence by using a space-filling curve, and the global information of the point cloud is extracted by a plurality of cascaded mamba modules to obtain the global consistent feature F G .

[0081] The global consistent feature and the local consistent feature are fused to obtain a comprehensive representative feature, wherein a formula for calculating the comprehensive representative feature is:

[0082] F = w G F G +w L F L

[0083] In the formula, F is the comprehensive representative feature, w G and w L are weight coefficients of the global consistent feature and the local consistent feature respectively, F G and F L are the global consistent feature and the local consistent feature respectively, wherein the weight coefficient w G of the global consistent feature is calculated according to the formula:

[0084]

[0085] In the formula, F gri and F lri represent the global rotation invariant feature and the local rotation invariant feature of the i-th point in the point cloud data, wherein w L and w G are calculated in the same way.

[0086] The global information is modeled by multiple mamba modules, the unstructured point cloud is converted into a regular sequence by using a space-filling curve, and the global information of the point cloud is extracted by multiple cascaded mamba modules, wherein the calculation formula of the mamba module is:

[0087] F′ L-1 = LN(F L-1 )

[0088] F′ L = sigma (DW (Linear (F′ L-1 )))

[0089] F″ L = sigma (Linear (F′ L-1 ))

[0090] F L = Linear (SelectiveSSM (F′ L ) o F″ L ) + F L-1

[0091] In the formula, F L-1 represents the global rotation invariant feature input of the L-1 layer of the mamba module, F′ L-1 represents the input feature FL-1 The result after layer normalization is denoted as LN, DW represents a depth separable convolution, Linear represents a linear transformation, and σ is a SiLU activation function. L F' = LN(DW(F)) L-1 F" represents the global feature after the depth separable convolution and linear transformation processing. L F' = LN(DW(F)) L-1 The global feature after directly performing the linear transformation and activation operation is denoted as F'', and SelectiveSSM is a selective state space model.

[0092] Step 4: Obtain the oral scan point cloud data to be segmented, and after preprocessing, tooth centroid prediction, rotation invariant feature extraction, and feature fusion, obtain the target comprehensive representative feature. The target comprehensive representative feature is input into the tooth segmentation prediction model that has been trained to obtain the tooth segmentation contour corresponding to the oral scan point cloud data to be segmented, and tooth segmentation is completed.

[0093] In the same way as described above, the target comprehensive representative feature is obtained, and the tooth segmentation is performed on the oral scan point cloud data to be segmented based on the target comprehensive representative feature.

[0094] The above formulas are all dimensionless numerical calculations, and the formulas are obtained by software simulation of a large amount of data to obtain a formula closest to the actual situation. The preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0095] The above embodiments can be realized wholly or partially by software, hardware, firmware, or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on the specific application and design constraints of the technical solutions.

[0096] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, which can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiments according to actual needs.

[0097] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A point cloud tooth segmentation method based on a rotation invariant feature, characterized in that, The specific steps include: Obtain a plurality of oral scanning point cloud data of known tooth feature information, normalize and preprocess the point cloud data, map the normalized and preprocessed oral scanning point cloud data to the corresponding tooth feature information one by one, generate an oral scanning data set, and the tooth feature information includes the centroid position of each tooth and the tooth segmentation contour; Based on the preprocessed oral scanning point cloud data in the oral scanning data set, the rotation invariant features of the point cloud data are extracted, the rotation invariant features include local rotation invariant features and global rotation invariant features, a centroid prediction model is established, the global rotation invariant features are taken as the input of the centroid prediction model, the corresponding tooth centroid position is taken as the label, and the centroid prediction model is trained; The extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model are input into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, the obtained global consistent features and local consistent features are fused to obtain comprehensive representative features, based on the obtained comprehensive representative features, a tooth segmentation prediction model is established, the comprehensive representative features are taken as the input, and the corresponding tooth segmentation contour in the oral scanning data set is taken as the label, and the tooth segmentation prediction model is trained; The point cloud tooth segmentation encoder is specifically composed of two branches, including a local branch and a global branch, the input received by the local branch is the splicing data of the local rotation invariant features and the predicted tooth centroid, a plurality of edge convolutions are cascaded to extract local consistent features, and the outputs of the plurality of edge convolutions are spliced in the feature dimension to obtain the local consistent features; The global rotation invariant features are input into the global branch, the global information is modeled through a plurality of mamba modules, the unstructured point cloud is converted into a regular sequence by using a space-filling curve, and the global information of the point cloud is extracted by a plurality of cascaded mamba modules to obtain the global consistent features; Obtain the oral scanning point cloud data to be segmented, after preprocessing, tooth centroid prediction, rotation invariant feature extraction and feature fusion, obtain the target comprehensive representative features, input the target comprehensive representative features into the tooth segmentation prediction model which is completed training, obtain the tooth segmentation contour corresponding to the oral scanning point cloud data to be segmented, and complete the tooth segmentation.

2. The point cloud tooth segmentation method based on the rotation invariant feature according to claim 1, characterized in that: The specific method for performing rotation invariant feature extraction on the normalized preprocessed point cloud data is: using the coordinates of the points in the input point cloud in a 3-dimensional space, calculating the inner product of each point with the coordinates of other points to obtain a global rotation invariant feature F gri where the global rotation invariant feature F gri The specific formula for calculation is: In the formula, F gri is the global rotation invariant feature of the i-th point in the point cloud data, X represents the point cloud block, x i represents the coordinates of the i-th point in the point cloud data in a 3-dimensional space, wherein i is the index of the point in the point cloud data, i=1, 2, …, n, n is the total number of points in the point cloud data, X∈R 3×K , R represents a set of real numbers, and K is the column number of the point cloud block; The KNN algorithm is used to find U nearest neighbor points for each point in the point cloud, and a local region is constructed for each point. In each local region, the inner product between each point and other points is calculated to obtain a local rotation invariant feature F lri The specific calculation method is consistent with that of the global rotation invariant feature F gri , and U is a positive integer.

3. The point cloud tooth segmentation method based on the rotation invariant feature according to claim 2, characterized in that: The loss function expression of the centroid prediction model is set as: In the formula, is a loss function of the centroid prediction model, is a smoothing loss function; wherein the smoothing loss function The specific expression is: In the formula, p j is a true label value of the jth sample data, is a label value predicted by the centroid prediction model for the jth sample data, j is an index of the sample data, and G is the total number of sample data.

4. The point cloud tooth segmentation method based on the rotation invariant feature according to claim 3, characterized in that: The extracted rotation invariant features and the tooth centroid position obtained by the centroid prediction model are input into the point cloud tooth segmentation encoder to obtain global consistent features and local consistent features, and the point cloud tooth segmentation encoder includes a local branch and a global branch; Different data augmentations are performed on the point cloud data, the point cloud tooth segmentation encoder is pre-trained based on the augmented data, and the pre-training includes: inputting the augmented point cloud data into the encoder, extracting a plurality of view feature vectors, mapping the features to a specific space through a projection head, the data augmentation includes rotation and translation, calculating a contrast loss based on the plurality of view feature vectors, adjusting the point cloud tooth segmentation encoder through the contrast loss to minimize the contrast loss, and the expression of the contrast loss function of the point cloud tooth segmentation encoder is: In the formula, L cvq is the contrastive loss function of the point cloud tooth segmentation encoder, s(.) represents the cosine similarity function, τ represents the temperature coefficient, B represents the batch size, and respectively represent the projection vectors obtained by the point cloud tooth segmentation encoder and the projection head for the qth point cloud data in the same batch, T1 and T2 respectively represent data augmentation by translation and rotation on the data, q is the index of the point cloud data in the same batch, a is the index of the point cloud data in the entire batch of the two kinds of data after data augmentation, a = 1, 2, …, 2B-1, q = 1, 2, …, B-1.

5. The method of claim 2, wherein: The local branch connection is through cascading multiple edge convolutions to extract local consistency features, and finally the outputs of the multiple edge convolutions are spliced in the feature dimension to obtain the local consistency features F L wherein the formula of the edge convolution is as follows: e il = h Θ (x i , x l - x i ) where e il represents the concatenated feature of the local rotation invariant feature of the l-th connected point corresponding to the i-th point cloud data point and the tooth centroid feature data, x i represents the concatenated feature of the local rotation invariant feature of the i-th point and the tooth centroid feature data, x l represents the concatenated feature of the local rotation invariant feature of the l-th connected point and the tooth centroid feature data, h Θ represents a nonlinear function with a set of learnable parameters Θ, represents the updated local feature of the i-th point after edge convolution, represents the neighbor point set of point i, containing all points connected to point i by edges, l is the index of the point connected to point i by edges, e il represents the concatenated feature of the local rotation invariant feature of the l-th connected point corresponding to the i-th point cloud data point and the tooth centroid feature data.

6. The point cloud tooth segmentation method based on the rotation invariant feature according to claim 5, characterized in that: The global consistent features and the local consistent features are fused to obtain comprehensive representative features, and a formula for calculating the comprehensive representative features is: F = w G F G +w L F L In the formula, F is a comprehensive representative feature, w G and w L are weight coefficients of global consistent features and local consistent features respectively, F G and F L are global consistent features and local consistent features respectively, wherein the weight coefficient w G of the global consistent features is calculated according to the formula: In the formula, F gri and F lri represent the global rotation invariant feature and the local rotation invariant feature of the i-th point in the point cloud data, wherein w L is calculated in the same way as w G .

7. The point cloud tooth segmentation method based on the rotation invariant feature according to claim 6, characterized in that: Global information is modeled through multiple mamba modules, and unstructured point clouds are converted into regular sequences through space-filling curves, and global information of the point clouds is extracted through multiple cascaded mamba modules, and a calculation formula of the mamba module is: F′ L-1 = LN(F L-1 ) F′ L = σ(DW(Linear(F′ L-1 ))) F" L = σ(Linear(F' L-1 )) F L = Linear(SelectiveSSM(F' L ) ⊙ F" L ) + F L-1 ​ In the formula, F L-1 represents the global rotation invariant feature input of the L-1 layer of the mamba module, F′ L-1 represents the input feature F L-1 after layer normalization, LN represents the layer normalization operation, DW represents the depth separable convolution, Linear represents the linear transformation, and σ is the SiLU activation function, F′ L represents the global feature after the depth separable convolution and linear transformation processing on F′ L-1 F″ L represents the global feature after the depth separable convolution and linear transformation processing on F′ L-1 F″ represents the global feature after the direct linear transformation and activation operation, SelectiveSSM is the selective state space model, and ⊙ represents the element-wise multiplication.

Images