An automatic segmentation method and system of a three-dimensional digital dental model based on superpoints

Abstract

The present application belongs to the field of computer vision, and relates to a kind of automatic segmentation method and system of three-dimensional digital dental model based on super point.The method comprises the following steps: obtaining a three-dimensional dental model by three-dimensional oral scanning through a three-dimensional oral scanner;inputting the three-dimensional dental model into a three-dimensional dental model automatic segmentation network based on super point to obtain the FDI label of each point of the three-dimensional dental model;according to the FDI label of the three-dimensional dental model, and according to the label color mapping table, each point of the three-dimensional dental model is colored and visualized, and the final segmentation model is output.The present application can efficiently capture the local geometric information and global semantic relationship of teeth, improve the segmentation accuracy, reduce memory consumption, and support higher precision tooth model segmentation tasks.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to an automatic segmentation method and system for a three-dimensional digital dental model based on superpoints. Background Technology

[0002] A three-dimensional digital dental model refers to a three-dimensional dental mesh model of a patient obtained using a three-dimensional scanning device in a computer-aided orthodontic system. The segmentation of the dental model follows the FDI World Dental Federationnumbering system (ISO 2016) standard, dividing the dental mesh model into gingiva and different categories of teeth, with the categories based on the FDI standard. Specifically, each point on the tooth model is assigned a label, which, according to FDI, can be divided into 33 categories (1+16+16), namely gingiva + 16 categories for upper teeth + 16 categories for lower teeth. The FDI World Dental Federationnumbering system (ISO 2016) standard is mainly used for tooth marking and identification, employing a two-digit numbering system to divide teeth into four quadrants, with left and right sides for both the upper and lower jaws.

[0003] Hyper-points (or super-points) in point cloud segmentation and mesh segmentation methods typically represent a fixed number of auxiliary points that are independent of the point cloud or network. They are usually learned through an additional feature learning module, and their features are used to assist the segmentation network to improve the segmentation effect.

[0004] The attention mechanism is a method that mimics the human visual and cognitive system, allowing neural networks to focus their attention on relevant parts of the input data. By introducing the attention mechanism, neural networks can automatically learn and selectively focus on important information in the input, thereby improving the model's performance and generalization ability.

[0005] Accurate 3D dental model segmentation is crucial for computer-aided orthodontic treatment. However, due to subtle shape differences between adjacent teeth, indistinct gingival boundaries, and the complexity of orthodontic patient samples, existing automatic 3D dental model segmentation methods cannot produce accurate segmentation results. Existing deep learning-based automatic 3D dental model segmentation methods can be divided into 2D and 3D segmentation methods. 2D segmentation methods typically render the 3D dental model from different perspectives onto a 2D image plane, then use these 2D images to train an image segmentation network, and map the image segmentation results back to the 3D model to obtain the final 3D model segmentation result. 3D segmentation methods typically train the segmentation network directly on the 3D model, using the point cloud or mesh of the 3D model as direct input to train a 3D tooth segmentation network with the original 3D information as input.

[0006] 2D segmentation methods, by converting 3D model segmentation into image segmentation, lack the inherent geometric information of the 3D model itself, resulting in poor segmentation performance. Existing 3D segmentation methods mainly focus on capturing the geometric features of the tooth model using local self-attention mechanisms, ignoring the global semantic information of the tooth model (such as the relative relationships, spatial distribution, and category distribution of teeth), leading to poor segmentation results and a tendency to classify teeth with similar appearances into the same category. Even though some methods are now starting to use global self-attention mechanisms to capture the global features of the tooth model, these methods require calculating attention across all points of the tooth model, resulting in high memory consumption and failing to support high-precision tooth model segmentation. Summary of the Invention

[0007] To address the aforementioned problems, this invention provides an automatic segmentation method and system for three-dimensional digital dental models based on superpoints, which can achieve efficient and accurate automatic segmentation of three-dimensional dental models.

[0008] The technical solution adopted in this invention is as follows:

[0009] An automatic segmentation method for a three-dimensional digital dental model based on superpoints includes the following steps:

[0010] A three-dimensional dental model is obtained by performing a three-dimensional oral scan using a three-dimensional oral scanner.

[0011] The three-dimensional dental model is input into the superpoint-based automatic segmentation network for the three-dimensional dental model to obtain the FDI label of each point in the three-dimensional dental model.

[0012] Based on the FDI labels of the 3D dental model, and according to the label color mapping table, each point of the 3D dental model is colored and visualized, and the final segmentation model is output.

[0013] Furthermore, the automatic segmentation network for the three-dimensional dental model based on superpoints includes:

[0014] Initialization module: First, the input 3D dental model is downsampled to obtain N*6 dimensional initial feature information, where N represents the number of point clouds and 6 dimensions include the 3D coordinates and 3D normal vectors of the points. Then, the initial feature information is sent to the encoder group; at the same time, the superpoint feature vector is initialized and sent to the corresponding encoder group.

[0015] Encoder group: It consists of several tooth center sensing encoders, each tooth center sensing encoder is connected in series, and the last tooth center sensing encoder is connected to the first decoder;

[0016] Decoder group: consists of several decoders, each decoder is connected in series, and each decoder is connected to the encoder symmetrically in the encoder group through skip connections;

[0017] The post-processing module consists of a semantic branch, an offset branch, and a fine-tuning module. It first receives the point cloud features output from the decoder group, and obtains the semantic probability distribution and offset vector of each point in the point cloud from the corresponding tooth center through the semantic branch and the offset branch, respectively. Then, the semantic probability distribution and offset vector of each point from the corresponding tooth center are input into the fine-tuning module to obtain the final segmentation result.

[0018] Furthermore, the tooth center perception encoder includes a downsampling module, a multilayer perceptron module, a superpoint double cross-attention module, a point cloud double cross-attention module, a grouping module, and a point cloud local attention module; the downsampling module of the i-th layer tooth center perception encoder downsamples the point cloud features of the (i-1)-th layer using the farthest point sampling method, and the multilayer perceptron module reduces the feature dimension from Ci to Ci. i-1 Upgraded to C i After processing, the initial point cloud features of layer i are obtained. Then, the initial point cloud features of layer i, together with the superpoint features output from layer (i-1) and the initial superpoint features of layer i, are input into the superpoint dual cross-attention module to obtain the superpoint features of layer i. At the same time, the initial point cloud features of layer i are input into the grouping module, where ball query is used to obtain the K nearest neighbor features of each point. Then, the superpoint features of layer i, the initial point cloud features of layer i, and the grouping features obtained from the grouping module are input into the point cloud dual cross-attention module. The processed point cloud features are then input into L combination modules composed of the grouping module and the point cloud local attention module to obtain the final point cloud encoding features of layer i.

[0019] Furthermore, the superpoint dual cross-attention module includes a superpoint inter-layer cross-attention module and a superpoint global cross-attention module. The superpoint inter-layer cross-attention module at layer i uses the superpoint features output from layer (i-1) as the key and value vectors for channel cross-attention, and the initialized initial superpoint features of layer i as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the superpoint inter-layer cross-attention module at layer i. The superpoint global cross-attention module at layer i uses the initial point cloud features of layer i as the key and value vectors for channel cross-attention, and the initialized initial superpoint features of layer i as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the superpoint global cross-attention module at layer i. The output of the superpoint inter-layer cross-attention module at layer i and the output of the superpoint global cross-attention module at layer i are added together and output to obtain the output of the superpoint dual cross-attention module at layer i, thereby obtaining the superpoint features of layer i.

[0020] Furthermore, the point cloud dual cross-attention module includes a point cloud global cross-attention module and a point cloud local cross-attention module. The point cloud global cross-attention module at layer i uses the superpoint features of layer i as the key vector and value vector for channel cross-attention, and the initial point cloud features of layer i as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the point cloud global cross-attention module at layer i. The point cloud local cross-attention module at layer i uses the K nearest neighbor features of each point obtained after the grouping layer as the key vector and value vector for channel cross-attention, and the initial point cloud features of layer i as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the point cloud local cross-attention module at layer i. The output of the point cloud global cross-attention module at layer i and the output of the point cloud local cross-attention module at layer i are added together and output to obtain the output of the point cloud dual cross-attention module at layer i.

[0021] Furthermore, each decoder in the decoder group consists of an upsampling layer and several multilayer perceptron layers. In the i-th layer decoder, the output of the i-1 layer decoder is first linearly interpolated through the upsampling layer. The upsampled result is then connected to the output of the encoder symmetrical to that layer decoder. The connected features are then decoded through several multilayer perceptrons to obtain the final decoding result of the i-th layer. The fine-tuning module in the post-processing module first adds the offset of the offset branch to the original spatial coordinates of the point cloud to obtain the offset point cloud. The offset point cloud is then clustered using a clustering algorithm. Based on the clustered labels, tooth instance labels are obtained. The point set of each tooth instance is found based on the tooth instance labels. Based on the semantic branch results of the point set, the semantic category of the current tooth instance is obtained using a majority voting method.

[0022] Furthermore, the automatic segmentation network for the 3D dental model based on superpoints is constrained and optimized using the following loss function:

[0023]

[0024] in, Using the cross-entropy loss function to measure the semantic branch prediction results With truth value y i The differences between them; Combined chamfer distance loss and smoothing absolute error loss To measure the results of the offset branch prediction The difference between the true value O and the actual value O; The smoothed absolute error loss is used to measure the difference between the 3D coordinates p of the superpoint and the 3D coordinates g of the tooth centroid, where i,j represent the point set indices, and MP(p,g) is the maximum matching pair calculated using the Hungarian algorithm on the point sets p,g; ω cd ,ω l1 ,ω seg ,ω tcp These are the weights of the loss function.

[0025] An automatic segmentation system for a three-dimensional digital dental model based on superpoints, comprising:

[0026] The 3D dental model acquisition module is used to obtain a 3D dental model by performing a 3D oral scan using a 3D oral scanner.

[0027] The automatic segmentation module for 3D dental and jaw models is used to input 3D dental and jaw models into a superpoint-based automatic segmentation network for 3D dental and jaw models to obtain the FDI label of each point in the 3D dental and jaw model.

[0028] The visualization module is used to perform color visualization processing on each point of the 3D dental model based on the FDI labels of the 3D dental model and according to the label color mapping table, and output the final segmentation model.

[0029] The beneficial effects of this invention are as follows:

[0030] This invention proposes a 3D tooth model segmentation network based on superpoints, which can efficiently capture local geometric information and global semantic relationships of teeth. Compared with 2D segmentation methods, this invention can capture geometric information; compared with segmentation methods that utilize local attention, this invention can model global semantic relationships, improving segmentation accuracy; compared with methods that utilize global self-attention, the method of this invention only uses a set of superpoints to calculate attention to model global information, reducing memory consumption and supporting higher-precision tooth model segmentation tasks. Attached Figure Description

[0031] Figure 1 It is the overall flowchart of the automatic segmentation method for three-dimensional digital dental models of the present invention.

[0032] Figure 2 It is a schematic diagram of the processing process of the automatic segmentation method for three-dimensional digital dental models of the present invention.

[0033] Figure 3 It is the segmentation effect diagram realized by using the method of the present invention. Specific embodiments

[0034] To make the above objects, features and advantages of the present invention more obvious and understandable, the following further describes the present invention in detail through specific embodiments and drawings.

[0035] The key points and inventive points of the present invention include the following:

[0036] 1) A method for automatically segmenting three-dimensional dental models by simultaneously using local attention and global attention: For the three-dimensional model automatic segmentation technology, among the methods taking the three-dimensional model itself as the input, usually the downsampling technology is first used to sample a fixed number of point clouds from the original model as the input, and then a neural network module is trained to extract the point-by-point features of the point clouds. Based on the extracted point-by-point features, finally a multi-layer perceptron layer is used to obtain the final point-by-point labels, so as to realize the automatic segmentation of the three-dimensional model. Generally, when training the neural network module (point-by-point feature extraction network), usually only local attention or only global attention is used, while the present invention works to simultaneously use the two kinds of attention to realize the automatic segmentation of the dental model, using local attention to realize the perception of the geometric features of the dental model and using global attention to model the global information of the dental model.

[0037] 2) A strategy for efficiently injecting global attention into the point-by-point feature extraction network by using superpoints: The learning strategy of global attention in the present invention is different from the general working learning strategy, with less memory consumption and higher efficiency. When generally implementing global attention, usually a self-attention mechanism with a computational complexity of O(N 2 ) level is adopted. Specifically, assuming the number of input points is N, when generally calculating global attention, each point will calculate the attention weights with these N points. The method of the present invention is to introduce a set of additional auxiliary points (superpoints), the number of which is fixed at M, and in the method of the present invention, M = 17 is set. First, the present invention enables the superpoints to have global features through a superpoint double cross-attention module. Then the present invention enables N points to calculate the attention weights with M points respectively, so as to realize the injection of global attention in the point cloud double cross-attention module. The computational complexity of this strategy is O(MN), where M << N, so the method of the present invention has less memory consumption and higher computational efficiency.

[0038] 3) A super-point dual cross-attention module: When training the point cloud dual cross-attention module, a super-point dual cross-attention module needs to be trained simultaneously for super-point feature learning. Specifically, this super-point dual cross-attention module includes two attention mechanisms. The first attention mechanism is super-point inter-layer cross-attention, which is the attention between super-points and upper-layer super-points, used to achieve consistency in super-point feature learning. The second attention mechanism is super-point global cross-attention, which is the attention between super-points and the input point cloud, used to aggregate information from the input point cloud, thereby achieving global information aggregation.

[0039] 4) A superpoint location constraint strategy: In order to enable the superpoint features to have global semantic information between teeth (such as relative relationship, spatial distribution, and category distribution of teeth), the superpoint features are first obtained through the superpoint feature learning module. Then, the similarity between the superpoint features and the input point cloud is used, and the spatial coordinates of the superpoint are calculated by interpolation. Then, the loss is calculated by comparing the superpoint features with the ground truth of the tooth center point, thereby constraining the superpoint position to the tooth center point. By constraining the spatial position of the superpoint through this loss, the superpoint feature learning module can be indirectly constrained, enabling it to effectively learn the global semantic information of the input point cloud.

[0040] like Figure 1 As shown, the automatic segmentation method for a three-dimensional digital dental model based on superpoints of the present invention includes the following steps:

[0041] Step 1: First, a 3D oral cavity scan is performed using a 3D oral scanner to obtain a 3D model of the teeth and jaws;

[0042] Step 2: Input the 3D dental model into the superpoint-based automatic segmentation network for the 3D dental model, and finally obtain the FDI label of each point on the 3D dental model;

[0043] Step 3: Based on the FDI labels of the 3D dental model, and according to the label color mapping table, perform color visualization processing on each point of the 3D dental model, and output the final segmentation model.

[0044] The label color map can be a manually defined color map, where each FDI label corresponds to a color, and different labels have different colors. The coloring and visualization process involves predicting the FDI label of each point on the dental model using this method, then querying the manually defined label color map to obtain its corresponding color, then coloring the point cloud, and finally rendering the point cloud using a visualization library.

[0045] like Figure 2 As shown, the automatic segmentation network for three-dimensional dental and jaw models based on superpoints proposed in this invention mainly includes: an initialization module, an encoder group, a decoder group, and a post-processing module.

[0046] 1) Initialization Module: The initialization module first downsamples the input 3D dental model to obtain N*6 dimensional initial feature information, where N represents the number of point clouds, and 6 dimensions include the 3D coordinates and 3D normal vectors of the points. Then, the initial feature information is sent to the encoder group. At the same time, the superpoint feature vectors are initialized and sent to the corresponding encoder group.

[0047] 2) Encoder group: The encoder group consists of several tooth center sensing encoders, each tooth center sensing encoder is connected in series, and the last tooth center sensing encoder is connected to the first decoder.

[0048] 3) Decoder group: The decoder group consists of several decoders, each decoder is connected in series, and each decoder is connected to the encoder symmetrically in the encoder group through skip connect.

[0049] 4) Post-processing module: The post-processing module mainly consists of a semantic branch, an offset branch, and a fine-tuning module. First, it receives the point cloud features output from the decoder group. Then, it obtains the semantic probability distribution and the offset vector from the corresponding tooth center for each point in the point cloud through the semantic branch and the offset branch, respectively. Their dimensions can be represented as N×17 and N×3. Next, the semantic probability distribution and the offset vector from the corresponding tooth center for each point are input into the fine-tuning module to obtain the FDI label for each point on the 3D dental model.

[0050] Specifically:

[0051] 1.1) The downsampling module in the initialization module and the downsampling module in the whole invention all use the farthest point sampling algorithm. The only difference between the different downsampling modules is the downsampling rate.

[0052] 2.1) The tooth center perception encoder consists of a downsampling module, a multilayer perceptron module, a superpoint double cross-attention module, a point cloud double cross-attention module, a grouping module, and a point cloud local attention module. The downsampling module of the i-th layer tooth center perception encoder downsamples the point cloud features of the (i-1)-th layer using the farthest point sampling method, with a downsampling rate of 4. Furthermore, the multilayer perceptron module reduces the feature dimension from Ci to Ci. i-1 Upgraded to C i C i-1 This represents the feature dimension of the (i-1)th layer. After processing, the initial point cloud features of the i-th layer are obtained, and their dimension can be represented as N. i ×C i Next, the initial point cloud features of layer i, combined with the superpoint features output from layer (i-1) and the initial superpoint features of layer i, are input into the superpoint dual cross-attention module to obtain the superpoint features of layer i, whose feature dimension can be represented as M×C.i Where M = 17. Simultaneously, the initial point cloud features of the i-th layer are input to the grouping module. In the grouping module, a ball query is used to obtain the K nearest neighbor features for each point, and the feature dimension can be represented as N. i ×K×C i Next, the superpoint features of layer i, the initial point cloud features of layer i, and the grouped features obtained from the grouping module are input into the point cloud dual cross-attention module. Then, the processed point cloud features are input into L combination modules composed of the grouping module and the point cloud local attention module to obtain the final point cloud encoding features of layer i, with a dimension of N×C. i .

[0053] 2.1.1) Channel cross-attention. All cross-attention in this invention refers to channel cross-attention.

[0054] 2.1.2) The superpoint dual cross attention module consists of the superpoint inter-layer cross attention module and the superpoint global cross attention module.

[0055] In the superpoint interlayer cross-attention module of layer i, the superpoint features output by layer i-1 are used as the key vector and value vector of channel cross-attention, and the initial superpoint features of layer i are used as the query vector of channel cross-attention. Then, channel cross-attention is performed to obtain the output result of the superpoint interlayer cross-attention module of layer i.

[0056] In the i-th layer, the superpoint global cross-attention module uses the initial point cloud features of the i-th layer as the key vector and value vector of the channel cross-attention, and the initialized initial superpoint features of the i-th layer as the query vector of the channel cross-attention. Then, channel cross-attention is performed to obtain the output result of the superpoint global cross-attention module of the i-th layer.

[0057] The output of the superpoint interlayer cross-attention module in layer i and the output of the superpoint global cross-attention module in layer i are added together and output to obtain the output of the superpoint double cross-attention module in layer i, thus obtaining the superpoint feature of layer i. Its feature dimension can be represented as M×C. i .

[0058] 2.1.3) The point cloud dual cross attention module consists of a point cloud global cross attention module and a point cloud local cross attention module.

[0059] In the i-th layer, the global cross-attention module of the point cloud uses the superpoint features of the i-th layer as the key vector and value vector of the channel cross-attention, and the initial point cloud features of the i-th layer as the query vector of the channel cross-attention. Then, the channel cross-attention is performed to obtain the output of the global cross-attention of the i-th layer of the point cloud.

[0060] In the point cloud local cross-attention module at layer i, the K nearest neighbor features of each point obtained after grouping layers are used as the key vector and value vector of channel cross-attention, and the initial point cloud features of layer i are used as the query vector of channel cross-attention. Then, channel cross-attention is performed to obtain the output of the point cloud local cross-attention module at layer i.

[0061] The output of the global cross-attention module of the i-th layer point cloud and the output of the local cross-attention module of the i-th layer point cloud are added together and output to obtain the output of the dual cross-attention module of the i-th layer point cloud, whose feature dimension can be represented as N. i ×C i .

[0062] exist Figure 2 In the superpoint double cross attention module and the point cloud double cross attention module, R represents matrix subtraction, the normalized exponential function refers to SoftMax, "⊙" represents matrix multiplication, pooling refers to Maxpooling, and "⊕" represents matrix addition.

[0063] 3.1) Decoder: Each decoder consists of an upsampling layer and several multilayer perceptron layers. In the i-th layer decoder, the output of the (i-1)-th layer decoder is first linearly interpolated through the upsampling layer with an upsampling rate of 4. The upsampled result is then concatenated with the output of the encoder symmetrical to that layer decoder. The concatenated feature is then decoded through several multilayer perceptrons to obtain the final decoding result of the i-th layer.

[0064] 4.1) The post-processing module mainly consists of three modules: semantic branching, offset branching, and fine-tuning. In the fine-tuning module, the original spatial coordinates of the point cloud are first added to the offset of the offset branch to obtain the offset point cloud. The DBSCAN clustering algorithm is used to cluster the offset point cloud, and the tooth instance labels are obtained based on the clustered labels. The point set of each tooth instance is found based on the tooth instance labels. Based on the semantic branching results of the point set, the semantic category (i.e., FDI label) of the current tooth instance is obtained using the majority voting method.

[0065] 5.1) Superpoint Spatial Location Calculation: In the i-th layer encoder, the superpoint feature H is obtained through the superpoint double cross-attention module. i Then, the three-dimensional spatial position Y of the superpoint is obtained by interpolating it with the initial point cloud of the i-th layer. i The process can be represented as follows:

[0066]

[0067] Where X i This represents the three-dimensional spatial position of the point cloud in the i-th encoder layer. This represents the transpose of the feature matrix of the point cloud in the i-th layer encoder.

[0068] 5.2) Loss Function: To constrain and optimize network learning, this invention primarily utilizes the following loss function to constrain the network learning process:

[0069]

[0070]

[0071]

[0072]

[0073] in Using the cross-entropy loss function to measure the semantic branch prediction results With truth value y i The differences between them;

[0074] in The combined chamfer distance loss and smoothing absolute error loss (i.e., in the formula) To measure the results of the offset branch prediction The difference between the true value O and the actual value O;

[0075] in The smoothed absolute error loss is used to measure the difference between the 3D coordinates p of the superpoint and the 3D coordinates g of the tooth centroid. Here, i,j represent the indexes of the point set, and MP(p,g) is the maximum matching pair calculated on the point set p,g using the Hungarian algorithm.

[0076] The weights of the loss function are defined as follows: ω cd =0.15,ω l1 =0.03,ω seg =1,ω tcp =0.1.

[0077] like Figure 3 The segmentation results shown demonstrate that, compared to existing segmentation methods such as PointTransformer, SpoTr, DBGANet, PointNet++, and DGCNN, the method of this invention (referred to as TCATSeg) can produce more accurate segmentation results.

[0078] Key Point 1: The present invention provides a method for automatic segmentation of a three-dimensional dental model that simultaneously utilizes local attention and global attention. Its beneficial technical effect is that simultaneously utilizing local attention and global attention can improve the segmentation network's ability to capture local geometric details of teeth and perceive global semantic information, which can significantly improve the segmentation effect compared to using local attention or global attention alone.

[0079] Key Point 2: This invention provides a strategy for efficiently injecting global attention into a point-by-point feature extraction network using superpoints. Its beneficial technical effect is that injecting global attention using superpoints effectively solves the problem of high memory consumption in global attention, reducing computational efficiency from O(N^2) to O(N^2). 2 ) decreased to O(MN).

[0080] Key Point 3: The beneficial technical effect of the super-point feature learning module of the present invention is that the module can achieve consistency in super-point feature learning and efficiently aggregate features from the input point cloud, thereby realizing the aggregation of global information.

[0081] Key Point 4: The beneficial technical effect of the superpoint position constraint strategy of the present invention is that by constraining the superpoint to the tooth centroid, the ability of superpoint features to aggregate global information can be further improved, because the distribution of the tooth centroid can naturally represent the relative relationship and semantic distribution between teeth.

[0082] In other embodiments of the present invention, other suitable methods may be used to replace the downsampling algorithm in 1.1) or modify the downsampling rate, the cross-attention module in 2.1.1) may be replaced with other types of cross-attention modules, branches may be added or removed in 4.1), the DBSCAN clustering algorithm may be replaced with other clustering algorithms in 4.1), and loss functions may be added or removed or the weight values ​​of the loss function may be modified in 5.2).

[0083] Another embodiment of the present invention provides an automatic segmentation system for a three-dimensional digital dental model based on superpoints, comprising:

[0084] The 3D dental model acquisition module is used to obtain a 3D dental model by performing a 3D oral scan using a 3D oral scanner.

[0085] The automatic segmentation module for 3D dental and jaw models is used to input 3D dental and jaw models into a superpoint-based automatic segmentation network for 3D dental and jaw models to obtain the FDI label of each point in the 3D dental and jaw model.

[0086] The visualization module is used to perform color visualization processing on each point of the 3D dental model based on the FDI labels of the 3D dental model and according to the label color mapping table, and output the final segmentation model.

[0087] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding process in the aforementioned method embodiments, and will not be repeated here.

[0088] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0089] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0090] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. An automatic segmentation method for a three-dimensional digital dental model based on superpoints, characterized in that, Includes the following steps: A three-dimensional dental model is obtained by performing a three-dimensional oral scan using a three-dimensional oral scanner. The three-dimensional dental model is input into the superpoint-based automatic segmentation network for the three-dimensional dental model to obtain the FDI label of each point in the three-dimensional dental model. Based on the FDI labels of the 3D dental model, and according to the label color mapping table, each point of the 3D dental model is colored and visualized, and the final segmentation model is output. The automatic segmentation network for the three-dimensional dental model based on superpoints includes: Initialization module: First, the input 3D dental model is downsampled to obtain N*6 dimensional initial feature information, where N represents the number of point clouds and 6 dimensions include the 3D coordinates and 3D normal vectors of the points. Then, the initial feature information is sent to the encoder group; at the same time, the superpoint feature vector is initialized and sent to the corresponding encoder group. Encoder group: It consists of several tooth center sensing encoders, each tooth center sensing encoder is connected in series, and the last tooth center sensing encoder is connected to the first decoder; Decoder group: consists of several decoders, each decoder is connected in series, and each decoder is connected to the encoder symmetrically in the encoder group through skip connections; The post-processing module consists of a semantic branch, an offset branch, and a fine-tuning module. It first receives the point cloud features output from the decoder group, and obtains the semantic probability distribution and offset vector of each point in the point cloud from the corresponding tooth center through the semantic branch and the offset branch, respectively. Then, the semantic probability distribution and offset vector of each point from the corresponding tooth center are input into the fine-tuning module to obtain the final segmentation result.

2. The method according to claim 1, characterized in that, The tooth center perception encoder includes a downsampling module, a multilayer perceptron module, a superpoint double cross-attention module, a point cloud double cross-attention module, a grouping module, and a point cloud local attention module. The downsampling module of the i-th layer tooth center perception encoder downsamples the point cloud features of the (i-1)-th layer using the farthest point sampling method, and the multilayer perceptron module reduces the feature dimension from Ci to Ci. i-1 Upgraded to C i After processing, the initial point cloud features of layer i are obtained. Then, the initial point cloud features of layer i, together with the superpoint features output from layer (i-1) and the initial superpoint features of layer i, are input into the superpoint dual cross-attention module to obtain the superpoint features of layer i. At the same time, the initial point cloud features of layer i are input into the grouping module, where ball query is used to obtain the K nearest neighbor features of each point. Then, the superpoint features of layer i, the initial point cloud features of layer i, and the grouping features obtained from the grouping module are input into the point cloud dual cross-attention module. The processed point cloud features are then input into L combination modules composed of the grouping module and the point cloud local attention module to obtain the final point cloud encoding features of layer i.

3. The method according to claim 2, characterized in that, The superpoint dual cross-attention module includes a superpoint inter-layer cross-attention module and a superpoint global cross-attention module. In the superpoint inter-layer cross-attention module at layer i, the superpoint features output from layer (i-1) are used as the key and value vectors for channel cross-attention, and the initialized initial superpoint features of layer i are used as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the superpoint inter-layer cross-attention module at layer i. In the superpoint global cross-attention module at layer i, the initial point cloud features of layer i are used as the key and value vectors for channel cross-attention, and the initialized initial superpoint features of layer i are used as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the superpoint global cross-attention module at layer i. The outputs of the superpoint inter-layer cross-attention module and the superpoint global cross-attention module at layer i are added together and output to obtain the output of the superpoint dual cross-attention module at layer i, thereby obtaining the superpoint features of layer i.

4. The method according to claim 2, characterized in that, The point cloud dual cross-attention module includes a point cloud global cross-attention module and a point cloud local cross-attention module. In the i-th layer, the point cloud global cross-attention module uses the superpoint features of the i-th layer as the key and value vectors for channel cross-attention, and the initial point cloud features of the i-th layer as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the i-th layer point cloud global cross-attention module. In the i-th layer, the point cloud local cross-attention module uses the K nearest neighbor features of each point obtained after grouping as the key and value vectors for channel cross-attention, and the initial point cloud features of the i-th layer as the query vector for channel cross-attention. Then, channel cross-attention is performed to obtain the output of the i-th layer point cloud local cross-attention module. The output of the i-th layer point cloud global cross-attention module and the output of the i-th layer point cloud local cross-attention module are added together and output to obtain the output of the i-th layer point cloud dual cross-attention module.

5. The method according to claim 1, characterized in that, Each decoder in the decoder group consists of an upsampling layer and several multilayer perceptron layers. In the i-th layer decoder, the output of the i-1 layer decoder is first linearly interpolated through the upsampling layer. The upsampled result is then connected to the output of the encoder symmetrical to the decoder of this layer. The connected features are then decoded through several multilayer perceptrons to obtain the final decoding result of the i-th layer. The fine-tuning module in the post-processing module first adds the offset of the offset branch to the original spatial coordinates of the point cloud to obtain the offset point cloud. Then, it uses a clustering algorithm to cluster the offset point cloud, obtains the tooth instance label based on the clustered label, finds the point set of each tooth instance based on the tooth instance label, and obtains the semantic category of the current tooth instance based on the semantic branch result of the point set using the majority voting method.

6. The method according to claim 1, characterized in that, The following loss function is used to constrain and optimize the automatic segmentation network of the 3D dental model based on superpoints: in, Using the cross-entropy loss function to measure the semantic branch prediction results With truth value y i The differences between them; Combined chamfer distance loss and smoothing absolute error loss To measure the results of the offset branch prediction The difference between the true value O and the actual value O; The smoothed absolute error loss is used to measure the difference between the 3D coordinates p of the superpoint and the 3D coordinates g of the tooth centroid, where i,j represent the point set indices, and MP(p,g) is the maximum matching pair calculated using the Hungarian algorithm on the point sets p,g; ω cd ,ω l1 ,ω seg ,ω tcp These are the weights of the loss function.

7. An automatic segmentation system for a three-dimensional digital dental model based on superpoints, characterized in that, include: The 3D dental model acquisition module is used to obtain a 3D dental model by performing a 3D oral scan using a 3D oral scanner. The automatic segmentation module for 3D dental and jaw models is used to input 3D dental and jaw models into a superpoint-based automatic segmentation network for 3D dental and jaw models to obtain the FDI label of each point in the 3D dental and jaw model. The visualization module is used to perform color visualization processing on each point of the 3D dental model based on the FDI labels of the 3D dental model and according to the label color mapping table, and output the final segmentation model. The automatic segmentation network for the three-dimensional dental model based on superpoints includes: Initialization module: First, the input 3D dental model is downsampled to obtain N*6 dimensional initial feature information, where N represents the number of point clouds and 6 dimensions include the 3D coordinates and 3D normal vectors of the points. Then, the initial feature information is sent to the encoder group; at the same time, the superpoint feature vector is initialized and sent to the corresponding encoder group. Encoder group: It consists of several tooth center sensing encoders, each tooth center sensing encoder is connected in series, and the last tooth center sensing encoder is connected to the first decoder; Decoder group: consists of several decoders, each decoder is connected in series, and each decoder is connected to the encoder symmetrically in the encoder group through skip connections; The post-processing module consists of a semantic branch, an offset branch, and a fine-tuning module. It first receives the point cloud features output from the decoder group, and obtains the semantic probability distribution and offset vector of each point in the point cloud from the corresponding tooth center through the semantic branch and the offset branch, respectively. Then, the semantic probability distribution and offset vector of each point from the corresponding tooth center are input into the fine-tuning module to obtain the final segmentation result.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 6.

Images