Automatic segmentation method and system of three-dimensional digital tooth jaw model based on superpoint

Abstract

The invention belongs to the field of computer vision, and relates to an automatic segmentation method and system of a three-dimensional digital dental model based on super points. The method comprises the following steps: carrying out oral cavity three-dimensional scanning through a three-dimensional oral cavity scanner to obtain a three-dimensional dental model; inputting the three-dimensional tooth jaw model into a three-dimensional tooth jaw model automatic segmentation network based on super points to obtain an FDI label of each point of the three-dimensional tooth jaw model; and according to the FDI label of the three-dimensional dental model and the label color mapping table, performing color assignment visualization processing on each point of the three-dimensional dental model, and outputting to obtain a final segmentation model. According to the method, the local geometric information and the global semantic relationship of the tooth can be efficiently captured, the segmentation precision can be improved, the memory consumption can be reduced, and a tooth model segmentation task with higher precision can be supported.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and in particular relates to a method and system for automatically segmenting a three-dimensional digital dental jaw model based on superpoints. Background Art

[0002] The three-dimensional digital dental model refers to the three-dimensional dental mesh model of the patient obtained by using a three-dimensional scanning device in a computer-assisted orthodontic system. The segmentation of the dental model is based on the FDI World Dental Federationnumbering system (ISO 2016) standard, which divides the dental mesh model into gums and different types of teeth. The classification is based on the FDI standard. Specifically, each point on the tooth model is assigned a label, which can be divided into 33 categories (1+16+16) according to FDI, namely gums + 16 types of upper teeth + 16 types of lower teeth. Among them, the FDI World Dental Federationnumbering system (ISO 2016) standard is mainly used for the marking and identification of teeth. It uses a two-digit numbering method to divide the teeth into four quadrants, with the upper and lower jaws each having a left and right side.

[0003] Hyper-points (or Super-points), in point cloud segmentation and mesh segmentation methods, usually represent a fixed number of auxiliary points independent of the point cloud or network. Usually, additional feature learning modules are used to learn the features of super-points, and the features of super-points are used to assist the segmentation network to improve the segmentation effect.

[0004] The attention mechanism is a method that mimics the human visual and cognitive system, allowing neural networks to focus on relevant parts when processing input data. By introducing the attention mechanism, neural networks can automatically learn and selectively focus on important information in the input, thereby improving the performance and generalization ability of the model.

[0005] Accurate 3D dental model segmentation is crucial for computer-assisted orthodontic treatment. However, due to the subtle shape differences between adjacent teeth, unclear gingival boundaries and the complexity of orthodontic patient samples, existing 3D dental model automatic segmentation methods cannot produce accurate segmentation results. Existing deep learning-based 3D dental model automatic segmentation methods can be divided into 2D segmentation methods and 3D segmentation methods. 2D segmentation methods usually render 3D dental models from different perspectives to 2D image planes, then use these 2D images to train image segmentation networks, and map the image segmentation results back to the 3D model to obtain the final 3D model segmentation results. 3D segmentation methods usually train the segmentation network directly on the 3D model, use the point cloud or mesh of the 3D model directly as the model input, and train the 3D tooth segmentation network with the original 3D information as input.

[0006] Among them, the 2D segmentation method converts the 3D model segmentation into image segmentation, lacking the inherent geometric information of the 3D model itself, resulting in poor segmentation effect. Existing 3D segmentation methods mainly focus on using local self-attention mechanisms to capture the geometric features of tooth models, ignoring the global semantic information of tooth models (such as the relative relationship, spatial distribution, and category distribution of teeth), resulting in poor segmentation effects and easily classifying teeth with similar appearance into one category. Even though some methods are now beginning to use global self-attention mechanisms to capture the global features of tooth models, such methods need to calculate attention between all points in the tooth model, resulting in large memory consumption and cannot support high-precision tooth model segmentation. Summary of the invention

[0007] In view of the above problems, the present invention provides a method and system for automatic segmentation of a three-dimensional digital dental model based on superpoints, which can realize efficient and accurate automatic segmentation of a three-dimensional dental model.

[0008] The technical solution adopted by the present invention is as follows:

[0009] A method for automatic segmentation of a three-dimensional digital dental model based on superpoints comprises the following steps:

[0010] Perform a three-dimensional oral scan using a three-dimensional oral scanner to obtain a three-dimensional dental model;

[0011] The three-dimensional dental model is input into the three-dimensional dental model automatic segmentation network based on super points to obtain the FDI label of each point of the three-dimensional dental model;

[0012] According to the FDI label of the 3D dental model and the label color mapping table, each point of the 3D dental model is colored and visualized, and the final segmentation model is output.

[0013] Furthermore, the superpoint-based three-dimensional dental model automatic segmentation network includes:

[0014] Initialization module: First, the input 3D dental model is downsampled to obtain N*6-dimensional initial feature information, where N represents the number of point clouds, and 6 dimensions include the 3D coordinates of the points and the 3D normal vectors. Then, the initial feature information is sent to the encoder group; at the same time, the super-point feature vector is initialized and sent to the corresponding encoder group;

[0015] Encoder group: composed of a plurality of tooth center perception encoders, each of which is connected in series, and the last tooth center perception encoder is connected to the first decoder;

[0016] Decoder group: consists of several decoders, each decoder is connected in series, and each decoder is connected to the symmetrical encoder in the encoder group through a skip connection;

[0017] Post-processing module: It consists of a semantic branch, an offset branch and a fine-tuning module. It first accepts the point cloud features output by the decoder group, and obtains the semantic probability distribution of each point in the point cloud and the offset vector of the distance to the corresponding tooth center through the semantic branch and the offset branch respectively. Then, the semantic probability distribution of each point and the offset vector of the distance to the corresponding tooth center are input into the fine-tuning module to obtain the final segmentation result.

[0018] Furthermore, the tooth center perception encoder includes a downsampling module, a multi-layer perceptron module, a super-point double cross attention module, a point cloud double cross attention module, a grouping module, and a point cloud local attention module; the downsampling module of the tooth center perception encoder of the i-th layer downsamples the point cloud features of the i-1th layer by the farthest point sampling method, and converts the feature dimension from C to 1 through the multi-layer perceptron module. i-1 Raised to C i , after processing, the initial point cloud features of the i-th layer are obtained, and then the initial point cloud features of the i-th layer are combined with the super-point features output by the i-1-th layer and the initial super-point features of the i-th layer and input into the super-point double cross attention module to obtain the super-point features of the i-th layer; at the same time, the initial point cloud features of the i-th layer are input into the grouping module, and the K nearest neighbor features of each point are obtained by ball query in the grouping module; then the super-point features of the i-th layer, the initial point cloud features of the i-th layer and the grouping features obtained by the grouping module are input into the point cloud double cross attention module, and the processed point cloud features are then input into L combination modules composed of the grouping module and the point cloud local attention module to obtain the final i-th layer point cloud encoding features.

[0019] Furthermore, the super-point dual cross-attention module includes a super-point inter-layer cross-attention module and a super-point global cross-attention module; the super-point inter-layer cross-attention module at the i-th layer uses the super-point features output by the i-1th layer as the key vector and value vector of the channel cross-attention, and uses the initialized initial super-point features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output result of the super-point inter-layer cross-attention module at the i-th layer; the super-point global cross-attention module at the i-th layer uses the initial point cloud features of the i-th layer as the key vector and value vector of the channel cross-attention, and uses the initialized initial super-point features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output result of the super-point global cross-attention module at the i-th layer; the output result of the super-point inter-layer cross-attention module at the i-th layer and the output result of the super-point global cross-attention module at the i-th layer are added and output to obtain the output of the super-point dual cross-attention module at the i-th layer, thereby obtaining the super-point features at the i-th layer.

[0020] Furthermore, the point cloud dual cross-attention module includes a point cloud global cross-attention module and a point cloud local cross-attention module; the point cloud global cross-attention module at the i-th layer uses the super-point features of the i-th layer as the key vector and value vector of the channel cross-attention, and uses the initial point cloud features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output of the point cloud global cross-attention at the i-th layer; the point cloud local cross-attention module at the i-th layer uses the K nearest neighbor features of each point obtained after the grouping layer as the key vector and value vector of the channel cross-attention, and uses the initial point cloud features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output of the point cloud local cross-attention module at the i-th layer; the output result of the point cloud global cross-attention at the i-th layer and the output result of the point cloud local cross-attention module at the i-th layer are added and output to obtain the output of the point cloud dual cross-attention module at the i-th layer.

[0021] Furthermore, each decoder in the decoder group is composed of an upsampling layer and several multi-layer perceptron layers; in the i-th layer decoder, the output of the i-1 layer decoder is first linearly interpolated through the upsampling layer, the up-sampled result is connected with the output of the encoder symmetrical to the decoder of this layer, and the connected features are decoded through several multi-layer perceptrons to obtain the final decoding result of the i-th layer; the fine-tuning module in the post-processing module first adds the original spatial coordinates of the point cloud to the offset of the offset branch to obtain the offset point cloud, clusters the offset point cloud using a clustering algorithm, obtains the tooth instance label according to the clustered label, finds the point set of each tooth instance according to the tooth instance label, and obtains the semantic category of the current tooth instance according to the semantic branch result of the point set using the majority voting method.

[0022] Furthermore, the following loss function is used to constrain and optimize the superpoint-based 3D dental model automatic segmentation network:

[0023]

[0024] in, Using cross entropy loss function to measure semantic branch prediction results and the true value y i The difference between Combined chamfer distance loss and smooth absolute error loss To measure the offset branch prediction result and the difference between the true value O; The smooth absolute error loss is used to measure the difference between the three-dimensional coordinates of the superpoint p and the three-dimensional coordinates of the tooth centroid g, where i, j represent the point set subscripts, and MP(p,g) is the maximum matching pair calculated using the Hungarian algorithm on the point set p,g; ω cd ,ω l1 ,ω seg ,ω tcp is the weight of the loss function.

[0025] An automatic segmentation system for a three-dimensional digital dental model based on superpoints, comprising:

[0026] A three-dimensional dental and jaw model acquisition module is used to perform a three-dimensional oral scan using a three-dimensional oral scanner to obtain a three-dimensional dental and jaw model;

[0027] A three-dimensional dental model automatic segmentation module is used to input the three-dimensional dental model into a three-dimensional dental model automatic segmentation network based on super points to obtain an FDI label for each point of the three-dimensional dental model;

[0028] The visualization processing module is used to perform color visualization processing on each point of the three-dimensional dental model according to the FDI label of the three-dimensional dental model and the label color mapping table, and output the final segmentation model.

[0029] The beneficial effects of the present invention are as follows:

[0030] The present invention proposes a superpoint-based 3D tooth model segmentation network, which can efficiently capture the local geometric information and global semantic relationship of teeth. Compared with the 2D segmentation method, the present invention can capture geometric information; compared with the segmentation method using local attention, the present invention can model global semantic relationships and improve segmentation accuracy; compared with the method using global self-attention, the method of the present invention only uses a set of superpoints to calculate attention to model global information, which reduces memory consumption and can support higher-precision tooth model segmentation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is the overall flowchart of the automatic segmentation method for three-dimensional digital dental models of the present invention.

[0032] Figure 2 It is a schematic diagram of the processing process of the automatic segmentation method for three-dimensional digital dental models of the present invention.

[0033] Figure 3 It is the segmentation effect diagram achieved by using the method of the present invention. Specific embodiments

[0034] To make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below through specific embodiments and accompanying drawings.

[0035] The key points and inventive points of the present invention include the following:

[0036] 1) A method for automatically segmenting three-dimensional dental models by simultaneously using local attention and global attention: For three-dimensional model automatic segmentation technology, in the method where the three-dimensional model itself is used as the input, usually the downsampling technology is first used to sample a fixed number of point clouds from the original model as the input, and then a neural network module is trained to extract the point-by-point features of the point cloud. Based on the extracted point-by-point features, finally, a multi-layer perceptron layer is used to obtain the final point-by-point labels, thereby realizing the automatic segmentation of the three-dimensional model. Generally, when training the neural network module (point-by-point feature extraction network), only local attention or only global attention is usually used, while the present invention works to simultaneously use both attentions to achieve the automatic segmentation of dental models, using local attention to perceive the geometric features of dental models and using global attention to model the global information of dental models.

[0037] 2) A strategy for efficiently injecting global attention into the point-by-point feature extraction network using superpoints: The learning strategy of global attention in the present invention is different from the general working learning strategy, with less memory consumption and higher efficiency. When implementing global attention in general work, the self-attention mechanism with a computational complexity of O(N 2 ) level is usually adopted. Specifically, assuming the number of input points is N, when generally calculating global attention, each point will calculate the attention weight with these N points. The method of the present invention is to introduce a set of additional auxiliary points (superpoints), the number of which is fixed at M. In the method of the present invention, M = 17 is set. First, the present invention makes the superpoints have global features through a superpoint double cross-attention module. Then the present invention makes the N points calculate the attention weights with the M points respectively, thereby realizing the injection of global attention in the point cloud double cross-attention module. The computational complexity of this strategy is O(MN), where M << N, so the method of the present invention has less memory consumption and higher computational efficiency.

[0038] 3) A super-point dual cross attention module: When training the point cloud dual cross attention module, it is necessary to simultaneously train the super-point dual cross attention module for learning super-point features. Specifically, the super-point dual cross attention module includes two attention mechanisms. The first attention mechanism is the super-point inter-layer cross attention, which is the attention between the super-point and the upper-layer super-point, and is used to achieve consistency in super-point feature learning. The second attention mechanism is the super-point global cross attention, which is the attention between the super-point and the input point cloud, and is used to aggregate information from the input point cloud, thereby achieving global information aggregation.

[0039] 4) A super-point position constraint strategy: In order to make the super-point features have the global semantic information between teeth (such as the relative relationship, spatial distribution, and category distribution of teeth), the super-point features are first obtained through the super-point feature learning module. Then, the similarity between the super-point features and the input point cloud is used, and the spatial coordinates of the super-point are calculated by interpolation. Then, the loss is calculated by comparing it with the true value of the tooth center point to constrain the super-point position to the tooth center point. Constraining the super-point spatial position through this loss can indirectly constrain the super-point feature learning module, so that it can effectively learn the global semantic information of the input point cloud.

[0040] like Figure 1 As shown, the automatic segmentation method of the three-dimensional digital dental model based on super points of the present invention comprises the following steps:

[0041] Step 1: First, perform a 3D oral scan using a 3D oral scanner to obtain a 3D model of the jaws;

[0042] Step 2: Take the three-dimensional dental model as input and input it into the three-dimensional dental model automatic segmentation network based on super points, and finally obtain the FDI label of each point on the three-dimensional dental model;

[0043] Step 3: According to the FDI label of the three-dimensional model of the tooth and jaw, each point of the three-dimensional model of the tooth and jaw is colored and visualized according to the label color mapping table, and the final segmentation model is output.

[0044] The label color mapping table can be a manually defined color mapping table, in which each label of FDI corresponds to a color, and different labels have different colors. The coloring visualization process predicts the FDI label of each point on the dental jaw model according to the method, then queries the manually defined label color mapping table to obtain its corresponding color, then colors the point cloud, and finally renders the point cloud through the visualization library.

[0045] like Figure 2 As shown, the super-point-based three-dimensional dental model automatic segmentation network proposed in the present invention mainly includes: an initialization module, an encoder group, a decoder group and a post-processing module.

[0046] 1) Initialization module: The initialization module first downsamples the input 3D dental model to obtain N*6-dimensional initial feature information, where N represents the number of point clouds and 6 dimensions include the 3D coordinates and 3D normal vectors of the points. Then the initial feature information is sent to the encoder group. At the same time, the super-point feature vector is initialized and sent to the corresponding encoder group.

[0047] 2) Encoder group: The encoder group consists of a number of tooth center perception encoders, each of which is connected in series, and the last tooth center perception encoder is connected to the first decoder.

[0048] 3) Decoder group: The decoder group consists of several decoders, each of which is connected in series, and each decoder is connected to a symmetrical encoder in the encoder group through a skip connection.

[0049] 4) Post-processing module: The post-processing module is mainly composed of semantic branch, offset branch and fine-tuning module. First, the point cloud features output by the decoder group are received, and the semantic probability distribution of each point in the point cloud and the offset vector of the distance to the corresponding tooth center are obtained through the semantic branch and the offset branch, respectively. The dimensions can be expressed as N×17 and N×3, respectively. Then, the semantic probability distribution of each point and the offset vector of the distance to the corresponding tooth center are input into the fine-tuning module to obtain the FDI label of each point on the three-dimensional model of the jaw.

[0050] Specifically:

[0051] 1.1) The downsampling modules in the initialization module and the entire invention all adopt the farthest point sampling algorithm, and different downsampling modules only have different downsampling rates.

[0052] 2.1) The tooth center perception encoder is composed of a downsampling module, a multi-layer perceptron module, a super-point double cross attention module, a point cloud double cross attention module, a grouping module, and a point cloud local attention module. The downsampling module of the tooth center perception encoder of the i-th layer downsamples the point cloud features of the i-1th layer by the farthest point sampling method, with a downsampling rate of 4, and uses the multi-layer perceptron module to convert the feature dimension from C to i-1 Raised to C i , where C i-1 Represents the feature dimension of the i-1th layer. After processing, the initial point cloud feature of the i-th layer is obtained, and its dimension can be expressed as N i ×C i Then, the initial point cloud features of the i-th layer are combined with the super-point features output by the i-1th layer and the initial super-point features of the i-th layer and input into the super-point double cross attention module to obtain the super-point features of the i-th layer, whose feature dimension can be expressed as M×Ci , where M = 17. At the same time, the initial point cloud features of the i-th layer are input into the grouping module, and the K nearest neighbor features of each point are obtained by ball query in the grouping module. The feature dimension can be expressed as N i ×K×C i Then, the super-point features of the i-th layer, the initial point cloud features of the i-th layer, and the grouping features obtained by the grouping module are input into the point cloud double cross attention module, and then the processed point cloud features are input into L combination modules composed of the grouping module and the point cloud local attention module to obtain the final i-th layer point cloud encoding features, whose dimension is N×C i .

[0053] 2.1.1) Channel Cross Attention: The cross attention in the present invention is channel cross attention.

[0054] 2.1.2) The super-point dual cross attention module consists of a super-point inter-layer cross attention module and a super-point global cross attention module.

[0055] The super-point inter-layer cross-attention module in the i-th layer uses the super-point features output by the i-1th layer as the key vector and value vector of the channel cross-attention, and uses the initialized initial super-point features of the i-th layer as the query vector of the channel cross-attention, and then performs the channel cross-attention to obtain the output result of the super-point inter-layer cross-attention module in the i-th layer;

[0056] The super-point global cross-attention module in the i-th layer uses the initial point cloud features of the i-th layer as the key vector and value vector of the channel cross-attention, and uses the initialized initial super-point features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output result of the super-point global cross-attention module in the i-th layer.

[0057] The output result of the super-point inter-layer cross attention module of the i-th layer and the output result of the super-point global cross attention module of the i-th layer are added and output to obtain the output of the super-point double cross attention module of the i-th layer, thereby obtaining the super-point feature of the i-th layer, and its feature dimension can be expressed as M×C i .

[0058] 2.1.3) The point cloud dual cross attention module consists of a point cloud global cross attention module and a point cloud local cross attention module.

[0059] The point cloud global cross attention module at the i-th layer uses the super-point features of the i-th layer as the key vector and value vector of the channel cross attention, and uses the initial point cloud features of the i-th layer as the query vector of the channel cross attention, and then performs channel cross attention to obtain the output of the point cloud global cross attention at the i-th layer;

[0060] The point cloud local cross-attention module in the i-th layer uses the K nearest neighbor features of each point obtained after the grouping layer as the key vector and value vector of the channel cross-attention, and uses the initial point cloud features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output of the point cloud local cross-attention module in the i-th layer.

[0061] The output result of the global cross attention of the i-th layer point cloud and the output result of the local cross attention module of the i-th layer point cloud are added and output to obtain the output of the i-th layer point cloud double cross attention module, whose feature dimension can be expressed as N i ×C i .

[0062] exist Figure 2 In the super-point dual cross attention module and the point cloud dual cross attention module, R represents matrix subtraction operation, the normalized exponential function refers to SoftMax, “⊙” represents matrix multiplication operation, pooling refers to Maxpooling (maximum pooling), and “⊕” represents matrix addition operation.

[0063] 3.1) Decoder, each decoder consists of an upsampling layer and several multi-layer perceptron layers. In the i-th layer decoder, the output of the i-1 layer decoder is first linearly interpolated through the upsampling layer with an upsampling rate of 4. The upsampling result is connected with the output of the encoder symmetrical to the decoder of this layer, and the connected features are decoded through several multi-layer perceptrons to obtain the final decoding result of the i-th layer.

[0064] 4.1) The post-processing module mainly consists of three modules: semantic branch, offset branch and fine-tuning. In the fine-tuning module, the original spatial coordinates of the point cloud are first added to the offset of the offset branch to obtain the offset point cloud, and the offset point cloud is clustered using the DBSCAN clustering algorithm. The tooth instance label is obtained based on the clustered label, and the point set of each tooth instance is found based on the tooth instance label. Based on the semantic branch result of the point set, the semantic category of the current tooth instance (i.e., FDI label) is obtained using the majority voting method.

[0065] 5.1) Superpoint spatial position calculation: In the i-th layer encoder, the superpoint feature H is obtained through the superpoint double cross attention module i Then, the three-dimensional spatial position Y of the super point is obtained by interpolating the super point and the initial point cloud of the i-th layer. i , the process can be expressed as follows:

[0066]

[0067] Where X i represents the 3D spatial position of the point cloud in the i-th layer encoder, Represents the transpose of the feature matrix of the point cloud in the i-th layer encoder.

[0068] 5.2) Loss function: In order to constrain and optimize network learning, the present invention mainly uses the following loss function to constrain the above network learning:

[0069]

[0070]

[0071]

[0072]

[0073] in Using cross entropy loss function to measure semantic branch prediction results and the true value y i The difference between

[0074] in Combined chamfer distance loss and smooth absolute error loss (Smooth L1 loss, that is, ) to measure the offset branch prediction result and the difference between the true value O;

[0075] in The smooth absolute error loss is used to measure the difference between the three-dimensional coordinates of the superpoint p and the three-dimensional coordinates of the tooth centroid g. Where i, j represents the point set subscript, and MP(p,g) is the maximum matching pair calculated using the Hungarian algorithm on the point set p,g.

[0076] The weight of the loss function is agreed as follows: cd =0.15,ω l1 =0.03,ω seg =1,ω tcp =0.1.

[0077] like Figure 3 As shown in the segmentation effect diagram, compared with the existing segmentation methods PointTransformer, SpoTr, DBGANet, PointNet++, and DGCNN, the method of the present invention (called TCATSeg) can produce a more accurate segmentation effect.

[0078] Key point 1: The method of the present invention uses local attention and global attention simultaneously to automatically segment a three-dimensional dental model. Its beneficial technical effect is that the simultaneous use of local attention and global attention can enhance the segmentation network's capture of local geometric details of teeth and perception of global semantic information, which can significantly improve the segmentation effect compared to methods that use local attention or global attention alone.

[0079] Key point 2: The present invention uses a strategy of using superpoints to efficiently inject global attention into the point-by-point feature extraction network. Its beneficial technical effect is: using superpoints to inject global attention can effectively solve the problem of high memory consumption of global attention, and the computational efficiency is reduced from O(N 2 ) is reduced to O(MN).

[0080] Key point 3: A super-point feature learning module of the present invention has the beneficial technical effect that the module can achieve consistency in super-point feature learning and efficiently aggregate features from the input point cloud, thereby aggregating global information.

[0081] Key point 4: The beneficial technical effect of a super-point position constraint strategy of the present invention is that by constraining the super-point to the tooth centroid, the ability of the super-point feature to aggregate global information can be further enhanced, because the distribution of the tooth centroid can naturally represent the relative relationship and semantic distribution between teeth.

[0082] In other embodiments of the present invention, other suitable methods may be used to replace the downsampling algorithm in 1.1) or modify the downsampling rate, the cross-attention module in 2.1.1) may be replaced by other types of cross-attention modules, branches may be added or deleted in 4.1), the DBSCAN clustering algorithm may be replaced by other clustering algorithms in 4.1), the loss function may be added or deleted in 5.2), or the weight value of the loss function may be modified.

[0083] Another embodiment of the present invention provides an automatic segmentation system for a three-dimensional digital dental model based on superpoints, comprising:

[0084] A three-dimensional dental and jaw model acquisition module is used to perform a three-dimensional oral scan using a three-dimensional oral scanner to obtain a three-dimensional dental and jaw model;

[0085] A three-dimensional dental model automatic segmentation module is used to input the three-dimensional dental model into a three-dimensional dental model automatic segmentation network based on super points to obtain an FDI label for each point of the three-dimensional dental model;

[0086] The visualization processing module is used to perform color visualization processing on each point of the three-dimensional dental model according to the FDI label of the three-dimensional dental model and the label color mapping table, and output the final segmentation model.

[0087] The division of the above modules is only an example. In actual applications, the above functions can be assigned to different functional modules as needed to complete all or part of the functions described in the above method. The specific working process of each module can refer to the corresponding process in the above method embodiment, which will not be repeated here.

[0088] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0089] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0090] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A method for automatic segmentation of a three-dimensional digital dental model based on superpoints, characterized in that: The following steps are involved: Perform a three-dimensional oral scan using a three-dimensional oral scanner to obtain a three-dimensional dental model; The three-dimensional dental model is input into the three-dimensional dental model automatic segmentation network based on super points to obtain the FDI label of each point of the three-dimensional dental model; According to the FDI label of the 3D dental model and the label color mapping table, each point of the 3D dental model is colored and visualized, and the final segmentation model is output.

2. The method according to claim 1, characterized in that The superpoint-based three-dimensional dental jaw model automatic segmentation network includes: Initialization module: First, the input 3D dental model is downsampled to obtain N*6-dimensional initial feature information, where N represents the number of point clouds, and 6 dimensions include the 3D coordinates of the points and the 3D normal vectors. Then, the initial feature information is sent to the encoder group; at the same time, the super-point feature vector is initialized and sent to the corresponding encoder group; Encoder group: composed of a plurality of tooth center perception encoders, each of which is connected in series, and the last tooth center perception encoder is connected to the first decoder; Decoder group: consists of several decoders, each decoder is connected in series, and each decoder is connected to the symmetrical encoder in the encoder group through a skip connection; Post-processing module: It consists of a semantic branch, an offset branch and a fine-tuning module. It first accepts the point cloud features output by the decoder group, and obtains the semantic probability distribution of each point in the point cloud and the offset vector of the distance to the corresponding tooth center through the semantic branch and the offset branch respectively. Then, the semantic probability distribution of each point and the offset vector of the distance to the corresponding tooth center are input into the fine-tuning module to obtain the final segmentation result.

3. The method according to claim 2, characterized in that The tooth center perception encoder comprises a downsampling module, a multi-layer perceptron module, a super-point double cross attention module, a point cloud double cross attention module, a grouping module, and a point cloud local attention module; the downsampling module of the tooth center perception encoder of the i-th layer downsamples the point cloud features of the i-1th layer by the farthest point sampling method, and converts the feature dimension from C to i-1 Raised to C i , after processing, the initial point cloud features of the i-th layer are obtained, and then the initial point cloud features of the i-th layer are combined with the super-point features output by the i-1-th layer and the initial super-point features of the i-th layer and input into the super-point double cross attention module to obtain the super-point features of the i-th layer; at the same time, the initial point cloud features of the i-th layer are input into the grouping module, and the K nearest neighbor features of each point are obtained by ball query in the grouping module; then the super-point features of the i-th layer, the initial point cloud features of the i-th layer and the grouping features obtained by the grouping module are input into the point cloud double cross attention module, and the processed point cloud features are then input into L combination modules composed of the grouping module and the point cloud local attention module to obtain the final i-th layer point cloud encoding features.

4. The method according to claim 3, characterized in that The super-point dual cross-attention module includes a super-point inter-layer cross-attention module and a super-point global cross-attention module; the super-point inter-layer cross-attention module at the i-th layer uses the super-point features output by the i-1th layer as the key vector and value vector of the channel cross-attention, and uses the initialized initial super-point features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output result of the super-point inter-layer cross-attention module at the i-th layer; the super-point global cross-attention module at the i-th layer uses the initial point cloud features of the i-th layer as the key vector and value vector of the channel cross-attention, and uses the initialized initial super-point features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output result of the super-point global cross-attention module at the i-th layer; the output result of the super-point inter-layer cross-attention module at the i-th layer and the output result of the super-point global cross-attention module at the i-th layer are added and output to obtain the output of the super-point dual cross-attention module at the i-th layer, thereby obtaining the super-point features at the i-th layer.

5. The method according to claim 3, characterized in that: The point cloud dual cross-attention module includes a point cloud global cross-attention module and a point cloud local cross-attention module; the point cloud global cross-attention module at the i-th layer uses the super-point features of the i-th layer as the key vector and value vector of the channel cross-attention, uses the initial point cloud features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output of the i-th layer point cloud global cross-attention; the point cloud local cross-attention module at the i-th layer uses the K nearest neighbor features of each point obtained after the grouping layer as the key vector and value vector of the channel cross-attention, uses the initial point cloud features of the i-th layer as the query vector of the channel cross-attention, and then performs channel cross-attention to obtain the output of the i-th layer point cloud local cross-attention module; the output result of the i-th layer point cloud global cross-attention and the output result of the i-th layer point cloud local cross-attention module are added and output to obtain the output of the i-th layer point cloud dual cross-attention module.

6. The method according to claim 2, characterized in that Each decoder in the decoder group is composed of an upsampling layer and several multi-layer perceptron layers; in the i-th layer decoder, the output of the i-1 layer decoder is firstly linearly interpolated through the upsampling layer, the up-sampled result is connected with the output of the encoder symmetrical to the decoder of this layer, and the connected features are decoded through several multi-layer perceptrons to obtain the final decoding result of the i-th layer; The fine-tuning module in the post-processing module first adds the original spatial coordinates of the point cloud to the offset of the offset branch to obtain the offset point cloud, clusters the offset point cloud using a clustering algorithm, obtains the tooth instance label based on the clustered label, finds the point set of each tooth instance based on the tooth instance label, and obtains the semantic category of the current tooth instance based on the semantic branch result of the point set using the majority voting method.

7. The method according to claim 2, characterized in that The following loss function is used to constrain and optimize the superpoint-based 3D dental model automatic segmentation network: in, Using cross entropy loss function to measure semantic branch prediction results The difference between yi and the true value yi; Combined chamfer distance loss and smooth absolute error loss To measure the offset branch prediction result and the difference between the true value O; The smooth absolute error loss is used to measure the difference between the three-dimensional coordinates of the superpoint p and the three-dimensional coordinates of the tooth centroid g, where i, j represent the point set subscripts, MP(p,g) is the maximum matching pair calculated using the Hungarian algorithm on the point sets p,g; ωcd, ωl1, ωseg, ωtcp are the weights of the loss function.

8. An automatic segmentation system for three-dimensional digital dental models based on superpoints, characterized in that: include: A three-dimensional dental and jaw model acquisition module is used to perform a three-dimensional oral scan using a three-dimensional oral scanner to obtain a three-dimensional dental and jaw model; A three-dimensional dental model automatic segmentation module is used to input the three-dimensional dental model into a three-dimensional dental model automatic segmentation network based on super points to obtain an FDI label for each point of the three-dimensional dental model; The visualization processing module is used to perform color visualization processing on each point of the three-dimensional dental model according to the FDI label of the three-dimensional dental model and the label color mapping table, and output the final segmentation model.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images