A tooth change detection method, system, electronic device and storage medium

Abstract

The present invention discloses a tooth change detection method, system, electronic device, and storage medium. The method comprises: collecting and preprocessing three-dimensional tooth data to construct a three-dimensional tooth dataset; sequentially constructing a point cloud feature extraction module, an attention network module, a deconvolution module, and a fully connected layer based on PointNet++ to obtain a change detection model to be trained with a twin network structure; constructing a composite loss function, the composite loss function consisting of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function; using the three-dimensional tooth dataset and the composite loss function to train the change detection model to obtain a trained change detection model; and inputting the tooth point cloud data to be processed into the trained change detection model to obtain a tooth change detection result. The present invention solves the shortcomings of existing technologies, such as poor targeting of three-dimensional tooth targets, low robustness, weak generalization ability, and low detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to a tooth change detection method, system, electronic equipment and storage medium. Background Art

[0002] With the increase in residents' income and the popularization of oral health knowledge, people's acceptance of dental beauty is increasing year by year, and its industry and corresponding market have maintained a rapid growth trend in recent years.

[0003] However, the current level of automation in the cosmetic dental process is still insufficient. For example, orthodontic personnel often need to manually determine the degree of change and difference in teeth before and after correction, which is subject to strong subjectivity and large errors, and can easily cause discomfort to the client, such as pain in the abutment teeth and changes in chewing patterns. Furthermore, the manual tooth correction process is time-consuming and labor-intensive, prone to fatigue, and consumes a lot of manpower and material resources. Therefore, achieving automation and intelligence in tooth correction is crucial for the development of orthodontics. However, many commercial change detection software programs currently use algorithms based on traditional manually designed operators, which have low performance and poor robustness, and need to be improved.

[0004] On the other hand, new theories and technologies such as deep learning have achieved success in the field of object detection in recent years due to their advantages in automatically learning features. However, existing deep learning-based change detection methods are generally applied to two-dimensional objects, and research on change detection of more complex three-dimensional objects (such as teeth) is still rare. In addition, teeth are relatively complex three-dimensional objects, characterized by subtle spatial structural changes. Therefore, it is necessary to study how to use deep learning to achieve three-dimensional tooth change detection for this specific three-dimensional object.

[0005] Through the above analysis, the problems and defects of the existing technology are as follows:

[0006] (1) Currently, the tooth change detection process in automatic dental cosmetic procedures is time-consuming, labor-intensive, and has a low degree of automation, and is heavily dependent on human experience.

[0007] (2) Many existing automatic tooth change detection methods use algorithms based on traditional manual operators, while there are fewer change detection methods based on deep learning;

[0008] (3) Existing deep learning-based target change detection methods are rarely designed for three-dimensional targets. How to design a three-dimensional target change detection network is a current research difficulty. Summary of the Invention

[0009] In order to solve the above problems existing in the prior art, the present invention provides a method, system, electronic device and storage medium for detecting tooth changes. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0010] In a first aspect, the present invention provides a method for detecting tooth changes, comprising:

[0011] Collecting three-dimensional tooth data and performing preprocessing to construct a three-dimensional tooth dataset, wherein the preprocessing includes point cloud format conversion, coordinate normalization, and data expansion;

[0012] A point cloud feature extraction module based on PointNet++, an attention network module based on self-attention and cross-attention mechanisms, a deconvolution module, and a fully connected layer are sequentially constructed to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure;

[0013] Constructing a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function;

[0014] Using the three-dimensional tooth dataset and the composite loss function, training the change detection model to be trained to obtain a trained change detection model;

[0015] The tooth point cloud data to be processed is input into the trained change detection model to obtain a tooth change detection result.

[0016] In one embodiment of the present invention, the step of inputting the tooth point cloud data to be processed into the trained change detection model to obtain the tooth change detection result includes:

[0017] The three-dimensional tooth data is obtained by scanning with a dental oral scanning device, and the three-dimensional tooth data is converted into a point cloud format, normalized into a coordinate system, and expanded. The point cloud format conversion converts the three-dimensional tooth data from a ply format to a pcd format, the coordinate normalization normalizes the coordinate value of each dimension to between [-1, 1], and the data expansion uses rotation and translation transformations.

[0018] The preprocessed three-dimensional tooth point cloud data is input into the trained change detection model, so that the preprocessed three-dimensional tooth point cloud data passes through the point cloud feature extraction module, the attention network module, the deconvolution module and the fully connected layer in sequence to obtain a binary classification result, thereby realizing the detection of the three-dimensional tooth change area.

[0019] In one embodiment of the present invention, the point cloud feature extraction module includes: a sampling layer, a grouping layer and a PointNet layer;

[0020] The sampling layer is used to downsample the preprocessed three-dimensional tooth point cloud data and select a number of center points using an iterative farthest point sampling algorithm;

[0021] The grouping layer is used to divide the downsampled three-dimensional tooth point cloud data into a plurality of regions using the plurality of center points;

[0022] The PointNet layer is used to extract features from each of the regions using a PointNet network to obtain global features.

[0023] In one embodiment of the present invention, the sampling layer is specifically used to: after downsampling the preprocessed tooth 3D point cloud data, randomly select a tooth 3D point cloud data as an initial point to form a selected sampling point set; calculate the distance between each unselected tooth 3D point cloud data and each selected tooth 3D point cloud data in the selected sampling point set, and add the unselected tooth 3D point cloud data corresponding to the maximum distance to the selected sampling point set until a preset number of center points are obtained;

[0024] The grouping layer is specifically used to: for each of the center points, draw three circular areas around the center point at three different preset scales, each of which has a different radius and contains different amounts of three-dimensional tooth point cloud data; extract features from the circular areas of different scales, and then splice the extracted features to obtain the corresponding features of each of the center points;

[0025] The PointNet layer is specifically used to: perform dimensionality enhancement on the corresponding features with a dimension of B*N*K*(d+C) to obtain output point cloud features with a dimension of B*N*K*C1, further perform a pooling operation on each circular area, and then use a multi-layer perceptron MLP to fully connect the pooling results to obtain global features; where B is the size of each batch, N and K are the number of point clouds and coordinates respectively, d+C is the number of channels, and C1 is the new number of channels after dimensionality enhancement.

[0026] In one embodiment of the present invention, the attention network module includes: a self-attention submodule and a cross-attention submodule. The introduction of the attention module enables the change detection network to be trained to focus on the places where the tooth wear changes;

[0027] The self-attention submodule is used to update the output features respectively using the two branches of the Siamese network structure based on the self-attention mechanism, and calculate the first correlation between different points in the point cloud data in the updated output features;

[0028] The cross-attention submodule is used to calculate the second correlation between the point cloud data in the output features after the two branches are updated by using the cross-attention mechanism and the first correlation.

[0029] In one embodiment of the present invention, the preset composite loss function is the sum of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function.

[0030] In one embodiment of the present invention, the binary cross entropy loss function is:

[0031]

[0032] Wherein, N represents the number of point clouds contained in the point cloud data, and M represents the number of points contained in a single point cloud in the point cloud data; ij Represents the label of the jth point in the i-th point cloud, y ij When the value is 0, it means that the jth point in the i-th point cloud has not changed, y ij When the value is 1, it means that the jth point in the i-th point cloud has changed; p(y ij ) indicates that the jth point in the i-th point cloud belongs to the label y ij probability;

[0033] The Tversky focal loss function is:

[0034]

[0035] Among them, α and β represent hyperparameters, p 0i 、p 1i Respectively represent the probability that the i-th point cloud in the true value belongs to the first category and the second category, and represent the probability that the i-th point cloud in the predicted value belongs to the first category, g 1i Indicates the probability that the i-th point cloud in the predicted value belongs to the second category;

[0036] The generalized Dice loss function is:

[0037]

[0038] Where, ω j represents the weight of each category, C represents the total number of categories, y ij Represents the label of the jth point in the i-th point cloud.

[0039] In a second aspect, the present invention further provides a tooth change detection system, comprising:

[0040] A data set construction module is used to collect three-dimensional tooth data and perform preprocessing to construct a three-dimensional tooth data set. The preprocessing includes point cloud format conversion, coordinate normalization and data expansion;

[0041] A detection model construction module is used to sequentially construct a point cloud feature extraction module based on PointNet++, an attention network module based on self-attention mechanism and cross-attention mechanism, a deconvolution module, and a fully connected layer to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure;

[0042] A loss function construction module is used to construct a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function;

[0043] a detection model training module, configured to train the change detection model to be trained using the three-dimensional tooth dataset and the composite loss function to obtain a trained change detection model;

[0044] The change detection module is used to input the tooth point cloud data to be processed into the trained change detection model to obtain the tooth change detection result.

[0045] In a third aspect, the present invention provides an electronic device comprising a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the tooth change detection method described in any one of the first aspects.

[0046] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program, wherein when the computer program is processed and executed, the processor executes the tooth change detection method described in any one of the first aspects.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The embodiments of the present invention propose a three-dimensional tooth change detection method, system, electronic device and storage medium, which can solve the defects of low robustness, poor generalization ability of general deep learning methods, poor robustness to noisy interference and complex models in the existing technology, and can be effectively applied to actual situations; the embodiments of the present invention provide new theoretical and new method support for the practical application of the three-dimensional tooth change detection method, making the three-dimensional tooth change detection method practical and universal, and can be widely used in dental beauty fields such as orthodontics and tooth restoration.

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of a tooth change detection method provided by an embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of a tooth change detection method provided by an embodiment of the present invention;

[0052] Figure 3 Schematic diagram of the structure of the point cloud feature extraction module provided by an embodiment of the present invention;

[0053] Figure 4 1 is a schematic diagram of the structure of a multilayer perceptron provided by an embodiment of the present invention;

[0054] Figure 5a This is a schematic diagram of real tooth changes provided by an embodiment of the present invention;

[0055] Figure 5b is a schematic diagram of tooth change detection results provided by an embodiment of the present invention;

[0056] Figure 6 is a structural diagram of a tooth change detection system provided by an embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present invention;

[0058] Figure 8 It is a schematic diagram of a readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0060] Currently, many commercial 3D tooth change detection software still uses traditional detection methods, which are mainly suitable for good backgrounds and user cooperation. They often perform poorly in complex environments and when the tooth model spans a long time span. Due to the outstanding advantages of deep neural networks in autonomous learning and abstract feature representation, how to use deep learning algorithms for 3D tooth change detection has become a pressing problem for those skilled in the art.

[0061] In view of this, the present invention provides a tooth change detection method, system, electronic device and storage medium.

[0062] Figure 1 This is a flow chart of a tooth change detection method provided by an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a method for detecting tooth changes, comprising:

[0063] S1. Collect three-dimensional tooth data and perform preprocessing such as point cloud format conversion, coordinate normalization, and data expansion to construct a three-dimensional tooth dataset. Point cloud format conversion converts the tooth data in ply format to pcd format, coordinate normalization normalizes the coordinate values ​​of each dimension to between [-1, 1], and data expansion uses transformations such as rotation and translation.

[0064] S2. Sequentially construct a point cloud feature extraction module based on PointNet++, an attention network module based on the self-attention mechanism and the cross-attention mechanism, a deconvolution module, and a fully connected layer to obtain a 3D tooth change detection model to be trained; the 3D tooth change detection model has a twin network structure;

[0065] S3. Construct a composite loss function, which consists of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function;

[0066] S4. Using the three-dimensional tooth dataset and the composite loss function, the change detection model to be trained is trained to obtain a trained change detection model;

[0067] S5. Input the tooth point cloud data to be processed into the trained change detection model to obtain a tooth change detection result.

[0068] In this embodiment, 3D point cloud data of an individual's teeth is collected and, after irrelevant background is removed using a segmentation algorithm, such as a RANSAC-based tooth segmentation algorithm, the 3D point cloud data is further amplified to construct a 3D dental dataset. Specifically, after obtaining the raw 3D point cloud data using an oral scanning device, background removal is performed on the tooth point cloud. Translation and rotation operations are then used to expand the 3D tooth change detection dataset. Point cloud downsampling techniques are then used to reduce the density of the tooth point cloud to meet subsequent network input requirements. Finally, the processed point cloud data is divided into a training set and a test set to construct the 3D dental dataset.

[0069] Figure 3 is a schematic diagram of the structure of the point cloud feature extraction module provided by an embodiment of the present invention. Figure 3 As shown, the point cloud feature extraction module includes: a sampling layer, a grouping layer and a PointNet layer;

[0070] The sampling layer is used to downsample the preprocessed three-dimensional tooth point cloud data and select a number of center points using an iterative farthest point sampling algorithm;

[0071] The grouping layer is used to divide the downsampled three-dimensional tooth point cloud data into a plurality of regions using the plurality of center points;

[0072] The PointNet layer is used to extract features from each of the regions using a PointNet network to obtain global features.

[0073] Specifically, the point cloud feature extraction module consists of three parts: a sampling layer, a grouping layer, and a PointNet layer. The sampling layer selects a series of center points from the preprocessed three-dimensional tooth point cloud data to define the center of a local region. Compared to random sampling algorithms, the present invention uses an iterative farthest point sampling algorithm to determine the center points, which can more completely sample the global point cloud from the regional center points. Furthermore, the grouping layer aims to construct local regions based on the center points for feature extraction. When the point cloud distribution is inconsistent, multi-scale grouping can be used to combine point cloud features of different densities. The PointNet layer performs local global feature extraction on the sampled and grouped point cloud data. The PointNet layer directly uses an unordered point set as the network input, uses symmetric functions to address the disorder of the point cloud, and uses a transformation matrix to address the rotational invariance of the point cloud. For example, for each input point cloud data point, the PointNet layer first spatially aligns it using a transformation matrix, then maps it through a fully connected multi-layer perceptron (MLP) process. After alignment, it is again mapped to a higher-dimensional space and then subjected to a maximum pooling operation.

[0074] In this embodiment, the parameters of each layer of the point cloud feature extraction module are detailed in Table 1, and the padding method during the convolution process is 0 padding.

[0075] Table 1 Detailed parameters of point cloud feature extraction module

[0076]

[0077]

[0078] Optionally, the sampling layer is specifically used to: after downsampling the preprocessed tooth three-dimensional point cloud data, randomly select a tooth three-dimensional point cloud data as an initial point to form a selected sampling point set; calculate the distance between each unselected tooth three-dimensional point cloud data and each selected tooth three-dimensional point cloud data in the selected sampling point set, and add the unselected tooth three-dimensional point cloud data corresponding to the maximum distance to the selected sampling point set, until a preset number of center points are obtained;

[0079] The grouping layer is specifically used to: for each of the center points, draw three circular areas around the center point at three different preset scales, each of which has a different radius and contains different amounts of three-dimensional tooth point cloud data; extract features from the circular areas of different scales, and then splice the extracted features to obtain the corresponding features of each of the center points;

[0080] The PointNet layer is specifically used to: perform dimensionality enhancement on the corresponding features with a dimension of B*N*K*(d+C) to obtain output point cloud features with a dimension of B*N*K*C1, further perform a pooling operation on each circular area, and then use a multi-layer perceptron MLP to fully connect the pooling results to obtain global features; where B is the size of each batch, N and K are the number of point clouds and coordinates respectively, d+C is the number of channels, and C1 is the new number of channels after dimensionality enhancement.

[0081] Figure 4 This is a schematic diagram of the structure of the multi-layer perceptron provided by the embodiment of the present invention. Figure 4 As shown, the parameters of each layer in the multi-layer perceptron MLP are detailed in Table 2:

[0082] Table 2 Detailed parameters of the multi-layer perceptron MLP

[0083]

[0084]

[0085] It should be noted that, in this embodiment, the attention network module can utilize the attention mechanism to make the change detection network to be trained focus on the places where the teeth have undergone wear changes.

[0086] The self-attention submodule is used to update the output features respectively using the two branches of the Siamese network structure based on the self-attention mechanism, and calculate the first correlation between different points in the point cloud data in the updated output features;

[0087] The cross-attention submodule is used to calculate the second correlation between the point cloud data in the output features after the two branches are updated by using the cross-attention mechanism and the first correlation.

[0088] Specifically, we first use geometry-based self-attention. Given a set of superpoints in point cloud data The input feature matrix is Then the output feature matrix It can be expressed as a weighted sum of all input feature projections:

[0089]

[0090] Among them, x j and z i are the row vectors of the input and output matrices respectively; is the size of the superpoint set (i.e., the total number of superpoints), d t Indicates the size of the second dimension of the superpoint; W V is the projection matrix corresponding to the attention value (Value); α i,jis the weight, which can be obtained by the attention score e i,j Calculated by the softmax rule.

[0091] Attention score e i,j Defined as:

[0092]

[0093] Where, is geometric structure embedding; W Q , W K , W R The corresponding projection matrices are query, key, and geometric structure embedding. The purpose of geometric structure embedding is to encode the rotation invariance of superpoints. Its core idea is to use the consistency of superpoints in different point clouds of the same scene to calculate distances and angles.

[0094] Given two superpoints P i and P j , whose geometric structures are embedded in r i,j Embedded by pairwise distances and 3D angle embedding Composition, expressed as:

[0095]

[0096] Where W D and W A The projection matrices representing the distance embedding and 3D angle embedding respectively. for The K nearest neighbors of the superpoint are k = 1, 2, ..., K. This geometric structure is embedded and then subjected to a maximum pooling operation, which can improve the robustness to different nearest neighbors of the superpoint due to self-occlusion.

[0097] Next, we use feature-based cross attention. Assume that two superpoint sets and The self-attention feature matrices are and but The cross attention feature matrix Can be used The calculation formula is:

[0098]

[0099] in, and The matrices and The row vector of ; Superpoint set The size of . Similarly, the weight By cross attention score Calculation, the formula is:

[0100]

[0101] The geometric self-attention module encodes a transformation-invariant geometric structure for each individual point cloud, while the feature-based cross-attention module models the geometric consistency between two point clouds, generating hybrid features that are invariant. Therefore, the cross-attention mechanism can better reason about the transformation of the correspondence between point clouds, making the neural network more robust.

[0102] In order to improve the accuracy of the change detection model, this embodiment constructs a composite loss function for training. Optionally, the preset composite loss function is composed of three loss function items, namely, binary cross entropy loss (Cross-Entropy Loss, CEL), Tversky focal loss (Focal Tversky Loss, FTL) and generalized Dice loss (Generalized Dice Loss, GDL). The preset composite loss function is the sum of the binary cross entropy loss function, the Tversky focal loss function and the generalized Dice loss function.

[0103] The binary cross entropy loss CEL is generally used to quantify the difference between two probability distributions and is often used for classification problems. The binary cross entropy loss function is:

[0104]

[0105] Wherein, N represents the number of point clouds contained in the point cloud data, M represents the number of points contained in a certain point cloud in the point cloud data, and y ij Represents the label of the jth point in the i-th point cloud, y ij When the value is 0, it means that the jth point in the i-th point cloud has not changed, y ij When the value is 1, it means that the jth point in the i-th point cloud has changed, p(y ij ) indicates that the jth point in the i-th point cloud belongs to the label y ij probability;

[0106] Tversky focal loss FTL further balances false negatives and false positives based on Tversky Loss. The formula of FTL used is:

[0107]

[0108] Where γ is the modulation coefficient.

[0109] The Tversky focal loss function is:

[0110]

[0111] Among them, α and β represent hyperparameters, p 0i 、p 1i Respectively represent the probability that the i-th point cloud in the true value belongs to the first category and the second category, and represent the probability that the i-th point cloud in the predicted value belongs to the first category, g 1i Indicates the probability that the i-th point cloud in the predicted value belongs to the second category;

[0112] The generalized Dice loss function is:

[0113]

[0114] Where, ω j represents the weight of each category; C represents the total number of categories, and in this embodiment, C is 2; y ij Represents the label of the jth point in the i-th point cloud.

[0115] When using the 3D tooth dataset to train the change detection model to be trained, it can be divided into a training set and a test set. Specifically, the 3D tooth point cloud data in the training set is input into the change detection model to be trained for feature learning and training to obtain a trained 3D tooth change detection model. The trained change detection model is then tested using the 3D tooth point cloud data in the test set to verify its performance. A total of 200 epochs are set in the entire training process. Multiple batches are set according to the total number of 3D tooth point cloud data. In each epoch, multiple batches are input in sequence for training. The size of each batch is set to 8, and the learning rate is 0.001. During the training process, the learning rate will be constant at e -4 Decreasingly, the parameters are updated using the Adam optimizer throughout the training process.

[0116] After obtaining the trained change detection model, the step of inputting the tooth point cloud data to be processed into the trained change detection model to obtain the tooth change detection result includes:

[0117] Scanning the three-dimensional tooth data using an oral scanning device, and performing pre-processing such as point cloud format conversion, coordinate normalization, and data expansion on the three-dimensional tooth data;

[0118] The preprocessed three-dimensional tooth point cloud data is input into the trained change detection model, so that the preprocessed three-dimensional tooth point cloud data passes through the point cloud feature extraction module, the attention network module, the deconvolution module and the fully connected layer in sequence to obtain a binary classification result, thereby realizing the detection of the three-dimensional tooth change area.

[0119] In summary, the three-dimensional tooth change detection method provided by the embodiment of the present invention uses a deep neural network to realize the change detection of three-dimensional spatial points of tooth point cloud. Figure 5a This is a real diagram of tooth changes provided by an embodiment of the present invention. Figure 5b is a schematic diagram of the tooth change detection result provided by an embodiment of the present invention. Obviously, Figure 5a-5b As shown, the tooth change detection method provided by the present invention can obtain a detection result that is relatively close to the true value.

[0120] The present invention employs multiple sets of experiments to comprehensively demonstrate the performance of various change detection methods. Specifically, to demonstrate the high performance of the 3D tooth change detection model, it is compared with other change detection methods. The following representative performance metrics are selected and calculated: mean intersection over union (mIOU), accuracy, recall, precision, and F1-score.

[0121] It is defined as follows:

[0122] mIOU is the mean of the intersection-over-union (mIOU) between the true label and the predicted value (k is the category):

[0123]

[0124] The calculation formula for Accuracy is:

[0125]

[0126] The calculation formula for Recall is:

[0127]

[0128] The calculation formula for Precision is:

[0129]

[0130] The calculation formula of F1-Score is:

[0131]

[0132] Among them, TP: true positive, which means the number of samples that are actually positive examples and predicted as positive examples;

[0133] FP: false positive, which indicates the number of samples that are actually negative but predicted to be positive;

[0134] TN: true negative, which indicates the number of samples that are actually negative and predicted as negative;

[0135] FN: false negative, which indicates the number of samples that are actually positive but predicted to be negative.

[0136] Table 3 compares the performance of the tooth change detection method provided by the present invention with non-end-to-end methods that first align and then perform subtraction (such as OverlapPredator) and end-to-end methods based entirely on deep learning (using PointNet++ as an example). As can be clearly seen in Table 3, the tooth change detection method provided by the present invention surpasses other mainstream methods in the existing literature, such as OverlapPredator and PointNet++, in all the aforementioned change detection performance indicators (including mIOU, accuracy, recall, precision, and F1-Score), demonstrating the superiority of the present method.

[0137] Table 3 Comparison of the present invention with other methods on five tooth segmentation indicators

[0138]

[0139] The embodiment of the present invention proposes a three-dimensional tooth change detection method, which can solve the defects of low robustness, poor generalization ability of general deep learning methods, poor robustness to noisy interference and complex models in the existing technology, and can be effectively applied to actual situations; the embodiment of the present invention provides new theoretical and new method support for the practical application of the three-dimensional tooth change detection method, making the three-dimensional tooth change detection method practical and universal, and can be widely used in the field of dental cosmetic restoration.

[0140] An embodiment of the present invention further provides a tooth change detection system, comprising:

[0141] A data set construction module 610 is used to collect three-dimensional tooth data and perform preprocessing to construct a three-dimensional tooth data set. The preprocessing includes point cloud format conversion, coordinate normalization, and data expansion.

[0142] A detection model construction module 620 is configured to sequentially construct a point cloud feature extraction module based on PointNet++, an attention network module based on self-attention and cross-attention mechanisms, a deconvolution module, and a fully connected layer to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure;

[0143] A loss function construction module 630 is used to construct a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function;

[0144] A detection model training module 640 is configured to train the change detection model to be trained using the three-dimensional tooth dataset and the composite loss function to obtain a trained change detection model;

[0145] The change detection module 650 is used to input the tooth point cloud data to be processed into the trained change detection model to obtain a tooth change detection result.

[0146] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0147] Memory 703, for storing computer programs;

[0148] The processor 701 is configured to execute the program stored in the memory 703, and implement the following steps:

[0149] Collecting three-dimensional tooth data and performing preprocessing to construct a three-dimensional tooth dataset, wherein the preprocessing includes point cloud format conversion, coordinate normalization, and data expansion;

[0150] A point cloud feature extraction module based on PointNet++, an attention network module based on self-attention and cross-attention mechanisms, a deconvolution module, and a fully connected layer are sequentially constructed to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure;

[0151] Constructing a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function;

[0152] Using the three-dimensional tooth dataset and the composite loss function, training the change detection model to be trained to obtain a trained change detection model;

[0153] The tooth point cloud data to be processed is input into the trained change detection model to obtain a tooth change detection result.

[0154] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0155] The communication interface is used for communication between the above electronic device and other devices.

[0156] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage system located away from the processor.

[0157] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0158] Figure 8 Schematic diagram of a readable storage medium provided by an embodiment of the present invention. Figure 8 As shown, an embodiment of the present invention further provides a storage medium storing a computer program, which, when processed and executed, implements the following steps:

[0159] Collecting three-dimensional tooth data and performing preprocessing to construct a three-dimensional tooth dataset, wherein the preprocessing includes point cloud format conversion, coordinate normalization, and data expansion;

[0160] A point cloud feature extraction module based on PointNet++, an attention network module based on self-attention and cross-attention mechanisms, a deconvolution module, and a fully connected layer are sequentially constructed to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure;

[0161] Constructing a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function;

[0162] Using the three-dimensional tooth dataset and the composite loss function, training the change detection model to be trained to obtain a trained change detection model;

[0163] The tooth point cloud data to be processed is input into the trained change detection model to obtain a tooth change detection result.

[0164] As for the system / electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] It should be noted that the system, electronic device and storage medium of the embodiments of the present invention are respectively the system, electronic device and storage medium applying the above-mentioned tooth change detection method. All embodiments of the above-mentioned tooth change detection method are applicable to the system, electronic device and storage medium, and can achieve the same or similar beneficial effects.

[0166] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0167] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims.

[0168] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for detecting tooth changes, characterized in that: include: Collecting three-dimensional tooth data and performing preprocessing to construct a three-dimensional tooth dataset, wherein the preprocessing includes point cloud format conversion, coordinate normalization, and data expansion; A point cloud feature extraction module based on PointNet++, an attention network module based on self-attention and cross-attention mechanisms, a deconvolution module, and a fully connected layer are sequentially constructed to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure; Constructing a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function; Using the three-dimensional tooth dataset and the composite loss function, training the change detection model to be trained to obtain a trained change detection model; Inputting the tooth point cloud data to be processed into the trained change detection model to obtain a tooth change detection result; The step of inputting the tooth point cloud data to be processed into the trained change detection model to obtain the tooth change detection result includes: Obtaining and preprocessing the three-dimensional tooth point cloud data to be processed, and inputting the preprocessed three-dimensional tooth point cloud data into the trained change detection model, so that the preprocessed three-dimensional tooth point cloud data is sequentially passed through a point cloud feature extraction module, an attention network module, a deconvolution module, and a fully connected layer to obtain a binary classification result, thereby realizing detection of three-dimensional tooth change areas; The point cloud feature extraction module based on PointNet++ includes: a sampling layer, a grouping layer and a PointNet layer; The sampling layer is used to downsample the preprocessed three-dimensional tooth point cloud data and select a number of center points using an iterative farthest point sampling algorithm; The grouping layer is used to divide the downsampled three-dimensional tooth point cloud data into a plurality of regions using the plurality of center points; The PointNet layer is used to extract features from each region using a PointNet network to obtain global features; The sampling layer is specifically configured to: after downsampling the preprocessed tooth 3D point cloud data, randomly select a tooth 3D point cloud data as an initial point to form a selected sampling point set; calculate the distance between each unselected tooth 3D point cloud data and each selected tooth 3D point cloud data in the selected sampling point set, and add the unselected tooth 3D point cloud data corresponding to the maximum distance to the selected sampling point set until a preset number of center points are obtained; The grouping layer is specifically used to: for each of the center points, draw three circular areas around the center point at three different preset scales, each of which has a different radius and contains different amounts of three-dimensional tooth point cloud data; extract features from the circular areas of different scales, and then splice the extracted features to obtain the corresponding features of each of the center points; The PointNet layer is specifically used to: perform dimensionality enhancement on the corresponding features with a dimension of B*N*K*(d+C) to obtain output point cloud features with a dimension of B*N*K*C1, further perform a pooling operation on each circular area, and then use a multi-layer perceptron MLP to fully connect the pooling results to obtain global features; where B is the size of each batch, N and K are the number of point clouds and coordinates respectively, d+C is the number of channels, and C1 is the new number of channels after dimensionality enhancement.

2. The tooth change detection method according to claim 1, characterized in that: The steps of obtaining the tooth 3D point cloud data to be processed and performing preprocessing include: The three-dimensional tooth data is obtained by scanning with a dental oral scanning device, and the three-dimensional tooth data is converted into a point cloud format, normalized in coordinates, and expanded. The point cloud format conversion converts the three-dimensional tooth data from the ply format to the pcd format, the coordinate normalization normalizes the coordinate value of each dimension to between [-1, 1], and the data expansion uses rotation and translation transformations.

3. The tooth change detection method according to claim 1, wherein: The attention network module includes: a self-attention submodule and a cross-attention submodule, wherein the attention network module is used to enhance the attention level of high-discrimination tooth point cloud features; The self-attention submodule is used to update the output features respectively using the two branches of the Siamese network structure, and calculate the first correlation between different points in the point cloud data in the updated output features; The cross-attention submodule is used to calculate the second correlation between the point cloud data in the output features after the two branches are updated by using the cross-attention mechanism and the first correlation.

4. The tooth change detection method according to claim 1, wherein: The composite loss function is the sum of the binary cross entropy loss function, the Tversky focal loss function and the generalized Dice loss function.

5. The tooth change detection method according to claim 4, characterized in that: The binary cross entropy loss function is: Wherein, N represents the number of point clouds contained in the point cloud data, M represents the number of points contained in a single point cloud in the point cloud data; ij Represents the label of the jth point in the i-th point cloud, y ij When the value is 0, it means that the jth point in the i-th point cloud has not changed, y ij When the value is 1, it means that the jth point in the i-th point cloud has changed; p(y ij ) indicates that the jth point in the i-th point cloud belongs to the label y ij probability; The Tversky focal loss function is: Among them, α and β represent hyperparameters, p 0i 、p 1i Respectively represent the probability that the i-th point cloud in the true value belongs to the first category and the second category, and represent the probability that the i-th point cloud in the predicted value belongs to the first category, g 1i Indicates the probability that the i-th point cloud in the predicted value belongs to the second category; The generalized Dice loss function is: Where, ω j represents the weight of each category, C represents the total number of categories, y ij Represents the label of the jth point in the i-th point cloud data.

6. A tooth change detection system, characterized in that: include: A data set construction module is used to collect three-dimensional tooth data and perform preprocessing to construct a three-dimensional tooth data set. The preprocessing includes point cloud format conversion, coordinate normalization and data expansion; A detection model construction module is used to sequentially construct a point cloud feature extraction module based on PointNet++, an attention network module based on self-attention mechanism and cross-attention mechanism, a deconvolution module, and a fully connected layer to obtain a three-dimensional tooth change detection model to be trained; the three-dimensional tooth change detection model has a twin network structure; A loss function construction module is used to construct a composite loss function, wherein the composite loss function is composed of a binary cross entropy loss function, a Tversky focal loss function, and a generalized Dice loss function; a detection model training module, configured to train the change detection model to be trained using the three-dimensional tooth dataset and the composite loss function to obtain a trained change detection model; a change detection module, configured to input the tooth point cloud data to be processed into the trained change detection model to obtain a tooth change detection result; The change detection module is specifically used to obtain and preprocess the three-dimensional tooth point cloud data to be processed, and input the preprocessed three-dimensional tooth point cloud data into the trained change detection model, so that the preprocessed three-dimensional tooth point cloud data is sequentially passed through the point cloud feature extraction module, the attention network module, the deconvolution module and the fully connected layer to obtain a binary classification result, thereby realizing the detection of three-dimensional tooth change areas; The point cloud feature extraction module based on PointNet++ includes: a sampling layer, a grouping layer and a PointNet layer; The sampling layer is used to downsample the preprocessed three-dimensional tooth point cloud data and select a number of center points using an iterative farthest point sampling algorithm; The grouping layer is used to divide the downsampled three-dimensional tooth point cloud data into a plurality of regions using the plurality of center points; The PointNet layer is used to extract features from each region using a PointNet network to obtain global features; The sampling layer is specifically configured to: after downsampling the preprocessed tooth 3D point cloud data, randomly select a tooth 3D point cloud data as an initial point to form a selected sampling point set; calculate the distance between each unselected tooth 3D point cloud data and each selected tooth 3D point cloud data in the selected sampling point set, and add the unselected tooth 3D point cloud data corresponding to the maximum distance to the selected sampling point set until a preset number of center points are obtained; The grouping layer is specifically used to: for each of the center points, draw three circular areas around the center point at three different preset scales, each of which has a different radius and contains different amounts of three-dimensional tooth point cloud data; extract features from the circular areas of different scales, and then splice the extracted features to obtain the corresponding features of each of the center points; The PointNet layer is specifically used to: perform dimensionality enhancement on the corresponding features with a dimension of B*N*K*(d+C) to obtain output point cloud features with a dimension of B*N*K*C1, further perform a pooling operation on each circular area, and then use a multi-layer perceptron MLP to fully connect the pooling results to obtain global features; where B is the size of each batch, N and K are the number of point clouds and coordinates respectively, d+C is the number of channels, and C1 is the new number of channels after dimensionality enhancement.

7. An electronic device, characterized in that: The electronic device includes a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the tooth change detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the tooth change detection method according to any one of claims 1 to 5.

Images