3D oral cavity scanning data malformed tooth segmentation method and device based on implicit and explicit enhancement

Abstract

The invention discloses an implicit and explicit enhanced 3D oral cavity scanning data malformed tooth segmentation method and device, and belongs to the technical field of 3D image segmentation, and the method comprises the steps: carrying out the preprocessing of 3D oral cavity scanning data; extracting advanced features by using a double-flow trunk; implicit enhancement is carried out on the coordinate flow features and the normal flow features; performing preliminary segmentation by using the optimized features; and performing explicit enhancement on the initial probability distribution diagram, and outputting a final deformed tooth segmentation result. According to the method, an implicit enhancement module and an explicit enhancement module are introduced on the basis of an original double-flow architecture TSGCNet, and the problems of incomplete form segmentation and speckle noise are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of 3D image segmentation, and in particular relates to a method and device for segmenting deformed teeth in 3D oral scan data based on implicit and explicit enhancement. Background Art

[0002] In computer-assisted orthodontic diagnosis and treatment, accurately segmenting tooth structure from 3D oral scan data is a key step in enabling intelligent analysis and intervention. Traditional 3D tooth segmentation methods face numerous challenges, particularly in scenarios involving malformed teeth, severely impacting their clinical practicality.

[0003] Point cloud and mesh-based deep networks, such as PointNet, PointNet++, PointCNN, and MeshSegNet, have improved the ability to model 3D tooth morphology. However, these methods generally target standard dentition (full-arch) scenarios and are not specifically optimized for tooth deformities. Specifically, when models are applied to deformed teeth, the following problems often arise:

[0004] Incomplete segmentation: Because deformed teeth have only slight geometric differences from normal teeth (such as tilt, rotation, and local depression), the model has difficulty capturing morphological details, resulting in incomplete tooth boundaries or missing structures.

[0005] Severe speckle noise: Existing methods often make misjudgments when distinguishing teeth from gingival tissue or when processing local morphological changes in deformed teeth, generating a large number of discrete incorrectly labeled regions, affecting the overall segmentation quality;

[0006] In addition, the two-stream architecture proposed in recent years (such as TSGCNet and HiCA) has improved segmentation accuracy to a certain extent by processing different geometric attributes such as coordinates and normal vectors separately. However, when dealing with deformed teeth with complex morphology and irregular distribution, problems such as incomplete structure recognition and insufficient noise suppression still exist.

[0007] Therefore, existing technologies for segmenting deformed teeth still lack a specialized solution that can comprehensively capture morphological details, enhance semantic consistency, and suppress speckle noise. To address these issues, a new segmentation method that can integrate multi-level feature enhancement mechanisms is urgently needed. Summary of the Invention

[0008] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a method and device for segmenting deformed teeth based on 3D oral scan data based on implicit and explicit enhancement.

[0009] A first aspect of the present invention relates to a method for segmenting deformed teeth from 3D oral scan data based on implicit and explicit enhancement, comprising the following steps:

[0010] S1 pre-processes the 3D oral scan data to obtain dual-channel input features containing coordinate information and normal information;

[0011] S2 extracts coordinate flow features and normal flow features respectively through a dual-stream backbone network;

[0012] S3 implicitly enhances the coordinate flow features and the normal flow features, and constructs a K-nearest neighbor graph through the full dental arch segmentation data to perform morphological feature constraints;

[0013] S4 uses the optimized features to perform preliminary segmentation and generate an initial probability distribution map including deformed teeth, normal teeth and gums;

[0014] S5 explicitly enhances the initial probability distribution map, performs noise suppression and boundary optimization through the architecture-level constraints of the auxiliary full dental arch segmentation branch, and outputs the final deformed tooth segmentation result.

[0015] Preferably, the pre-processing of the 3D oral scan data in step S1 includes:

[0016] S11 converts the original oral scan data into a mesh representation containing vertex coordinates and normals;

[0017] S12 extracts the coordinates and normal information of the center point and three vertices of each grid cell and constructs a 24-dimensional feature vector (including a 12-dimensional coordinate vector and a 12-dimensional normal vector);

[0018] S13 performs data augmentation operations, including random rotation, translation, and grid density normalization.

[0019] Preferably, the step S2 of extracting the coordinate flow features and the normal flow features respectively through the dual-stream backbone network includes:

[0020] The S21 coordinate stream branch performs a three-layer graph attention operation on the input coordinate features, and the output dimension of each layer is [64, 128, 256];

[0021] The S22 normal flow branch performs a three-layer graph channel attention operation on the input normal features. Each layer includes a hierarchical feature aggregation operation, an adaptive dynamic channel attention mechanism to optimize feature expression operations, and a maximum pooling operation.

[0022] Preferably, the adaptive dynamic channel attention mechanism optimization feature expression described in step S22 includes:

[0023] S221 performs global average pooling on the input features to generate channel statistics;

[0024] S222 learns the nonlinear relationship between channels through a two-level fully connected layer and generates a channel weight vector;

[0025] S223 multiplies the weight vector by the original feature channel by channel to achieve feature recalibration.

[0026] Preferably, the implicit enhancement of the coordinate flow feature and the normal flow feature in step S3 includes:

[0027] S31 combines the coordinate flow and normal flow features and then reduces the dimension through a multi-layer perceptron to generate high-dimensional fusion features;

[0028] S32 performs matrix multiplication using high-dimensional features for full arch segmentation tasks Generate full arch prediction results;

[0029] S33 builds a KNN graph based on the prediction results

[0030] S34 enhances the coordinate flow features and normal flow features described in step S31 based on the constructed KNN graph, and calculates the geometric difference features between the grid point and the neighborhood and geometric difference characteristics of the normal and the neighborhood

[0031] Preferably, the step S5 of explicitly enhancing the initial probability distribution map includes:

[0032] S51 uses the auxiliary branch (using the HiCA network) to perform full dental arch segmentation and obtain the full dental arch segmentation result;

[0033] S52 identifies each tooth area and gum area according to the full dental arch segmentation result;

[0034] S53 calculates the proportion of deformed meshes in each tooth area and determines the deformity attribute of the entire tooth based on a preset threshold (35%);

[0035] S54 retains the segmentation result of the gingival area (RGB = (255, 255, 255, 255)) unchanged and fuses the reclassified tooth attributes to generate an optimized segmentation map.

[0036] The second aspect of the present invention relates to a device for segmenting deformed teeth based on 3D oral scan data based on implicit and explicit enhancement, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the method for segmenting deformed teeth based on 3D oral scan data based on implicit and explicit enhancement of the present invention.

[0037] The innovative features of the present invention are:

[0038] (1) Dual-stream feature enhancement mechanism: In the dual-stream backbone network, the normal stream branch innovatively introduces an adaptive dynamic channel attention mechanism (ACA), which significantly improves the feature expression ability of subtle normal changes of deformed teeth through global average pooling and dynamic learning of channel weights in two-level fully connected layers.

[0039] (2) Implicit enhancement module: the first to construct a K-nearest neighbor graph through the full dental arch segmentation task Calculation using geometric difference features and Implement morphological constraints and strengthen implicit learning of the complete structure of teeth.

[0040] (3) Explicit enhancement module: Based on the full dental arch segmentation results, through tooth region-level reclassification (deformed mesh proportion > 35% to determine deformed attributes), the gingival region preservation (RGB = (255, 255, 255, 255)) and tooth attribute optimization are integrated at the decision layer to achieve speckle noise suppression and precise boundary correction.

[0041] The working principle of the present invention is:

[0042] Collaboratively improve segmentation accuracy through a two-stage enhancement framework.

[0043] (1) Implicit enhancement stage (feature space optimization): The dual-stream backbone network extracts high-dimensional features of the coordinate stream (3-layer graph attention) and the normal stream (3-layer ACA mechanism) respectively. After splicing the dual-stream features, the KNN graph (G full ), calculate the geometric difference characteristics between the center point and the neighborhood (F c_knn / F n_knn ), constraining the model to learn the overall tooth morphology prior.

[0044] (2) Explicit enhancement stage (decision space optimization): Using the initial probability distribution map generated by the preliminary segmentation, full arch segmentation is performed through the auxiliary branch (HiCA network). Based on the full arch segmentation results, independent tooth regions are identified, the proportion of deformed meshes is counted and a threshold is applied (>35% is marked as RGB = (255, 0, 0, 255)), and the invariant characteristics of the gingival region are integrated to output a segmentation result with complete structure and clear boundaries.

[0045] The advantages of the present invention are: it improves the accuracy of automatic segmentation of deformed teeth in 3D oral scan data, especially when facing deformed teeth with complex morphology and irregular spatial distribution, it can significantly reduce the problems of incomplete morphological segmentation and speckle noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the method of the present invention;

[0047] Figure 2 This is a schematic diagram of the deep learning model structure of the present invention;

[0048] Figure 3 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0051] Example 1

[0052] In order to overcome the problems of incomplete tooth morphology segmentation and speckle noise in the existing technology for deformed tooth segmentation in 3D oral scan data, this embodiment provides a deformed tooth segmentation method for 3D oral scan data based on implicit and explicit enhancement to improve the accuracy of deformed tooth segmentation.

[0053] As shown in the figure, the method of the present invention includes the following steps:

[0054] S1 pre-processes the 3D oral scan data to obtain dual-channel input features containing coordinate information and normal information;

[0055] S2 extracts coordinate flow features and normal flow features respectively through a dual-stream backbone network;

[0056] S3 implicitly enhances the coordinate flow features and the normal flow features, and constructs a K-nearest neighbor graph through the full dental arch segmentation data to perform morphological feature constraints;

[0057] S4 uses the optimized features to perform preliminary segmentation and generate an initial probability distribution map including deformed teeth, normal teeth and gums;

[0058] S5 explicitly enhances the initial probability distribution map, performs noise suppression and boundary optimization through the architecture-level constraints of the auxiliary full dental arch segmentation branch, and outputs the final deformed tooth segmentation result.

[0059] The pre-processing of the 3D oral scan data in step S1 includes:

[0060] S11 converts the original oral scan data into a mesh representation containing vertex coordinates and normals;

[0061] S12 extracts the coordinates and normal information of the center point and three vertices of each grid cell and constructs a 24-dimensional feature vector (including a 12-dimensional coordinate vector and a 12-dimensional normal vector);

[0062] S13 performs data augmentation operations, including random rotation, translation, and grid density normalization.

[0063] like Figure 2 As shown, the dual-stream backbone of the deep learning model of the present invention follows the dual-stream graph convolutional network architecture proposed by TSGCNet to learn more discriminative feature representations from the geometric features of the 3D tooth model. This architecture includes two parallel learning streams, processing 3D coordinate and normal vector data respectively. This design aims to solve the mutual interference problem that may be caused by mixed geometric inputs, while making full use of the complementary geometric information provided by coordinates and normal vectors. The extraction of coordinate stream features and normal stream features respectively through the dual-stream backbone network in step S2 includes:

[0064] The S21 coordinate stream branch performs a three-layer graph attention operation on the input coordinate features, and the output dimension of each layer is [64, 128, 256];

[0065] The S22 normal flow branch performs a three-layer graph channel attention operation on the input normal features. Each layer includes a hierarchical feature aggregation operation, an adaptive dynamic channel attention mechanism to optimize feature expression operations, and a maximum pooling operation.

[0066] In order to fully capture the information of complex geometric forms and give full play to the implicit enhancement effect of the coordinate flow features and normal flow features described in step S3, the model introduces a dynamic channel attention mechanism (ACA) in the N-stream (normal flow) of the dual-stream backbone. This mechanism adjusts the importance of each channel in an adaptive manner, thereby improving the performance of the model. The adaptive dynamic channel attention mechanism described in step S22 optimizes the feature expression including:

[0067] S221 performs global average pooling on the input features to generate channel statistics. B×C×H×W Perform a global average pooling operation to obtain a tensor of size B × C × 1 × 1. This operation calculates the average value of each channel and compresses the spatial dimensions to obtain global information at the channel level.

[0068] Y=AvgPool2d(X)∈R B×C×1×1

[0069] S222 uses two fully connected layers to learn the nonlinear relationship between channels and generate a channel weight vector. The average pooling results are processed through a two-layer fully connected network. Specifically, the input Y is reduced from the number of channels C to C / reduction_ratio through the first fully connected layer. The ReLU activation function is then applied, and the original number of channels C is restored through the second fully connected layer. Finally, the output is mapped to the range [0, 1] using the Sigmoid function to generate adaptive weights for each channel.

[0070] S = σ(W2·ReLU(W1·Y))∈R B×C×1×1

[0071] Among them, W1 and W2 are the weight matrices of the two fully connected layers, σ is the Sigmoid activation function, and S is the channel adaptive weight;

[0072] S223 multiplies the weight vector by the original feature channel by channel to achieve feature recalibration. The input feature map X is multiplied by the generated channel adaptive weight S to adjust the importance of each channel. This process can be expressed as:

[0073] X out =X⊙S

[0074] Where ⊙ represents element-by-element multiplication. Finally, the output feature map X after the dynamic channel attention mechanism is processed out The original dimension order is returned for use by subsequent network layers.

[0075] The core idea of ​​this mechanism is to improve the model's ability to focus on key information by adaptively adjusting the weight of each channel, thereby improving overall performance. In implementation, ACA combines global average pooling with a simple fully connected network structure to dynamically generate channel weights, with low computational complexity.

[0076] like Figure 2 As shown, in order to improve the ability to capture the complete morphology of teeth in deformed teeth segmentation, the present invention designs an implicit enhancement module, which provides implicit constraints on the deformed teeth segmentation task by utilizing the auxiliary information of the full dental arch segmentation. The implicit enhancement module uses the full dental arch segmentation data as the loss term of the prototype space to constrain the deformed teeth segmentation loss to enhance the model's ability to capture the complete morphology of teeth and make full use of the relationship between the deformed teeth segmentation task and the implicit full dental arch segmentation task, that is, before distinguishing deformed teeth from normal teeth, each tooth needs to be distinguished first. The implicit enhancement of the coordinate flow features and the normal flow features described in step S3 includes:

[0077] S31 combines the coordinate flow and normal flow features and then reduces the dimension through a multi-layer perceptron to generate high-dimensional fusion features. coor∈R N×512 and F nor ∈R N×512 Merge into a feature vector F∈R through concatenation N×1024 , where N is the number of grids in the input grid data. Then, three different MLPs are used to reduce the dimension and obtain the high-level features F high ∈R N ×128 ;

[0078] S32 performs matrix multiplication using high-dimensional features for full arch segmentation tasks Generate the full dental arch prediction result. The hidden layer and the classification layer respectively use MLP to analyze the high-level features F in step S31. high ∈R N×128 Activate and normalize to get F hidden ∈R N×128 and F class ∈R N×33 , where 33 represents the number of categories for full arch segmentation. hidden and F class Through multiple matrix multiplications, the prediction result N of the full dental arch segmentation is obtained c ∈R N ×33 :

[0079]

[0080] in, represents matrix multiplication, T represents matrix transpose;

[0081] S33 builds a KNN graph based on the prediction results By matrix multiplication again, the first 16 nearest neighbor points of each point are selected to generate the KNN graph G with the full dental arch information. full ∈R N×16 :

[0082]

[0083] The full arch auxiliary information is then used in subsequent calculations in the form of a KNN graph, enhancing the model's ability to capture the complete tooth morphology. This is the core of implicit enhancement. Constraining the segmentation of deformed teeth using a KNN graph for full arch segmentation is more effective than directly constraining the segmentation of deformed teeth using features. This approach does not deviate from the model's primary task of segmenting deformed teeth, while still providing a good constraint.

[0084] S34 enhances the coordinate flow features and normal flow features described in step S31 based on the constructed KNN graph, and calculates the geometric difference features between the grid point and the neighborhood and geometric difference characteristics of the normal and the neighborhood Specifically, the high-level features F of the C-stream and N-stream inputs of the module coor ∈R N×512 and F nor ∈R N×512 New features will be calculated based on the KNN graph with auxiliary information of the entire dental arch. Using the symbol C i 、C ij and N i 、N ij Represent the input features of the central node and its neighbors of the KNN graph of C-stream and N-stream respectively, and calculate the new C-stream feature F c_knn ∈R N ×1024 and N-stream feature F n_knn ∈R N×1024 :

[0085]

[0086] F c_knn ∈R N×1024 and F n_knn ∈R N×1024 Then, MLP is used to reduce the dimension and obtain the new feature F c_knn ∈R N ×512 and F n_knn ∈R N×512 In C flow F c_knn ∈R N×512 The feature expression ability is further enhanced through the graph attention mechanism to obtain the enhanced feature F c_en ∈R N×512 :

[0087] F c_en =f GAT (F c_knn ,G full )

[0088] Among them, f GAT It is a multi-head attention mechanism based on graph (K=16), through the adjacency relationship G full Calculate the weighted feature representation. In N streams, directly perform maximum pooling on the neighborhood features to obtain the enhanced feature F n_en ∈R N×512 :

[0089] F n_en =max(F n_knn ,dim=-1)

[0090] At this point, the input feature F of the implicit enhancement module coor ∈R N×512 and F nor ∈R N×512After the module, the feature F after the full dental arch auxiliary information enhancement is obtained c_en ∈R N×512 and F n_en ∈R N×512 .

[0091] like Figure 2 As shown, the explicit enhancement module is a bridge connecting the main path of deformed tooth segmentation and the branch of full dental arch segmentation. The goal of the explicit enhancement module is to classify and optimize the RGB values ​​of the target area by combining the prediction results of the full dental arch segmentation network with the deformed tooth segmentation results, thereby eliminating point noise and improving the segmentation performance of the deformed teeth. The explicit enhancement of the initial probability distribution map described in step S5 includes:

[0092] S51 uses the auxiliary branch (using the HiCA network) to perform full dental arch segmentation and obtain the full dental arch segmentation result;

[0093] S52 identifies each tooth region and gum region based on the full dental arch segmentation results. The meshes with the same RGB values ​​in the full dental arch segmentation results constitute the gums and each tooth. The explicit enhancement module first classifies and counts these meshes.

[0094] S53 calculates the proportion of deformed meshes in each tooth area, and determines the deformity attribute of the entire tooth based on a preset threshold (35%). In the prediction results of the deformed tooth segmentation network, the mesh marked in red (255, 0, 0, 255) represents the part that is judged to be a deformed tooth. The explicit enhancement module counts the proportion of the meshes of each tooth in the full dental arch segmentation result that are judged to be deformed by the deformed tooth segmentation network. If this proportion exceeds a certain threshold (for example, 35%), the tooth is considered to be a deformed tooth, and it is marked as red (255, 0, 0, 255) as a whole; otherwise, it is considered to be a normal tooth, and it is marked as green (0, 255, 0, 255) as a whole;

[0095] S54 retains the segmentation result of the gingival area (RGB = (255, 255, 255, 255)) unchanged and fuses the reclassified tooth attributes to generate an optimized segmentation map.

[0096] Example 2

[0097] like Figure 2 This embodiment relates to a device for segmenting deformed teeth based on 3D oral scan data based on implicit and explicit enhancement, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the method for segmenting deformed teeth based on 3D oral scan data based on implicit and explicit enhancement of Example 1.

[0098] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for segmenting deformed teeth from 3D oral scan data based on implicit and explicit enhancement, characterized in that: The steps include: S1 pre-processes the 3D oral scan data to obtain dual-channel input features containing coordinate information and normal information; S2 extracts coordinate flow features and normal flow features respectively through a dual-stream backbone network; S3 implicitly enhances the coordinate flow features and the normal flow features, and constructs a K-nearest neighbor graph through the full dental arch segmentation data to perform morphological feature constraints; S4 uses the optimized features to perform preliminary segmentation and generate an initial probability distribution map including deformed teeth, normal teeth and gums; S5 explicitly enhances the initial probability distribution map, performs noise suppression and boundary optimization through the architecture-level constraints of the auxiliary full dental arch segmentation branch, and outputs the final deformed tooth segmentation result.

2. The method for segmenting deformed teeth based on implicit and explicit enhancement in 3D oral scan data according to claim 1, wherein: The pre-processing of the 3D oral scan data in step S1 includes: S11 converts the original oral scan data into a mesh representation containing vertex coordinates and normals; S12 extracts the coordinates and normal information of the center point and three vertices of each grid cell and constructs a 24-dimensional feature vector (including a 12-dimensional coordinate vector and a 12-dimensional normal vector); S13 performs data augmentation operations, including random rotation, translation, and grid density normalization.

3. The method for segmenting deformed teeth based on implicit and explicit enhancement in 3D oral scan data according to claim 1, wherein: The extraction of coordinate flow features and normal flow features by the dual-stream backbone network in step S2 includes: The S21 coordinate stream branch performs a three-layer graph attention operation on the input coordinate features, and the output dimension of each layer is [64, 128, 256]; The S22 normal flow branch performs a three-layer graph channel attention operation on the input normal features. Each layer includes a hierarchical feature aggregation operation, an adaptive dynamic channel attention mechanism to optimize feature expression operations, and a maximum pooling operation.

4. The method for segmenting deformed teeth based on implicit and explicit enhancement in 3D oral scan data according to claim 3, wherein: The adaptive dynamic channel attention mechanism optimization feature expression described in step S22 includes: S221 performs global average pooling on the input features to generate channel statistics; S222 learns the nonlinear relationship between channels through a two-level fully connected layer and generates a channel weight vector; S223 multiplies the weight vector by the original feature channel by channel to achieve feature recalibration.

5. The method for segmenting deformed teeth based on implicit and explicit enhancement in 3D oral scan data according to claim 1, wherein: The implicit enhancement of the coordinate flow feature and the normal flow feature in step S3 includes: S31 combines the coordinate flow and normal flow features and then reduces the dimension through a multi-layer perceptron to generate high-dimensional fusion features; S32 performs matrix multiplication using high-dimensional features for full arch segmentation tasks Generate full arch prediction results; S33 builds a KNN graph based on the prediction results. S34 enhances the coordinate flow features and normal flow features described in step S31 based on the constructed KNN graph, and calculates the geometric difference features between the grid point and the neighborhood and geometric difference characteristics of the normal and the neighborhood 6. The method for segmenting deformed teeth based on implicit and explicit enhancement in 3D oral scan data according to claim 1, wherein: The explicit enhancement of the initial probability distribution map in step S5 includes: S51 uses auxiliary branches to perform full arch segmentation and obtain full arch segmentation results; S52 identifies each tooth area and gum area according to the full dental arch segmentation result; S53 calculates the proportion of deformed meshes in each tooth area and determines the deformity attribute of the entire tooth based on a preset threshold (35%); S54 retains the segmentation result of the gingival area (RGB = (255, 255, 255, 255)) unchanged and fuses the reclassified tooth attributes to generate an optimized segmentation map.

7. A device for segmenting deformed teeth based on implicit and explicit enhancement of 3D oral scan data, characterized by: The method comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the method for segmenting deformed teeth based on implicit and explicit enhancement of 3D oral scan data according to any one of claims 1 to 7.

Images