Two-stage CBCT tooth segmentation method based on multi-direction multi-scale feature fusion network

Abstract

The present application belongs to the technical field of dental image segmentation, and particularly relates to a two-stage CBCT tooth segmentation method based on a multi-direction multi-scale feature fusion network, comprising the following steps: obtaining an oral CBCT data set and performing pretreatment; constructing a multi-direction multi-scale feature fusion module M2FF-Block; constructing two-stage multi-direction multi-scale feature fusion networks M2FF-UNet based on the M2FF-Block; performing tooth recognition under a global perspective to obtain the position of the centroid of each tooth; cutting the original CBCT image according to the centroid position to obtain tooth block data corresponding to each tooth, and performing fine segmentation on a single tooth block in combination with a center focusing mechanism; constructing a post-processing algorithm suitable for tooth segmentation problems, and correcting the segmentation result to obtain a final segmentation result. The present application can effectively utilize the inherent multi-direction multi-scale features of teeth, and also realizes a balance between context information capture and calculation efficiency, so that the positioning and segmentation of teeth are more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of tooth image segmentation, and in particular relates to a two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network. Background Art

[0002] Cone-beam computed tomography (CBCT), as an advanced three-dimensional imaging technology, has been widely used in dentistry. Compared with traditional two-dimensional images, CBCT provides higher-resolution three-dimensional images, helping doctors to more accurately evaluate teeth, bones, and surrounding tissues, and providing more comprehensive information support for clinical diagnosis and treatment planning. Therefore, high-precision tooth recognition and segmentation technology for CBCT images has become the basis for accurate diagnosis and personalized treatment. However, the natural arrangement of human teeth may lead to dense crowding or overlap between adjacent teeth, blurring tooth boundaries and making it difficult to identify the outline of each tooth. In addition, in actual applications, due to the presence of metal objects such as metal restorations and orthodontic brackets in the mouths of some patients, these metal objects will produce serious artifacts in CBCT images, resulting in a significant decrease in image quality. This makes it difficult for doctors to accurately identify and evaluate the tooth structure in the affected area, increasing the difficulty of diagnosis and treatment planning.

[0003] Research on tooth segmentation in CBCT images has made considerable progress. Traditional image processing methods rely primarily on techniques such as edge detection and threshold segmentation, but often struggle to achieve good segmentation results when processing complex structures or images with high noise levels. In recent years, the rapid development of deep learning technology, particularly the widespread application of convolutional neural networks (CNNs), has enabled the automatic learning of multi-level features in images through its powerful feature learning capabilities. This allows for a better understanding of image content, more accurate identification of complex structures, and ultimately high-precision segmentation.

[0004] However, current deep learning-based methods still have the following shortcomings: Existing methods struggle to capture the inherent directional characteristics of teeth, namely, the differences in tooth morphology and structure across different spatial directions, making accurate tooth recognition and segmentation difficult. Furthermore, current two-stage instance-based segmentation methods fail to adequately address the segmentation of central teeth during fine-grained segmentation of tooth blocks, resulting in adjacent teeth appearing glued together in the segmentation results. Existing technical solutions also struggle to achieve high-quality segmentation of densely packed, crowded, missing, misaligned teeth, and areas with metal artifacts. Summary of the Invention

[0005] In response to the technical problems existing in the above-mentioned traditional image processing methods, the present invention provides a two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network includes the following steps:

[0008] S1. Obtain oral CBCT dataset and preprocess it;

[0009] S2, build a multi-directional and multi-scale feature fusion module M2FF-Block;

[0010] S3, based on M2FF-Block, construct two stages of multi-directional and multi-scale feature fusion network M2FF-UNet;

[0011] S4. Perform tooth recognition from a global perspective to obtain the center of mass of each tooth;

[0012] S5. Crop the original CBCT image according to the centroid position to obtain the tooth block data corresponding to each tooth, and perform fine segmentation of the single tooth block in combination with the center focusing mechanism;

[0013] S6. Construct a post-processing algorithm suitable for the tooth segmentation problem, and correct the segmentation result to obtain the final segmentation result.

[0014] The method for constructing the multi-directional and multi-scale feature fusion module M2FF-Block in S2 is:

[0015] First, a multi-directional and multi-scale feature extraction module (M2FCM) is constructed to systematically extract multi-scale features in different directions. Then, a spatial focusing module (SFM) is constructed to adaptively emphasize key spatial positions. Then, the input and the output of the two modules are multiplied and added together to obtain enhanced multi-directional and multi-scale features.

[0016] The construction method of M2FCM in S3 is:

[0017] The input feature vector is divided into 5 branches. The first branch retains the original feature information without any processing. The second branch uses 3×3×3 depth convolution to extract local spatial features. The remaining three branches use large-kernel depth convolution blocks of different sizes of 1×1×k, 1×k×1, and k×1×1 to replace the k×k×k convolution block to extract multi-directional and multi-scale features along different directions. Large-kernel convolution is used to perceive contextual information, and depth convolution is used to improve computational efficiency. These multi-directional and multi-scale features are then aggregated through 1×1×1 convolution. Finally, all outputs are spliced ​​in the channel dimension to achieve feature fusion.

[0018] The structure of the M2FCM includes:

[0019] Branch 1: No processing;

[0020] Branch 2: Convolutional layer br2_conv1, with a kernel size of 3×3×3, a stride of 1, and the number of groups equal to the number of input channels;

[0021] Branch three: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer brb3_conv1 has a convolution kernel size of 1×1×7, a stride of 1, padding of (0, 0, 3), and the number of groups is the number of input channels; the second convolutional layer br3_conv2 has a convolution kernel size of 1×1×9, a stride of 1, padding of (0, 0, 4), and the number of groups is the number of input channels; the third convolutional layer br3_conv3 has a convolution kernel size of 1×1×11, a stride of 1, padding of (0, 0, 5), and the number of groups is the number of input channels;

[0022] Branch 4: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer br4_conv1 has a convolution kernel size of 1×7×1, a stride of 1, padding of (0, 3, 0), and the number of groups is the number of input channels; the second convolutional layer br4_conv2 has a convolution kernel size of 1×9×1, a stride of 1, padding of (0, 4, 0), and the number of groups is the number of input channels; the third convolutional layer br4_conv3 has a convolution kernel size of 1×11×1, a stride of 1, padding of (0, 5, 0), and the number of groups is the number of input channels;

[0023] Branch 5: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer br5_conv1 has a convolution kernel size of 7×1×1, a stride of 1, and padding of (3, 0, 0). The number of groups is the number of input channels. The second convolutional layer br5_conv2 has a convolution kernel size of 9×1×1, a stride of 1, and padding of (4, 0, 0). The number of groups is the number of input channels. The third convolutional layer br5_conv3 has a convolution kernel size of 11×1×1, a stride of 1, and padding of (5, 0, 0). The number of groups is the number of input channels.

[0024] Multi-directional and multi-scale feature fusion: The outputs of the third, fourth, and fifth branches are aggregated through a 1×1×1 convolutional layer with a convolution kernel of 1 and a stride of 1. The aggregated features are concatenated with the outputs of the first and second branches in the channel dimension.

[0025] The steps for constructing the spatial focusing module SFM are as follows:

[0026] First, a three-dimensional average pooling layer is used to compress spatial information, and a 1×1×1 convolution is used to refine the interaction and integration between channels. In order to capture cross-directional spatial information, 1×1×k, 1×k×1, and k×1×1 convolutions are applied along the height, width, and depth directions, respectively, to focus on feature changes specific to each spatial direction. Finally, an attention map is generated through 1×1×1 convolution and sigmoid activation to guide the focus on spatially important areas in the multi-directional and multi-scale features extracted by M2FCM.

[0027] The structure of the spatial focusing module SFM includes:

[0028] The first pooling layer, pool1, uses average pooling with a window size of 7, a stride of 1, and a padding of 3;

[0029] The first convolutional layer is conv1, with a convolution kernel size of 1×1×1 and a stride of 1;

[0030] The second convolutional layer conv2 has a convolution kernel size of 1×1×11, a stride of 1, padding of (0, 0, 5), and the number of groups is the number of input channels;

[0031] The third convolutional layer conv3 has a convolution kernel size of 1×11×1, a stride of 1, padding of (0, 5, 0), and the number of groups is the number of input channels;

[0032] The fourth convolutional layer conv4 has a kernel size of 11×1×1, a stride of 1, padding of (5, 0, 0), and the number of groups is the number of input channels;

[0033] The fifth convolutional layer conv5 has a convolution kernel size of 1×1×1 and a stride of 1;

[0034] Finally, the sigmoid function outputs an attention map from 0 to 1.

[0035] The method of the two-stage multi-directional and multi-scale feature fusion network M2FF-Une in S3 is as follows: first, the convolution blocks of the four encoder layers in the UNet architecture are replaced with different numbers of M2FF-Blocks; for the tooth recognition M2FF-UNet in the tooth recognition stage, two decoder branches are constructed, one branch is used to segment the global tooth area, and the other branch is used to perform the regression task of the offset of each voxel relative to its center of mass; for the tooth segmentation M2FF-UNet in the tooth segmentation stage, two decoder branches are constructed, one branch is used to segment the tooth area within the tooth block, and the other branch is used to segment the boundary area within the tooth block.

[0036] The method for obtaining the position of the centroid of each tooth in S4 is as follows: first obtain the global binary segmentation map and the global voxel centroid offset map output by the tooth recognition subnetwork, then map all predicted foreground tooth voxels to new coordinates according to the offset map, count the pointing frequencies of each new coordinate to obtain a density map P, then calculate the minimum distance from each voxel to the voxel with a larger density value, construct a distance map D, and finally select points with a density value greater than 60 and a distance value greater than 10 as the tooth centroid.

[0037] The center-focusing mechanism in S5 includes remapping the label category of the tooth block in the tooth segmentation stage and then inputting it into the tooth segmentation M2FF-UNet, setting the tooth category located at the center of the tooth block as the central tooth, and changing the corresponding label value to 1. All teeth except the central tooth in the tooth block are set as adjacent teeth, and the corresponding label value is changed to 2. The label value corresponding to the background remains unchanged at 0. The tooth block segmentation task is a three-category segmentation task, which will ensure that the model's attention is focused on the segmentation of the central tooth.

[0038] The construction method of the post-processing algorithm suitable for the tooth segmentation problem in S6 is: traverse each centroid point as the target centroid, find the coordinates of the 6 adjacent centroid points closest to the target centroid through the kd tree algorithm, and then calculate the Euclidean distance of all voxels in the tooth block to these 7 points, and obtain the centroid index corresponding to the minimum distance. All voxels corresponding to non-zero indexes are put into set 1, and then find all voxels in the tooth block that have the same predicted category as the target centroid and put them into set 2. The voxels common to set 1 and set 2 are deleted in set 1. Finally, all accelerated prediction categories in set 1 are modified to background.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The multi-directional and multi-scale feature fusion module M2FF-Block proposed in this paper can effectively utilize the inherent multi-directional and multi-scale features of teeth, and also achieve a balance between contextual information capture and computational efficiency, thereby making tooth positioning and segmentation more accurate.

[0041] 2. The center focusing mechanism proposed in the present invention can enable the model to focus on the segmentation of the central teeth, improve the segmentation effect on the boundaries of the central teeth, make the boundaries between adjacent teeth clearer, improve the refined segmentation accuracy of the tooth block, and thus improve the overall segmentation performance.

[0042] 3. The post-processing algorithm proposed in the present invention starts from the spatial relationship between the centroid and the nearest neighbor centroid, and adjusts the predicted category of the foreground voxels in the predicted fuzzy area according to the distance from the foreground voxels to the target centroid, further ensuring the accurate segmentation between adjacent teeth, thereby effectively solving the adhesion problem between adjacent teeth. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can, without inventive effort, derive other implementation drawings based on the provided drawings.

[0044] The structures, proportions, sizes, etc. illustrated in this specification are intended solely to complement the contents disclosed herein and to facilitate understanding and reading by persons skilled in the art. They are not intended to limit the conditions under which the present invention may be implemented and therefore have no substantive technical significance. Any structural modifications, changes in proportions, or adjustments in sizes, without affecting the efficacy and objectives of the present invention, shall remain within the scope of the technical contents disclosed herein.

[0045] Figure 1 is a flow chart of the method of the present invention;

[0046] Figure 2 This is a schematic diagram of the structure of the M2FF-Block proposed in the present invention;

[0047] Figure 3 This is the structural diagram of the M2FF-UNet proposed in this invention;

[0048] Figure 4 This is a flow chart of the post-processing method proposed in the present invention;

[0049] Figure 5 This is the segmentation result diagram of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of this application, not all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0051] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following embodiments are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0052] like Figures 1 to 5As shown, the present invention proposes a two-stage CBCT tooth instance segmentation method based on a multi-directional multi-scale feature fusion network (M2FF-UNet), comprising the following steps:

[0053] Step 1: Dataset Preparation

[0054] 1.1 Data Collection

[0055] A total of 129 CBCT datasets were collected from public databases, and 305 CBCT datasets were collected from the oral radiology department of a certain hospital's oral center with the patients' consent.

[0056] 1.2 Data Preprocessing

[0057] The CBCT dataset collected from the hospital was subjected to gold standard labeling by dentists. The physical resolution of all CBCT images was standardized to 0.4 mm, and the intensity values ​​were limited to 500 to 2500 before normalization.

[0058] Step 2: Design of the multi-directional and multi-scale feature fusion module M2FF-Block

[0059] 2.1 Design of Multi-directional and Multi-scale Feature Extraction Module M2FCM

[0060] Branch 1: No processing;

[0061] Branch 2: Convolutional layer br2_conv1, with a kernel size of 3×3×3, a stride of 1, and the number of groups equal to the number of input channels;

[0062] Branch three: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer brb3_conv1 has a convolution kernel size of 1×1×7, a stride of 1, padding of (0, 0, 3), and the number of groups is the number of input channels; the second convolutional layer br3_conv2 has a convolution kernel size of 1×1×9, a stride of 1, padding of (0, 0, 4), and the number of groups is the number of input channels; the third convolutional layer br3_conv3 has a convolution kernel size of 1×1×11, a stride of 1, padding of (0, 0, 5), and the number of groups is the number of input channels;

[0063] Branch 4: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer br4_conv1 has a convolution kernel size of 1×7×1, a stride of 1, padding of (0, 3, 0), and the number of groups is the number of input channels; the second convolutional layer br4_conv2 has a convolution kernel size of 1×9×1, a stride of 1, padding of (0, 4, 0), and the number of groups is the number of input channels; the third convolutional layer br4_conv3 has a convolution kernel size of 1×11×1, a stride of 1, padding of (0, 5, 0), and the number of groups is the number of input channels;

[0064] Branch 5: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer br5_conv1 has a convolution kernel size of 7×1×1, a stride of 1, and padding of (3, 0, 0). The number of groups is the number of input channels. The second convolutional layer br5_conv2 has a convolution kernel size of 9×1×1, a stride of 1, and padding of (4, 0, 0). The number of groups is the number of input channels. The third convolutional layer br5_conv3 has a convolution kernel size of 11×1×1, a stride of 1, and padding of (5, 0, 0). The number of groups is the number of input channels.

[0065] Multi-directional and multi-scale feature fusion: The outputs of the third, fourth, and fifth branches are aggregated through a 1×1×1 convolutional layer with a convolution kernel of 1 and a stride of 1. The aggregated features are concatenated with the outputs of the first and second branches in the channel dimension.

[0066] 2.2 Spatial Focusing Module (SFM) Design

[0067] The first pooling layer, pool1, uses average pooling with a window size of 7, a stride of 1, and a padding of 3;

[0068] The first convolutional layer is conv1, with a convolution kernel size of 1×1×1 and a stride of 1;

[0069] The second convolutional layer conv2 has a convolution kernel size of 1×1×11, a stride of 1, padding of (0, 0, 5), and the number of groups is the number of input channels;

[0070] The third convolutional layer conv3 has a convolution kernel size of 1×11×1, a stride of 1, padding of (0, 5, 0), and the number of groups is the number of input channels;

[0071] The fourth convolutional layer conv4 has a kernel size of 11×1×1, a stride of 1, padding of (5, 0, 0), and the number of groups is the number of input channels;

[0072] The fifth convolutional layer conv5 has a convolution kernel size of 1×1×1 and a stride of 1;

[0073] Finally, the sigmoid function outputs an attention map from 0 to 1.

[0074] 2.3 Overall structural design

[0075] The input feature vector is input to two branches at the same time. The first branch systematically extracts multi-scale features in different directions through the Multi-Directional Multi-Scale Feature Extraction Module (M2FCM). The second branch adaptively emphasizes key spatial positions through the Spatial Focusing Module (SFM). The enhanced features obtained by multiplying the input features with the output results of the two branches are then added together to obtain the enhanced multi-directional multi-scale features.

[0076] Step 3. Structural design of tooth recognition M2FF-UNet and tooth segmentation M2FF-UNet

[0077] The convolution blocks in the four encoder layers of UNet are replaced with the multi-directional and multi-scale feature integration module M2FF-Block. The first encoder layer contains 3 M2FF-Blocks, the second encoder layer contains 3 M2FF-Blocks, the third encoder layer contains 15 M2FF-Blocks, and the fourth encoder layer contains 3 M2FF-Blocks.

[0078] For tooth recognition M2FF-UNet, two different decoder branches are constructed. The first branch is used to segment the global tooth area, and the other branch is used to regress the offset of each voxel relative to its center of mass. The loss function of tooth recognition M2FF-UNet during training is:

[0079] Lid=αLbseg+βLoff

[0080] Lbseg=0.8×Ldice+0.2×Lce

[0081] Among them, L bseg is the global binary segmentation loss, L dice is the dice loss, L ce is the cross entropy loss,

[0082] L off For the regression offset loss, L1 loss is used with α being 0.65 and β being 0.35.

[0083] For tooth segmentation M2FF-UNet, two different decoder branches are constructed. The first branch is used to segment the tooth area within the tooth block, and the other branch is used to segment the boundary area within the tooth block. The loss function of tooth segmentation M2FF-UNet during training is:

[0084] Lseg=αLtooth+βLbd

[0085] Ltooth=0.6×Ldice+0.4×Lce

[0086] Lbd=0.6×Ldice+0.4×Lce

[0087] Among them, L tooth is the segmentation loss of the tooth region in the tooth block, L dice is the dice loss, L ce is the cross entropy loss, L bd is the boundary region segmentation loss in the tooth block, α is 0.65, and β is 0.35.

[0088] Step 4: Identify teeth from a global perspective

[0089] The specific steps for tooth recognition from a global perspective to obtain the center of mass of each tooth are:

[0090] 1. First, obtain the global binary segmentation map and the centroid offset map of all voxels output by the tooth recognition subnetwork;

[0091] 2. Accumulate all predicted foreground tooth voxels according to the offset map to obtain the mapped new coordinates;

[0092] 3. Count the pointing frequencies of each new coordinate to obtain the density map P;

[0093] 4. Retain voxels with density values ​​greater than 20 as potential centroids;

[0094] 5. Calculate the minimum distance from each voxel in the potential centroid to the voxel with a larger density value, construct a distance map D, and filter points with a density value greater than 60 and a distance value greater than 10 as the tooth centroid.

[0095] Step 5: Finely segment individual tooth blocks

[0096] Based on the position of the tooth centroid obtained in the tooth recognition stage, the original image is centrally cropped to obtain an 80×80×128 tooth block centered on each tooth. The present invention takes into account that the number of teeth contained in each tooth block is not fixed, and we are only concerned with the segmentation of the central teeth of the tooth block. Existing methods directly regard it as a two-category segmentation task, and then rely on some morphological algorithms or connected domain analysis to separate the central teeth. This will result in unclear boundaries between the central teeth and adjacent teeth, resulting in segmentation adhesion. Therefore, the label category of the tooth block must be remapped. The category of the tooth located at the center of the tooth block is set to "central tooth", and the corresponding label value is modified to 1. The category of all teeth in the tooth block except the central tooth is set to "adjacent teeth", and the corresponding label value is 2. The label value corresponding to "background" is 0. The tooth block segmentation task becomes a three-category segmentation task, ensuring that the model focuses on the segmentation of the central teeth.

[0097] Step 6: Construct a post-processing algorithm suitable for tooth segmentation and correct the segmentation results. The specific steps of the post-processing algorithm are as follows:

[0098] 1. Traverse each centroid point as the target centroid, and use the kd tree algorithm to find the 6 adjacent centroid points closest to the target centroid;

[0099] 2. Calculate the Euclidean distance from all voxels in the tooth block to these 7 points and obtain the centroid index corresponding to the minimum distance. If the index value corresponding to a voxel is 0, it means that the centroid closest to the voxel is the target centroid, and the voxel belongs to the center tooth and needs to be retained. Otherwise, the voxel does not belong to the center tooth, and the predicted label value should be changed to 0;

[0100] 3. Store all voxels corresponding to non-zero indexes into set 1;

[0101] 4. Find all voxels in the tooth block that have the same category as the target centroid prediction and store them in set 2;

[0102] 5. Delete the voxels common to Set 1 and Set 2 from Set 1. This step is to maintain the integrity of the output prediction segmentation results of the segmentation network.

[0103] Change the predicted class of all voxels in set 1 to background.

[0104] Step 7: Model Evaluation

[0105] The present invention uses four metrics to evaluate segmentation performance: Dice similarity coefficient, intersection over union (IoU), average symmetric surface distance (ASSD), and 95% Hausdorff distance (HD). Specifically, the Dice coefficient calculates the proportion of voxel-level segmentation overlap, while IoU quantifies the overlap between the predicted area and the ground truth area. HD measures the maximum distance between the closest points on two surfaces, and ASSD is mainly used to calculate the average distance between the ground truth and the segmented surface. The specific formula is as follows:

[0106]

[0107] Among them, V seg Represents the predicted segmentation result, V gt represents the true label, S seg represents the predicted surface point set, S gt represents the surface point set of the true label, and p is the predicted surface point set S seg In the point, g is a point in the real label surface point set. In addition, d(p,S gt ) is the distance from point p to set S gt The distance between the nearest points, d(g,S seg ) is the point g to the set S seg The distance between the nearest points in the image. TP, TN, FP, and FN are True Positive, True Negative, False Positive, and False Negative, respectively.

[0108] Six state-of-the-art methods and the proposed method were validated on public CBCT datasets and CBCT datasets collected from hospitals. The effects of different methods on the segmentation results were compared, and the Dice coefficient, IoU, 95% HD, and ASSD of different methods were obtained, as shown in Tables 1 and 2.

[0109] Table 1 Comparison of segmentation results of different methods on the CBCT dataset collected from the hospital

[0110]

[0111]

[0112] Table 2 Comparison of segmentation results of different methods on public CBCT datasets

[0113] method Dice IoU 95% HD ASSD 3DUNet 0.918 0.908 1.940 0.703 U-NetR 0.923 0.899 1.957 0.669 Swin U-NetR 0.931 0.911 1.836 0.612 3D UX-Net 0.944 0.919 1.688 0.472 HMG-Net 0.942 0.928 1.531 0.423 HFF-Net 0.953 0.939 1.307 0.349 The present invention 0.968 0.947 1.241 0.301

[0114] As shown in Tables 1 and 2, our proposed two-stage CBCT tooth instance segmentation method based on multi-directional and multi-scale feature fusion (M2FF-UNet) achieved optimal performance across all metrics on a hospital-collected dataset, with a Dice coefficient of 0.977, an Intersection over Union (IoU) of 0.954, a 95% HD of 1.013 mm, and an ASSD of 0.233. Notably, our proposed method improved the Dice coefficient by 3.4% and reduced the 95% HD by 0.398 mm, respectively. Furthermore, our method outperformed all other methods on a public dataset, achieving a Dice coefficient of 0.965, an IoU of 0.947, a 95% HD of 1.249 mm, and an ASSD of 0.301 mm. This demonstrates that our proposed method significantly improves the accuracy of tooth instance segmentation on CBCT datasets, significantly outperforming other methods. Furthermore, as shown in the segmentation results, our proposed method effectively addresses the problem of segmentation adhesion in complex regions, such as those with closely spaced teeth, achieving optimal segmentation results.

[0115] The present invention proposes a two-stage CBCT tooth instance segmentation method based on the multi-directional and multi-scale feature fusion network M2FF-UNet. The M2FF-UNet proposed in the present invention plays a key role in extracting multi-directional and multi-scale features, enabling the model to better adapt to the morphological size and spatial characteristics of tooth changes. The center-focusing mechanism proposed in the present invention achieves refined segmentation by prioritizing the central tooth within each tooth block and enhancing the segmentation boundary segmentation, especially in areas with dense or overlapping teeth. In addition, the present invention proposes a tooth segmentation post-processing algorithm to further optimize the segmentation results by redistributing voxels in areas with blurred predicted boundaries. A large number of experiments have demonstrated the effectiveness and robustness of the present invention, highlighting its potential to enhance diagnostic accuracy and optimize treatment strategies in clinical practice.

[0116] The above only describes in detail the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the scope of protection of the present invention.

Claims

1. A two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network, characterized by: The following steps are involved: S1. Obtain oral CBCT dataset and preprocess it; S2. Construct a multi-directional and multi-scale feature fusion module M2FF-Block. First, construct a multi-directional and multi-scale feature extraction module M2FCM to systematically extract multi-scale features in different directions. Then, construct a spatial focusing module SFM to adaptively emphasize key spatial positions. Then, the input is multiplied by the output of the two modules and the result is added to obtain enhanced multi-directional and multi-scale features. S3. Based on M2FF-Block, two stages of multi-directional and multi-scale feature fusion networks M2FF-UNet are constructed respectively; first, the convolution blocks of the four encoder layers in the UNet architecture are replaced with different numbers of M2FF-Blocks. For the tooth recognition stage, two decoder branches are constructed by M2FF-UNet, one branch is used to segment the global tooth area, and the other branch is used to regress the offset of each voxel relative to its center of mass; for the tooth segmentation stage, two decoder branches are constructed by M2FF-UNet, one branch is used to segment the tooth area within the tooth block, and the other branch is used to segment the boundary area within the tooth block; S4. Perform tooth recognition from a global perspective to obtain the center of mass of each tooth; S5. Crop the original CBCT image according to the centroid position to obtain the tooth block data corresponding to each tooth, and perform fine segmentation of the single tooth block in combination with the center focusing mechanism; S6. Construct a post-processing algorithm suitable for the tooth segmentation problem, and correct the segmentation result to obtain the final segmentation result.

2. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 1, characterized in that: The construction method of M2FCM in S3 is: The input feature vector is divided into 5 branches. The first branch retains the original feature information without any processing. The second branch uses 3×3×3 depth convolution to extract local spatial features. The remaining three branches use large-kernel depth convolution blocks of different sizes of 1×1×k, 1×k×1, and k×1×1 to replace the k×k×k convolution block to extract multi-directional and multi-scale features along different directions. Large-kernel convolution is used to perceive contextual information, and depth convolution is used to improve computational efficiency. These multi-directional and multi-scale features are then aggregated through 1×1×1 convolution. Finally, all outputs are spliced ​​in the channel dimension to achieve feature fusion.

3. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 2, characterized in that: The structure of the M2FCM includes: Branch 1: No processing; Branch 2: Convolutional layer br2_conv1, with a kernel size of 3×3×3, a stride of 1, and the number of groups equal to the number of input channels; Branch three: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer brb3_conv1 has a convolution kernel size of 1×1×7, a stride of 1, and padding of (0, 0, 3). The number of groups is the number of input channels. The second convolutional layer br3_conv2 has a convolution kernel size of 1×1×9, a stride of 1, and padding of (0, 0, 4). The number of groups is the number of input channels. The third convolutional layer br3_conv3 has a convolution kernel size of 1×1×11, a stride of 1, and padding of (0, 0, 5). The number of groups is the number of input channels. Branch 4: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer br4_conv1 has a convolution kernel size of 1×7×1, a stride of 1, and padding of (0, 3, 0). The number of groups is the number of input channels. The second convolutional layer br4_conv2 has a convolution kernel size of 1×9×1, a stride of 1, and padding of (0, 4, 0). The number of groups is the number of input channels. The third convolutional layer br4_conv3 has a convolution kernel size of 1×11×1, a stride of 1, and padding of (0, 5, 0). The number of groups is the number of input channels. Branch 5: After the normalization layer, there are three parallel convolutional layers. The first convolutional layer br5_conv1 has a convolution kernel size of 7×1×1, a stride of 1, and padding of (3, 0, 0). The number of groups is the number of input channels. The second convolutional layer br5_conv2 has a convolution kernel size of 9×1×1, a stride of 1, and padding of (4, 0, 0). The number of groups is the number of input channels. The third convolutional layer br5_conv3 has a convolution kernel size of 11×1×1, a stride of 1, and padding of (5, 0, 0). The number of groups is the number of input channels. Multi-directional and multi-scale feature fusion: The outputs of the third, fourth, and fifth branches are aggregated through a 1×1×1 convolutional layer with a convolution kernel of 1 and a stride of 1. The aggregated features are concatenated with the outputs of the first and second branches in the channel dimension.

4. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 1, characterized in that: The steps for constructing the spatial focusing module SFM are as follows: First, a three-dimensional average pooling layer is used to compress spatial information, and a 1×1×1 convolution is used to refine the interaction and integration between channels. In order to capture cross-directional spatial information, 1×1×k, 1×k×1, and k×1×1 convolutions are applied along the height, width, and depth directions, respectively, to focus on feature changes specific to each spatial direction. Finally, an attention map is generated through 1×1×1 convolution and sigmoid activation to guide the focus on spatially important areas in the multi-directional and multi-scale features extracted by M2FCM.

5. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 4, characterized in that: The structure of the spatial focusing module SFM includes: The first pooling layer, pool1, uses average pooling with a window size of 7, a stride of 1, and a padding of 3; The first convolutional layer is conv1, with a convolution kernel size of 1×1×1 and a stride of 1; The second convolutional layer conv2 has a convolution kernel size of 1×1×11, a stride of 1, padding of (0, 0, 5), and the number of groups is the number of input channels; The third convolutional layer conv3 has a convolution kernel size of 1×11×1, a stride of 1, padding of (0, 5, 0), and the number of groups is the number of input channels; The fourth convolutional layer conv4 has a kernel size of 11×1×1, a stride of 1, padding of (5, 0, 0), and the number of groups is the number of input channels. The fifth convolutional layer conv5 has a convolution kernel size of 1×1×1 and a stride of 1; Finally, the sigmoid function outputs an attention map from 0 to 1.

6. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 1, characterized in that: The method for obtaining the position of the centroid of each tooth in S4 is as follows: first obtain the global binary segmentation map and the global voxel centroid offset map output by the tooth recognition subnetwork, then map all predicted foreground tooth voxels to new coordinates according to the offset map, count the pointing frequencies of each new coordinate to obtain a density map P, then calculate the minimum distance from each voxel to the voxel with a larger density value, construct a distance map D, and finally select points with a density value greater than 60 and a distance value greater than 10 as the tooth centroid.

7. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 1, characterized in that: The center-focusing mechanism in S5 includes remapping the label category of the tooth block in the tooth segmentation stage and then inputting it into the tooth segmentation M2FF-UNet, setting the tooth category located at the center of the tooth block as the central tooth, and changing the corresponding label value to 1. All teeth except the central tooth in the tooth block are set as adjacent teeth, and the corresponding label value is changed to 2. The label value corresponding to the background remains unchanged at 0. The tooth block segmentation task is a three-category segmentation task, which will ensure that the model's attention is focused on the segmentation of the central tooth.

8. The two-stage CBCT tooth segmentation method based on a multi-directional and multi-scale feature fusion network according to claim 1, characterized in that: The construction method of the post-processing algorithm suitable for the tooth segmentation problem in S6 is: traverse each centroid point as the target centroid, find the coordinates of the 6 adjacent centroid points closest to the target centroid through the kd tree algorithm, and then calculate the Euclidean distance of all voxels in the tooth block to these 7 points, and obtain the centroid index corresponding to the minimum distance. All voxels corresponding to non-zero indexes are put into set 1, and then find all voxels in the tooth block that have the same predicted category as the target centroid and put them into set 2. The voxels common to set 1 and set 2 are deleted in set 1. Finally, all accelerated prediction categories in set 1 are modified to background.

Images