Underjaw CBCT image inferior tooth segmentation method based on deep learning

Abstract

The invention belongs to the field of image segmentation, and particularly relates to a mandibular CBCT image inferior tooth segmentation method based on deep learning, which comprises the following steps of: acquiring a mandibular CBCT image data set, slicing the data set from top to bottom, and performing data enhancement on the sliced data; performing pixel-level labeling on the enhanced data, and dividing the labeled data into a training set, a verification set and a test set according to a proportion; the method comprises the following steps: constructing a U-shaped network comprising an input module, a DCTA module, an output module and cavity convolution, namely a DCTA-DilUnet deep learning model; inputting the training set and the verification set into the constructed U-shaped network, and training the network; the verification set is verified through the trained U-shaped network, and segmentation of lower teeth in the lower jaw CBCT image is achieved through the verified U-shaped network. The method is suitable for the field of image segmentation and deep learning, an attention mechanism based on a frequency domain is adopted to replace original jump connection, and then feature fusion is carried out on a feature map output through the attention mechanism and a feature map after adoption.

Description

Technical Field

[0001] The present invention belongs to the field of image segmentation, and in particular to a method for segmenting lower teeth in mandibular CBCT images based on deep learning. Background Art

[0002] With the development of digital technology, cone beam computed tomography (CBCT) is widely used in dental imaging due to its high resolution and low radiation dose, which can provide detailed information about tooth and jaw structures. Tooth segmentation in CBCT images is a basic step in dental applications, such as tooth position analysis and root canal treatment. However, traditional methods based on threshold or edge detection have difficulty in accurately segmenting tooth areas, resulting in insufficient processing accuracy and efficiency.

[0003] Existing tooth segmentation methods rely on manually designed rules, cannot effectively process complex image structures, and usually require a lot of manual intervention, making it difficult to achieve automated processing. Therefore, there is an urgent need for an automated tooth segmentation method based on deep learning to fully utilize the information of CBCT images and accurately and efficiently extract tooth areas. Summary of the invention

[0004] The purpose of the present invention is to provide a method for segmenting lower teeth in mandibular CBCT images based on deep learning. The method has high segmentation accuracy and uses convolution with multiple expansion rates and a method for segmenting lower teeth in mandibular CBCT images based on an attention mechanism module.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a method for segmenting lower teeth in mandibular CBCT images based on deep learning, comprising the following steps:

[0006] Step 1: Obtain the mandibular CBCT image dataset, slice the dataset from top to bottom, and perform data enhancement on the sliced ​​data;

[0007] Step 2: Annotate the enhanced data at the pixel level and divide the annotated data into training set, validation set and test set in proportion;

[0008] Step 3: Construct a U-type network including an input module, a DCTA module, an output module and a dilated convolution, namely a DCTA-DilUnet deep learning model; wherein the U-type network includes: a compression encoder path and a scalable decoder path;

[0009] Step 4: Input the training set and validation set obtained in step 2 into the constructed U-shaped network to train the network;

[0010] Step 5: The validation set obtained in step 2 is verified by the trained U-shaped network, and the verified U-shaped network is used to segment the lower teeth in the mandibular CBCT image.

[0011] The step 1 comprises the following steps:

[0012] Step 1.1: The mandibular CBCT image in the data set is converted from the nii format, i.e., the initial format, into a two-dimensional png format image by slicing from high to low in the vertical direction;

[0013] Step 1.2: Normalize the pixel values ​​of the slice to the range of 0 to 255, and reduce the bit depth of the two-dimensional png format image to 8 through the cvtColor function;

[0014] Step 1.3: Crop or fill the image obtained in step 1.2 to a uniform size, process the image obtained by affine transformation and rotation, and complete the data enhancement of the sliced ​​data.

[0015] The step 2 is specifically:

[0016] Step 2.1: Image annotation: Use labelme software to annotate the tooth region of a single image in the image dataset after data enhancement. Set L as the label representing the tooth region. Save the annotation file to the same path as the image. Generate an annotation file in json format that includes the target box information of the tooth region annotation.

[0017] Step 2.2: Extract all the annotation point information from the obtained json file through the array function, fill the polygon on the png image through the fillPoly function, specify the pixel value as 0 to represent the background area, and specify the pixel value as 255 to represent the target area, and obtain the annotated image;

[0018] Step 2.3: Use the data-enhanced image as the original image, and the annotated image as the label image. The original image and the label image are named one-to-one and stored in different folders.

[0019] Step 2.4: Randomly divide the dataset containing the original images and the labeled images into training sets, validation sets, and test sets in a ratio of 8:1:1.

[0020] The step 3 comprises the following steps:

[0021] Step 3.1: Input module: The encoding and decoding structure based on the image segmentation network U-Net is the initial framework of the model; a preprocessor input module is added at the beginning of the image input model, where the input module consists of a 3×3 size Conv2d convolution operation with expansion rates of 3, 2, 1 and 0, BatchNorm2d normalization, Relu activation function and feature fusion concat;

[0022] Step 3.2: A dilated convolution kernel is constructed by inserting holes with fixed intervals into the convolution kernel to replace the ordinary convolution in the original U-Net model. The ordinary convolution used in each layer is replaced by a combination of dilated convolutions with dilation rates of 3, 2, 1, and 0, forming a dilated U-Net model.

[0023] Step 3.3: Integrate the frequency domain attention mechanism into the constructed void U-Net model;

[0024] Step 3.4: Integrate the output module into the constructed U-Net model, and improve the image segmentation accuracy by weighting the features at different levels and then fusing the output, ultimately forming a U-shaped network, namely the DCTA-DilUnet deep learning model.

[0025] The step 3.3 is specifically as follows:

[0026] The DCTA module includes: an information extraction stage and an information fusion stage;

[0027] (1) In the information extraction stage, the input feature map is subjected to discrete cosine transform to convert it from the spatial domain to the frequency domain. The discrete cosine transform operation maps the spatial information of the input feature map to the frequency domain so that the frequency domain features can better capture the frequency components in the image. In the frequency domain representation, different dilation rates, i.e., 3×3 convolution kernels with Rate=2 and Rate=1, are used for feature extraction to obtain feature information of different frequencies.

[0028] (2) In the information fusion stage, the feature maps are connected through channels to fuse the feature information extracted by convolutions with different dilation rates. The fused features are processed nonlinearly through the ReLU activation function. In order to further fuse the feature information in the information extraction stage, a convolution layer with a kernel size of 1×1 is used, and then the tanh function is applied to generate a frequency domain attention map to more effectively capture and adjust the frequency domain features.

[0029] (3) Perform element-by-element multiplication of the generated frequency domain attention map and the frequency domain representation of the input feature map;

[0030] (4) The processed feature map is converted from the frequency domain back to the spatial domain through inverse discrete cosine transform, and the frequency domain attention mechanism is integrated into the U-Net model.

[0031] The step 3.4 is specifically as follows:

[0032] a. In the DCTA-DilUnet model, the feature map of each decoder module is upsampled to restore the low-resolution feature map to the same spatial resolution as the original image.

[0033] b. Applying dilated convolutions with different dilation rates on the upsampled feature map, wherein the dilated convolutions have different dilation rates; since each decoder module is at a different depth of the network and uses a different dilation rate, the decoder module can acquire and process multi-level tooth information.

[0034] c. The output of each decoder module is weighted to reflect the relative importance of each feature map; the weighted output is cascaded with the final DC block output to form the final prediction result.

[0035] The step 4 is specifically as follows:

[0036] Step 4.1: Set hyperparameters, loss function, learning rate and training step size, initialize the optimizer, and initialize the DCTA-DilUnet deep learning model for image segmentation;

[0037] Step 4.2: Send the obtained training set data and its labels into the DCTA-DilUnet deep learning model for model training, calculate the loss between the predicted results and the true values, and then back-propagate to update the network parameters;

[0038] Step 4.3: Use the validation set to validate the DCTA-DilUnet deep learning model after updating the network parameters, calculate the Dice coefficient and IOU of the current model on the validation set, and if the IOU coefficient is better than the historical maximum value, save the current model parameters and update the historical optimal IOU value. The formula is:

[0039]

[0040] Among them, |A∩B| represents the number of elements in the intersection between A and B, |A∪B| represents the union of the true set and the predicted set, |A| and |B| represent the number of elements in A and B respectively;

[0041] Step 4.4: When the number of iterations reaches the set number of training steps, the training ends.

[0042] In step 4.1, the loss function is a two-dimensional cross entropy, which is expressed as:

[0043]

[0044] Where N is the number of pixels, y i is a label, p i is the probability of predicting the positive class.

[0045] The step 5 is specifically as follows:

[0046] Step 5.1: Input the test set data into the trained DCTA-DilUnet deep learning model for prediction and obtain the prediction result image;

[0047] Step 5.2: Evaluate the test set labels and prediction result images using the Dice, Iou, and Acc evaluation indicators;

[0048] The Acc evaluation index is:

[0049]

[0050] Among them, TP represents the part that is predicted correctly, FP represents the part that is falsely detected, FN represents the part that is missed, and TN represents that the predicted value is consistent with the true value and both are negative, that is, although the prediction is correct, it is not the area that is ultimately required;

[0051] Step 5.3: Segment the lower teeth in the mandibular CBCT image using the verified U-shaped network.

[0052] The present invention has the following beneficial effects and advantages:

[0053] 1. The present invention proposes to replace the original single ordinary convolution with spatial convolutions with multiple expansion rates, and uses multiple hole convolutions with different expansion rates at different levels to combine and extract features, and perform feature fusion after the convolution at each level is completed.

[0054] 2. The present invention adopts a frequency-domain-based attention mechanism (DCTA module) to replace the original jump connection, and then fuses the feature map output by the attention mechanism with the feature map after the above adoption.

[0055] 3. The present invention introduces an output module, which convolves feature maps of different levels into a uniform size and sets different weights for features of different levels before performing feature fusion to output a predicted image.

[0056] 4. The technical solution adopted by the present invention is based on the encoding and decoding structure of the deep learning image segmentation network U-Net as the initial framework of the model. By inserting holes with different intervals in the convolution kernel to form hole convolutions with different expansion rates, all conventional convolutions are replaced. At the same time, the frequency domain-based attention mechanism (DCTA module) is introduced in the jump connection layer to replace the original ordinary jump connection. In the feature map output stage, an output module is introduced to weight the features of different levels and then fuse them into the final predicted image to construct a DCTA-DilUnet model. It was compared with 6 classic or advanced methods on the mandibular CBCT dataset. The DCTA-DilUnet model has advantages in different segmentation indicators, can effectively segment the tooth area, is more friendly to the boundary fuzzy problem, and is better than other methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 It is an overall overview of the mandibular CBCT image lower teeth segmentation model (DCTA-DilUnet) of the present invention;

[0059] Figure 2 It is an improved version of the frequency domain attention mechanism model (DCTA) diagram described in the present invention;

[0060] Figure 3 is a schematic diagram of the input module of the present invention;

[0061] Figure 4 It is the principle diagram of the output module described in the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0063] The present invention adopts a deep learning network to segment the lower teeth area in the mandibular CBCT image.

[0064] The present invention discloses a method for segmenting lower teeth in mandibular CBCT images based on deep learning. Figure 1-Figure 4 The segmentation network includes an input module (Input Block, referred to as Input module), a frequency-domain attention mechanism module (Discrete Cosine Transform Attention Block, referred to as DCTA module), an output module (Output Block, referred to as Output module) that fuses weighted features at different levels, and a combination of dilated convolutions with different expansion rates to replace the original ordinary convolution.

[0065] Normalization (Batch Normalization): After the image is convolved, the feature distribution of the input data will become unstable, that is, the internal covariate displacement problem. At this time, batch normalization is performed appropriately after the feature map convolution to standardize the input data. Use learnable scaling factors and offsets to adjust the standardized data to stabilize the input feature distribution of this layer.

[0066] Normalization formula:

[0067]

[0068] Among them, X min and X max Represents this set of data X i The minimum and maximum values ​​in .

[0069] In order to make the purpose, technical solutions and advantages of the implementation methods of the present application clearer, the technical solutions in the implementation methods of the present application will be clearly and completely described below in conjunction with the drawings in the implementation methods of the present application. Obviously, the described implementation methods are only part of the implementation methods of the present application, not all of the implementation methods.

[0070] Embodiment 1 of the present invention provides a method for segmenting lower teeth in mandibular CBCT images based on an attention module. The method mainly comprises the following steps:

[0071] Embodiment 1: This embodiment provides a method for segmenting lower teeth in mandibular CBCT images based on an attention module, the method comprising the following steps:

[0072] Step 1: Download the public mandibular CBCT image dataset from the challenge (https: / / ditto.ing.unimore.it / maxillo / ), slice the dataset from top to bottom, and perform data augmentation on the sliced ​​data.

[0073] Step 2: Annotate the enhanced data at the pixel level and divide the annotated data into training set, validation set and test set in the ratio of 8:1:1.

[0074] Step 3: Construct a U-shaped network consisting of an Input module, a DCTA module, an Output module, and a dilated convolution, which includes a compression encoder path and a scalable decoder path.

[0075] Step 4: Input the training set and validation set obtained in step 2 into the U-shaped network constructed in step 3 to train the network.

[0076] Step 5: Use the U-shaped network trained in step 4 to test and verify the test set obtained in step 2, and use the verified U-shaped network to segment the lower teeth in the mandibular CBCT image.

[0077] Implementation 2: This implementation further limits the method for segmenting lower teeth in mandibular CBCT images based on deep learning described in Implementation 1. The method for obtaining a data set in step 1 and processing it into a data set that can be used for annotation is:

[0078] Download the dataset used in this example from the official website of the challenge, and convert the mandibular CBCT image from the initial nii format into a two-dimensional PNG format image by vertically slicing from high to low.

[0079] 1.2 Use the cvtColor function in the python third-party library OpenCV to reduce the bit depth of the image obtained in step 1.1 of implementation method 2 to 8 bits.

[0080] 1.3 The image obtained in step 1.2 of implementation mode 2 is cropped or filled to a uniform size of 360×360, and the image is enhanced by affine transformation, rotation and other methods.

[0081] Implementation method 3: This implementation method further limits the method for segmenting lower teeth in mandibular CBCT images based on deep learning described in implementation method 1. The method for data annotation and division of training set, validation set and test set in step 2 is:

[0082] 2.1 Use Labelme software to label the lower teeth area in the image data processed in step 1.3 of implementation mode 2, and set labels to represent the tooth area. The labeling file will be saved in the same path as the image, and the generated JSON format labeling file will contain the target box information of the tooth area.

[0083] 2.2 Use the array function in the python third-party library NumPy to extract all the annotation point information from the JSON file obtained in step 2.1 of implementation method 3, and fill the polygon on the PNG image through the fillPoly function in the python third-party library OpenCV. The specified pixel value is 0 (black) to represent the background area, and the specified pixel value is 255 (white) to represent the target area.

[0084] 2.3 Use the image obtained in step 1.3 of implementation mode 2 as the original image, and use the image obtained in step 2.2 of implementation mode 3 as the label. Ensure that each image in the two parts maintains a one-to-one corresponding naming relationship and is stored in different folders.

[0085] 2.4 The data set obtained in step 2.3 of implementation mode 3 is randomly divided, and the ratio of the training set, validation set and test set after the division is 8:1:1.

[0086] Implementation 4: This implementation further limits the method for mandibular CBCT image lower teeth segmentation based on deep learning described in Implementation 1. In step 3, a U-shaped network including an Input module, a DCTA module, an Output module and a dilated convolution is constructed. The U-shaped network includes a compression encoder path and an expandable decoder path. The method is as follows:

[0087] 3.1 In the encoding-decoding architecture based on the image segmentation network U-Net, we introduced an improved model framework. Specifically, in the image input stage, we added a preprocessor Input module (see Appendix for details). Figure 3 ). The Input module includes a Conv2d convolution operation with a kernel size of 3×3 and expansion rates of 3, 2, 1, and 0. In addition, the module also contains a BatchNorm2d normalization layer, a ReLU activation function, and a feature fusion operation (concat) to optimize the feature extraction process.

[0088] 3.1.1 further applies convolution kernels with different expansion rates of 0, 1, 2, and 3 before the image is input to the model. This design focuses on retaining the detailed information of the image while ensuring a large receptive field. After each convolution operation, we use the ReLU activation function for nonlinear mapping to enhance feature expression. The main purpose of introducing this module is to address the limitations of traditional single convolution kernels in processing raw image features, which can significantly improve the comprehensive extraction of image detail information and global information. Through this improvement, we can more effectively capture the complex patterns and subtle features in the image, thereby improving the overall performance of the model.

[0089] 3.2 In order to enhance the performance of the model when processing complex images, we optimized the traditional U-Net model and used a combination of dilated convolutions with different dilation rates to replace ordinary convolutions. Specifically, each layer of ordinary convolution in the traditional U-Net model was replaced by dilated convolutions with dilation rates of 3, 2, 1, and 0, thereby constructing an improved dilated U-Net model. This improvement uses a combination of dilated convolutions with different dilation rates to expand the receptive field while taking into account the model's ability to obtain detailed information, thereby improving the ability to extract features.

[0090] 3.2.1 Further considering the blurred boundaries between teeth and adjacent tissues (such as bone), especially in the lesion area or under the gums, this increases the difficulty of the segmentation algorithm in judging the boundaries. In order to solve this problem, this paper focuses on feature extraction in the neural network encoding part, optimization based on the frequency domain attention mechanism, and improvement of feature fusion capabilities. By applying dilated convolutions with different dilation rates at each level to replace the traditional ordinary convolution operation in U-Net. This method not only expands the receptive field of the model, but also increases the weight of edge information in the features, thereby enhancing the recognition of details and boundaries. With this improvement, the model performs better in dealing with complex boundaries and subtle structures, improving the accuracy and effect of overall image segmentation.

[0091] 3.3 The frequency domain attention mechanism proposed in this invention (see Appendix Figure 2 ) is integrated into the hole U-Net model constructed in step 3.2 to further enhance the performance of the model in the image segmentation task.

[0092] 3.3.1 The DCTA module proposed in the present invention mainly includes the information extraction (IE) stage and the information fusion (IF) stage. In the IE stage, the input feature map is first subjected to discrete cosine transform (DCT) to convert it from the spatial domain to the frequency domain representation. Specifically, the DCT operation maps the spatial information of the input feature map to the frequency domain, so that the frequency domain features can better capture the frequency components in the image. In the frequency domain representation, 3×3 convolution kernels with different expansion rates (Rate=2 and Rate=1) are used for feature extraction to obtain feature information of different frequencies.

[0093] 3.3.2 In the information fusion (IF) stage, the feature map is first channel-connected to fuse the feature information extracted by convolutions with different dilation rates. Then the ReLU activation function is used to perform nonlinear processing on the fused features. In order to further fuse the feature information in the IE stage, the present invention uses a convolution layer with a kernel size of 1×1, and then applies the tanh function to generate a frequency domain attention map. Unlike the attention mechanism using the sigmoid function, the reason why the present invention chooses the tanh function is that the amplitude and phase of the attention map need to be adaptively adjusted in order to more effectively capture and adjust the frequency domain features. Experimental results show that the adjustment of the phase has a significant effect on improving the segmentation accuracy.

[0094] 3.3.3 The generated frequency domain attention map is then element-wise multiplied with the frequency domain representation of the input feature map. Finally, the processed feature map is converted from the frequency domain back to the spatial domain using the inverse discrete cosine transform (IDCT). This process demonstrates the potential advantages of the frequency domain attention mechanism, indicating that the attention mechanism developed in the frequency domain can provide better performance than the spatial domain method.

[0095] 3.4 In the model constructed by step 3.3 in implementation method 4, we further integrated a new Output module to form the DCTA-DilUnet model we proposed. This integrated module enables the model to more effectively combine feature information at each level, thereby improving the final segmentation accuracy and robustness.

[0096] 3.4.1 In the DCTA-DilUnet model, we upsample the feature maps of each decoder module. The purpose of upsampling is to restore the low-resolution feature maps to the same spatial resolution as the original image so that the final prediction can be made more accurately. To further improve the decoding process, we apply dilated convolutions with different dilation rates to the upsampled feature maps. These dilated convolutions have different dilation rates and can extract multi-level feature information in different receptive fields, thereby capturing rich details of the teeth and their surrounding structures. Since each decoder module is at a different depth of the network and uses a different dilation rate, they are able to obtain and process multi-level tooth information.

[0097] 3.4.2 The output of each decoder module is weighted to reflect the relative importance of each feature map. This step ensures that the model can effectively fuse the feature information of each level and avoid over-reliance on a specific level. The weighted output is cascaded with the final direct convolution block output to form the final prediction result. Through this cascade method, the DCTA-DilUnet model can comprehensively consider the feature information from different levels and make full use of the advantages of each level of the model, thereby improving the ability to segment complex structures and details.

[0098] 3.4.3 The final Output module not only enhances the model's comprehensive processing capabilities for multi-level features, but also makes the final prediction results more comprehensive and accurate. This integrated solution ensures that the DCTA-DilUnet model can fully utilize its powerful feature extraction and fusion capabilities when processing tooth image segmentation tasks.

[0099] Implementation 5: This implementation is a further limitation of the mandibular CBCT image lower teeth segmentation method based on the attention module described in Implementation 1. In step 4, the method of sending the training set and the validation set of step 2 in Implementation 1 to the U-shaped network constructed in step 3 in Implementation 1 for training is:

[0100] 4.1 Set hyperparameters, loss function, learning rate, number of training steps and initialize the optimizer to initialize the DCTA-DilUnet deep learning model for image segmentation;

[0101] 4.2 The training set data and its labels obtained in step 2.4 of implementation method three are sent to the DCTA-DilUnet deep learning model described in implementation method four to train the model, obtain the segmentation result, calculate the Loss between the predicted result and the true value, and then back propagate to update the network parameters. The loss function is a commonly used binary cross entropy loss function (BCEWithLogitsLoss), and its formula is:

[0102]

[0103] Among them, p represents the predicted probability, y represents the true label, 0 represents a negative sample, and 1 represents a positive sample.

[0104] 4.3 Use the validation set data obtained in step 2.4 of implementation method three to verify the DCTA-DilUnet deep learning model after updating the network parameters in step 4.2 of implementation method five, and calculate the Dice coefficient and IOU of the current model on the validation set. If the IOU coefficient is better than the historical maximum value, save the current model parameters and update the historical optimal IOU value. The formulas for IOU and Dice coefficient are as follows:

[0105]

[0106] Among them, |A∩B| represents the number of elements in the intersection between A and B, |A∪B| represents the union of the true set and the predicted set, and |A| and |B| represent the number of elements in A and B respectively.

[0107] 4.4 When the number of iterations reaches the set number of training steps, the training ends.

[0108] Implementation 6: This implementation is a further limitation of the method for segmenting lower teeth in mandibular CBCT images based on an attention module described in Implementation 1. In step 5, the U-shaped network trained in step 4 is tested and verified using the test set obtained in step 2. The method for segmenting lower teeth in mandibular CBCT images using the verified U-shaped network is as follows:

[0109] 5.1 The test set data obtained in step 2.4 of implementation method three is input into the DCTA-DilUnet deep learning model trained in implementation method five for prediction.

[0110] 5.2 The test set labels obtained in step 2.4 of implementation mode 3 and the test set prediction result images obtained in step 5.1 of implementation mode 6 are evaluated using evaluation indicators such as Dice, Iou and Acc. The formula is:

[0111]

[0112] Among them, |A∩B| represents the number of elements in the intersection between A and B, |A∪B| represents the union of the true set and the predicted set, and |A| and |B| represent the number of elements in A and B respectively.

[0113]

[0114] Among them, TP (True Positive) indicates the correctly predicted part, FP (False Positive) indicates the falsely detected part, FN (False Negative) indicates the missed part, and TN (True Negative) indicates that the predicted value is consistent with the true value and both are negative, that is, although the prediction is correct, it is not the area we want.

[0115] Example 2 verifies the superiority of the present invention in segmenting lower teeth in mandibular CBCT images:

[0116] 1. Dataset, training parameters and evaluation indicators

[0117] 1) Dataset Description

[0118] The dataset used in this paper is a mandibular CBCT dataset downloaded from an open challenge (https: / / ditto.ing.unimore.it / maxillo / ). After excluding the data with severe metal artifacts, more than 100 data sets were left. 50 datasets were randomly selected from them. The data were converted from the initial nii format into two-dimensional PNG format images in vertical direction from high to low slices. Each data set has about 160 slices, totaling 8012. In this paper, the 8012 images are unified into 360*360 pixel size and divided into training set, validation set and test set in a ratio of 8:1:1.

[0119] Specific parameter settings when training the model:

[0120] This experiment was trained for 200 epochs. The batch size of all data sets was 64, the optimizer was Adam, and the learning rate was 0.0001. The programming language of the experimental environment was python3.10, the deep learning framework was PyTorch 2.4.0, CUDA12.3, and the main packages were Numpy1.26, opencv-python 4.10.0.84, and pands2.2.2. The loss function was the two-dimensional cross entropy (BCE) loss, and the formula was as follows:

[0121]

[0122] Where N is the number of pixels, y i is a label, p i is the probability of predicting the positive class.

[0123] 2. Comparative experimental results

[0124] In order to evaluate the superiority of DCTA-DilUnet over other methods, it is compared with six cutting-edge or classic methods: U-net, Unet++, DilUnet, Attention u-net, and the contents applied in the two papers.

[0125] 1)Inferior Alveolar Nerve Canal Segmentation on CBCT Using U-Net withFrequency Attentions[J].Bioengineering, 2024,11(4):354.

[0126] 2)Encoder-decoder with atrousseparable convolution for semantic imagesegmentation[C]Proceedings of the European conference on computer vision(ECCV).2018:801-818;

[0127] Table 1: Comparison of the performance of different methods for segmenting teeth in mandibular CBCT images

[0128]

[0129] "↓" means the smaller the better, "↑" means the larger the better, red fonts indicate the best results, and blue fonts indicate the suboptimal results. As shown in Table 1:

[0130] Compared with the above six methods, DCTA-Diluet showed the best results in the three segmentation indicators, among which IOU, Dice and Acc increased by 0.0218, 0.0139 and 0.0021 respectively, which shows that DCTA-Diluet is very effective for the segmentation of the tooth area of ​​mandibular CBCT images. Through this comparative experiment, it can be seen that the DCTA-Diluet model combines the advantages of the hollow convolution and frequency attention mechanism, and shows good segmentation accuracy for the segmentation of the tooth area of ​​mandibular CBCT images.

[0131] In summary, the present invention proposes a mandibular CBCT image lower teeth segmentation method based on deep learning, which adopts a U-shaped structure, can better handle complex structures and noise, improve the accuracy and stability of segmentation, and realize efficient segmentation on large-scale CBCT image data, thus promoting the development of digital dental technology.

[0132] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0133] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for segmenting lower teeth in mandibular CBCT images based on deep learning, characterized in that: The following steps are involved: Step 1: Obtain the mandibular CBCT image dataset, slice the dataset from top to bottom, and perform data enhancement on the sliced ​​data; Step 2: Annotate the enhanced data at the pixel level and divide the annotated data into training set, validation set and test set in proportion; Step 3: Construct a U-type network including an input module, a DCTA module, an output module and a dilated convolution, namely a DCTA-DilUnet deep learning model; wherein the U-type network includes: a compression encoder path and a scalable decoder path; Step 4: Input the training set and validation set obtained in step 2 into the constructed U-shaped network to train the network; Step 5: The validation set obtained in step 2 is verified by the trained U-shaped network, and the verified U-shaped network is used to segment the lower teeth in the mandibular CBCT image.

2. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1: The mandibular CBCT image in the data set is converted from the nii format, i.e., the initial format, into a two-dimensional png format image by slicing from high to low in the vertical direction; Step 1.2: Normalize the pixel values ​​of the slice to the range of 0 to 255, and reduce the bit depth of the two-dimensional png format image to 8 through the cvtColor function; Step 1.3: Crop or fill the image obtained in step 1.2 to a uniform size, process the image obtained by affine transformation and rotation, and complete the data enhancement of the sliced ​​data.

3. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 1, characterized in that: The step 2 is specifically: Step 2.1: Image annotation: Use labelme software to annotate the tooth region of a single image in the image dataset after data enhancement. Set L as the label representing the tooth region. Save the annotation file to the same path as the image. Generate an annotation file in json format that includes the target box information of the tooth region annotation. Step 2.2: Extract all the annotation point information from the obtained json file through the array function, fill the polygon on the png image through the fillPoly function, specify the pixel value as 0 to represent the background area, and specify the pixel value as 255 to represent the target area, and obtain the annotated image; Step 2.3: Use the data-enhanced image as the original image, and the annotated image as the label image. The original image and the label image are named one-to-one and stored in different folders. Step 2.4: Randomly divide the dataset containing the original images and the labeled images into training sets, validation sets, and test sets in a ratio of 8:1:

1.

4. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1: Input module: The encoding and decoding structure based on the image segmentation network U-Net is the initial framework of the model; a preprocessor input module is added at the beginning of the image input model, where the input module consists of a 3×3 size Conv2d convolution operation with expansion rates of 3, 2, 1 and 0, BatchNorm2d normalization, Relu activation function and feature fusion concat; Step 3.2: A dilated convolution kernel is constructed by inserting holes with fixed intervals into the convolution kernel to replace the ordinary convolution in the original U-Net model. The ordinary convolution used in each layer is replaced by a combination of dilated convolutions with dilation rates of 3, 2, 1, and 0, forming a dilated U-Net model. Step 3.3: Integrate the frequency domain attention mechanism into the constructed void U-Net model; Step 3.4: Integrate the output module into the constructed U-Net model, and improve the image segmentation accuracy by weighting the features at different levels and then fusing the output, ultimately forming a U-shaped network, namely the DCTA-DilUnet deep learning model.

5. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 4, characterized in that: The step 3.3 is specifically as follows: The DCTA module includes: an information extraction stage and an information fusion stage; (1) In the information extraction stage, the input feature map is subjected to discrete cosine transform to convert it from the spatial domain to the frequency domain. The discrete cosine transform operation maps the spatial information of the input feature map to the frequency domain so that the frequency domain features can better capture the frequency components in the image. In the frequency domain representation, different dilation rates, i.e., 3×3 convolution kernels with Rate=2 and Rate=1, are used for feature extraction to obtain feature information of different frequencies. (2) In the information fusion stage, the feature maps are connected through channels to fuse the feature information extracted by convolutions with different dilation rates. The fused features are processed nonlinearly through the ReLU activation function. In order to further fuse the feature information in the information extraction stage, a convolution layer with a kernel size of 1×1 is used, and then the tanh function is applied to generate a frequency domain attention map to more effectively capture and adjust the frequency domain features. (3) Perform element-by-element multiplication of the generated frequency domain attention map and the frequency domain representation of the input feature map; (4) The processed feature map is converted from the frequency domain back to the spatial domain through inverse discrete cosine transform, and the frequency domain attention mechanism is integrated into the U-Net model.

6. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 4, characterized in that: The step 3.4 is specifically as follows: a. In the DCTA-DilUnet model, the feature map of each decoder module is upsampled to restore the low-resolution feature map to the same spatial resolution as the original image. b. Applying dilated convolutions with different dilation rates on the upsampled feature map, wherein the dilated convolutions have different dilation rates; since each decoder module is at a different depth of the network and uses a different dilation rate, the decoder module can acquire and process multi-level tooth information. c. Weight the output of each decoder module to reflect the relative importance of each feature map; The weighted output is cascaded with the final DC block output to form the final prediction result.

7. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 1, characterized in that: The step 4 is specifically as follows: Step 4.1: Set hyperparameters, loss function, learning rate and training step size, initialize the optimizer, and initialize the DCTA-DilUnet deep learning model for image segmentation; Step 4.2: Send the obtained training set data and its labels into the DCTA-DilUnet deep learning model for model training, calculate the loss between the predicted results and the true values, and then back-propagate to update the network parameters; Step 4.3: Use the validation set to validate the DCTA-DilUnet deep learning model after updating the network parameters, calculate the Dice coefficient and IOU of the current model on the validation set, and if the IOU coefficient is better than the historical maximum value, save the current model parameters and update the historical optimal IOU value. The formula is: Among them, |A∩B| represents the number of elements in the intersection between A and B, |A∪B| represents the union of the true set and the predicted set, |A| and |B| represent the number of elements in A and B respectively; Step 4.4: When the number of iterations reaches the set number of training steps, the training ends.

8. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 7, characterized in that: In step 4.1, the loss function is a two-dimensional cross entropy, which is expressed as: Where N is the number of pixels, y i is a label, p i is the probability of predicting the positive class.

9. The method for segmenting lower teeth in mandibular CBCT images based on deep learning according to claim 1, characterized in that: The step 5 is specifically as follows: Step 5.1: Input the test set data into the trained DCTA-DilUnet deep learning model for prediction and obtain the prediction result image; Step 5.2: Evaluate the test set labels and prediction result images using the Dice, Iou, and Acc evaluation indicators; The Acc evaluation index is: Among them, TP represents the correctly predicted part, FP represents the falsely detected part, FN represents the missed part, and TN represents that the predicted value is consistent with the true value and both are negative, that is, although the prediction is correct, it is not the area ultimately required; Step 5.3: Segment the lower teeth in the mandibular CBCT image through the verified U-shaped network.

Images