Tooth Image Segmentation Model Training Method, Segmentation Method, Device, and Storage Medium

Abstract

The present invention relates to a method for training a dental image segmentation model, a segmentation method, a device, and a storage medium. The method for training a dental image segmentation model includes the following steps: obtaining a labeled dental image data set; constructing a dental image segmentation model including an encoder, a bottleneck layer, a decoder, and a skip connection layer, and training the dental image segmentation model using the labeled dental image data set; wherein, in the encoder, performing encoding operations in x stages on the input labeled dental image, and outputting feature maps in each stage; wherein x is a positive integer greater than 2; in the skip connection layer, performing feature fusion based on channel gating on the output feature maps of three stages of the encoder respectively; in the bottleneck layer, performing feature fusion on the output feature map of the x-th stage of the encoder; and in the decoder, performing decoding based on the output feature maps of the bottleneck layer and the skip connection layer. The trained dental image segmentation model has excellent performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically relates to a method for training a tooth image segmentation model, a segmentation method, a device, and a storage medium. Background Art

[0002] Through intelligent recognition and analysis of dental image information, artificial intelligence technology can significantly improve the diagnostic efficiency and accuracy of oral diseases. Currently, U-shaped neural networks have been used in medical image segmentation tasks. A U-shaped neural network generally consists of an encoder, a bottleneck layer, a decoder, and skip connections, such as Unet, Unet++, and Unet3+. Many networks based on the U-shaped structure have achieved excellent performance in many medical image fields, but there are still relatively few networks specifically designed for oral tasks.

[0003] Since panoramic dental films contain very complex oral structure information and there are very large differences in clinical data, existing tooth image segmentation methods may be difficult to fully capture the details in the images. For example, limitations of medical imaging equipment itself or some metal objects (such as dental braces, restorations, etc.) that may exist in the oral cavity may cause obvious artifacts in CT scan images, and these artifacts usually damage the readability of the images. In addition, there may be large differences in tooth morphology, position, and size among different individuals, which easily causes overfitting in complex image segmentation network models, making it difficult for the image segmentation network model to be applicable to tooth images of different individuals. Summary of the Invention

[0004] Based on this, the present invention provides a method for training a tooth image segmentation model, a segmentation method, a device, and a storage medium, which solve at least one problem in the prior art.

[0005] In a first aspect, the present invention provides a method for training a tooth image segmentation model, which includes the following steps:

[0006] Obtain a labeled tooth image dataset;

[0007] Construct a tooth image segmentation model including an encoder, a bottleneck layer, a decoder, and a skip connection layer, and use the labeled tooth image dataset to train the tooth image segmentation model;

[0008] Wherein, in the encoder, perform encoding operations in x stages on the input labeled tooth image, and each stage outputs a feature map; where x is a positive integer greater than 2;

[0009] In the skip connection layer, perform feature fusion based on channel gating on the output feature maps of three stages of the encoder respectively;

[0010] In the bottleneck layer, perform feature fusion on the output feature map of the x-th stage of the encoder;

[0011] In the decoder, decoding is performed based on the output feature maps of the bottleneck layer and the skip connection layer.

[0012] It should be noted that the labeled tooth image dataset contains tooth images and corresponding labels. Among them, the tooth image can be an oral panoramic image (abbreviated as dental film), and the label can be a mask of the tooth area in the tooth image. The encoder, bottleneck layer, and decoder constitute a U-shaped structure neural network, and the structures of the encoder and decoder are symmetric (the number of encoding and decoding stages is the same).

[0013] In some alternative embodiments, performing channel gating-based feature fusion on the output feature maps of the three stages of the encoder respectively includes the following steps:

[0014] Reshape the size of the output feature map of the first stage of the encoder from R C×H×W to R C×N , serving as the global channel feature F 1; perform upsampling and convolution operations on the output feature maps of the second and third stages of the encoder respectively to obtain f 1 and f 2; perform matrix multiplication on the transposes of f 1 and f 2 ( ), and use the SoftMax function to generate the channel gate G 1; multiply the channel gate G 1 by the global channel feature F 1 to obtain the result f r_1 , perform a reshaping operation, and perform a residual connection on the output feature map of the first stage of the encoder to obtain the output F output_1 ;

[0015] Reshape the size of the output feature map of the x-th stage of the encoder from R C×H×W to R C×N , serving as the global channel feature F x ; perform downsampling on the output feature maps of the (x - 1)-th and (x - 2)-th stages of the encoder respectively to obtain f 1 and f 2; perform matrix multiplication on the transposes of f 1 and f 2 ( ), and use the SoftMax function to generate the channel gate G x ; use the channel gate Gx Multiply with the global channel feature F x to obtain a result f r_x , perform a reshaping operation, and perform a residual connection on the output feature map of the x-th stage of the encoder to obtain an output F output_x ;

[0016] Reshape the size of the output feature map of the y-th stage of the encoder from R C×H×W to R C×N , which serves as the global channel feature F y , where y is a positive integer and 1 < y < x; downsample the output feature map of the (y - 1)-th stage of the encoder to obtain f 1; perform upsampling and convolution operations on the output feature map of the (y + 1)-th stage of the encoder to obtain f 2; perform matrix multiplication on the transpose of f 1 and f 2 ( ), and use the SoftMax function to generate a channel gate G y ; use the channel gate G y to multiply with the global channel feature F y to obtain a result f r_y , perform a reshaping operation, and perform a residual connection on the output feature map of the y-th stage of the encoder to obtain an output F output_y ;

[0017] where C represents the number of channels, H represents the length of the feature map, W represents the width of the feature map, N = H × W, and R represents the mapping space.

[0018] In some alternative embodiments, x is equal to 4. That is, the encoder includes 4 LGFF modules and performs encoding in 4 stages.

[0019] In some alternative embodiments, the encoder includes an LGFF module. In the LGFF module, local feature extraction and global feature extraction are respectively performed on the input feature map (tooth image), and the extracted local feature map and global feature map are concatenated. After concatenation, depthwise separable convolution is performed to obtain a fused feature map (the output feature map of the LGFF module), and finally downsampling is performed through a max pooling operation.

[0020] In some alternative embodiments, local feature extraction includes the following steps: performing 3×3 convolution and 1×1 convolution on the input feature map respectively, adding them element-wise after normalization respectively; activating with the ReLU function, performing 1×1 convolution and normalization and then adding them element-wise; then performing 3×3 convolution and 1×1 convolution respectively, adding them element-wise after normalization respectively.

[0021] In some alternative embodiments, global feature extraction includes the following steps: reconstructing the input feature map into a multi-dimensional feature map, then performing 3×3 convolution and multi-head self-attention operation on the features of each dimension respectively; activating with normalization and the ReLU function; then performing 3×3 convolution, activating with the ReLU function after normalization; finally reconstructing the multi-dimensional feature map into the size of the input feature map.

[0022] In some alternative embodiments, the bottleneck layer includes an LGFF module. In the LGFF module, local feature extraction and global feature extraction are performed on the feature maps from the encoder respectively, and the extracted local feature map and global feature map are concatenated, and then depthwise separable convolution is performed to obtain a fused feature map and send it into the decoder.

[0023] In some alternative embodiments, the decoder includes an LGFF module. First, the features from the bottleneck layer are upsampled, the features output by the skip connection layer are concatenated, and then the features are sent into the LGFF module for local feature extraction and global feature extraction. The extracted local feature map and global feature map are concatenated, and then depthwise separable convolution is performed to obtain a fused feature map.

[0024] In a second aspect, the present invention provides a tooth image segmentation method, which includes the following steps:

[0025] Inputting the tooth image into a tooth image segmentation model, and outputting a target image by the tooth image segmentation model;

[0026] wherein, the tooth image segmentation model is obtained by the tooth image segmentation model training method as described above.

[0027] In a third aspect, the present invention provides a tooth image segmentation device, which includes:

[0028] At least one processor;

[0029] And a memory communicatively connected to at least one processor;

[0030] wherein, the memory stores instructions, and when the instructions are executed by at least one processor, the tooth image segmentation method as described above is implemented.

[0031] Fourthly, the present invention provides a computer-readable storage medium storing instructions which, when executed by a processor, implement the tooth image segmentation method as described above.

[0032] Due to the above technical solutions, the embodiments of the present invention have at least the following beneficial effects:

[0033] (1) The symmetric structure of the encoder and the decoder can ensure that the information flow between the two is similar, which helps to maintain the semantic consistency between the encoder and the decoder;

[0034] (2) A lighter self-attention module is used to replace the ordinary self-attention module, and the self-attention is calculated within a single block, which not only reduces the computational and storage costs, but also can effectively capture the global information of the image;

[0035] (3) The multi-scale channel gating module effectively fuses the features from different stages of the encoder, solves the semantic gap problem between different stages of the encoder, enables the model to more comprehensively understand the structure and features of the oral cavity image, and improves the performance and robustness of the model in the tooth image segmentation task;

[0036] (4) The channel gating mechanism can help the model to more effectively utilize the important feature channels and suppress the channels irrelevant to the task, thereby improving the accuracy and efficiency of tooth image segmentation. Description of the Drawings

[0037] Figure 1 It is a schematic diagram of the overall architecture of the tooth image segmentation model in the embodiment of the present invention.

[0038] Figure 2 It is a schematic diagram of the feature fusion module in the embodiment of the present invention.

[0039] Figure 3 It is a schematic diagram of the local feature extraction module in the embodiment of the present invention.

[0040] Figure 4 It is a schematic diagram of the global feature extraction module in the embodiment of the present invention.

[0041] Figure 5 It is a schematic diagram of the first stage of the MSCG module in the embodiment of the present invention.

[0042] Figure 6 Some images in the MICCAI dataset and the corresponding masks are shown.

[0043] Figure 7 Some visualization results of each model for tooth image segmentation are shown. Detailed Embodiments

[0044] The concept of the present invention and the resulting technical effects will be clearly and completely described below to fully elaborate the purpose, solution, and effects of the present invention.

[0045] As Figure 1 shown, the tooth image segmentation model of an embodiment of the present invention consists of an encoder, a bottleneck layer, a decoder, and a skip connection layer. Among them, the encoder, the bottleneck layer, and the decoder all contain a Local and Global Feature Fusion (LGFF) module, and the skip connection layer contains a Mult-Scale Channel Gated (MSCG) module. In the encoder, the LGFF module is used to extract high-dimensional features locally and globally and adaptively fuse the two, and then the output fused feature map is downsampled by max pooling. In the bottleneck layer, the LGFF module is used to extract high-dimensional features locally and globally from the feature map from the encoder and adaptively fuse the two to obtain a fused feature map and send it into the decoder. To maintain the semantic consistency between the encoder and the decoder, the decoder adopts a structure symmetric to the encoder and gradually restores the size of the feature map through upsampling. In the skip connection layer, the MSCG module is used to fuse multi-layer features from the encoder to eliminate the semantic gap between multi-layer features, and enable the model to better understand the semantic information of input features at different levels, capture effective information, and suppress irrelevant noise. The output features of the MSCG module are then concatenated with the corresponding feature maps in the decoder, so that the decoder receives richer semantic information and better restores the input image. Finally, 1×1 convolution is performed to obtain the prediction mask.

[0046] As Figure 2 shown, in the LGFF module of the encoder, local feature extraction and global feature extraction are respectively performed on the input feature map and the two are adaptively fused (that is, the extracted local feature map and global feature map are concatenated, and depthwise separable convolution is performed after concatenation) to obtain a fused feature map, as shown in Equations (1) and (2):

[0047]

[0048] Among them, F in represents the input feature map, LFF (·) represents local feature extraction, GFF (·) represents global feature extraction, L represents the local feature map obtained by local feature extraction, G represents the global feature map obtained by global feature extraction, concat (·) represents the concatenation operation, DsConv (·) represents depthwise separable convolution, Fout Represents the fused feature map output by the LGFF module.

[0049] Specifically, as Figure 3 shown, the local feature extraction includes the following steps: performing 3×3 convolution and 1×1 convolution on the input feature map respectively, adding them element-wise after normalization respectively; activating with the ReLU function, performing 1×1 convolution and normalization and then adding them element-wise; then performing 3×3 convolution and 1×1 convolution respectively, adding them element-wise after normalization respectively, and then activating with the ReLU function; as shown in Equations (3) to (7):

[0050]

[0051]

[0052] Among them, F in represents the input feature map, f 3×3 (·) represents the 3×3 convolution operation, f 1×1 (·) represents the 1×1 convolution operation, BN (·) represents normalization, ReLU (·) represents the ReLu activation function, f 1 and f 2 respectively represent the features obtained by performing 3×3 convolution and 1×1 convolution on the input feature and then normalizing, F 1 represents the feature obtained by adding 1 and f 1 and f 2 element-wise and then activating with the ReLU function, F 2 represents the feature obtained by performing 1×1 convolution and normalization on F 1 and then adding them element-wise, f 3 and f 4 respectively represent the features obtained by performing 3×3 convolution and 1×1 convolution on F 2 and then normalizing, L represents the local feature map obtained by local feature extraction. Here, two parallel convolutions with different kernel sizes are used twice successively to extract multi-scale information in the features, and more extensive and detailed information in the features can be extracted.

[0053] Specifically, as Figure 4 shown, the global feature extraction includes the following steps: reconstructing the input feature map into a multi-dimensional feature map, and then performing 3×3 convolution and multi-head self-attention operation on each dimension of the feature map respectively; using normalization and the ReLU function to activate; then performing 3×3 convolution, normalizing and then using the ReLU function to activate; finally reconstructing the multi-dimensional feature map into the size of the input feature map; as shown in Equations (8) to (10):

[0054]

[0055] Among them, F i represents the input feature of the i-th dimension of the multi-dimensional feature, where i is a positive integer, f 3×3 (·) represents a 3×3 convolution operation, f 1×1 (·) represents a 1×1 convolution operation, MHSA (·) represents a multi-head self-attention operation, BN (·) represents normalization, ReLU (·) represents the ReLu activation function, F 1 represents the feature obtained by performing a 3×3 convolution on the input feature, F 2 represents the feature obtained by performing multi-head self-attention on F 1 and then performing normalization and ReLU function activation, F out_i represents the output feature of the i-th dimension. Here, window-based multi-head self-attention ( MHSA ) does not need to consider the attention relationship of the entire sequence when processing long sequences, so it reduces the requirement for the pre-trained model and also reduces the requirement for computing resources. At the same time, convolution ( f 3×3 (·)) is introduced to assist window-based multi-head self-attention to obtain global features at a lower complexity.

[0056] Specifically, when applying window-based multi-head self-attention, the input feature map is divided into n non-overlapping blocks of the same size , where , , P represents the block size, C represents the number of channels, H represents the length of the input feature map, and W represents the width of the input feature map. In this embodiment, P = 4. Within each block, learnable projection matrices W Q , W K , W V are used to project each input block to obtain the corresponding , , , that is, the query, key, and value matrices. The same self-attention operation is performed h times on each block, where h is a positive integer. First , perform similarity calculation to obtain the attention weight matrix, then add the bias term matrix B to the attention weight matrix, and finally use the obtained attention weight matrix for Perform weighting to obtain the final attention feature map. The computation formula for the

[0057]

[0058] j-th self-attention operation is as follows: F i represents the input feature; W j Q and W j K and W j V respectively represent the learnable matrices W Q and W K and W V for the j-th self-attention operation; Q ij , K ij and V ij represent the query, key, and value matrices corresponding to the j-th self-attention operation of the input feature; SoftMax (·) represents SoftMax the F iout_j function; .

[0059] Then, the outputs obtained from multiple self-attention operations are concatenated to obtain the output of block , and the specific computation formula is as follows:

[0060]

[0061] where represents the learnable projection matrix, Concat (·) represents the concatenation operation, MultiHead ( Q i , K i , V i ) represents the output of block F i .

[0062] It should be noted that the complexity of calculating the attention matrix globally for the feature map is as shown in Equation (14), while the complexity of calculating the attention matrix within each block of the feature map is as shown in Equation (15). It can be seen that when calculating the attention matrix globally, the complexity of the algorithm has a quadratic relationship with the image size; while when calculating the attention only within the block, the complexity of the algorithm has a linear relationship with the image size, greatly reducing the computational cost.

[0063]

[0064] Among them, H and W respectively represent the length and width of the input feature map, P represents the block size, and C represents the number of channels.

[0065] In the MSCG module, the feature outputs from three consecutive stages of the encoder are used as inputs. Define as the convolution operation with a convolution kernel size of n and a stride of i, as the upsampling with a magnification factor of j, where n, i, and j are all positive integers. Input the features of three adjacent layers of the encoder into the MSCG module. For the size differences of the feature outputs at different stages of the encoder, there are the following three cases in the feature fusion process:

[0066] For the first stage, perform upsampling and convolution operations on the features of the subsequent two stages. In this embodiment, the feature of the current stage is , and the features of the two stages for feature fusion are , , respectively perform upsampling and convolution operations on , to obtain , as shown in Equations (16) and (17):

[0067]

[0068] For the last stage, perform downsampling on the features of the previous two stages. In this embodiment, the feature of the current stage is , then the features of the two stages for feature fusion are , , respectively perform downsampling on and using a convolution with a stride of 2 to obtain , as shown in Equations (18) and (19):

[0069]

[0070] For the middle stage, perform downsampling on the feature of the previous stage and perform upsampling and convolution operations on the feature of the subsequent stage. In this embodiment, the feature of the current stage is or Taking the current stage features as an example, the features of the two stages for feature fusion are and . For , perform downsampling using a convolution with a stride of 2. For , perform upsampling and convolution operations to obtain , as shown in Equations (20) and (21):

[0071]

[0072] For each stage, transform the size of the input feature map of this stage from R C×H×W to R C×N as the global channel feature F ; and perform matrix multiplication on the transpose of f 1 and f 2 ( ), and use the SoftMax function to generate the channel gate G ; multiply with the global channel feature to get the result f , perform a reshaping operation, and finally perform a residual connection on the input feature map to obtain the output of the MSCG module. As Figure 5 shown, taking the first stage as an example, transform the size of the feature from to . The transformed is regarded as the global channel feature ; then perform matrix multiplication on the transpose of the obtained and , and use the SoftMax function to generate the channel gate ; multiply with the global channel feature , reshape the result f to , and then perform a residual connection on to obtain the final feature, as shown in Equations (22) to (24):

[0073]

[0074] Among them, f 1 and f 2 respectively represent the results after feature fusion of the output feature maps of the second and third stages of the encoder; F 1 represents the input feature map (i.e., the output feature map of the first stage of the encoder);reshape (·) represents a deformation operation; f R_1 and f R_2 respectively represent the deformed f 1 and f 2; F represents the global channel feature; SoftMax (·) represents the SoftMax function, G represents the channel gate, F output represents the output feature map.

[0075] It should be noted that the MSCG module introduces a gating mechanism, which combines shallow detail information with deep semantic information, enabling features of different scales to more comprehensively participate in cross-layer information transmission to enrich the information at each stage. In addition, the channel gating mechanism can enhance the model's perception ability of important features in different channels of oral images, helping the network to more effectively utilize the important feature channels in the image. Therefore, a more refined segmentation result can be obtained when performing the tooth segmentation task, improving the accuracy and robustness of the model.

[0076] The feature fusion module (LGFF module) in the bottleneck layer is the same as the feature fusion module in the encoder. The LGFF module in the bottleneck layer respectively performs local feature extraction and global feature extraction on the output feature map of the encoder, splices the extracted local feature map and global feature map, and then performs depthwise separable convolution to obtain the fused feature map and sends it into the decoder.

[0077] The decoder receives the output feature map from the bottleneck layer and performs upsampling. After splicing the multi-scale features output from the MSCG module, it respectively performs local feature extraction and global feature extraction, and splices and performs depthwise separable convolution on the extracted local feature map and global feature map to obtain the prediction result map.

[0078] The following describes the process and results of testing the performance of the tooth image segmentation model in the embodiments of the present invention through experiments.

[0079] The publicly available dataset (MICCAI dataset) used in the experiment comes from the MICCAI 2023 Tooth Image Segmentation Task Challenge. This dataset contains a total of 2000 panoramic oral images and corresponding masks, Figure 6 showing some images and corresponding masks. The resolution of each image is 640×320 pixels. The dataset is randomly divided into a training set and a test set, with an allocation ratio of 19:1, and then the abnormal data existing in the training set and the test set are removed, such as Figure 6 shown in (d). Finally, the training set contains 1900 images, and the test set contains 98 images.

[0080] Seven evaluation metrics were used in the experiment, namely Dice coefficient (Dice), Hausdorff distance (HD), Recall, Specificity (Spe), Precision (Pre), Accuracy (Acc), and Intersection over Union (IoU). Their definitions are as follows:

[0081]

[0082] Among them, A represents the predicted pixel set, B represents the actual pixel set. TP represents true positives, that is, the number of samples where the model correctly predicts the positive class; FP represents false positives, that is, the number of samples where the model incorrectly predicts the positive class; TN represents true negatives, that is, the number of samples where the model correctly predicts the negative class; FN represents false negatives, that is, the number of samples where the model incorrectly predicts the negative class. d (·) represents calculating the Euclidean distance between two pixels.

[0083] The Dice coefficient is used to evaluate the similarity between the model prediction and the true label. HD is used to measure the dissimilarity between the predicted pixel set and the actual pixel set, mainly measuring the precision of the boundary. The smaller the value of HD, the better. Recall is used to evaluate the efficiency of the model in successfully segmenting teeth. Spe is used to evaluate the model's ability to identify non-target regions. Pre is used to evaluate the precision of the model in segmenting teeth. Acc is used to evaluate the correctness of the prediction. Similar to the Dice coefficient, IoU is also used to measure the similarity between the predicted segmentation region and the true segmentation region, but with different focuses. IoU pays more attention to measuring the overlapping degree, while the Dice coefficient emphasizes the proportion of the intersection region in the total area.

[0084] In the experiment, the tooth image segmentation model in the embodiments of the present invention was constructed based on Pytorch (version 2.3.1), and the computing device used was an Nvidia GeForce RTX 4080s GPU. The cross-entropy loss function was selected as the loss function. The smaller the loss function, the smaller the difference between the predicted value and the actual value of the model, as shown in Equation (32):

[0085]

[0086] Among them, represents the actual value, represents the model predicted value.

[0087] The hyperparameters of the model are set as follows:

[0088] MICCAI dataset: input size = 224×224, max epoch = 200, batch size = 6, lr = 5e-5, and optimizer = Adam (weight decay = 0.0001);

[0089] MICCAI dataset: input size = 512×512, max epoch = 100, batch size = 3, lr = 5e-5, and optimizer = Adam (weight decay = 0.0001).

[0090] To verify the effectiveness of each module of the tooth image segmentation model in the embodiments of the present invention, ablation experiments were carried out on the MICCAI dataset, and the tooth images were input according to the size of 224×224.

[0091] As shown in Table 1, after simultaneously extracting the global and local features in the feature map in the encoder, the segmentation performance has improved, indicating that combining local and global features can make the feature representation more comprehensive. In addition, when the dual-path module is used in both the encoder and the decoder, a Dice score of 92.47% is obtained, which also shows that using a symmetric structure can ensure that the information flow between the encoder and the decoder is similar, helping to maintain the semantic consistency between the encoder and the decoder. After using the LGFF module, the performance of the network has been improved to a certain extent.

[0092] Table 1 Influence of the LGFF module on the model performance

[0093]

[0094] In Table 1, U-Net means that the model uses the encoder or decoder in U-Net; the dual-path module means that the model uses the local feature extraction module and the global feature extraction module, but only simply adds the extracted local and global features (that is, the part remaining after removing the splicing and depthwise separable convolution in the LGFF module).

[0095] Under the condition of using the LGFF module, the MSCG module is added, and the results are shown in Table 2. It can be seen from the table that after fusing multi-scale information through the MSCG module, the segmentation performance of the model has improved, indicating that the MSCG module can alleviate the semantic gap existing in the network, enabling the model to more comprehensively understand the structure and features of oral images.

[0096] Table 2 Influence of the MSCG module on the model performance

[0097]

[0098] In addition, the influence of different numbers of channels on the model performance was also tested. C1, C2, C3, C4, and C5 in [C1, C2, C3, C4, C5] respectively represent the numbers of channels of the features output by the four stages and the bottleneck layer in the encoder. The results are shown in Table 3. It can be seen from the table that when the number of channels is [64, 128, 256, 512, 1024], the model achieves better performance, and the Dice values are 92.55% and 93.28% respectively. Reducing the number of channels will significantly decrease the parameters of the model, thereby reducing the computational burden and accelerating the training speed, but reducing the number of channels has a certain impact on the model performance.

[0099] Table 3 Influence of the number of channels on the model performance

[0100]

[0101] To further verify the effectiveness of the tooth image segmentation model in the embodiments of the present invention, it was compared with some models in the prior art. To ensure fairness, when comparing with other models, all parameters were implemented according to their official codes. The MICCAI dataset was used for the experiment. When the resolution of the input image was 224×224, the results are shown in Table 4.

[0102] Table 4 Performance of each model when the resolution of the input image is 224×224

[0103]

[0104] It can be seen that the tooth image segmentation model in the embodiments of the present invention obtained the highest Dice score of 92.55%, and the improvement range of Dice was from 1.06% to 7.22%. More specifically, compared with other models, the tooth image segmentation model in the embodiments of the present invention obtained the highest scores in 6 evaluation metrics (Dice, HD, Spe, Pre, Acc, and IoU), and the remaining one evaluation metric (Recall) obtained a competitive result.

[0105] The MICCAI dataset was used for the experiment. When the resolution of the input image was 512×512, the results are shown in Table 5.

[0106] Table 5 Performance of each model when the resolution of the input image is 512×512

[0107]

[0108] It can be seen that when the resolution is larger, the feature map contains more detailed information. In this case, all the metrics of the tooth image segmentation model in the embodiments of the present invention obtained the highest scores. For example, the tooth image segmentation model in the embodiments of the present invention obtained the highest Dice score of 93.46%, with the improvement range being from 0.23% to 5.31%. Except for SwinUnet, the performance of other models has a certain improvement compared to that at a smaller resolution because only the pre-trained model with a resolution of 224×224 is provided in the official code of SwinUnet. When the pre-trained model is not used, its model performance will be greatly affected. It should be noted that TransUNet, SwinUnet, and U-ConvNext all use pre-trained models, while the tooth image segmentation model in the embodiments of the present invention does not rely on a pre-trained model with pre-trained weights. Whether at a low resolution or a high resolution, the tooth image segmentation model in the embodiments of the present invention shows more excellent performance.

[0109] Some segmentation results for an input image with a resolution of 512×512 are visualized as Figure 7 shown. In the case of lacking tooth targets, such as in the second column, other models cannot accurately predict the actual tooth targets, but the tooth image segmentation model in the embodiments still shows excellent prediction ability. Generally speaking, the tooth image segmentation model in the embodiments produces fewer noise points during prediction and is closer to the annotation results of doctors.

[0110] As mentioned above, it is only the preferred embodiment of the present invention. The present invention is not limited to the above-mentioned implementation manner. As long as it achieves the technical effects of the present invention by the same or equivalent means, it should fall within the protection scope of the present invention. Within the protection scope of the present invention, various different modifications and changes can be made to its technical solution and / or implementation manner.

Claims

1. A method for training a dental image segmentation model, characterized in that It includes the following steps: Obtain a labeled dental image dataset; Construct a dental image segmentation model including an encoder, a bottleneck layer, a decoder, and a skip connection layer, and train the dental image segmentation model using the labeled dental image dataset; Among them, in the encoder, perform encoding operations in x stages on the input labeled dental image, and each stage outputs a feature map; where x is a positive integer greater than 2; the encoder includes an LGFF module. In the LGFF module, perform local feature extraction and global feature extraction on the input feature map respectively, splice the extracted local feature map and global feature map, perform depthwise separable convolution after splicing to obtain a fused feature map, and finally perform downsampling through a max pooling operation; In the skip connection layer, perform channel-gated feature fusion on the output feature maps of three stages of the encoder respectively; In the bottleneck layer, perform feature fusion on the output feature map of the x-th stage of the encoder; the bottleneck layer includes an LGFF module. In the LGFF module, perform local feature extraction and global feature extraction on the feature map from the encoder respectively, splice the extracted local feature map and global feature map, and then perform depthwise separable convolution to obtain a fused feature map and send it into the decoder; In the decoder, perform decoding based on the output feature maps of the bottleneck layer and the skip connection layer; the decoder includes an LGFF module. First, upsample the feature from the bottleneck layer, splice the feature output by the skip connection layer, then send the feature into the LGFF module for local feature extraction and global feature extraction, splice the extracted local feature map and global feature map, and then perform depthwise separable convolution to obtain a fused feature map; Among them, performing channel-gated feature fusion on the output feature maps of three stages of the encoder respectively includes the following steps: Reshape the size of the output feature map of the first stage of the encoder from R C×H×W to R C×N , which serves as the global channel feature F 1; Upsample and perform convolution operations on the output feature maps of the second and third stages of the encoder respectively to obtain f 1 and f 2; Perform matrix multiplication on the transposes of f 1 and f 2, and use the SoftMax function to generate the channel gate G 1; Multiply the channel gate G 1 with the global channel feature F 1 to obtain the result f r_1 , perform a reshaping operation, and perform a residual connection on the output feature map of the first stage of the encoder to obtain the output F output_1 ; Reshape the size of the output feature map of the x-th stage of the encoder from R C×H×W to R C×N , which serves as the global channel feature F x ; Downsample the output feature maps of the (x - 1)-th and (x - 2)-th stages of the encoder respectively to obtain f 1 and f 2; Perform matrix multiplication on the transposes of f 1 and f 2, and use the SoftMax function to generate the channel gate G x ; Multiply the channel gate G x with the global channel feature F x to obtain the result f r_x , perform a reshaping operation, and perform a residual connection on the output feature map of the x-th stage of the encoder to obtain the output F output_x ; Resize the output feature map of the y-th stage of the encoder from R C×H×W to R C×N , which serves as the global channel feature F y , where y is a positive integer and 1 < y < x; downsample the output feature map of the (y - 1)-th stage of the encoder to obtain f 1; upsample and perform a convolution operation on the output feature map of the (y + 1)-th stage of the encoder to obtain f 2; perform a matrix multiplication operation on the transposes of f 1 and f 2, and use the SoftMax function to generate the channel gate G y ; use the channel gate G y to multiply with the global channel feature F y to obtain the result f r_y , perform a reshaping operation, and perform a residual connection on the output feature map of the y-th stage of the encoder to obtain the output F output_y ; Where C represents the number of channels, H represents the length of the feature map, W represents the width of the feature map, N = H×W, and R represents the mapping space.

2. The method according to claim 1, wherein x is equal to 4.

3. The method according to claim 1, characterized in that, Local feature extraction includes the following steps: perform 3×3 convolution and 1×1 convolution on the input feature map respectively, perform element-wise addition after normalization respectively; activate using the ReLU function, perform 1×1 convolution and normalization and then perform element-wise addition; then perform 3×3 convolution and 1×1 convolution respectively, perform element-wise addition after normalization respectively.

4. The method according to claim 3, wherein Global feature extraction includes the following steps: reconstruct the input feature map into a multi-dimensional feature map, and then perform 3×3 convolution and multi-head self-attention operation on the features of each dimension respectively; perform normalization and activation using the ReLU function; then perform 3×3 convolution, perform normalization and activation using the ReLU function; finally, reconstruct the multi-dimensional feature map into the size of the input feature map.

5. A method for tooth image segmentation, characterized in that, It includes the following steps: Input the dental image into the dental image segmentation model, and the dental image segmentation model outputs a target image; Among them, the dental image segmentation model is obtained by the dental image segmentation model training method described in any one of claims 1-4.

6. A dental image segmentation device, characterized in that, It includes: At least one processor; And a memory communicatively connected to the at least one processor; Among them, the memory stores instructions that, when executed by at least one processor, implement the tooth image segmentation method described in claim 5.

7. A computer-readable storage medium, characterized in that, Stores instructions that, when executed by a processor, implement the tooth image segmentation method described in claim 5.

Images