Tooth image segmentation model training method, segmentation method, equipment and storage medium

Abstract

The invention relates to a tooth image segmentation model training method, a segmentation method, equipment and a storage medium. The tooth image segmentation model training method comprises the following steps: acquiring a tooth image data set with labels; constructing a tooth image segmentation model comprising an encoder, a bottleneck layer, a decoder and a jump connection layer, and training the tooth image segmentation model by using the labeled tooth image data set; wherein in the encoder, encoding operation of x stages is executed on an input tooth image with a label, and a feature map is output in each stage; wherein x is a positive integer greater than 2; in the jump connection layer, feature fusion based on channel gating is carried out on output feature maps of the three stages of the encoder; in the bottleneck layer, carrying out feature fusion on the output feature map of the xth stage of the encoder; in a decoder, decoding is performed based on the output feature maps of the bottleneck layer and the jump connection layer. The tooth image segmentation model obtained through training has excellent performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a tooth image segmentation model training method, a segmentation method, a device and a storage medium. Background Art

[0002] Artificial intelligence technology can significantly improve the efficiency and accuracy of oral disease diagnosis by intelligently identifying and analyzing dental image information. At present, U-shaped neural networks have been used for medical image segmentation tasks. U-shaped neural networks are generally composed of encoders, bottleneck layers, decoders, and jump connections, such as Unet, Unet++, and Unet3+. Many networks based on the U-shaped structure have achieved excellent performance in many medical image fields, but networks specifically designed for oral tasks are still relatively scarce.

[0003] Since oral panoramic radiographs contain very complex oral structure information and clinical data vary greatly, existing dental image segmentation methods may have difficulty in fully capturing the details in the image. For example, the limitations of the medical imaging equipment itself or some metal objects that may exist in the oral cavity (such as braces, restorations, etc.) may cause obvious artifacts in CT scan images, which usually impair the readability of the image. In addition, there may be large differences in tooth morphology, position, and size between different individuals, which can easily cause overfitting of complex image segmentation network models, making it difficult for image segmentation network models to apply to dental images of different individuals. Summary of the invention

[0004] Based on this, the present invention provides a tooth image segmentation model training method, a segmentation method, a device and a storage medium, which at least solve one problem in the prior art.

[0005] In a first aspect, the present invention provides a method for training a tooth image segmentation model, which comprises the following steps: Get a labeled dental image dataset; Build a tooth image segmentation model including an encoder, a bottleneck layer, a decoder, and a skip connection layer, and use a labeled tooth image dataset to train the tooth image segmentation model; In the encoder, an x-stage encoding operation is performed on the input labeled tooth image, and each stage outputs a feature map; wherein x is a positive integer greater than 2; In the skip connection layer, the output feature maps of the three stages of the encoder are fused based on channel gating; In the bottleneck layer, feature fusion is performed on the output feature map of the encoder at stage x; In the decoder, decoding is performed based on the output feature maps of the bottleneck layer and the skip connection layer.

[0006] It should be noted that the labeled dental image dataset contains dental images and corresponding labels, where the dental image can be a panoramic oral image (referred to as a dental film), and the label can be a mask of the dental area in the dental image. The encoder, bottleneck layer and decoder form a U-shaped neural network, and the encoder and decoder structures are symmetrical (the number of encoding and decoding stages is the same).

[0007] In some optional embodiments, performing channel gating-based feature fusion on the output feature maps of the three stages of the encoder respectively includes the following steps: The size of the output feature map of the first stage of the encoder is changed from R C×H×W Transformed into R C×N , as the global channel feature F 1; Upsample and convolve the output feature maps of the encoder stage 2 and stage 3 respectively to obtain f 1 and f 2. f 1 and f The transpose of 2 ( ) performs matrix multiplication and uses the SoftMax function to generate channel gates G 1. Use the access door G 1 and global channel characteristics F 1 multiplied by 1, we get the result f r_1 , perform a reshaping operation, and perform a residual connection on the output feature map of the first stage of the encoder to obtain the output F output_1 ; The size of the output feature map of the encoder at stage x is R C×H×W Transformed into R C×N , as the global channel feature F x ; Downsample the output feature maps of the encoder stage x-1 and stage x-2 respectively, and obtain f 1 and f 2. f 1 and f The transpose of 2 ( ) performs matrix multiplication and uses the SoftMax function to generate channel gates G x ; Use access door G x With global channel characteristics F x Multiply and get the result f r_x, perform a reshaping operation and perform a residual connection on the output feature map of the x-th stage of the encoder to obtain the output F output_x ; Reshape the size of the output feature map of the y-th stage of the encoder from R C×H×W to R C×N , as the global channel feature F y , where y is a positive integer and 1 < y < x; downsample the output feature map of the (y - 1)-th stage of the encoder to obtain f 1; perform upsampling and convolution operations on the output feature map of the (y + 1)-th stage of the encoder to obtain f 2; perform matrix multiplication on the transpose of f 1 and f 2 ( ), and use the SoftMax function to generate the channel gate G y ; use the channel gate G y to multiply with the global channel feature F y to obtain the result f r_y , perform a reshaping operation and perform a residual connection on the output feature map of the y-th stage of the encoder to obtain the output F output_y ; where C represents the number of channels, H represents the length of the feature map, W represents the width of the feature map, N = H × W, and R represents the mapping space.

[0008] In some alternative embodiments, x is equal to 4. That is, the encoder includes 4 LGFF modules and performs encoding in 4 stages.

[0009] In some alternative embodiments, the encoder includes an LGFF module. In the LGFF module, local feature extraction and global feature extraction are respectively performed on the input feature map (tooth image), and the extracted local feature map and global feature map are concatenated. After concatenation, depthwise separable convolution is performed to obtain the fused feature map (the output feature map of the LGFF module), and finally downsampling is performed through a max pooling operation.

[0010] In some alternative embodiments, the local feature extraction includes the following steps: perform 3×3 convolution and 1×1 convolution on the input feature map respectively, add them element-wise after normalization respectively; activate using the ReLU function, perform 1×1 convolution and normalization and then add them element-wise; then perform 3×3 convolution and 1×1 convolution respectively, and add them element-wise after normalization respectively.

[0011] In some optional embodiments, global feature extraction includes the following steps: reconstructing the input feature map into a multidimensional feature map, and then performing 3×3 convolution and multi-head self-attention operations on the features of each dimension; using normalization and ReLU function activation; then performing 3×3 convolution, normalizing and using ReLU function activation; finally, reconstructing the multidimensional feature map to the size of the input feature map.

[0012] In some optional embodiments, the bottleneck layer includes a LGFF module, in which local feature extraction and global feature extraction are performed on the feature maps from the encoder, respectively, and the extracted local feature maps and global feature maps are spliced, and then depth-separable convolution is performed to obtain a fused feature map and send it to the decoder.

[0013] In some optional embodiments, the decoder includes a LGFF module, which first upsamples the features from the bottleneck layer, concatenates the features output by the jump connection layer, and then sends the features to the LGFF module for local feature extraction and global feature extraction, concatenates the extracted local feature map and the global feature map, and then performs depth-separable convolution to obtain a fused feature map.

[0014] In a second aspect, the present invention provides a tooth image segmentation method, which comprises the following steps: The tooth image is input into the tooth image segmentation model, and the tooth image segmentation model outputs the target image; The tooth image segmentation model is obtained by the tooth image segmentation model training method as described above.

[0015] In a third aspect, the present invention provides a tooth image segmentation device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions, which, when executed by at least one processor, implement the tooth image segmentation method as described above.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium storing instructions, which, when executed by a processor, implement the tooth image segmentation method as described above.

[0017] Due to the adoption of the above technical solution, the embodiments of the present invention have at least the following beneficial effects: (1) The symmetric structure of the encoder and decoder ensures that the information flow between them is similar, which helps maintain the semantic consistency between the encoder and decoder; (2) A lighter self-attention module is used to replace the ordinary self-attention module. Self-attention is calculated within a single block, which not only reduces the computation and storage costs, but also effectively captures the global information of the image; (3) The multi-scale channel gating module effectively integrates features from different stages of the encoder and solves the semantic gap problem between different stages of the encoder, so that the model can more comprehensively understand the structure and characteristics of oral images and improve the performance and robustness of the model in the tooth image segmentation task; (4) The channel gating mechanism can help the model more effectively utilize important feature channels and suppress channels that are irrelevant to the task, thereby improving the accuracy and efficiency of tooth image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the overall architecture of the tooth image segmentation model in an embodiment of the present invention.

[0019] Figure 2 Schematic diagram of a feature fusion module in an embodiment of the present invention.

[0020] Figure 3 Schematic diagram of a local feature extraction module in an embodiment of the present invention.

[0021] Figure 4 Schematic diagram of a global feature extraction module in an embodiment of the present invention.

[0022] Figure 5 Schematic diagram of the first stage of the MSCG module in an embodiment of the present invention.

[0023] Figure 6 Some images and corresponding masks from the MICCAI dataset are shown.

[0024] Figure 7 Some visualization results of tooth image segmentation by each model are shown. DETAILED DESCRIPTION

[0025] The following is a clear and complete description of the concept of the present invention and the technical effects produced, so as to fully explain the purpose, scheme and effect of the present invention.

[0026] like Figure 1As shown, a tooth image segmentation model of an embodiment of the present invention is composed of an encoder, a bottleneck layer, a decoder and a jump connection layer, wherein the encoder, the bottleneck layer and the decoder all include a feature fusion (Local and Global Feature Fusion, referred to as LGFF) module, and the jump connection layer includes a multi-scale channel gating (Mult-Scale Channel Gated, referred to as MSCG) module. In the encoder, the LGFF module is used to extract high-dimensional features locally and globally and the two are adaptively fused, and then the output fusion feature map is downsampled by maximum pooling. In the bottleneck layer, the LGFF module is used to extract high-dimensional features locally and globally from the feature map from the encoder and the two are adaptively fused to obtain a fused feature map and send it to the decoder. In order to maintain the semantic consistency between the encoder and the decoder, the decoder adopts a symmetrical structure with the encoder, and gradually restores the size of the feature map by upsampling. In the jump connection layer, the MSCG module is used to fuse the multi-layer features from the encoder to eliminate the semantic gap between the multi-layer features, and enable the model to better understand the semantic information of input features at different levels, capture effective information and suppress irrelevant noise. The output features of the MSCG module are then concatenated with the corresponding feature maps in the decoder, so that the decoder receives richer semantic information and better restores the input image. Finally, a 1×1 convolution is performed to obtain the predicted mask.

[0027] like Figure 2 As shown in FIG. 1 , in the LGFF module of the encoder, local feature extraction and global feature extraction are performed on the input feature map respectively and the two are adaptively fused (i.e., the extracted local feature map and the global feature map are concatenated and then subjected to depthwise separable convolution) to obtain a fused feature map, as shown in equations (1) and (2): in, F in represents the input feature map, LFF (·) represents local feature extraction, GFF (·) represents global feature extraction, L represents the local feature map obtained by local feature extraction, G represents the global feature map obtained by global feature extraction, concat (·) indicates a concatenation operation, DsConv (·) represents depthwise separable convolution, F out Represents the fused feature map output by the LGFF module.

[0028] Specifically, Figure 3As shown in , local feature extraction includes the following steps: perform 3×3 convolution and 1×1 convolution on the input feature map respectively, normalize them respectively and then add them element by element; use ReLU function to activate, perform 1×1 convolution and normalize them and then add them element by element; then perform 3×3 convolution and 1×1 convolution respectively, normalize them respectively and then add them element by element, and then use ReLU function to activate; as shown in equations (3) to (7): in, F in represents the input feature map, f 3×3 (·) represents a 3×3 convolution operation, f 1×1 (·) represents a 1×1 convolution operation, BN (·) indicates normalization, ReLU (·) represents the ReLu activation function, f 1 and f 2 represents the features obtained by performing 3×3 convolution and 1×1 convolution on the input features and then normalizing them. F 1 means f 1 and f 2 The features activated by ReLU function after element-wise addition, F 2 means F 1 is the feature that is 1×1 convolved and normalized and then added element by element. f 3 and f 4 respectively represent F 2. The features obtained after 3×3 convolution and 1×1 convolution and then normalization, L Represents the local feature map obtained by local feature extraction. Here, two parallel convolutions with different convolution kernel sizes are used twice to extract multi-scale information in the features, which can extract more extensive and detailed information in the features.

[0029] Specifically, Figure 4 As shown in , global feature extraction includes the following steps: reconstructing the input feature map into a multi-dimensional feature map, and then performing 3×3 convolution and multi-head self-attention operations on the feature map of each dimension; using normalization and ReLU function activation; then performing 3×3 convolution, normalizing and using ReLU function activation; finally, reconstructing the multi-dimensional feature map to the size of the input feature map; as shown in equations (8) to (10): in, F i Represents the input feature of the i-th dimension of the multidimensional feature, i is a positive integer, f3×3 (·) represents a 3×3 convolution operation, f 1×1 (·) represents a 1×1 convolution operation, MHSA (·) represents a multi-head self-attention operation, BN (·) indicates normalization, ReLU (·) represents the ReLu activation function, F 1 represents the feature obtained by performing 3×3 convolution on the input feature. F 2 means yes F 1. The features obtained by performing multi-head self-attention operation, normalization and ReLU function activation, F out_i Represents the output feature of the i-th dimension. Here, the window-based multi-head self-attention ( MHSA ) When processing long sequences, there is no need to consider the attention relationship of the entire sequence, thus reducing the need for pre-trained models and the requirements for computing resources. At the same time, convolution ( f 3×3 (·)) to assist the window-based multi-head self-attention and obtain global features with lower complexity.

[0030] Specifically, when applying window-based multi-head self-attention, the input feature map Divide into n non-overlapping blocks of equal size ,in , , P represents the block size, C represents the number of channels, H represents the length of the input feature map, and W represents the width of the input feature map. In this embodiment, P=4. In each block, a learnable projection matrix W is used Q , W K , W V For each block of input Projection is performed to obtain the corresponding , , , which is the query, key, and value matrix. Perform the same self-attention operation h times on each block, where h is a positive integer. First , The similarity calculation is performed to obtain the attention weight matrix, and then the bias matrix B is added to the attention weight matrix. Finally, the obtained attention weight matrix is ​​used to The weighted ones are then added to obtain the final attention feature map. No. The calculation formula for the self-attention operation is as follows: Where d represents the scaling factor, which is used to avoid the problem of gradient vanishing or gradient exploding; B represents the bias matrix, which is used to introduce relative position information; F i represents the input feature; W j Q , W j K and W j V They represent the learnable matrix W of the j-th self-attention operation respectively. Q , W K , W V ; Q ij , K ij and V ij Represents the query, key, and value matrix corresponding to the j-th self-attention operation of the input feature; SoftMax (·)express SoftMax function; F iout_j represents the output feature obtained in the jth self-attention operation. In this embodiment, .

[0031] Then concatenate the outputs of multiple self-attention operations to get a block The output is calculated as follows: in, represents the learnable projection matrix, Concat (·) indicates a concatenation operation, MultiHead ( Q i , K i , V i ) indicates a block F i Output.

[0032] It should be noted that the complexity of calculating the attention matrix globally in the feature map is shown in formula (14), and the complexity of calculating the attention matrix in each block in the feature map is shown in formula (15). It can be seen that when calculating the attention matrix globally, the complexity of the algorithm is quadratic with the image size; while when calculating the attention only in the block, the complexity of the algorithm is linear with the image size, which greatly reduces the computational cost.

[0033] Among them, H and W represent the length and width of the input feature map respectively, P represents the block size, and C represents the number of channels.

[0034] In the MSCG module, the feature outputs from three consecutive stages of the encoder are taken as input. It is defined as a convolution operation with kernel size n and step size i. is the upsampling with a factor of j, and n, i, and j are all positive integers. Input the features of the three adjacent layers of the encoder into the MSCG module. For the size difference of the output features of the encoder at different stages, there are three situations in the feature fusion process: For the first stage, upsampling and convolution operations are performed on the features of the next two stages. In this embodiment, the current stage feature is , the features of the two stages of feature fusion are , , respectively , Use upsampling and convolution operations to obtain , as shown in formula (16) and formula (17): For the last stage, the features of the previous two stages are downsampled. In this embodiment, the current stage features are , then the features of the two stages of feature fusion are , , respectively and Use convolution with a stride of 2 for downsampling to obtain , as shown in formula (18) and formula (19): For the intermediate stage, the features of the previous stage are downsampled, and the features of the next stage are upsampled and convolved. In this embodiment, the features of the current stage are or . Based on the current stage characteristics For example, the features of the two stages of feature fusion are , ,right Use convolution with a stride of 2 for downsampling. Using upsampling and convolution operations, we can obtain , as shown in formula (20) and formula (21): For each stage, the size of the input feature map of this stage is R C×H×W Transformed into R C×N , as the global channel feature F ; and the obtainedf 1 and f The transpose of 2 ( ) performs matrix multiplication and uses the SoftMax function to generate channel gates G ;use With global channel characteristics Multiply and get the result f , perform the reshaping operation, and finally perform residual connection on the input feature map to obtain the output of the MSCG module. Figure 5 As shown, taking the first stage as an example, the features The size of Transformed into , after deformation Considered as global channel features ; Then get and The transpose of is used for matrix multiplication and the channel gate is generated using the SoftMax function ;use With global channel characteristics Multiply and then add the result f Reshape , then Perform residual connection to obtain the final features, as shown in equations (22) to (24): in, f 1. f 2 respectively represent the results of feature fusion of the output feature maps of the second and third stages of the encoder; F 1 represents the input feature map (i.e., the output feature map of the first stage of the encoder); reshape (·) indicates a deformation operation; f R_1 and f R_2 Respectively represent the deformed f 1 and f 2; F Represents the global channel characteristics; SoftMax (·) represents the SoftMax function, G Indicates a passage door. F output Represents the output feature map.

[0035] It should be noted that the MSCG module introduces a gating mechanism to combine shallow detail information with deep semantic information. Features of different scales can participate more comprehensively in the transmission of cross-layer information to enrich information at each stage. In addition, the channel gating mechanism can enhance the model's perception of important features in different channels in oral images, helping the network to more effectively utilize important feature channels in the image. Therefore, more refined segmentation results can be obtained when performing tooth segmentation tasks, improving the accuracy and robustness of the model.

[0036] The feature fusion module (LGFF module) in the bottleneck layer is the same as the feature fusion module in the encoder. The LGFF module in the bottleneck layer extracts local features and global features from the output feature map of the encoder, concatenates the extracted local feature map and global feature map, and then performs depthwise separable convolution to obtain the fused feature map and send it to the decoder.

[0037] The decoder receives the output feature map from the bottleneck layer and performs upsampling. After splicing the multi-scale features output from the MSCG module, it performs local feature extraction and global feature extraction respectively, and splices and performs depth-separable convolution on the extracted local feature map and global feature map to obtain the prediction result map.

[0038] The following describes the process and results of testing the performance of the tooth image segmentation model in the embodiment of the present invention through experiments.

[0039] The public dataset (MICCAI dataset) used in the experiment comes from the MICCAI 2023 dental image segmentation task challenge. The dataset contains a total of 2,000 oral panoramic images and corresponding masks. Figure 6 Some images and corresponding masks are shown. The resolution of each image is 640×320 pixels. The dataset is randomly divided into training set and test set with a distribution ratio of 19:1, and then the abnormal data in the training set and test set are removed. Figure 6 As shown in (d), the training set contains 1900 images and the test set contains 98 images.

[0040] Seven evaluation indicators are used in the experiment, namely Dice coefficient (Dice), Hausdorff distance (HD), recall rate (Recall), specificity rate (Spe), precision rate (Pre), accuracy rate (Acc), and intersection over union (IoU). Their definitions are as follows: in, A represents the predicted pixel set, B Represents the actual set of pixels. TP represents true positive examples, that is, the number of samples for which the model correctly predicts the positive class; FPrepresents false positive examples, that is, the number of samples that the model incorrectly predicts as positive; TN represents true negative examples, that is, the number of samples correctly predicted by the model as negative; FN represents false negative examples, that is, the number of samples that the model incorrectly predicts as negative. d (·) means calculating the Euclidean distance between two pixels.

[0041] The Dice coefficient is used to evaluate the similarity between the model prediction and the true label. HD is used to measure the dissimilarity between the predicted pixel set and the actual pixel set, mainly measuring the accuracy of the boundary. The smaller the HD value, the better. Recall is used to evaluate the efficiency of the model in successfully segmenting teeth. Spe is used to evaluate the model's ability to recognize non-target areas. Pre is used to evaluate the accuracy of the model's segmentation of teeth. Acc is used to evaluate the correctness of the prediction. IoU, like the Dice coefficient, is used to measure the similarity between the predicted segmentation area and the true segmentation area, but the focus is different. IoU focuses more on measuring the degree of overlap, while the Dice coefficient emphasizes the proportion of the intersection area in the total area.

[0042] In the experiment, the tooth image segmentation model in the embodiment of the present invention was constructed based on Pytorch (version 2.3.1), and the computing device used was Nvidia GeForce RTX 4080s GPU. The cross entropy loss function was selected as the loss function. The smaller the loss function, the smaller the difference between the predicted value and the actual value of the model, as shown in formula (32): in, Indicates the actual value, Represents the model prediction value.

[0043] The hyperparameters of the model are set as follows: MICCAI data set: input size = 224×224, max epoch = 200, batch size = 6, lr = 5e-5, and optimizer = Adam(weight decay = 0.0001); MICCAI data set: input size = 512×512, max epoch = 100, batch size = 3, lr = 5e-5, and optimizer = Adam(weight decay = 0.0001).

[0044] In order to verify the effectiveness of each module of the tooth image segmentation model in the embodiment of the present invention, an ablation experiment was performed on the MICCAI dataset, and the tooth image was input in a size of 224×224.

[0045] As shown in Table 1, after extracting global and local features from the feature map in the encoder at the same time, the segmentation performance is improved, indicating that combining local and global features can make the feature representation more comprehensive. In addition, when the encoder and decoder use dual-path modules at the same time, a Dice score of 92.47% is obtained, which also shows that the use of a symmetrical structure can ensure that the information flow between the encoder and decoder is similar, which helps to maintain the semantic consistency between the encoder and decoder. After using the LGFF module, the performance of the network has been improved to a certain extent.

[0046] Table 1 The impact of LGFF module on model performance In Table 1, U-Net indicates that the model uses the encoder or decoder in U-Net; dual-path module indicates that the model uses a local feature extraction module and a global feature extraction module, but only performs a simple addition of the extracted local and global features (i.e., the remaining part after removing the splicing and depth-wise separable convolution in the LGFF module).

[0047] Under the condition of using the LGFF module, the MSCG module is added, and the results are shown in Table 2. As can be seen from the table, after fusing multi-scale information through the MSCG module, the segmentation performance of the model is improved, indicating that the MSCG module can alleviate the semantic gap in the network, so that the model can understand the structure and characteristics of oral images more comprehensively.

[0048] Table 2 Impact of MSCG module on model performance In addition, the impact of different numbers of channels on model performance was tested. C1, C2, C3, C4, and C5 in [C1, C2, C3, C4, and C5] represent the number of channels of the features output by the four stages and the bottleneck layer in the encoder, respectively. The results are shown in Table 3. As can be seen from the table, when the number of channels is [64, 128, 256, 512, 1024], the model achieves better performance, with Dice values ​​of 92.55% and 93.28% respectively. Reducing the number of channels will significantly reduce the parameters of the model, thereby reducing the computational burden and speeding up the training speed, but reducing the number of channels will have a certain impact on the performance of the model.

[0049] Table 3 The impact of the number of channels on model performance In order to further verify the effectiveness of the tooth image segmentation model in the embodiment of the present invention, it is compared with some models in the prior art. To ensure fairness, when comparing with other models, all parameters are implemented according to their official codes. The MICCAI dataset is used for experiments. When the resolution of the input image is 224×224, the results are shown in Table 4.

[0050] Table 4 Performance of each model when the input image resolution is 224×224 It can be seen that the tooth image segmentation model in the embodiment of the present invention achieved the highest Dice score of 92.55%, and the improvement range of Dice was 1.06% to 7.22%. More specifically, compared with other models, the tooth image segmentation model in the embodiment of the present invention achieved the highest scores in 6 evaluation indicators (Dice, HD, Spe, Pre, Acc and IoU), and achieved competitive results in the remaining evaluation indicator (Recall).

[0051] Experiments are conducted using the MICCAI dataset. When the resolution of the input image is 512×512, the results are shown in Table 5.

[0052] Table 5 Performance of each model when the input image resolution is 512×512 It can be seen that when the resolution is larger, the feature map contains more detail information. In this case, all indicators of the tooth image segmentation model in the embodiment of the present invention obtained the highest score. For example, the tooth image segmentation model in the embodiment of the present invention obtained the highest Dice score of 93.46%, with an improvement range of 0.23% to 5.31%. Except for SwinUnet, the performance of other models has been improved compared to the smaller resolution. This is because the official code of SwinUnet only provides a pre-trained model with a resolution of 224×224. When the pre-trained model is not used, the model performance will be greatly affected. It is worth noting that TransUNet, SwinUnet and U-ConvNext all use pre-trained models, while the tooth image segmentation model in the embodiment of the present invention does not rely on a pre-trained model with pre-trained weights. Whether in the case of low resolution or high resolution, the tooth image segmentation model in the embodiment of the present invention has shown better performance.

[0053] Some segmentation results when the input image resolution is 512×512 are visualized, such as Figure 7As shown in the figure. In the case of missing tooth targets, as shown in the second column, other models cannot accurately predict the actual tooth targets, but the tooth image segmentation model in the embodiment still shows excellent prediction ability. Overall, the tooth image segmentation model in the embodiment generates fewer noise points during prediction and is closer to the doctor's annotation results.

[0054] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same or equivalent means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A tooth image segmentation model training method, characterized in that: The following steps are involved: Get a labeled dental image dataset; Build a tooth image segmentation model including an encoder, a bottleneck layer, a decoder, and a skip connection layer, and use a labeled tooth image dataset to train the tooth image segmentation model; In the encoder, an x-stage encoding operation is performed on the input labeled tooth image, and each stage outputs a feature map; wherein x is a positive integer greater than 2; In the skip connection layer, the output feature maps of the three stages of the encoder are fused based on channel gating; In the bottleneck layer, feature fusion is performed on the output feature map of the encoder at stage x; In the decoder, decoding is performed based on the output feature maps of the bottleneck layer and the skip connection layer.

2. The method according to claim 1, characterized in that The channel-gated feature fusion of the output feature maps of the three stages of the encoder includes the following steps: The size of the output feature map of the first stage of the encoder is changed from R C×H×W Transformed into R C×N , as the global channel feature F 1; Upsample and convolve the output feature maps of the encoder stage 2 and stage 3 respectively to obtain f 1 and f 2. f 1 and f 2 is transposed for matrix multiplication, and the channel gate is generated using the SoftMax function G 1. Use the access door G 1 and global channel characteristics F 1 multiplied by 1, we get the result f r_1 , perform a reshaping operation, and perform a residual connection on the output feature map of the first stage of the encoder to obtain the output F output_1 ; The size of the output feature map of the encoder at stage x is R C×H×W Transformed into R C×N , as the global channel feature F x ; Downsample the output feature maps of the encoder stage x-1 and stage x-2 respectively, and obtain f 1 and f 2. f 1 and f 2 is transposed for matrix multiplication, and the channel gate is generated using the SoftMax function G x ; Use access door G x With global channel characteristics F x Multiply and get the result f r_x , perform a reshaping operation, and perform a residual connection on the output feature map of the encoder stage x to obtain the output F output_x ; Reshape the output feature map of the y-th stage of the encoder from R C×H×W to R C×N , which serves as the global channel feature F y , where y is a positive integer and 1 < y < x; downsample the output feature map of the (y - 1)-th stage of the encoder to obtain f 1; upsample and perform a convolution operation on the output feature map of the (y + 1)-th stage of the encoder to obtain f 2; perform a matrix multiplication operation on the transposes of f 1 and f 2, and use the SoftMax function to generate the channel gate G y ; use the channel gate G y to multiply with the global channel feature F y to obtain the result f r_y , perform a reshaping operation, and perform a residual connection on the output feature map of the y-th stage of the encoder to obtain the output F output_y ; Among them, C represents the number of channels, H represents the length of the feature map, W represents the width of the feature map, N=H×W, and R represents the mapping space.

3. The method according to claim 2, characterized in that x is equal to 4.

4. The method according to claim 1, characterized in that: The encoder includes an LGFF module, in which local feature extraction and global feature extraction are performed on the input feature map respectively, and the extracted local feature map and global feature map are spliced, and then depthwise separable convolution is performed after splicing to obtain a fused feature map, and finally down-sampled through a maximum pooling operation.

5. The method according to claim 4, characterized in that The local feature extraction includes the following steps: perform 3×3 convolution and 1×1 convolution on the input feature map, normalize them respectively and then add them element by element; use the ReLU function to activate, perform 1×1 convolution and normalize them and then add them element by element; then perform 3×3 convolution and 1×1 convolution, normalize them respectively and then add them element by element.

6. The method according to claim 4, characterized in that Global feature extraction includes the following steps: reconstructing the input feature map into a multi-dimensional feature map, and then performing 3×3 convolution and multi-head self-attention operations on the features of each dimension; using normalization and ReLU function activation; then performing 3×3 convolution, normalizing and using ReLU function activation; finally, reconstructing the multi-dimensional feature map to the size of the input feature map.

7. The method according to claim 1, characterized in that The bottleneck layer includes the LGFF module, in which local feature extraction and global feature extraction are performed on the feature maps from the encoder respectively, and the extracted local feature maps and global feature maps are concatenated, and then depth-wise separable convolution is performed to obtain a fused feature map and send it to the decoder.

8. The method according to claim 1, characterized in that The decoder includes an LGFF module, which first upsamples the features from the bottleneck layer, concatenates the features output by the jump connection layer, and then sends the features to the LGFF module for local feature extraction and global feature extraction. The extracted local feature map and global feature map are concatenated, and then a depth-wise separable convolution is performed to obtain a fused feature map.

9. A tooth image segmentation method, characterized in that: The following steps are involved: The tooth image is input into the tooth image segmentation model, and the tooth image segmentation model outputs the target image; Wherein, the tooth image segmentation model is obtained by the tooth image segmentation model training method described in any one of claims 1-8.

10. A tooth image segmentation device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions, which implement the tooth image segmentation method according to claim 9 when executed by at least one processor.

11. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed by the processor, the tooth image segmentation method according to claim 9 is implemented.

Images