A tooth image segmentation method based on root edge weighted loss

Abstract

The present invention discloses a tooth image segmentation method based on weighted root edge loss, which relates to the field of medical image processing. The method constructs and trains a tooth instance segmentation network with weighted root edge loss. The segmentation network mainly includes a backbone network, a refined feature enhancement module, a root edge feature enhancement module, and a root edge weighted cross entropy loss function. The best model obtained after training is used to obtain the tooth instance segmentation result. The method of the present invention deeply integrates the root boundary clues and the detailed features of the teeth to produce a fine tooth mask, overcomes the limitations of traditional algorithms in the face of root boundary uncertainty and due to the wide variety of teeth, and uses a weighted cross entropy loss function to optimize the model training process during the training process, thereby achieving high-precision recognition and segmentation of teeth.

Description

Technical Field

[0001] The present invention belongs to the technical fields of deep learning, medical images, and semantic segmentation, and specifically relates to a tooth image segmentation method based on root edge weighted loss. Background Art

[0002] With the development of artificial intelligence, computer vision and deep learning technologies have made significant progress in automatic panoramic X-ray analysis, aiming to assist dentists in making accurate diagnoses and significantly improve the diagnostic efficiency and accuracy of panoramic X-rays. Convolutional neural networks (CNNs) have changed the landscape of medical image analysis by automatically learning and extracting hierarchical features from images. They are widely used in image classification, segmentation, and detection tasks, especially in the detection of breast tumors, lung lesions, and heart disease. In recent years, deep learning has also made progress in the field of dental image segmentation. Faster R-CNN and Mask R-CNN are widely used in tooth segmentation, but these methods rely on pre-positioned teeth. MSLPNet solves the boundary prediction problem and achieves accurate positioning of teeth by introducing a multi-scale structure. In addition, TransUNet combines the U-Net and Transformer architectures to improve image segmentation performance.

[0003] Current tooth instance segmentation methods mainly focus on segmenting the core area of ​​teeth, which often faces two major problems: first, the instance segmentation accuracy of tooth images is low; second, due to the wide variety of teeth, different types of teeth are difficult to distinguish, which makes the boundary of tooth segmentation blurred and the segmentation accuracy needs to be improved urgently. Therefore, incorporating detailed information features of the edge is particularly important for improving the accuracy of tooth segmentation. Summary of the invention

[0004] In order to solve the shortcomings of the prior art, improve the accuracy of tooth instance segmentation, and avoid the blurring of tooth root boundaries in instance segmentation, the present invention adopts the following technical solutions:

[0005] A tooth image segmentation method based on tooth root edge weighted loss comprises the following steps:

[0006] Step 1: Obtain oral dental X-ray images and divide the data set into training set, validation set and test set;

[0007] Step 2: By embedding the backbone network Resnet-101 with the refined feature enhancement module, feature extraction is performed on the oral dental X-ray image to obtain a tooth feature map, and then the tooth feature map after root feature enhancement is obtained by the tooth root edge feature enhancement module; by embedding the refined feature enhancement module in the Resnet-101 network, the model's ability to focus on and extract important features is further improved;

[0008] Step 3: The tooth feature map with feature extraction and root edge feature enhancement is used to generate the coordinates of the candidate region of interest ROI of the potential prediction object through the region of interest prediction module;

[0009] Step 4: After the candidate area is passed through the mask segmentation network, combined with the root edge weighted cross entropy loss function, a clear pixel-level segmentation mask result of the tooth root is generated; the segmentation effect of the training model is evaluated, especially the root loss index MR. If it meets the requirements, the segmentation mask result is output, otherwise the mask segmentation is performed again;

[0010] Step 5: Pass the candidate region coordinates through the fully connected layer to obtain instance segmentation labels, bounding boxes and other information about each tooth; obtain the instance segmentation result of the tooth based on the clear segmentation mask result of the tooth root, the instance segmentation label and bounding box information of each tooth; input the tooth image to be segmented into the trained tooth image instance segmentation process, and finally output the instance segmentation result of the tooth.

[0011] Furthermore, the refined feature enhancement module in step 2 compresses the input valid features through an adaptive average pooling operation, aggregates the features of size H×W, obtains an aggregated information feature map, compresses the dimension through a linear layer, then passes through an activation function, and activates specific channel features through a channel weight allocation mechanism, and finally controls the activation of each channel, multiplies the input valid features by the corresponding channel weight, and obtains enhanced features. The refined feature enhancement module is mainly used to enhance the importance of each channel, optimize feature extraction through a residual learning mechanism, and enable the model to more accurately identify and utilize valuable information.

[0012] Furthermore, the calculation formula for executing the refined feature enhancement module is as follows:

[0013]

[0014]

[0015]

[0016]

[0017] Among them, F represents the effective features of the input, Avg() represents the average pooling operation on the input feature vector, and R s1 () indicates adaptive average pooling, Linear 1 () represents the first linear transformation layer, It represents the feature map of F after adaptive average pooling and linear layer processing, which is used to compress the feature map F to obtain aggregate features. Relu represents the ReLU activation function. Linear 2 () represents the second linear transformation layer, express The feature map after activation and linear layer processing, P CBR-2 Indicates zero padding processing, that is, after 2 Conv-BatchNormational-Relu operations, Conv 1 Represents a convolution operation with a convolution kernel size of 1. Sigmoid represents the Sigmoid activation function, which is used to map the result of the convolution to the interval [0,1]. express After zero padding, convolution and activation operations, the feature map Indicates the enhanced features;

[0018] The average pooling operation calculates the average value of the pixel values ​​covered by a k*k pooling window. The specific expression is as follows:

[0019] .

[0020] Furthermore, the tooth root edge feature enhancement module in step 2 includes the following steps:

[0021] Step 2.1: Obtain the tooth feature map and the root area mask to obtain the four-dimensional tensor F RF With R m , in order to obtain their common feature information, the tensor F RF With R m After element-by-element multiplication, we get the merged tensor R f ;

[0022] Step 2.2: For the tensor F RF A 1×1 convolution is performed to compress its channels, and a nonlinearity is introduced through the RELU activation function to enable the model to fit more complex relationships. A 3×3 convolution is then introduced to capture spatial information, especially the characteristics of the root area. After the RELU activation function, a 1×1 convolution is performed again to restore the number of channels to the original input channel number. The Sigmoid activation function is then used to generate feature weights for each channel of the input feature map. Finally, the feature weights F se With the original input tensor F RF Perform element-by-element multiplication to obtain feature F out ; The specific expression method is as follows:

[0023]

[0024] Among them Represented as the weight of the first 1×1 convolution, represents the weight of the second 1×1 convolution, Represented as the weight of 3×3 convolution, RELU and Sigmoid are both activation functions;

[0025] Step 2.3: Transform feature F out And the combined tensor R f The fused feature F is obtained through the adder out1 , to fuse the relevant information of these two features, so that the fused feature combines the correlation of the tooth target feature and the root feature, providing richer feature information for subsequent steps. out1 Perform 1×1 convolution to obtain feature F out2 , to adjust the number of channels and introduce nonlinearity. Through this operation, the model can extract deeper feature representations while reducing computational complexity. out1 With feature F out2 The feature F is obtained by multiplication out3 , to enhance the complementarity of features. This process further enhances the expressiveness of features by element-by-element multiplication, retaining F out1 With F out2 The complementarity between them, the feature F out3 The feature F is obtained by the inverter out4 , transform the features spatially to enhance the distinguishability of the features, F out4 The feature map processed by the inverter can effectively enhance the edge information and details of the feature, and transform the feature F out3 With feature F out4 The feature F after the root edge enhancement is obtained by the adder out5 , the edge features of the tooth root are enhanced, the information of the tooth and root features is integrated, and the model's perception of the detailed features of the tooth root is improved; the specific expression method is as follows:

[0026]

[0027]

[0028]

[0029]

[0030] .

[0031] Furthermore, in step 3, all regions of interest ROIs are transformed into the same output size by a region of interest alignment module, including the following steps:

[0032] Step 3.1: Since the resolution of the feature map is lower than that of the original image, the coordinates of the region of interest ROI need to be scaled to the feature map first;

[0033] Step 3.2: For each region of interest ROI, a fixed number of grid samples are performed within the region;

[0034] Step 3.3: Since the coordinate points of the region of interest (ROI) are not integers, bilinear interpolation is used to calculate the value of the region of interest in the original image through weighted calculation of multiple adjacent points;

[0035] Step 3.4: Aggregate the sampling results. For each sampling grid in each ROI grid, use the value as the feature value of the position to obtain a ROI of the same size.

[0036] Furthermore, in step 3.1, the input image size is height H × width W, and the corresponding feature map size is H f ×W f ;

[0037]

[0038] Among them, (x, y) represents the coordinates of the region of interest ROI in the original image, (x f ,y f ) represents the corresponding coordinates in the feature map;

[0039] In step 3.2, for the output H f ×W f The feature map of size will be divided into H f ×W f small grids, each of which needs to sample a value from the feature map;

[0040] In step 3.4, for each region of interest ROI, f ×W f For each sampling grid in the grid, the value obtained using bilinear interpolation will be used as the feature value of that position, and these features are concatenated into a unified feature map of size K*K.

[0041] Furthermore, in step 3.3, bilinear interpolation is used to accurately calculate the value formula for the target position (x, y) through weighted calculation of four adjacent points:

[0042]

[0043] Among them, V represents the value on the feature map obtained by bilinear interpolation, (x, y) represents the coordinates of the region of interest ROI in the original image, Δx and Δy are the distances between the floating point coordinates and their integer points. This formula calculates the distance between the target position (x, y) relative to the other three points (x 1 ,y 1 ),(x 2 ,y 1 ),(x 1 ,y 2 ) and (x 2 ,y 2 )’s position changes.

[0044] Furthermore, in step 4, the candidate region is processed by the mask segmentation network U-KAN, and the predicted mask of the candidate region is obtained through the processing of the encoder and the decoder. The predicted mask and the true mask are jointly calculated with the weighted cross entropy loss and the shape inconsistency penalty term, and the two are combined to obtain the root edge loss function, and the weight W is adaptively changed based on the degree of root edge mismatch. q After a loop operation, a pixel-level mask of the teeth is finally generated.

[0045] Furthermore, the weighted cross entropy loss is defined as:

[0046]

[0047] in, represents the probability that pixel q in the true mask belongs to the tooth root area, represents the predicted probability that pixel q belongs to the tooth root area, root represents the tooth root pixel set, W q represents the adaptive change weight based on the mismatch degree of the tooth root edge. If the pixel q belongs to the tooth root area, a high weight is assigned to the adaptive change weight W. q =W root ,otherwise:

[0048]

[0049] Among them, β(q,σ(q)) represents the shape inconsistency between the pixel q used to measure the tooth root and the predicted tooth root pixel σ(q), and λ represents the hyperparameter controlling the penalty intensity;

[0050] The shape inconsistency penalty term is defined as a nonlinear combination of shape inconsistency and overall loss, increasing the penalty for high error areas in the form of a power function:

[0051]

[0052] Where G represents the tooth pixel set, k represents the parameter controlling the penalty strength, when k>1, large shape inconsistency will receive a larger penalty, β(o,σ(o)) represents the shape inconsistency degree of the tooth pixel o and the predicted tooth pixel σ(o) used to measure the tooth;

[0053] The root edge loss function REloss is defined as a weighted cross entropy loss function, and the pixel positions of the real object and the predicted object in the root edge area are assigned a high weight W root (Usually W root >1) to enhance the model’s attention to the root edge pixels. The formula is as follows:

[0054]

[0055] Among them, α represents the relative weight of the weighted cross entropy and shape penalty terms, and γ represents the hyperparameter that controls the degree of nonlinearity, which is used to significantly increase the overall penalty when the loss value is large.

[0056] To further improve the accuracy of segmentation, the root edge loss function RELoss adopts an adaptive weight mechanism, that is, the weight of the root area is no longer fixed, but dynamically adjusted according to the shape inconsistency β(q,σ(q)), so as to better capture the mismatched area; in addition, the root edge loss function RELoss uses a power function to control the overall loss through a nonlinear combination, imposes a greater penalty on high error areas, so that the model can more effectively reduce the segmentation errors of the root area; to deal with the problem of shape inconsistency, RELoss also introduces a shape penalty significant enhancement mechanism, which further strengthens the model's learning of shape inconsistent areas through the power function of β(o,σ(o)).

[0057] Furthermore, in step 4, the tooth root loss index MR is introduced to evaluate the segmentation effect, and the MIoU metric, Dice coefficient, and NSD index are introduced for auxiliary evaluation. If the evaluation effect meets the requirements, the segmented image is output, otherwise, mask segmentation is performed again;

[0058] The tooth root loss index MR is used to calculate the ratio of the tooth root loss portion in the statistical prediction mask image to the number of tooth roots in the correct mask image, and the formula is as follows:

[0059]

[0060] Among them, S represents the number of root canals missing from the predicted mask, and C represents the number of root canals in the real mask; the larger the MR ratio, the lower the accuracy of root segmentation;

[0061] The Dice coefficient indicator is as follows:

[0062]

[0063] Among them, X represents the predicted result, and Y represents the true result. When the Dice coefficient is larger, the predicted correct result is closer to the true result, indicating that the accuracy of tooth segmentation is higher.

[0064] The MioU metric is as follows:

[0065]

[0066] Among them, k represents the number of categories, the number of backgrounds and targets in the image, i represents the true value, j represents the predicted value, and p ij It indicates the value when i is predicted as j. When the MIoU value is larger, the predicted correct result is closer to the true result, which means the accuracy of tooth segmentation is higher.

[0067] The NSD indicator has the following formula:

[0068]

[0069] Where N represents the total number of data points, f i represents the value of function f at the i-th data point, g i Represents the value of the function g at the i-th data point.

[0070] The advantages and beneficial effects of the present invention are:

[0071] (1) The present invention combines a refined feature enhancement module on the basis of the ResNet-101 network, compresses the features through an adaptive average pooling operation, and effectively improves the accuracy of tooth segmentation;

[0072] (2) The tooth root edge feature enhancement module proposed in the present invention introduces the tooth root boundary contour to supervise the training process, and uses a specific complex calculation method to enhance the tooth root contour features according to the boundary clues, which significantly improves the extraction of the root edge features;

[0073] (3) The present invention proposes a root edge loss function to supervise the training process and improves the training effect of the model by jointly calculating the weighted cross entropy and the shape inconsistency penalty term.

[0074] (4) The present invention proposes a tooth root loss index MR for evaluation to evaluate the accuracy of tooth mask segmentation, especially to determine whether there is any root loss in the tooth root part after segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a flow chart of a method according to an embodiment of the present invention.

[0076] Figure 2 It is a structural diagram of a refined feature enhancement module in an embodiment of the present invention.

[0077] Figure 3 Schematic diagram of the structure of the tooth root edge feature enhancement module in an embodiment of the present invention.

[0078] Figure 4 Schematic diagram of the root edge weighted cross entropy loss function used in an embodiment of the present invention.

[0079] Figure 5 It is an experimental visualization comparison diagram in the embodiment of the present invention.

[0080] Figure 6 It is a visualization comparison diagram of tooth root experiments in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0082] To make up for the shortcomings of existing tooth instance segmentation techniques, such as Figure 1 As shown, the present invention provides a tooth image segmentation method based on root edge weighted loss, which improves the accuracy of tooth instance segmentation by fusing root boundary information, and solves the problem of root boundary fuzziness in tooth instance segmentation by fusing root edge feature enhancement module and combining root edge loss function, and specifically includes the following steps:

[0083] Step 1: Collect oral dental X-ray images and divide the dataset into training set, validation set and test set;

[0084] In this embodiment of the present invention, the data is D={dataset 1, data set 2}, where data set 1 contains 1000 tooth images, data set 2 contains 200 tooth images, and 700 and 140 tooth images are selected from data set 1 and data set 2 respectively as training sets, 200 and 40 as validation sets, and the rest as test sets.

[0085] Step 2: Input a panoramic X-ray of the oral cavity, extract features through the backbone network Resnet-101 embedded with the feature refinement and enhancement module, and then embed the root edge feature enhancement module to obtain the tooth feature map after the root features are enhanced; by embedding the refined feature enhancement module in the Resnet-101 network, the model's ability to focus on and extract important features is further improved.

[0086] Among them, Figure 2 As shown, the implementation of the refined feature enhancement module is as follows:

[0087] First, the effective input feature is defined as F, and the feature F is compressed through the adaptive average pooling operation. The features of size H×W are aggregated to obtain the aggregated information feature map. After the dimension is compressed by the linear layer, it passes through the activation function and activates specific channel features through the channel weight allocation mechanism. Finally, the activation of each channel is controlled, and the feature F is multiplied by the corresponding channel weight to obtain the enhanced result F. RF ;

[0088] The relevant calculation formula of this module is as follows:

[0089]

[0090]

[0091]

[0092]

[0093] in, It is represented as the feature map after adaptive average pooling and linear layer processing, R s1 () represents adaptive average pooling, which is used to compress the feature map F to obtain aggregate features. Avg() represents the average pooling operation on the input feature vector. Linear() represents the linear transformation layer. Relu represents the ReLU activation function. Conv 1 Indicates a convolution operation with a convolution kernel size of 1. Sigmoid indicates the Sigmoid activation function, which is used to map the result of the convolution to the interval [0,1]. CBR-2 Indicates that the input image is zero-padded before convolution.

[0094] like Figure 3 As shown, the execution of the root edge feature enhancement module includes the following steps:

[0095] Step 2.1: Input the tooth feature map and the root area mask to obtain the four-dimensional tensor F RF With R m , in order to obtain their common feature information, F RF With R m After element-by-element multiplication, we get R f ;

[0096] Step 2.2: F RFA 1×1 convolution is performed to compress its channels. A nonlinearity is introduced through the RELU activation function to enable the model to fit more complex relationships. A 3×3 convolution is introduced to capture spatial information, especially the characteristics of the root area. After the RELU activation function, a 1×1 convolution is performed again to restore the number of channels to the original input channel number. The Sigmoid function is then used to generate feature weights for each channel of the input feature map. Finally, the feature weights F se With the original input feature F RF Perform element-wise multiplication to get F out ;

[0097] Step 2.3: Type F out With R f To the adder for processing, get F out1 , for F out1 Perform 1×1 convolution to get F out2 , enter F out1 With F out2 To the multiplier for processing, get F out3 , enter F out3 To the inverter for processing, get F out4 , enter F out3 With F out4 To the adder for processing, get F out5 , thus completing the root edge feature enhancement.

[0098] Step 3: After the tooth feature map is extracted and the root edge feature is enhanced, the coordinates of the candidate regions of interest (ROI) of potential prediction objects are generated by the region of interest prediction module, and all ROIs are transformed into the same output size by the region of interest alignment module.

[0099] The execution of the region interest alignment module includes the following steps:

[0100] Step 3.1: Since the resolution of the feature map is lower than that of the original image, the coordinates of the ROI need to be scaled to the feature map. The input image size is H × W, and the corresponding feature map size is H. f ×W f ;

[0101]

[0102] Among them, (x, y) is the ROI coordinate in the original image, (x f ,y f ) are the corresponding coordinates in the feature map;

[0103] Step 3.2: For each ROI, perform a fixed number of grid sampling within its area; for the output H f ×W f The feature map of size will be divided into H f ×W f small grids, each of which needs to sample a value from the feature map;

[0104] Step 3.3: Since the ROI coordinates are not integers, bilinear interpolation is used to calculate the value of the target position (x, y) accurately through the weighted calculation of the four adjacent points:

[0105]

[0106] Among them, Δx and Δy are the distances between the floating point coordinates and their integer points. This formula calculates the distance between the target position (x, y) relative to the other three points (x 1 ,y 1 ),(x 2 ,y 1 ),(x 1 ,y 2 ) and (x 2 ,y 2 )’s position change;

[0107] Step 3.4: Aggregate sampling results, for each ROI H f ×W f For each sampling grid in the grid, the value obtained using bilinear interpolation will be used as the feature value of that position, and these features are concatenated into a unified feature map of size K*K.

[0108] Step 4: The potential object candidate region is segmented by a mask network and combined with a root edge weighted cross entropy loss function to generate a pixel-level clear mask of the tooth root, and the segmentation results of the training model are evaluated, especially the root loss index MR;

[0109] Specifically, Figure 4 As shown in FIG. 1 , the potential object candidate region obtained in step 3 is processed by the mask segmentation network U-KAN through the encoder and decoder to obtain the predicted mask of the potential object candidate region. The weighted cross entropy loss and shape inconsistency penalty term are calculated together with the real mask. The root edge loss function is obtained by combining the two, and then W is used to calculate the root edge loss function. q The loop operation finally generates a pixel-level mask about the teeth;

[0110] The weighted cross entropy loss is defined as:

[0111]

[0112] Among them, W q is the adaptive weight based on the mismatch degree of the tooth root edge. If the pixel q belongs to the tooth root area, then W q =W root ,otherwise:

[0113]

[0114] Among them, β(q,σ(q)) is used to measure the shape inconsistency between the tooth root pixel q and the predicted σ(q), and λ is a hyperparameter that controls the penalty intensity;

[0115] The shape inconsistency penalty term is defined as a nonlinear combination of shape inconsistency and overall loss, mainly considering increasing the penalty for high error areas in the form of a power function:

[0116]

[0117] Where k is a parameter that controls the intensity of the penalty. When k>1, larger shape inconsistencies will receive larger penalties. β(o,σ(o)) is used to measure the shape inconsistency between the tooth pixel o and the predicted σ(o).

[0118] The root edge loss function (REloss) is defined as a weighted cross entropy loss function, where the pixel positions of the real object and the predicted object in the root edge area are given a higher weight W root (Usually W root >1) to enhance the model’s attention to the root edge pixels. The formula is as follows:

[0119]

[0120] Among them, α is the relative weight of the weighted cross entropy and shape penalty terms, and γ is a hyperparameter used to control the degree of nonlinearity, which is used to significantly increase the overall penalty when the loss value is large;

[0121] At the same time, in order to further improve the accuracy of segmentation, RELoss adopts an adaptive weight mechanism, that is, the weight of the root area is no longer fixed, but dynamically adjusted according to the shape inconsistency β(q,σ(q)), so as to better capture the mismatched area; in addition, RELoss uses a power function to control the overall loss through nonlinear combination, imposes a greater penalty on high error areas, so that the model can more effectively reduce the segmentation errors in the root area; to deal with the problem of shape inconsistency, RELoss also introduces a shape penalty significant enhancement mechanism, through the power function of β(o,σ(o)), to further strengthen the model's learning of shape inconsistent areas.

[0122] After collecting the pixel-level masks, instance segmentation labels and bounding boxes of the teeth, the tooth segmentation image based on the weighted loss of the root edge is output to evaluate the tooth segmentation effect. In particular, the root loss index MR is introduced for evaluation. At the same time, the MIoU metric, Dice coefficient, NSD and other indicators are introduced for auxiliary evaluation. If the evaluation effect is excellent, the tooth segmentation image is output. Otherwise, the tooth mask segmentation is re-performed.

[0123] The root loss index MR is used to calculate the ratio of the root loss part in the statistical prediction mask image to the number of roots in the correct mask image. The formula is as follows:

[0124]

[0125] Among them, S is the number of root canals missing from the predicted mask, and C is the number of root canals with the real mask. The larger the MR ratio, the lower the accuracy of the model for root segmentation.

[0126] The Dice coefficient indicator, the formula is as follows:

[0127]

[0128] Among them, X is the predicted result, and Y is the true result. When the Dice coefficient is larger, the predicted correct result is closer to the true result, and the model has higher accuracy in tooth segmentation.

[0129] The MioU metric is as follows:

[0130]

[0131] Among them, i represents the true value, j represents the predicted value, and p ij It indicates the value when i is predicted as j. When the MIoU value is larger, the predicted correct result is closer to the true result, and the model has higher accuracy in tooth segmentation.

[0132] The NSD indicator has the following formula:

[0133]

[0134] Where N represents the total number of data points, f i represents the value of function f at the i-th data point, g i Represents the value of the function g at the i-th data point.

[0135] Step 5: The coordinates of the potential object candidate region obtained in step 3 are passed through the fully connected layer to obtain the instance segmentation label, bounding box and other information about each tooth; the clear mask, instance segmentation label and bounding box of the tooth root are collected, and finally the instance segmentation result of the tooth is output.

[0136] The tooth image to be segmented is input into the trained tooth instance segmentation model, and the segmentation mask is output.

[0137] The coordinates of the potential object candidate region obtained in step 2 are passed through the fully connected layer to obtain the instance segmentation label, bounding box and other information about each tooth; the clear mask, instance segmentation label and bounding box of the tooth root are collected to finally output the instance segmentation result of the tooth.

[0138] To illustrate the effectiveness of this method, Figure 5 As shown in the figure, the datasets used in the experiment are from the public datasets of the MICCAI2024 and MICCAI2023 competitions. The labeled datasets provided by the competition total 200 and 40 images, and after data enhancement, a total of 1000 and 200 images are formed. The dataset will be divided into training set, validation set and test set. The training set is used to optimize the parameters of the neural network, and the validation set is used to evaluate the applicability of the model to data outside the training set and determine the number of iterations required. After the model training is completed, the test set will be used to evaluate the quality indicators. The segmentation of the dataset is based on the typical value range commonly used in the task. Specifically, the images will be divided into training set, validation set and test set at a ratio of 70%, 10% and 20%; the size of the input dataset is 512×512. The test results are compared with the mainstream tooth segmentation algorithm in the prior art. The proposed model framework is implemented in PyTorch and trained using an NVIDIA GeForce RTX 4060ti GPU with 8GB memory. The Adam optimization algorithm is used to optimize the overall parameters, and the learning rate is set to 0.01e-4. The test results of the public data sets of the MICCAI2024 and MICCAI2023 competitions are shown in Table 1:

[0139] Table 1 Test results of tooth segmentation of the present invention and other five methods

[0140] frame ↑Dice_Instance ↑Dice_image ↑NSD_instance ↓NSD_image ↑MIoU_instance ↑MIoU_image ↓MR MaskRCNN 0.650 0.786 0.220 0.838 0.159 0.654 0.125 PANet 0.620 0.703 0.214 0.738 0.167 0.560 0.0625 SOLOv2 0.640 0.718 0.216 0.746 0.189 0.565 0.1875 Unet 0.600 0.450 0.089 0.484 0.057 0.292 0.25 SegFormer 0.630 0.720 0.203 0.749 0.177 0.566 0.09375 DAE-Net (this invention) 0.700 0.741 0.274 0.37 0.244 0.595 0.03125

[0141] Among them, the present invention is the best value.

[0142] All of the 7 evaluation indicators are commonly used in tooth instance segmentation tasks. The upward arrow indicates that the larger the value, the more accurate it is, and vice versa.

[0143] Figure 6 This is a visualization comparison diagram of the experimental results of the method in the present invention and other methods. From the results, it can be concluded that the tooth instance segmentation method proposed in the present invention can obtain a more complete and accurate segmentation result of the tooth root boundary.

[0144] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tooth image segmentation method based on root edge weighted loss, characterized in that The steps include: Step 1: Obtain oral dental X-ray images; Step 2: extracting features from the oral dental X-ray image through the backbone network Resnet embedded with the refined feature enhancement module to obtain a tooth feature map, and then extracting features from the tooth root edge feature enhancement module to obtain a tooth feature map after root feature enhancement; The refined feature enhancement module compresses the input valid features through an adaptive average pooling operation, aggregates the features to obtain an aggregated information feature map, compresses the dimensions through a linear layer, passes through an activation function, and activates specific channel features through a channel weight allocation mechanism. Finally, the activation of each channel is controlled, and the input valid features are multiplied by the corresponding channel weight to obtain the enhanced features. The execution process of the tooth root edge feature enhancement module includes the following steps: Step 2.1: Obtain the tooth feature map and the root area mask to obtain the tensor F RF With R m , the tensor F RF With R m After element-by-element multiplication, we get the merged tensor R f ; Step 2.2: For the tensor F RF Channel compression is performed, nonlinearity is introduced through the RELU activation function, and convolution is introduced to capture spatial information, especially the characteristics of the root area. After the RELU activation function, the number of channels is restored to the original number of input channels; then the Sigmoid activation function is used to generate feature weights for each channel of the input feature map, and finally the feature weights F se With the original input tensor F RF Perform element-by-element multiplication to obtain feature F out ; Step 2.3: Transform feature F out And the combined tensor R f The fused feature F is obtained through the adder out1 , for the merged feature F out1 Perform convolution processing to obtain feature F out2 , and then the merged feature F out1 With feature F out2 The feature F is obtained by multiplication out3 , the feature F out3 The feature F is obtained by the inverter out4 , transform the feature in space and transform the feature F out3 With feature F out4 The feature F after the root edge enhancement is obtained by the adder out5 ; Step 3: The tooth feature map with feature extraction and root edge feature enhancement is used to generate a candidate region of interest for potential prediction objects through a region of interest prediction module; Step 4: After the candidate area is passed through the mask segmentation network, combined with the root edge loss function, a segmentation mask result of the tooth root is generated; Evaluate the segmentation effect. If it meets the requirements, output the segmentation mask result. Otherwise, perform mask segmentation again. Step 5: Pass the candidate region through the fully connected layer to obtain instance segmentation labels and bounding box information for each tooth; Based on the segmentation mask result of the tooth root, the instance segmentation label and bounding box information of each tooth, obtaining the instance segmentation result of the tooth; The tooth image to be segmented is input into the trained tooth image instance segmentation process, and finally the tooth instance segmentation result is output.

2. The tooth image segmentation method based on root edge weighted loss according to claim 1, characterized in that: The calculation formula for executing the refined feature enhancement module is as follows: F c =linear1(R s1 (Avg(F))) F′ c =linear2(relu(F c )) Among them, F represents the effective features of the input, Avg() represents the average pooling operation on the input feature vector, and R s1 () represents adaptive average pooling, Linear1() represents the first linear transformation layer, F c represents the feature map of F after adaptive average pooling and linear layer processing, relu represents the ReLU activation function, Linear2() represents the second linear transformation layer, and F′ c Indicates F c The feature map after activation and linear layer processing, P CBR-2 Indicates zero padding processing, Conv1 indicates a convolution operation with a convolution kernel size of 1, Sigmoid indicates the Sigmoid activation function, F′ c After zero padding, convolution and activation operations, the feature map F RF Represents the enhanced features.

3. The tooth image segmentation method based on root edge weighted loss according to claim 1, characterized in that: In step 3, all regions of interest are transformed into the same output size by the region of interest alignment module, including the following steps: Step 3.1: Scale the coordinates of the region of interest to the feature map; Step 3.2: For each region of interest, a fixed number of grid samples are performed within the region; Step 3.3: Use bilinear interpolation to calculate the value of the region of interest in the original image through weighted calculation of multiple adjacent points; Step 3.4: Aggregate the sampling results. For each sampling grid in each region of interest grid, use the value as the feature value of the position to obtain a region of interest of the same size.

4. The tooth image segmentation method based on root edge weighted loss according to claim 3, characterized in that: In step 3.1, the input image size is H × W, and the corresponding feature map size is H f ×W f ; Among them, (x, y) represents the coordinates of the region of interest in the original image, (x f ,y f ) represents the corresponding coordinates in the feature map; In step 3.2, for the output H f ×W f The feature map of size will be divided into H f ×W f small grids, each of which needs to sample a value from the feature map; In step 3.4, for each region of interest, f ×W f For each sampling grid in the grid, the value obtained using bilinear interpolation is used as the feature value of that position, and these features are spliced ​​into a unified feature map.

5. The tooth image segmentation method based on root edge weighted loss according to claim 3, characterized in that: In step 3.3, bilinear interpolation is used to calculate the value of the target position (x, y) accurately through weighted calculation of four adjacent points: V(x,y)=(1-Δx)(1-Δy)×V(x1,y1)+Δx(1-Δy)×V(x2,y1)+(1-Δx)Δy×V(x1,y2)+ΔxΔy×V(x2,y2) Among them, V represents the value on the feature map obtained by bilinear interpolation, (x, y) represents the coordinates of the region of interest in the original image, Δx and Δy are the distances between the floating-point coordinates and their integer points. This formula calculates the position change of the target position (x, y) relative to the other three points (x1, y1), (x2, y1), (x1, y2) and (x2, y2).

6. The tooth image segmentation method based on root edge weighted loss according to claim 1, characterized in that: In step 4, the candidate area is processed by the mask segmentation network, the encoder and the decoder to obtain the predicted mask of the candidate area, the predicted mask and the real mask are jointly calculated with the weighted cross entropy loss and the shape inconsistency penalty term, and the two are combined to obtain the root edge loss function, and the weight W is adaptively changed based on the degree of root edge mismatch. q After a loop operation, a mask about the teeth is finally generated.

7. The tooth image segmentation method based on root edge weighted loss according to claim 6, characterized in that: The weighted cross entropy loss is defined as: Among them, p q represents the probability that pixel q in the true mask belongs to the tooth root area, represents the predicted probability that pixel q belongs to the tooth root area, root represents the tooth root pixel set, W q represents the adaptive change weight based on the mismatch degree of the tooth root edge. If the pixel q belongs to the tooth root area, a high weight is assigned to the adaptive change weight W. q =W root ,otherwise: w q =1+λ·β(q,σ(q)) Among them, β(q,σ(q)) represents the shape inconsistency between the pixel q used to measure the tooth root and the predicted tooth root pixel σ(q), and λ represents the hyperparameter controlling the penalty intensity; The shape inconsistency penalty term is defined as a nonlinear combination of shape inconsistency and overall loss, increasing the penalty for high error areas in the form of a power function: Where G represents the tooth pixel set, k represents the parameter controlling the penalty strength, when k>1, large shape inconsistency will receive a larger penalty, β(o,σ(o)) represents the shape inconsistency degree of the tooth pixel o and the predicted tooth pixel σ(o) used to measure the tooth; The root edge loss function is defined as a weighted cross entropy loss function, and the pixel positions of the real object and the predicted object in the root edge area are assigned a high weight W root , the formula is as follows: RootEdgeLossI=(ShapePenaltyI+α·CE root ) γ Among them, α represents the relative weight of the weighted cross entropy and shape penalty terms, and γ represents the hyperparameter that controls the degree of nonlinearity.

8. The tooth image segmentation method based on root edge weighted loss according to claim 1, characterized in that: In step 4, the tooth root loss index MR is introduced to evaluate the segmentation effect, and the MIoU metric, Dice coefficient, and NSD index are introduced for auxiliary evaluation. If the evaluation effect meets the requirements, the segmented image is output, otherwise, mask segmentation is performed again; The tooth root loss index MR is used to calculate the ratio of the tooth root loss portion in the statistical prediction mask image to the number of tooth roots in the correct mask image, and the formula is as follows: Among them, S represents the number of root canals missing from the predicted mask, and C represents the number of root canals in the real mask; the larger the MR ratio, the lower the accuracy of root segmentation; The Dice coefficient indicator is as follows: Among them, X represents the predicted result, and Y represents the true result. When the Dice coefficient is larger, the predicted correct result is closer to the true result, indicating that the accuracy of tooth segmentation is higher. The MioU metric is as follows: Among them, k represents the number of categories, the number of backgrounds and targets in the image, i represents the true value, j represents the predicted value, and p ij It indicates the value when i is predicted as j. When the MIoU value is larger, the predicted correct result is closer to the true result, which means the accuracy of tooth segmentation is higher. The NSD indicator has the following formula: Where N represents the total number of data points, f i represents the value of function f at the i-th data point, g i Represents the value of the function g at the i-th data point.

Images