Dental image segmentation method and device based on semi-supervised learning and nnU-Net

Abstract

The invention provides a tooth image segmentation method and device based on semi-supervised learning and nnU-Net, and the method comprises the steps: obtaining a to-be-segmented tooth image, carrying out the preprocessing of the to-be-segmented tooth image, inputting the preprocessed image into a tooth image segmentation model, and obtaining a tooth segmentation result, the tooth image segmentation model is obtained by training a pre-constructed image segmentation model according to a semi-supervised learning mechanism, and the pre-constructed image segmentation model is constructed based on an nnU-Net structure and an attention mechanism; according to the method, the dependence of the pre-constructed image segmentation model on the labeled data in the training process is reduced, that is, training of the pre-constructed image segmentation model can be completed through part of the labeled data and part of the unlabeled data, so that the data labeling cost is reduced; and the tooth image segmentation effect under the conditions of complex background, structure overlapping, unclear boundary and the like is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tooth segmentation, and particularly relates to a tooth image segmentation method and device based on semi-supervised learning and nnU-Net. BACKGROUND

[0002] Tooth image segmentation is a key link in oral medical diagnosis, treatment planning and effect evaluation. Accurate tooth image segmentation can help doctors more accurately identify tooth structure, lesion location and range, so as to develop more personalized treatment plans. However, in the related art, the tooth image segmentation method (for example, threshold segmentation, edge detection and region growing), when processing complex background, structure overlap and unclear boundary medical images, the segmentation effect is not ideal. With the development of image segmentation technology, the tooth image segmentation method in the related art is realized by model training, and model training often needs a large amount of labeled data, and because medical image labeling is costly and time-consuming, it limits its application in actual clinical practice.

[0003] Therefore, it is necessary to improve the tooth image segmentation algorithm in the related art to improve the accuracy of tooth image segmentation and reduce the cost of tooth image segmentation. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the present application provides a tooth image segmentation method and device based on semi-supervised learning and nnU-Net to solve the above-mentioned technical problems.

[0005] According to an aspect of an embodiment of the present application, a tooth image segmentation method based on semi-supervised learning and nnU-Net is provided, the method comprising: acquiring a tooth image to be segmented, and pre-processing the tooth image to be segmented to obtain a pre-processed image; the pre-processing method comprises: image size conversion; inputting the pre-processed image into a tooth image segmentation model to obtain a tooth segmentation result; the tooth image segmentation model is obtained by training a pre-constructed image segmentation model according to a semi-supervised learning mechanism; the pre-constructed image segmentation model is obtained by constructing based on an nnU-Net structure and an attention mechanism.

[0006] In an embodiment of the present application, the process of training the pre-constructed image segmentation model according to a semi-supervised learning mechanism to obtain the tooth image segmentation model includes: obtaining a sample image set for training the pre-constructed image segmentation model; the sample image set includes a labeled image set and an unlabeled image set; training the pre-constructed image segmentation model through the labeled image set to obtain a teacher model; dividing the sample image set into K training set and validation set combinations according to a K-fold cross-validation manner; and training the pre-constructed image segmentation model through the training set and the validation set in the K combinations to obtain a plurality of student models; performing an enhancement operation on the unlabeled images in the unlabeled image set, and inputting the unlabeled images after the enhancement operation into the teacher model to obtain a plurality of first pseudo-labeled images; the enhancement operation includes a vertical flip operation and a histogram equalization; inputting the plurality of first pseudo-labeled images into each student model to obtain a plurality of second pseudo-labeled images; and performing a fusion operation on the plurality of second pseudo-labeled images to obtain a third pseudo-labeled image; the fusion operation includes a pixel average operation, a binaryzation operation and a morphological operation; combining the plurality of first pseudo-labeled images, the third pseudo-labeled image and the labeled images in the labeled image set to obtain an updated image set; and training the pre-constructed image segmentation model through the updated image set to obtain the tooth image segmentation model.

[0007] In an embodiment of the present application, before obtaining the sample image set for training the pre-constructed image segmentation model, the method further includes: selecting a part of images in the sample image set as unlabeled images; and taking the remaining part of images in the sample image set as to-be-labeled images; the set composed of the unlabeled images is taken as the unlabeled image set; converting the to-be-labeled images into a preset bit single-channel grayscale image; the pixel value of each pixel point in the preset bit single-channel grayscale image is set to a preset pixel value; obtaining the contour point information and the classification category of each tooth in the to-be-labeled image; determining the pixel area of each tooth in the preset bit single-channel grayscale image according to the contour point information of each tooth in the to-be-labeled image; filling the pixel area of each tooth in the preset bit single-channel grayscale image as the corresponding classification label according to the corresponding relationship between the classification category and the classification label to obtain a labeled image, and taking the set composed of the labeled image as the labeled image set; the classification label is represented in the form of a numerical value.

[0008] In one embodiment of this application, after inputting the enhanced unlabeled image into the teacher model to obtain multiple first pseudo-labeled images, the method further includes: performing a binarization operation on the multiple first pseudo-labeled images to obtain multiple binarized images; determining the connected regions of each binarized image based on the multiple binarized images, and determining the contour of each tooth in each binarized image according to the connected regions of each binarized image; the number of contour pixels of each tooth is greater than or equal to a preset number of pixels; each tooth contour has a unique number; converting the contour of each tooth in each binarized image into a tooth category number according to the correspondence between the category number and the contour number of each tooth in each binarized image, to obtain multiple fourth pseudo-labeled images, and updating the multiple first pseudo-labeled images with the multiple fourth pseudo-labeled images, so as to input the updated multiple first pseudo-labeled images into each student model.

[0009] In one embodiment of this application, the process of training the pre-built image segmentation model using the updated image set to obtain the tooth image segmentation model includes: inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images; updating the parameters in the pre-built image segmentation model with the goal of minimizing the error between the sample images and the predicted images, until the error between the predicted images output by the updated image segmentation model and the sample images is less than a preset error threshold, or the number of times the parameters in the pre-built image segmentation model are updated is greater than or equal to a preset number threshold.

[0010] In one embodiment of this application, the error between the sample image and the predicted image is characterized by a loss function, the expression of which is as follows: ,in, This represents the error between the sample image and the predicted image. Indicates the number of sample images. Indicates the first One sample image, Indicates the first The predicted image of the nth sample image, the nth The sample image and the first The predicted image for each sample image is represented in the form of a binary vector. Indicates the first Labels for each sample image. Indicates the first The predicted probability of a sample image. This represents the loss weight of the Dice similarity term coefficient. The weights represent the binary cross-entropy loss. .

[0011] In an embodiment of the present application, if the pre-constructed image segmentation model comprises an input layer, a plurality of feature extraction branches, a feature fusion layer and an output layer, the process of inputting the sample image in the updated image set into the pre-constructed image segmentation model to obtain the predicted image of the sample image comprises: The input layer is configured to receive the sample image and convert the sample image into a preset size image; the plurality of feature extraction branches are configured to extract different dimension features in the preset size image to obtain different dimension feature maps; the feature fusion layer is configured to fuse the different dimension feature maps to obtain a fused feature map; and the output layer is configured to convert the fused feature map into a preset dimension feature vector to obtain the predicted image.

[0012] In an embodiment of the present application, the plurality of feature extraction branches include: a first feature extraction branch, a second feature extraction branch, a third feature extraction branch, a fourth feature extraction branch, a fifth feature extraction branch, and a sixth feature extraction branch; the first feature extraction branch is configured to extract first preset dimension features in the preset size image, and compress the first preset dimension features to obtain a first preset dimension feature map; the second feature extraction branch is configured to extract spatial features and channel features from the first preset dimension features to obtain a second preset dimension feature map; the second preset dimension is equal to the first preset dimension; the third feature extraction branch is configured to extract third preset dimension features from the first preset dimension features, and extract spatial features and channel features from the third preset dimension features to obtain a first spatial-channel feature; the first spatial-channel feature is subjected to a fusion operation, an up-sampling operation, and a compression operation to obtain a third preset dimension feature map; the third preset dimension is greater than the second preset dimension; the fourth feature extraction branch is configured to extract fourth preset dimension features from the third preset dimension features, and extract spatial features and channel features from the fourth preset dimension features to obtain a second spatial-channel feature map; the second spatial-channel feature map is subjected to a fusion operation, an up-sampling operation, and a compression operation to obtain a fourth preset dimension feature map; the fourth preset dimension is greater than the third preset dimension; the fifth feature extraction branch is configured to extract fifth preset dimension features from the fourth preset dimension features, and subject the fifth preset dimension features to a fusion operation, an up-sampling operation, and a compression operation to obtain a fifth preset dimension feature map; the fifth preset dimension is greater than the fourth preset dimension; the sixth feature extraction branch is configured to extract sixth preset dimension features from the fifth preset dimension feature map, and subject the sixth preset dimension features to an up-sampling operation and a compression operation to obtain a sixth preset dimension feature map; the sixth preset dimension is greater than the fifth preset dimension; the first preset dimension feature map, the second preset dimension feature map, the third preset dimension feature map, the fourth preset dimension feature map, the fifth preset dimension feature map, and the sixth preset dimension feature map are combined as different dimension feature maps.

[0013] In an embodiment of the present application, the first feature extraction branch includes: the first feature extraction branch includes: a first feature extraction module and a first RFEM module; the first feature extraction module is configured to extract first preset dimension features in the preset size image; the first RFEM module is configured to compress the first preset dimension features to obtain the first preset dimension feature map; the second feature extraction branch includes: a first CBMA module; the first CBMA module is configured to extract spatial features and channel features of the first preset dimension features to obtain the second preset dimension feature map; the third feature extraction branch includes: a second feature extraction module, a second CBMA module, a first feature fusion module, a first transposed convolution module, a second RFEM module and a second transposed convolution module; the second feature extraction module is configured to extract third preset dimension features from the first preset dimension features; the second CBMA module is configured to extract spatial features and channel features of the third preset dimension features to obtain the first spatial-channel feature; the first feature fusion module is configured to perform fusion operation on the first spatial-channel feature and the fourth preset dimension feature map to obtain first fusion features; the first transposed convolution module is configured to perform up-sampling operation on the first fusion features to obtain first up-sampling features; the second RFEM module is configured to compress the first up-sampling features to obtain first compressed features; the second transposed convolution module is configured to up-sample the first compressed features to obtain the third preset dimension feature map; the feature fusion layer is configured to fuse the first preset dimension feature map, the second preset dimension feature map and the third preset dimension feature map; the fourth feature extraction branch includes: the third feature extraction module, the third CBMA module, the second feature fusion module, the third transposed convolution module, the third RFEM module and the fourth transposed convolution module; the third feature extraction module is configured to extract fourth preset dimension features from the third preset dimension features; the third CBMA module is configured to extract spatial features and channel features of the fourth preset dimension features to obtain the second spatial-channel feature map; the second feature fusion module is configured to perform fusion operation on the second spatial-channel feature map and the fifth preset dimension feature map to obtain the second fusion feature map; the third transposed convolution module is configured to up-sample the second fusion features to obtain the second up-sampling feature map; the third RFEM module is configured to compress the second up-sampling features to obtain the second compressed feature map; the fourth transposed convolution module is configured to up-sample the second compressed features to obtain the fourth preset dimension feature map; the fifth feature extraction branch includes: the fourth feature extraction module, the third feature fusion module, the fifth transposed convolution module, the fourth RFEM module and the sixth transposed convolution module;The fourth feature extraction module is configured to extract a fifth preset dimension feature from the fourth preset dimension feature as a pooling layer feature map; the third feature fusion module is configured to perform a fusion operation on the pooling layer feature map and the sixth preset dimension feature map to obtain a third fusion feature map; the fifth transpose convolution module is configured to perform up-sampling on the third fusion feature map to obtain a third up-sampled feature map; the fourth RFEM module is configured to compress the third up-sampled feature map to obtain a third compressed feature map; the sixth transpose convolution module is configured to perform up-sampling on the third compressed feature map to obtain the fifth preset dimension feature map; the sixth feature extraction branch includes a convolution module, a seventh transpose convolution module, a fifth RFEM module, and an eighth transpose convolution module, the convolution module is configured to extract a sixth preset dimension feature from the fifth preset dimension feature map; the seventh transpose convolution module is configured to perform up-sampling on the sixth preset dimension feature to obtain a fourth up-sampled feature map; the fifth RFEM module is configured to compress the fourth up-sampled feature map to obtain a fourth compressed feature map; and the eighth transpose convolution module is configured to perform up-sampling on the fourth compressed feature map to obtain the sixth preset dimension feature map.

[0014] According to an aspect of the embodiment of the present application, a tooth image segmentation device based on semi-supervised learning and nnU-Net is provided, which comprises: an image acquisition module configured to acquire a tooth image to be segmented, and perform preprocessing on the tooth image to be segmented to obtain a preprocessed image; the preprocessing manner comprises: image size conversion; an image segmentation module configured to input the preprocessed image into a tooth image segmentation model to obtain a tooth segmentation result; the tooth image segmentation model is obtained by training a pre-constructed image segmentation model according to a semi-supervised learning mechanism; and the pre-constructed image segmentation model is obtained by constructing based on an nnU-Net structure and an attention mechanism.

[0015] The present application has the following advantages: the present application acquires a tooth image to be segmented, pre-processes the tooth image to be segmented, inputs the pre-processed image into a tooth image segmentation model, and obtains a tooth segmentation result; the tooth image segmentation model is obtained by training a pre-constructed image segmentation model according to a semi-supervised learning mechanism; the pre-constructed image segmentation model is obtained by constructing based on an nnU-Net structure and an attention mechanism; through the above process, the pre-constructed image segmentation model is constructed based on the nnU-Net structure and the attention mechanism, and the pre-constructed image segmentation model is trained according to the semi-supervised learning mechanism, which reduces the dependence of the pre-constructed image segmentation model on labeled data in the training process, i.e., the training of the pre-constructed image segmentation model can be completed by using part of the labeled data and part of the unlabeled data, thereby reducing the data labeling cost and improving the tooth image segmentation effect in complex backgrounds, structure overlaps, and unclear boundaries.

[0016] It should be understood that the general description above and the detailed description below are only exemplary and explanatory, and are not restrictive of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0017] The drawings incorporated in and forming a part of the specification, illustrate preferred embodiments of the present application and, together with the description, serve to explain the principles of the present application. It is pointed out that the drawings are only meant to illustrate some embodiments of the present application, and that other drawings can be derived from these drawings by those of ordinary skill in the art without incurring inventive effort. In the drawings: Figure 1 is a schematic diagram of an exemplary system architecture shown in accordance with an exemplary embodiment of the present application; Figure 2 is a flowchart of a dental image segmentation method based on semi-supervised learning and nnU-Net shown in accordance with an exemplary embodiment of the present application; Figure 3 is a flowchart of a dental image segmentation method based on semi-supervised learning and nnU-Net shown in accordance with another exemplary embodiment of the present application; Figure 4 is a structural schematic diagram of a pre-constructed image segmentation model shown in accordance with an exemplary embodiment of the present application; Figure 5 is a variation curve diagram of a loss function value in a training process of the pre-constructed image segmentation model shown in accordance with an exemplary embodiment of the present application; Figure 6 is a variation curve diagram of a Dice similarity coefficient in the training process of the pre-constructed image segmentation model shown in accordance with an exemplary embodiment of the present application; Figure 7 is an effect diagram of dental image segmentation shown in accordance with an exemplary embodiment of the present application; Figure 8 is an effect diagram of dental image segmentation shown in accordance with another exemplary embodiment of the present application; Figure 9 is a block diagram of a dental image segmentation device based on semi-supervised learning and nnU-Net shown in accordance with an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0018] The present application can be implemented or applied by other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0019] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner, and the drawings only show the components related to the present application, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change in type, number and proportion, and the component layout pattern may be more complex.

[0020] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, well-known structures and devices are shown in block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0021] Figure 1 is a schematic diagram of an exemplary system architecture shown by an exemplary embodiment of the present application.

[0022] Referring to Figure 1 As shown in the figure, the system architecture can include a collection device 101 and a data processing device 102. The data processing device 102 can be at least one of a desktop graphic processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. The relevant technical personnel can use the data processing device 102 to obtain a tooth image to be segmented, pre-process the tooth image to be segmented, input the pre-processed image into a tooth image segmentation model, and obtain a tooth segmentation result. The tooth image segmentation model is obtained by training a pre-constructed image segmentation model according to a semi-supervised learning mechanism, and the pre-constructed image segmentation model is obtained by constructing based on an nnU-Net structure and an attention mechanism. The collection device 101 is used for X-ray imaging of the tooth as a tooth image to be segmented and provided to the data processing device 102 for processing.

[0023] Illustratively, after obtaining the tooth image to be segmented in the acquisition device 101, the data processing device 102 inputs the preprocessed image into the tooth image segmentation model to obtain the tooth segmentation result. The tooth image segmentation model is trained according to a semi-supervised learning mechanism to obtain a pre-constructed image segmentation model, which is constructed based on an nnU-Net structure and an attention mechanism. Through the construction of the pre-constructed image segmentation model based on the nnU-Net structure and the attention mechanism and the training of the pre-constructed image segmentation model according to the semi-supervised learning mechanism, the dependence of the pre-constructed image segmentation model on labeled data in the training process is reduced, that is, the training of the pre-constructed image segmentation model can be completed by using part of the labeled data and part of the unlabeled data, thereby reducing the data labeling cost and improving the tooth image segmentation effect in complex backgrounds, structure overlap, and unclear boundaries.

[0024] The implementation details of the technical solutions of the embodiments of the present application are described in detail as follows: Figure 2 is a flowchart of a tooth image segmentation method based on semi-supervised learning and nnU-Net according to an example embodiment of the present application. Referring to Figure 2 , the tooth image segmentation method based on semi-supervised learning and nnU-Net includes at least steps S210 to S220, which are described in detail as follows: In step S210, a tooth image to be segmented is obtained, and the tooth image to be segmented is preprocessed to obtain a preprocessed image. In an embodiment of the present application, the tooth image to be segmented is an X-ray image. The preprocessing methods include image size conversion, image resolution adjustment, etc. The image size conversion, image resolution adjustment, etc. preprocessing methods improve the quality of the tooth image to be segmented and adapt to the size requirements of the tooth image segmentation model for the input image.

[0025] In step S220, the preprocessed image is input into the tooth image segmentation model to obtain a tooth segmentation result. In an embodiment of the present application, the tooth image segmentation model is trained according to a semi-supervised learning mechanism to obtain a pre-constructed image segmentation model, which is constructed based on an nnU-Net structure and an attention mechanism. The dependence of the pre-constructed image segmentation model on labeled data in the training process is reduced, that is, the training of the pre-constructed image segmentation model can be completed by using part of the labeled data and part of the unlabeled data, thereby reducing the data labeling cost and improving the tooth image segmentation effect in complex backgrounds, structure overlap, and unclear boundaries.

[0026] In an embodiment of the present application, the process of training the pre-constructed image segmentation model according to the semi-supervised learning mechanism to obtain the tooth image segmentation model includes: A sample image set for training the pre-constructed image segmentation model is obtained. In an embodiment of the present application, the sample image set includes a labeled image set and an unlabeled image set, for example, in the sample image set, the proportion of labeled images in the labeled image set is set to 10% of the number of sample images, and the proportion of unlabeled images in the unlabeled image set is set to 90% of the number of sample images. Before training the pre-constructed image segmentation model, the sample images in the sample image set need to be adjusted to a uniform resolution (for example, a resolution of 224x224), and the coordinate information is adjusted in proportion to ensure that the sample images input into the pre-constructed image segmentation model have consistent size and coordinate standards, thereby avoiding model training problems caused by inconsistent sample image sizes.

[0027] In an embodiment of the present application, in order to reduce the computational burden and improve the training efficiency, all sample images are uniformly scaled to a size of 320x640, which helps to reduce the consumption of computing resources, optimize memory usage, reduce GPU burden, and thus speed up the training process, which helps to improve the training efficiency and segmentation accuracy of the pre-constructed image segmentation model.

[0028] The pre-constructed image segmentation model is trained by the labeled image set to obtain a teacher model; the sample image set is divided into K training set and validation set combinations in a K-fold cross-validation manner; and the pre-constructed image segmentation model is trained by the training set and the validation set in the K combinations to obtain a plurality of student models. In an embodiment of the present application, the pre-constructed image segmentation model is trained by the training set and the validation set in the K combinations, so that the student model approaches the teacher model, improving the accuracy and precision of the student model. At the same time, in order to ensure the accuracy of the student model, the student model is updated and optimized regularly to strengthen the student model's ability to capture key features in the tooth image and ensure the stability of the student model in segmenting the tooth image.

[0029] The unlabeled images in the unlabeled image set are subjected to enhancement operations, and the unlabeled images subjected to the enhancement operations are input into the teacher model to obtain a plurality of first pseudo-labeled images. In an embodiment of the present application, the enhancement operations include vertical flipping and histogram equalization, and the enhancement operations also include rotation, scaling, etc. By performing enhancement operations on the unlabeled images in the unlabeled image set, the quality of the unlabeled images in the unlabeled image set is improved.

[0030] The multiple first pseudo-label images are input into each student model to obtain multiple second pseudo-label images; and the multiple second pseudo-label images are fused to obtain a third pseudo-label image. In an embodiment of the present application, the fusion operation includes pixel averaging operation, binaryzation operation and morphological operation; the process of fusing the multiple second pseudo-label images includes: performing pixel averaging calculation on each bit of pixels in the multiple second pseudo-label images to obtain a fusion image with pseudo-labels; performing binaryzation operation on the fusion image with pseudo-labels to obtain a binaryzation fusion image; and performing morphological operation (for example, dilation, erosion, opening operation, closing operation, etc.) on the binaryzation fusion image to obtain the third pseudo-label image. The above operations are beneficial to improving the quality of the third pseudo-label image and realizing accurate segmentation of the pseudo-label image.

[0031] The multiple first pseudo-label images, the third pseudo-label image and the labeled images in the labeled image set are combined to obtain an updated image set; and the pre-constructed image segmentation model is trained through the updated image set to obtain a tooth image segmentation model. In an embodiment of the present application, based on only a small amount of labeled images, the inference of the enhanced unlabeled images by the teacher model obtains the multiple first pseudo-label images, the prediction of the multiple first pseudo-label images by the student model cross-validation obtains the multiple second pseudo-label images, and the fusion operation on the multiple second pseudo-label images obtains the third pseudo-label image, thereby realizing the conversion of the unlabeled images into the pseudo-label images, improving the quality of the pseudo-label images, and constantly improving the generalization ability of the pre-constructed image segmentation model in the process of training the pre-constructed image segmentation model through the updated image set.

[0032] In an embodiment of the present application, before obtaining the sample image set for training the pre-constructed image segmentation model, the tooth image segmentation method based on semi-supervised learning and nnU-Net further includes: A part of images in the sample image set are selected as unlabeled images; and the remaining part of images in the sample image set are selected as to-be-labeled images; and the set of the unlabeled images is used as an unlabeled image set. In an embodiment of the present application, the division ratio of the unlabeled images and the to-be-labeled images in the sample image set can be set according to actual conditions, for example, the proportion of the unlabeled images in the sample image set is 90%, and the proportion of the to-be-labeled images in the sample image set is 10%.

[0033] The to-be-labeled images are converted into a preset bit single-channel grayscale image. In an embodiment of the present application, the pixel value of each pixel point in the preset bit single-channel grayscale image is set to a preset pixel value; the preset bit single-channel grayscale image is an 8-bit single-channel grayscale image, and the preset pixel value is set to 0.

[0034] The process involves obtaining the contour point information and classification category of each tooth in the image to be labeled. Based on the contour point information of each tooth in the image to be labeled, the pixel region of each tooth in a preset single-channel grayscale image is determined. According to the correspondence between classification category and classification label, the pixel region of each tooth in the preset single-channel grayscale image is filled with the corresponding classification label to obtain a labeled image. The set of labeled images is then used as the labeled image set. In one embodiment of this application, the classification label is represented in numerical form, for example, 1, 2, 3. The contour point information and classification category of each tooth in the image to be labeled are extracted from JSON annotations. The JSON annotations store the contour point information and classification category of each tooth in the image to be labeled. The contour point information and classification category of each tooth in the image to be labeled are annotated using the VGG ImageAnnotator (VIA) image annotation tool and saved in JSON format in the JSON annotations. The tooth classification categories include: anterior teeth, posterior teeth, supernumerary teeth, caries, etc. Each classification category is assigned a classification label number, thus obtaining the correspondence between classification categories and classification labels.

[0035] In one embodiment of this application, after inputting the enhanced unlabeled image into the teacher model to obtain multiple first pseudo-labeled images, the tooth image segmentation method based on semi-supervised learning and nnU-Net further includes: Binarization is performed on multiple first pseudo-label images to obtain multiple binarized images. In one embodiment of this application, the process of binarizing multiple first pseudo-label images includes: setting 0.5 as a pixel threshold, treating pixels with a value greater than or equal to 0.5 in each first pseudo-label image as tooth pixels, and treating pixels with a value less than 0.5 in each first pseudo-label image as background pixels, thereby determining the tooth pixels and background pixels in each first pseudo-label image, and setting the pixel value of the tooth pixels to 0 and the pixel value of the background pixels to 1. The above operations help to distinguish the tooth region from the background region.

[0036] Based on multiple binarized images, the connected regions of each binarized image are determined, and the contour of each tooth in each binarized image is determined according to the connected regions of each binarized image. In one embodiment of this application, the number of pixels in the contour of each tooth is greater than or equal to a preset number of pixels; the preset number of pixels is set to 45, and the contour of each tooth has a unique number. The method for determining the connected regions of each binarized image based on multiple binarized images is either the four-connectivity method or the eight-connectivity method. The process of determining the contour of each tooth in each binarized image according to the connected regions of each binarized image is the process of extracting the contour of the connected regions of each binarized image. The process of extracting the contour of the connected regions of each binarized image can be implemented by image edge detection algorithms (e.g., Canny edge detection algorithm, adaptive threshold segmentation algorithm) and boundary tracking algorithms (e.g., Suzuki algorithm, Boundary Tracing).

[0037] Based on the correspondence between the category number and contour number of each tooth in each binarized image, the contour of each tooth in each binarized image is converted into a tooth category number, resulting in multiple fourth pseudo-label images. These fourth pseudo-label images are then used to update multiple first pseudo-label images, which are then input into each student model. In one embodiment of this application, there is a one-to-one correspondence between the tooth category and the tooth contour. After obtaining the contour of each tooth, the corresponding contour number and category number are determined based on the contour. Updating the multiple first pseudo-label images with multiple fourth pseudo-label images and then inputting the updated first pseudo-label images into each student model improves the quality of the pseudo-label images. Compared to multiple first pseudo-label images, multiple fourth pseudo-label images provide more accurate segmentation label information, which is beneficial for improving the segmentation accuracy and reliability of the pre-built image segmentation model during subsequent training.

[0038] In one embodiment of this application, the process of training a pre-built image segmentation model by updating the image set to obtain a tooth image segmentation model includes: The process of inputting sample images from the updated image set into a pre-built image segmentation model to obtain predicted images of the sample images includes the following steps: In one embodiment of this application, if the pre-built image segmentation model includes an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: the input layer receives the sample images and converts them into images of a preset size; the multiple feature extraction branches extract features of different dimensions from the preset size image to obtain feature maps of different dimensions; the feature fusion layer fuses the feature maps of different dimensions to obtain a fused feature map; and the output layer converts the fused feature map into a feature vector of a preset dimension to obtain the predicted image. This process fully utilizes feature information of different dimensions, enabling the pre-built image segmentation model to have stronger feature extraction capabilities when processing complex tooth images. By effectively fusing these features of different dimensions, a more representative and discriminative fused feature map is formed. By converting the fused feature map into a feature vector of a preset dimension, a predicted image is obtained. This predicted image can accurately reflect the segmentation of the teeth, providing a reliable basis for subsequent model training and optimization.

[0039] The goal is to minimize the error between the sample image and the predicted image. The parameters of a pre-built image segmentation model are updated until the error between the predicted image and the sample image output by the updated image segmentation model is less than a preset error threshold, or the number of updates to the parameters of the pre-built image segmentation model is greater than or equal to a preset number of updates threshold. In one embodiment of this application, both the preset error threshold and the preset number of updates threshold are set according to actual conditions. The error between the sample image and the predicted image is represented by a loss function, the expression of which is shown below: Equation (1) in, This represents the error between the sample image and the predicted image. Indicates the number of sample images. Indicates the first One sample image, Indicates the first The predicted image of the nth sample image, the nth The sample image and the first The predicted image for each sample image is represented in the form of a binary vector. Indicates the first Labels for each sample image. Indicates the first The predicted probability of a sample image. This represents the loss weight of the Dice similarity term coefficient. The weights represent the binary cross-entropy loss. .

[0040] In one embodiment of this application, a loss function for training a pre-built image segmentation model is constructed using Dice similarity coefficient loss and binary cross-entropy loss. This is beneficial for enhancing the robustness of training the pre-built image segmentation model and continuously improving the performance and accuracy of the pre-built image segmentation model in tooth segmentation scenarios.

[0041] In one embodiment of this application, a pre-built image segmentation model is trained using a labeled image set to obtain the loss function of the teacher model. The pre-built image segmentation model is trained using the training set and validation set in K combinations to obtain the loss functions of multiple student models. All of these loss functions can be measured using the loss function of formula (1) to measure the error between the sample data and the predicted data.

[0042] In one embodiment of this application, if the pre-built image segmentation model includes an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, then the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: The input layer is used to receive sample images and convert them into images of a preset size. In one embodiment of this application, the preset size can be 360. 640, but other sizes are also possible.

[0043] Multiple feature extraction branches are used to extract features of different dimensions from an image of a preset size, resulting in feature maps of different dimensions. In one embodiment of this application, the different dimensions include: 16 channels, 32 channels, 64 channels, 128 channels, 256 channels, etc. By extracting features of different dimensions from an image of a preset size, the richness of features is improved.

[0044] The feature fusion layer is used to fuse feature maps of different dimensions to obtain a fused feature map. In one embodiment of this application, the feature fusion layer includes: a feature concatenation layer and two 3D features. 3 convolutional layers.

[0045] The output layer is used to convert the fused feature map into a feature vector of a preset dimension to obtain the predicted image. In one embodiment of this application, the preset dimension feature vector is a one-dimensional feature vector, which can concisely and efficiently express the key information in the fused feature map. The one-dimensional feature vector is transformed according to the spatial dimensions of the sample image to obtain the predicted image. After obtaining the predicted image, the labels of the predicted image and the sample image can be compared and analyzed to evaluate the performance of the pre-built image segmentation model, which facilitates subsequent optimization and adjustment of the pre-built image segmentation model to continuously improve the segmentation accuracy and reliability of the model.

[0046] In one embodiment of this application, the plurality of feature extraction branches include: a first feature extraction branch, a second feature extraction branch, a third feature extraction branch, a fourth feature extraction branch, a fifth feature extraction branch, and a sixth feature extraction branch; The first feature extraction branch is used to extract features of a first preset dimension from an image of a preset size, and to compress these features to obtain a feature map of the first preset dimension. In one embodiment of this application, the first preset dimension has 16 channels. The compression of the first preset dimension features is implemented using the RFEM (Refined Feature Enhancement Module), which helps to enhance the features in the image of the preset size, accurately focus on the key tooth region in the image of the preset size, reduce interference from irrelevant backgrounds, and thus improve the segmentation effect of the pre-built image segmentation model in complex backgrounds.

[0047] The second feature extraction branch is used to extract spatial and channel features from the first preset dimension features to obtain a second preset dimension feature map. In one embodiment of this application, the second preset dimension is equal to the first preset dimension. The process of extracting spatial and channel features from the first preset dimension features is implemented by the CBAM (Convolutional Block Attention Module) module. The CBAM module can automatically assign different attention weights to each convolutional block within the CBAM module, thereby highlighting important features in the preset size image and further mitigating the impact of boundary blurring and uncertainty.

[0048] The third feature extraction branch is used to extract third preset dimension features from the first preset dimension features, and to extract spatial and channel features from the third preset dimension features to obtain first spatial-channel features. The first spatial-channel features are then fused, upsampled, and compressed to obtain a third preset dimension feature map. In one embodiment of this application, the third preset dimension is larger than the second preset dimension, and the third preset dimension has 32 channels. The third feature extraction branch includes: a second feature extraction module, a second CBMA module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module. By setting the second CBMA module and the second RFEM module in the third feature extraction branch, the ability of the pre-built image segmentation model to capture and optimize the third preset dimension features is further enhanced.

[0049] The fourth feature extraction branch is used to extract features of the fourth preset dimension from the features of the third preset dimension, and to extract spatial and channel features from the features of the fourth preset dimension to obtain a second spatial-channel feature map. The second spatial-channel feature map is then subjected to fusion, upsampling, and compression operations to obtain the feature map of the fourth preset dimension. In one embodiment of this application, the fourth preset dimension is larger than the third preset dimension, and the fourth preset dimension has 64 channels. The fourth feature extraction branch includes: a third feature extraction module, a third CBMA module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module. By setting the second CBMA module and the second RFEM module in the third feature extraction branch, the ability of the pre-built image segmentation model to capture and optimize features of the fourth preset dimension is further enhanced.

[0050] The fifth feature extraction branch is used to extract features of the fifth preset dimension from the features of the fourth preset dimension, and to perform fusion, upsampling, and compression operations on the features of the fifth preset dimension to obtain a feature map of the fifth preset dimension. In one embodiment of this application, the fifth preset dimension is larger than the fourth preset dimension, and the fifth preset dimension has 128 channels. The fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module. By setting the fourth RFEM module in the fifth feature extraction branch, the features of the fifth preset dimension can be refined, effectively improving the model's ability to resolve high-resolution features. At the same time, the sixth transposed convolution module ensures the integrity of the information in the spatial dimension of the feature map through progressive upsampling operations.

[0051] The sixth feature extraction branch is used to extract features of the sixth preset dimension from the fifth preset dimension feature map, and to perform upsampling and compression operations on the sixth preset dimension features to obtain the sixth preset dimension feature map. In one embodiment of this application, the sixth preset dimension is larger than the fifth preset dimension, the sixth preset dimension has 256 channels, and the sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module includes two 3D convolution modules. The convolutional layer has 3 layers. The convolutional module is used to extract feature information from the feature map of the fifth preset dimension, providing a feature basis for subsequent processing.

[0052] The first, second, third, fourth, fifth, and sixth preset dimension feature maps are combined to form feature maps of different dimensions. In one embodiment of this application, the feature maps of different dimensions encompass rich feature information in an image of a preset size. By combining feature maps of different dimensions, the advantages of each dimension can be fully utilized to achieve complementarity of multi-dimensional information. When training a pre-built image segmentation model, the feature maps of different dimensions not only enhance the pre-built image segmentation model's ability to capture detailed features such as tooth edges and textures, but also improve the recognition accuracy of the pre-built image segmentation model for tooth regions in complex backgrounds. This results in the pre-built image segmentation model having higher robustness and accuracy in tooth image segmentation after training.

[0053] In one embodiment of this application, the first feature extraction branch includes a first feature extraction module and a first RFEM module. In one embodiment of this application, the first feature extraction module is used to extract a first preset dimension feature from an image of a preset size; the first RFEM module is used to compress the first preset dimension feature to obtain a first preset dimension feature map. The first feature extraction module includes two 3D features. The system consists of 3 convolutional layers and 1 feature concatenation layer. The first feature extraction module performs layer-by-layer convolution operations on the image of the preset size through its internal convolutional layers, thereby capturing the first preset dimension features in the image of the preset size. After being processed by the first RFEM module, these features are further compressed and optimized to generate a first preset dimension feature map with stronger expressive power. The combination of the first feature extraction module and the first RFEM module can efficiently extract and compress key features in the image of the preset size, which helps to improve the accuracy of subsequent image segmentation.

[0054] The second feature extraction branch includes a first CBMA module. The first CBMA module is used to extract spatial and channel features from the first preset dimension features to obtain a second preset dimension feature map. In one embodiment of this application, the first CBMA module is composed of an attention mechanism, which internally includes a channel attention submodule and a spatial attention submodule. The channel attention submodule mainly focuses on the importance of features in different dimensions of the image, while the spatial attention submodule focuses on features at different spatial locations in the image. By assigning different attention weights to each spatial location, it highlights the features of key locations such as the area where the teeth are located and the edges of the teeth. The first CBMA module organically combines the channel attention submodule and the spatial attention submodule to perform comprehensive feature extraction of the first preset dimension features, thereby obtaining a second preset dimension feature map that accurately reflects the features of the teeth.

[0055] The third feature extraction branch includes: a second feature extraction module, a second CBMA module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module. The second feature extraction module is used to extract a third preset dimension feature from a first preset dimension feature. The second CBMA module is used to extract spatial and channel features from the third preset dimension feature to obtain a first spatial-channel feature. The first feature fusion module is used to fuse the first spatial-channel feature and the fourth preset dimension feature map to obtain a first fused feature. The first transposed convolution module is used to upsample the first fused feature to obtain a first upsampled feature. The second RFEM module is used to compress the first upsampled feature to obtain a first compressed feature. The second transposed convolution module is used to upsample the first compressed feature to obtain a third preset dimension feature map. The feature fusion layer is used to fuse the first preset dimension feature map, the second preset dimension feature map, and the third preset dimension feature map. In one embodiment of this application, a 2-layer configuration is provided between the second feature extraction module and the first feature extraction module. The pooling layer 2, combined with the pooling layer, the second feature extraction module, the second CBMA module, the first feature fusion module, the first transposed convolution module, the second RFEM module, and the second transposed convolution module, not only enhances the model's ability to capture detailed features such as tooth edges and textures, but also improves the recognition accuracy of the pre-built image segmentation model for tooth regions in complex backgrounds, making the pre-built image segmentation model more robust and accurate for tooth image segmentation tasks.

[0056] The fourth feature extraction branch includes: a third feature extraction module, a third CBMA module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module. The third feature extraction module extracts fourth-preset-dimensional features from third-preset-dimensional features. The third CBMA module extracts spatial and channel features from the fourth-preset-dimensional features to obtain a second spatial-channel feature map. The second feature fusion module fuses the second spatial-channel feature map and the fifth-preset-dimensional feature map to obtain a second fused feature map. The third transposed convolution module upsamples the second fused features to obtain a second upsampled feature map. The third RFEM module compresses the second upsampled features to obtain a second compressed feature map. The fourth transposed convolution module upsamples the second compressed features to obtain a fourth-preset-dimensional feature map. In one embodiment of this application, a 2-column configuration is provided between the third feature extraction module and the second feature extraction module. The pooling layer 2, combined with the pooling layer, the third feature extraction module, the third CBMA module, the second feature fusion module, the third transposed convolution module, the third RFEM module, and the fourth transposed convolution module, significantly improves the performance and accuracy of the pre-built image segmentation model on the tooth image segmentation task.

[0057] The fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module. The fourth feature extraction module extracts features of a fifth preset dimension from features of a fourth preset dimension, serving as a pooling layer feature map. The third feature fusion module fuses the pooling layer feature map and the sixth preset dimension feature map to obtain a third fused feature map. The fifth transposed convolution module upsamples the third fused feature map to obtain a third upsampled feature map. The fourth RFEM module compresses the third upsampled feature map to obtain a third compressed feature map. The sixth transposed convolution module upsamples the third compressed feature map to obtain a fifth preset dimension feature map. In one embodiment of this application, a 2-dimensional array is set between the fourth feature extraction module and the third feature extraction module. The pooling layer 2, through the combination of the pooling layer, the fourth feature extraction module, the third feature fusion module, the fifth transposed convolution module, the fourth RFEM module, and the sixth transposed convolution module, greatly improves the adaptability and segmentation accuracy of the pre-built image segmentation model in complex dental image scenarios, providing more reliable and efficient technical support for clinical dental image analysis.

[0058] The sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module is used to extract the sixth preset dimension features from the fifth preset dimension feature map; the seventh transposed convolution module is used to upsample the sixth preset dimension features to obtain a fourth upsampled feature map; the fifth RFEM module is used to compress the fourth upsampled feature map to obtain a fourth compressed feature map; and the eighth transposed convolution module is used to upsample the fourth compressed feature map to obtain the sixth preset dimension feature map. In one embodiment of this application, the convolution module includes two 3D convolution modules. A convolutional layer of 3, with a 2-level connection between the convolutional module and the fourth feature extraction module. The pooling layer (2), combined with convolutional modules, the seventh transposed convolutional module, the fifth RFEM module, and the eighth transposed convolutional module, further enhances the pre-built image segmentation model's ability to perceive subtle structures in tooth images. When processing images with complex tooth shapes and blurred boundaries between adjacent teeth, it can more accurately delineate tooth contours and effectively distinguish between different individual teeth. Simultaneously, it enhances the pre-built image segmentation model's resistance to image noise and interference factors, maintaining high segmentation accuracy and stability even under adverse conditions such as image blurring and uneven lighting. Furthermore, through the collaborative work of multiple modules, tooth image features are extracted, fused, and optimized from different angles and levels. This results in a sixth-preset dimension feature map that comprehensively and accurately reflects various key information of the tooth image, providing a solid and reliable feature foundation for subsequent tooth image segmentation tasks. This significantly improves the performance and effectiveness of tooth image segmentation, enabling it to better meet practical needs in clinical applications and providing stronger support for the diagnosis and treatment of dental diseases.

[0059] In one embodiment of this application, all feature extraction modules include two 3D modules. The system consists of a 3x3 convolutional layer and a concatenation layer. All feature fusion modules have the same structure as the feature fusion layer, including one concatenation layer and two 3x3 convolutional layers. A convolutional layer of 3.

[0060] Figure 3 This is a flowchart illustrating another exemplary embodiment of the present application of a tooth image segmentation method based on semi-supervised learning and nnU-Net. Figure 3In this paper, the tooth image segmentation method based on semi-supervised learning and nnU-Net includes: (1) training a pre-built image segmentation model with a set of labeled images to obtain a teacher model, where Ground Truth represents the real label data in the labeled images; (2) performing enhancement operations on the unlabeled images in the unlabeled image set, and inputting the enhanced unlabeled images into the teacher model to obtain multiple first pseudo-labeled images; (3) dividing the sample image set into K training and validation sets according to K-fold cross-validation; and training the pre-built image segmentation model with the training and validation sets in the K combinations to obtain multiple student models (N student models); (4) inputting multiple first pseudo-labeled images into each student model to obtain multiple second pseudo-labeled images (Pred 1-Pred 2). K); and perform a fusion operation on multiple second pseudo-label images to obtain a third pseudo-label image; the fusion operation includes: pixel averaging operation, binarization operation and morphological operation; (5) combine multiple first pseudo-label images, third pseudo-label images and labeled images in the labeled image set to obtain an updated image set; train the pre-built image segmentation model through the updated image set (final training) to obtain a tooth image segmentation model.

[0061] Figure 4 This is a schematic diagram illustrating the structure of a pre-built image segmentation model as shown in an exemplary embodiment of this application. Figure 4 The pre-built image segmentation model includes: an input layer, a feature fusion layer, an output layer, a first feature extraction branch, a second feature extraction branch, a third feature extraction branch, a fourth feature extraction branch, a fifth feature extraction branch, and a sixth feature extraction branch. The first, second, third, fourth, fifth, and sixth feature extraction branches are arranged sequentially from top to bottom. The first feature extraction branch includes: a first feature extraction module and a first RFEM module; the second feature extraction branch includes: a first CBMA module; the third feature extraction branch includes: a second... The system comprises: a feature extraction module, a second CBMA module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module; a fourth feature extraction branch includes: a third feature extraction module, a third CBMA module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module; a fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module; and a sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module.

[0062] In one embodiment of this application, a first feature extraction module extracts channel feature information from an image of a preset size according to a 16-channel dimension, a second feature extraction module extracts channel feature information from an image of a preset size according to a 32-channel dimension, and the 16-channel and 32-channel feature information are finally fused at a feature fusion layer; a third feature extraction module extracts channel feature information from an image of a preset size according to a 64-channel dimension; the 32-channel and 64-channel feature information are fused at a first feature fusion module; a fourth feature extraction module extracts channel feature information from an image of a preset size according to a 128-channel dimension; the 128-channel and 64-channel feature information are fused at a second feature fusion module; and a convolution module extracts channel feature information from an image of a preset size according to a 256-channel dimension, and the 256-channel and 128-channel feature information are fused at a third feature fusion module. The above methods achieve the fusion of 16-channel, 32-channel, 64-channel, 128-channel, and 256-channel feature information.

[0063] This application uses a computer with 64GB of solid-state drive memory, an NVIDIA RTX 4090 graphics card, and an Intel(R) Core(TM) i9-10900K @ 3.70GHz processor. The software environment used includes a Python 3.9.0 operating system, a PyTorch 2.0.0 deep learning framework, and CUDA version 11.8 as the experimental environment for training the pre-built image segmentation model. This application selects training, validation, and test sets from the STS-2D-Tooth dataset to train, validate, and test the pre-built image segmentation model. The training set contains 2380 panoramic dental X-ray images (including 30 labeled images and 2350 unlabeled images), the validation set contains 100 panoramic dental X-ray images (including 50 labeled images and 50 unlabeled images), and the test set contains 20 panoramic dental X-ray images (including 0 labeled images and 20 unlabeled images).

[0064] Figure 5 This is a schematic diagram illustrating the change curve of the loss function value during the training process of a pre-built image segmentation model, as shown in an exemplary embodiment of this application. Figure 5 In the diagram, the horizontal axis represents the training epochs of the pre-built image segmentation model, and the vertical axis represents the loss function value calculated using formula (1). Figure 5As can be seen, with the increase of training rounds, the training loss gradually decreases from about 0.3804 to 0.0082, while the validation loss decreases from about 0.4204 to 0.01. Throughout the training process, both the training loss curve and the validation loss curve show a smooth downward trend with the increase of training rounds, and the downward trends of the training loss curve and the validation loss curve are consistent, indicating that the pre-built image segmentation model has been effectively optimized and overfitting has been avoided during the training process.

[0065] Figure 6 This is a schematic diagram illustrating the change curve of the Dice similarity coefficient during the training of a pre-built image segmentation model, as shown in an exemplary embodiment of this application. Figure 6 In the diagram, the horizontal axis represents the training epochs of the pre-built image segmentation model, and the vertical axis represents the Dice similarity coefficient value. Figure 6 As can be seen, the Dice similarity coefficient value steadily increases with the increase of training rounds. The Dice similarity coefficient value of the training set increases from about 0.3 to 0.92, while the Dice similarity coefficient value of the validation set eventually reaches about 0.89. Moreover, the gap between the Dice similarity coefficient value change curve of the training set and the Dice similarity coefficient value of the validation set remains within the predetermined range, which further verifies the good generalization ability of the image segmentation model during training and the image segmentation model after training on unseen data.

[0066] Combination Figure 5 and Figure 6 It can be seen that the trained image segmentation model has good training effect and can resist the decline in validation performance while maintaining high segmentation accuracy. Therefore, it can have higher segmentation accuracy when facing complex anatomical structures and overlapping areas in dental panoramic X-ray images.

[0067] Figure 7 This is a schematic diagram illustrating the effect of tooth image segmentation in an exemplary embodiment of this application. Figure 7The results of segmenting the same image using the FixMatch method, CPS (Cross Pseudo Supervision), MT (Mersenne Twister) method, and DeepLab V3+ method in related technologies are compared with those of the image segmentation method in this application. The image segmentation method in this application achieves high-precision segmentation on most tooth instances, with complete contours and clear instance boundaries, accurately distinguishing adjacent individual teeth. This demonstrates that the image segmentation method in this application has strong segmentation capabilities in densely structured regions and complex morphological backgrounds. However, the FixMatch method, CPS (Cross Pseudo Supervision), MT (Mersenne Twister) method, and DeepLab V3+ method in related technologies... In some cases, the V3+ method still has certain segmentation errors (as marked by the red circle), mainly including the following situations: (1) Due to the high degree of overlap between teeth or the low contrast of X-ray images, the determination of the boundary of individual teeth is blurred, causing instance fusion or edge contour shift; (2) Some root areas have incomplete instance separation due to bone overlap or image artifact interference; (3) Individual small teeth or abnormal teeth have limited recognition ability due to the scarcity of training samples.

[0068] Figure 8 This is a schematic diagram illustrating the effect of tooth image segmentation, shown in another exemplary embodiment of this application. Figure 8 The paper demonstrates the segmentation performance of the tooth image segmentation model in this application for difficult cases. Compared with the mask image of the image to be segmented, the segmented image output by the tooth image segmentation model in this application shows segmentation deviation at only a few locations. Furthermore, the Dice similarity coefficient between the segmented image output by the tooth image segmentation model in this application and the mask image of the image to be segmented reaches 92.05%, the Normalized Surface Distance (NSD) reaches 94.72%, the Intersection over Union (IoU) reaches 86.27%, and the Recognition Accuracy (IA) reaches 85.48%.

[0069] The following describes an apparatus embodiment of this application, which can be used to execute the tooth image segmentation method based on semi-supervised learning and nnU-Net described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the tooth image segmentation method based on semi-supervised learning and nnU-Net described above.

[0070] Figure 9This is a block diagram illustrating a tooth image segmentation apparatus based on semi-supervised learning and nnU-Net, as shown in an exemplary embodiment of this application.

[0071] like Figure 9 As shown, this exemplary tooth image segmentation device 900 based on semi-supervised learning and nnU-Net includes: The image acquisition module 901 is used to acquire images of teeth to be segmented and to preprocess these images to obtain preprocessed images.

[0072] The image segmentation module 902 is used to input the preprocessed image into the tooth image segmentation model to obtain the tooth segmentation result.

[0073] In one embodiment of this application, the tooth image to be segmented is an X-ray image. Preprocessing methods include image resizing and resolution adjustment, which improve the quality of the tooth image to be segmented and adapt it to the size requirements of the input image for the tooth image segmentation model.

[0074] In one embodiment of this application, the tooth image segmentation model is trained on a pre-built image segmentation model using a semi-supervised learning mechanism. The pre-built image segmentation model is constructed based on the nnU-Net structure and attention mechanism, which reduces the dependence of the pre-built image segmentation model on labeled data during training. That is, the pre-built image segmentation model can be trained using a combination of labeled and unlabeled data, reducing data labeling costs and improving the performance of tooth image segmentation in cases of complex backgrounds, overlapping structures, and unclear boundaries.

[0075] It should be noted that the tooth image segmentation device based on semi-supervised learning and nnU-Net provided in the above embodiments and the tooth image segmentation method based on semi-supervised learning and nnU-Net provided in the above embodiments belong to the same concept. The specific way in which each module and unit performs its operation has been described in detail in the method embodiments, and will not be repeated here. In practical applications, the tooth image segmentation device based on semi-supervised learning and nnU-Net provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0076] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A tooth image segmentation method based on semi-supervised learning and nnU-Net, characterized in that, The method includes: Acquire an image of the tooth to be segmented, and preprocess the image to obtain a preprocessed image; the preprocessing method includes: image size conversion; The preprocessed image is input into the tooth image segmentation model to obtain the tooth segmentation result; the tooth image segmentation model is trained on the pre-constructed image segmentation model according to the semi-supervised learning mechanism; the pre-constructed image segmentation model is constructed based on the nnU-Net structure and attention mechanism.

2. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 1, characterized in that, The process of training a pre-built image segmentation model using a semi-supervised learning mechanism to obtain the tooth image segmentation model includes: Obtain a set of sample images for training the pre-constructed image segmentation model; the set of sample images includes: a set of labeled images and a set of unlabeled images; The pre-built image segmentation model is trained using the labeled image set to obtain a teacher model; the sample image set is divided into K training and validation sets using K-fold cross-validation; and the pre-built image segmentation model is trained using the training and validation sets in the K combinations to obtain multiple student models. The unlabeled images in the unlabeled image set are enhanced, and the enhanced unlabeled images are input into the teacher model to obtain multiple first pseudo-labeled images; the enhancement operation includes: vertical flip operation and histogram equalization; Multiple first pseudo-label images are input into each student model to obtain multiple second pseudo-label images; and the multiple second pseudo-label images are fused to obtain a third pseudo-label image; the fusion operation includes: pixel averaging, binarization and morphological operations; Multiple first pseudo-labeled images, the third pseudo-labeled image, and labeled images from the labeled image set are combined to obtain an updated image set; the pre-built image segmentation model is trained using the updated image set to obtain the tooth image segmentation model.

3. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 2, characterized in that, Before obtaining the sample image set for training the pre-built image segmentation model, the method further includes: A subset of images from the sample image set is selected as unlabeled images; the remaining images from the sample image set are selected as images to be labeled; the set of unlabeled images is selected as the unlabeled image set. The image to be labeled is converted into a preset-bit single-channel grayscale image; the pixel value of each pixel in the preset-bit single-channel grayscale image is set to a preset pixel value. The contour point information and classification category of each tooth in the image to be labeled are obtained. Based on the contour point information of each tooth in the image to be labeled, the pixel region of each tooth in the preset single-channel grayscale image is determined. According to the correspondence between the classification category and the classification label, the pixel region of each tooth in the preset single-channel grayscale image is filled with the corresponding classification label to obtain a labeled image. The set of labeled images is used as the labeled image set. The classification label is represented in numerical form.

4. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 2, characterized in that, After inputting the enhanced unlabeled image into the teacher model to obtain multiple first pseudo-labeled images, the method further includes: Binarization is performed on multiple first pseudo-label images to obtain multiple binarized images; Based on multiple binarized images, the connected regions of each binarized image are determined, and the contour of each tooth in each binarized image is determined according to the connected regions of each binarized image; the number of pixels in the contour of each tooth is greater than or equal to the preset number of pixels; the contour of each tooth has a unique number. Based on the correspondence between the category number and the contour number of each tooth in each binarized image, the contour of each tooth in each binarized image is converted into the category number of the tooth, resulting in multiple fourth pseudo-label images. Multiple first pseudo-label images are then updated using these multiple fourth pseudo-label images, and the updated multiple first pseudo-label images are input into each student model.

5. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 2, characterized in that, The process of training the pre-built image segmentation model using the updated image set to obtain the tooth image segmentation model includes: The sample images from the updated image set are input into the pre-built image segmentation model to obtain the predicted images of the sample images; With the goal of minimizing the error between the sample image and the predicted image, the parameters in the pre-built image segmentation model are updated until the error between the predicted image output by the updated image segmentation model and the sample image is less than a preset error threshold, or the number of times the parameters in the pre-built image segmentation model are updated is greater than or equal to a preset number threshold.

6. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 5, characterized in that, The error between the sample image and the predicted image is represented by a loss function, the expression of which is as follows: ， in, This represents the error between the sample image and the predicted image. Indicates the number of sample images. Indicates the first One sample image, Indicates the first The predicted image of the nth sample image, the nth The sample image and the first The predicted image for each sample image is represented in the form of a binary vector. Indicates the first Labels for each sample image. Indicates the first The predicted probability of a sample image. This represents the loss weight of the Dice similarity term coefficient. The weights represent the binary cross-entropy loss. .

7. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 5, characterized in that, If the pre-built image segmentation model includes an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, then the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: The input layer is used to receive the sample image and convert the sample image into an image of a preset size; Multiple feature extraction branches are used to extract features of different dimensions from the image of the preset size, resulting in feature maps of different dimensions; The feature fusion layer is used to fuse feature maps of different dimensions to obtain a fused feature map; The output layer is used to convert the fused feature map into a feature vector of a preset dimension to obtain the predicted image.

8. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 7, characterized in that, The multiple feature extraction branches include: the first feature extraction branch, the second feature extraction branch, the third feature extraction branch, the fourth feature extraction branch, the fifth feature extraction branch, and the sixth feature extraction branch; The first feature extraction branch is used to extract the first preset dimension features in the preset size image and compress the first preset dimension features to obtain the first preset dimension feature map; The second feature extraction branch is used to extract spatial and channel features from the first preset dimension features to obtain a second preset dimension feature map; the second preset dimension is equal to the first preset dimension. The third feature extraction branch is used to extract a third preset dimension feature from the first preset dimension feature, and to extract spatial and channel features from the third preset dimension feature to obtain a first spatial-channel feature. The first spatial-channel feature is then subjected to fusion, upsampling, and compression operations to obtain a third preset dimension feature map; the third preset dimension is greater than the second preset dimension. The fourth feature extraction branch is used to extract the fourth preset dimension feature from the third preset dimension feature, and to extract the spatial feature and channel feature of the fourth preset dimension feature to obtain the second spatial-channel feature map. The second spatial-channel feature map is then subjected to fusion operation, upsampling operation and compression operation to obtain the fourth preset dimension feature map; the fourth preset dimension is greater than the third preset dimension. The fifth feature extraction branch is used to extract the fifth preset dimension feature from the fourth preset dimension feature, and to perform fusion, upsampling and compression operations on the fifth preset dimension feature to obtain the fifth preset dimension feature map; the fifth preset dimension is greater than the fourth preset dimension. The sixth feature extraction branch is used to extract the sixth preset dimension feature from the fifth preset dimension feature map, and to perform upsampling and compression operations on the sixth preset dimension feature to obtain the sixth preset dimension feature map; the sixth preset dimension is greater than the fifth preset dimension; The first preset dimension feature map, the second preset dimension feature map, the third preset dimension feature map, the fourth preset dimension feature map, the fifth preset dimension feature map, and the sixth preset dimension feature map are combined to form different dimension feature maps.

9. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 8, characterized in that, The first feature extraction branch includes: a first feature extraction module and a first RFEM module; the first feature extraction module is used to extract a first preset dimension feature from the preset size image; the first RFEM module is used to compress the first preset dimension feature to obtain the first preset dimension feature map; The second feature extraction branch includes: a first CBMA module; the first CBMA module is used to extract spatial features and channel features from the first preset dimension features to obtain the second preset dimension feature map; The third feature extraction branch includes: a second feature extraction module, a second CBMA module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module; the second feature extraction module is used to extract a third preset dimension feature from the first preset dimension feature; the second CBMA module is used to extract spatial and channel features from the third preset dimension feature to obtain the first spatial-channel feature; the first feature fusion module is used to fuse the first spatial-channel feature and the fourth preset dimension feature map to obtain a first fused feature; the first transposed convolution module is used to upsample the first fused feature to obtain a first upsampled feature; the second RFEM module is used to compress the first upsampled feature to obtain a first compressed feature; the second transposed convolution module is used to upsample the first compressed feature to obtain the third preset dimension feature map; the feature fusion layer is used to fuse the first preset dimension feature map, the second preset dimension feature map, and the third preset dimension feature map. The fourth feature extraction branch includes: the third feature extraction module, the third CBMA module, the second feature fusion module, the third transposed convolution module, the third RFEM module, and the fourth transposed convolution module; the third feature extraction module is used to extract fourth preset dimension features from the third preset dimension features; the third CBMA module is used to extract spatial and channel features from the fourth preset dimension features to obtain a second spatial-channel feature map; the second feature fusion module is used to fuse the second spatial-channel feature map and the fifth preset dimension feature map to obtain a second fused feature map; the third transposed convolution module is used to upsample the second fused features to obtain a second upsampled feature map; the third RFEM module is used to compress the second upsampled features to obtain a second compressed feature map; the fourth transposed convolution module is used to upsample the second compressed features to obtain the fourth preset dimension feature map; The fifth feature extraction branch includes: the fourth feature extraction module, the third feature fusion module, the fifth transposed convolution module, the fourth RFEM module, and the sixth transposed convolution module; the fourth feature extraction module is used to extract a fifth preset dimension feature from the fourth preset dimension feature, as a pooling layer feature map; the third feature fusion module is used to fuse the pooling layer feature map and the sixth preset dimension feature map to obtain a third fused feature map; the fifth transposed convolution module is used to upsample the third fused feature map to obtain a third upsampled feature map; the fourth RFEM module is used to compress the third upsampled feature map to obtain a third compressed feature map; the sixth transposed convolution module is used to upsample the third compressed feature map to obtain the fifth preset dimension feature map; The sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module is used to extract a sixth preset dimension feature from the fifth preset dimension feature map. The seventh transposed convolution module is used to upsample the sixth preset dimension feature to obtain a fourth upsampled feature map. The fifth RFEM module is used to compress the fourth upsampled feature map to obtain a fourth compressed feature map. The eighth transposed convolution module is used to upsample the fourth compressed feature map to obtain the sixth preset dimension feature map.

10. A tooth image segmentation device based on semi-supervised learning and nnU-Net, characterized in that, include: The image acquisition module is used to acquire images of teeth to be segmented and to preprocess the images of teeth to be segmented to obtain preprocessed images; Preprocessing methods include: image size conversion; The image segmentation module is used to input the preprocessed image into the tooth image segmentation model to obtain the tooth segmentation result; the tooth image segmentation model is trained on the pre-constructed image segmentation model according to the semi-supervised learning mechanism; the pre-constructed image segmentation model is constructed based on the nnU-Net structure and attention mechanism.

Images