A tooth image segmentation method and device based on semi-supervised learning and nnU-Net

Abstract

This application provides a method and apparatus for tooth image segmentation based on semi-supervised learning and nnU-Net. The method acquires a tooth image to be segmented, preprocesses the image, and inputs the preprocessed image into a tooth image segmentation model to obtain the tooth segmentation result. The tooth image segmentation model is trained on a pre-built image segmentation model using a semi-supervised learning mechanism. The pre-built image segmentation model is constructed based on the nnU-Net structure and attention mechanism. This application reduces the dependence of the pre-built image segmentation model on labeled data during training. That is, the pre-built image segmentation model can be trained using a combination of labeled and unlabeled data, reducing data labeling costs and improving the tooth image segmentation effect under complex backgrounds, overlapping structures, and unclear boundaries.

Description

Technical Field

[0001] This application relates to the field of tooth segmentation technology, and in particular to a tooth image segmentation method and apparatus based on semi-supervised learning and nnU-Net. Background Technology

[0002] Dental image segmentation is a crucial step in oral medicine diagnosis, treatment planning, and outcome evaluation. Accurate dental image segmentation helps dentists more precisely identify tooth structure, lesion location, and extent, thereby enabling the development of more personalized treatment plans. However, current dental image segmentation methods (e.g., thresholding, edge detection, and region growing) often fail to achieve ideal segmentation results when dealing with complex backgrounds, overlapping structures, and unclear boundaries in medical images. With advancements in image segmentation technology, dental image segmentation methods rely on model training. However, model training often requires large amounts of labeled data, and the high cost and time-consuming nature of medical image annotation limits its practical clinical application.

[0003] Therefore, it is necessary to improve the tooth image segmentation algorithm in related technologies to increase the accuracy of tooth image segmentation and reduce the cost of tooth image segmentation. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, this application provides a tooth image segmentation method and apparatus based on semi-supervised learning and nnU-Net to solve the above technical problems.

[0005] According to one aspect of the embodiments of this application, a tooth image segmentation method based on semi-supervised learning and nnU-Net is provided. The method includes: acquiring a tooth image to be segmented, and preprocessing the tooth image to be segmented to obtain a preprocessed image; the preprocessing includes: image size conversion; inputting the preprocessed image into a tooth image segmentation model to obtain a tooth segmentation result; the tooth image segmentation model is trained on a pre-constructed image segmentation model according to a semi-supervised learning mechanism; the pre-constructed image segmentation model is constructed based on the nnU-Net structure and attention mechanism.

[0006] In one embodiment of this application, the process of training a pre-constructed image segmentation model to obtain the tooth image segmentation model according to a semi-supervised learning mechanism includes: obtaining a sample image set for training the pre-constructed image segmentation model; the sample image set includes: a labeled image set and an unlabeled image set; training the pre-constructed image segmentation model using the labeled image set to obtain a teacher model; and dividing the sample image set into K training and validation set combinations using K-fold cross-validation. The pre-built image segmentation model is trained using training and validation sets from K combinations to obtain multiple student models. Unlabeled images from the unlabeled image set are augmented, and these augmented images are input into the teacher model to obtain multiple first pseudo-labeled images. The augmentation operations include vertical flipping and histogram equalization. These first pseudo-labeled images are input into each student model to obtain multiple second pseudo-labeled images. These second pseudo-labeled images are then fused to obtain a third pseudo-labeled image. The fusion operations include pixel averaging, binarization, and morphological operations. The multiple first pseudo-labeled images, the third pseudo-labeled images, and the labeled images from the labeled image set are combined to obtain an updated image set. The pre-built image segmentation model is trained using this updated image set to obtain the tooth image segmentation model.

[0007] In one embodiment of this application, before obtaining the sample image set for training the pre-constructed image segmentation model, the method further includes: selecting a portion of the images in the sample image set as unlabeled images; and using the remaining images in the sample image set as images to be labeled; the set of unlabeled images is used as the unlabeled image set; converting the images to be labeled into a preset-position single-channel grayscale image; setting the pixel value of each pixel in the preset-position single-channel grayscale image to a preset pixel value; obtaining the contour point information and classification category of each tooth in the image to be labeled, and determining the pixel region of each tooth in the preset-position single-channel grayscale image based on the contour point information of each tooth in the image to be labeled; filling the pixel region of each tooth in the preset-position single-channel grayscale image with the corresponding classification label according to the correspondence between the classification category and the classification label, obtaining labeled images, and using the set of labeled images as the labeled image set; the classification label is represented in numerical form.

[0008] In one embodiment of this application, after inputting the enhanced unlabeled image into the teacher model to obtain multiple first pseudo-labeled images, the method further includes: performing a binarization operation on the multiple first pseudo-labeled images to obtain multiple binarized images; determining the connected regions of each binarized image based on the multiple binarized images, and determining the contour of each tooth in each binarized image according to the connected regions of each binarized image; the number of contour pixels of each tooth is greater than or equal to a preset number of pixels; each tooth contour has a unique number; converting the contour of each tooth in each binarized image into a tooth category number according to the correspondence between the category number and the contour number of each tooth in each binarized image, to obtain multiple fourth pseudo-labeled images, and updating the multiple first pseudo-labeled images with the multiple fourth pseudo-labeled images, so as to input the updated multiple first pseudo-labeled images into each student model.

[0009] In one embodiment of this application, the process of training the pre-built image segmentation model using the updated image set to obtain the tooth image segmentation model includes: inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images; updating the parameters in the pre-built image segmentation model with the goal of minimizing the error between the sample images and the predicted images, until the error between the predicted images output by the updated image segmentation model and the sample images is less than a preset error threshold, or the number of times the parameters in the pre-built image segmentation model are updated is greater than or equal to a preset number threshold.

[0010] In one embodiment of this application, the error between the sample image and the predicted image is characterized by a loss function, the expression of which is as follows: ,in, This represents the error between the sample image and the predicted image. Indicates the number of sample images. Indicates the first One sample image, Indicates the first The predicted image of the nth sample image, the nth The sample image and the first The predicted image for each sample image is represented in the form of a binary vector. Indicates the first Labels for each sample image. Indicates the first The predicted probability of a sample image. This represents the loss weight of the Dice similarity term coefficient. The weights represent the binary cross-entropy loss. .

[0011] In one embodiment of this application, if the pre-built image segmentation model includes: an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, then the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: The input layer is used to receive the sample image and convert the sample image into an image of a preset size; multiple feature extraction branches are used to extract features of different dimensions from the image of the preset size to obtain feature maps of different dimensions; the feature fusion layer is used to fuse the feature maps of different dimensions to obtain a fused feature map; the output layer is used to convert the fused feature map into a feature vector of a preset dimension to obtain the predicted image.

[0012] In one embodiment of this application, multiple feature extraction branches include: a first feature extraction branch, a second feature extraction branch, a third feature extraction branch, a fourth feature extraction branch, a fifth feature extraction branch, and a sixth feature extraction branch; the first feature extraction branch is used to extract a first preset dimension feature from the image of the preset size, and compress the first preset dimension feature to obtain a first preset dimension feature map; the second feature extraction branch is used to extract spatial and channel features from the first preset dimension feature to obtain a second preset dimension feature map; the second preset dimension is equal to the first preset dimension; the third feature extraction branch is used to extract a third preset dimension feature from the first preset dimension feature, and extract spatial and channel features from the third preset dimension feature to obtain a first spatial-channel feature, and perform fusion, upsampling, and compression operations on the first spatial-channel feature to obtain a third preset dimension feature map; the third preset dimension is greater than the second preset dimension; the fourth feature extraction branch is used to extract a fourth preset dimension from the third preset dimension feature. The system first extracts features from the fourth preset dimension features, and then extracts spatial and channel features from the fourth preset dimension features to obtain a second spatial-channel feature map. The second spatial-channel feature map is then fused, upsampled, and compressed to obtain a fourth preset dimension feature map. The fourth preset dimension is greater than the third preset dimension. The fifth feature extraction branch extracts features from the fourth preset dimension features, and then performs fusion, upsampling, and compression operations on the fifth preset dimension features to obtain a fifth preset dimension feature map. The fifth preset dimension is greater than the fourth preset dimension. The sixth feature extraction branch extracts features from the fifth preset dimension feature map, and then performs upsampling and compression operations on the sixth preset dimension features to obtain a sixth preset dimension feature map. The sixth preset dimension is greater than the fifth preset dimension. The first, second, third, fourth, fifth, and sixth preset dimension feature maps are combined to form feature maps of different dimensions.

[0013] In one embodiment of this application, the first feature extraction branch includes: a first feature extraction module and a first RFEM module; the first feature extraction module is used to extract a first preset dimension feature from the preset size image; the first RFEM module is used to compress the first preset dimension feature to obtain a first preset dimension feature map; the second feature extraction branch includes: a first CBAM module; the first CBAM module is used to extract spatial and channel features from the first preset dimension feature to obtain a second preset dimension feature map; the third feature extraction branch includes: a second feature extraction module, a second CBAM module, and a first feature extraction module. The system comprises a fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module; a second feature extraction module is used to extract a third preset dimension feature from the first preset dimension feature; a second CBAM module is used to extract spatial and channel features from the third preset dimension feature to obtain the first spatial-channel feature; a first feature fusion module is used to fuse the first spatial-channel feature and the fourth preset dimension feature map to obtain a first fused feature; a first transposed convolution module is used to upsample the first fused feature to obtain a first upsampled feature; and a second RFEM module is used to compress the first upsampled feature to obtain a second fused feature. The system comprises: a first compressed feature; a second transposed convolutional module upsampling the first compressed feature to obtain the third preset dimension feature map; a feature fusion layer fusing the first preset dimension feature map, the second preset dimension feature map, and the third preset dimension feature map; and a fourth feature extraction branch including: a third feature extraction module, a third CBAM module, a second feature fusion module, a third transposed convolutional module, a third RFEM module, and a fourth transposed convolutional module; the third feature extraction module extracting fourth preset dimension features from the third preset dimension features; and the third CBAM module extracting spatial and channel features from the fourth preset dimension features. A second spatial-channel feature map is obtained; the second feature fusion module is used to fuse the second spatial-channel feature map and the fifth preset dimension feature map to obtain a second fused feature map; the third transposed convolution module is used to upsample the second fused feature to obtain a second upsampled feature map; the third RFEM module is used to compress the second upsampled feature to obtain a second compressed feature map; the fourth transposed convolution module is used to upsample the second compressed feature to obtain the fourth preset dimension feature map; the fifth feature extraction branch includes: the fourth feature extraction module, the third feature fusion module, the fifth transposed convolution module, the fourth RFEM module, and the sixth transposed convolution module;The fourth feature extraction module is used to extract a fifth preset dimension feature from the fourth preset dimension feature, as a pooling layer feature map; the third feature fusion module is used to fuse the pooling layer feature map and the sixth preset dimension feature map to obtain a third fused feature map; the fifth transposed convolution module is used to upsample the third fused feature map to obtain a third upsampled feature map; the fourth RFEM module is used to compress the third upsampled feature map to obtain a third compressed feature map; the sixth transposed convolution module is used to upsample the third compressed feature map to obtain the fifth preset dimension feature map. Let a dimensional feature map be defined. The sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module is used to extract the sixth preset dimension feature from the fifth preset dimension feature map. The seventh transposed convolution module is used to upsample the sixth preset dimension feature to obtain a fourth upsampled feature map. The fifth RFEM module is used to compress the fourth upsampled feature map to obtain a fourth compressed feature map. The eighth transposed convolution module is used to upsample the fourth compressed feature map to obtain the sixth preset dimension feature map.

[0014] According to one aspect of the embodiments of this application, a tooth image segmentation device based on semi-supervised learning and nnU-Net is provided, comprising: an image acquisition module for acquiring a tooth image to be segmented and preprocessing the tooth image to be segmented to obtain a preprocessed image; the preprocessing method includes: image size conversion; an image segmentation module for inputting the preprocessed image into a tooth image segmentation model to obtain a tooth segmentation result; the tooth image segmentation model is obtained by training a pre-constructed image segmentation model according to a semi-supervised learning mechanism; the pre-constructed image segmentation model is constructed based on the nnU-Net structure and attention mechanism.

[0015] The beneficial effects of this application are as follows: This application acquires a tooth image to be segmented, preprocesses the tooth image, inputs the preprocessed image into a tooth image segmentation model, and obtains the tooth segmentation result. The tooth image segmentation model is trained on a pre-built image segmentation model according to a semi-supervised learning mechanism. The pre-built image segmentation model is constructed based on the nnU-Net structure and attention mechanism. In the above process, by constructing a pre-built image segmentation model based on the nnU-Net structure and attention mechanism, and training the pre-built image segmentation model according to the semi-supervised learning mechanism, the dependence of the pre-built image segmentation model on labeled data during the training process is reduced. That is, the training of the pre-built image segmentation model can be completed with some labeled data and some unlabeled data, which reduces the data labeling cost and improves the tooth image segmentation effect under complex backgrounds, overlapping structures, and unclear boundaries.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating a tooth image segmentation method based on semi-supervised learning and nnU-Net, as shown in an exemplary embodiment of this application. Figure 3 This is a flowchart illustrating a tooth image segmentation method based on semi-supervised learning and nnU-Net, as shown in another exemplary embodiment of this application. Figure 4 This is a schematic diagram illustrating the structure of a pre-built image segmentation model as shown in an exemplary embodiment of this application; Figure 5 This is a schematic diagram illustrating the change curve of the loss function value during the training process of a pre-built image segmentation model, as shown in an exemplary embodiment of this application. Figure 6 This is a schematic diagram illustrating the change curve of the Dice similarity coefficient during the training of a pre-built image segmentation model, as shown in an exemplary embodiment of this application. Figure 7 This is a schematic diagram illustrating the effect of tooth image segmentation in an exemplary embodiment of this application; Figure 8 This is a schematic diagram illustrating the effect of tooth image segmentation, as shown in another exemplary embodiment of this application; Figure 9 This is a block diagram illustrating a tooth image segmentation apparatus based on semi-supervised learning and nnU-Net, as shown in an exemplary embodiment of this application. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0021] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0022] Reference Figure 1 As shown, the system architecture may include an acquisition device 101 and a data processing device 102. The data processing device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. Those skilled in the art can use the data processing device 102 to acquire images of the teeth to be segmented, preprocess these images, and input the preprocessed images into a tooth image segmentation model to obtain the tooth segmentation results. The tooth image segmentation model is trained using a semi-supervised learning mechanism on a pre-built image segmentation model, which is constructed based on the nnU-Net structure and attention mechanism. The acquisition device 101 is used to take X-ray images of the teeth, which are then used as images to be segmented and provided to the data processing device 102 for processing.

[0023] Indicatively, after acquiring the tooth image to be segmented from the acquisition device 101, the data processing device 102 inputs the preprocessed image into the tooth image segmentation model to obtain the tooth segmentation result. The tooth image segmentation model is trained on a pre-built image segmentation model using a semi-supervised learning mechanism. The pre-built image segmentation model is constructed based on the nnU-Net structure and attention mechanism. In the above process, by constructing a pre-built image segmentation model based on the nnU-Net structure and attention mechanism, and training the pre-built image segmentation model using a semi-supervised learning mechanism, the dependence of the pre-built image segmentation model on labeled data during training is reduced. That is, the training of the pre-built image segmentation model can be completed using a combination of labeled and unlabeled data, reducing data labeling costs and improving the tooth image segmentation effect under complex backgrounds, overlapping structures, and unclear boundaries.

[0024] The implementation details of the technical solutions in the embodiments of this application are described in detail below: Figure 2 This is a flowchart illustrating a tooth image segmentation method based on semi-supervised learning and nnU-Net, as shown in an exemplary embodiment of this application. (Refer to...) Figure 2 As shown, this tooth image segmentation method based on semi-supervised learning and nnU-Net includes at least steps S210 to S220, which are described in detail below: In step S210, an image of the tooth to be segmented is acquired and preprocessed to obtain a preprocessed image. In one embodiment of this application, the image of the tooth to be segmented is an X-ray image. The preprocessing methods include image size conversion, image resolution adjustment, etc., which improve the quality of the image of the tooth to be segmented and adapt to the size requirements of the input image for the tooth image segmentation model.

[0025] In step S220, the preprocessed image is input into the tooth image segmentation model to obtain the tooth segmentation result. In one embodiment of this application, the tooth image segmentation model is trained on a pre-built image segmentation model using a semi-supervised learning mechanism. The pre-built image segmentation model is constructed based on the nnU-Net structure and attention mechanism, which reduces the dependence of the pre-built image segmentation model on labeled data during training. That is, the pre-built image segmentation model can be trained using a combination of labeled and unlabeled data, reducing data labeling costs and improving the tooth image segmentation effect under complex backgrounds, overlapping structures, and unclear boundaries.

[0026] In one embodiment of this application, the process of training a pre-built image segmentation model to obtain a tooth image segmentation model according to a semi-supervised learning mechanism includes: A sample image set is obtained for training a pre-built image segmentation model. In one embodiment of this application, the sample image set includes a labeled image set and an unlabeled image set. For example, in the sample image set, the proportion of labeled images in the labeled image set is set to 10% of the total number of sample images, and the proportion of unlabeled images in the unlabeled image set is set to 90% of the total number of sample images. Before training the pre-built image segmentation model, the sample images in the sample image set need to be adjusted to a uniform resolution (e.g., 224×224 resolution), and the coordinate information needs to be adjusted proportionally to ensure that the sample images input to the pre-built image segmentation model have consistent size and coordinate standards, thereby avoiding model training problems caused by inconsistent sample image sizes.

[0027] In one embodiment of this application, in order to reduce the computational burden and improve training efficiency, all sample images are uniformly scaled to a size of 320×640. This operation helps to reduce the consumption of computing resources, optimize memory usage, reduce GPU burden, and thus accelerate the training process, which helps to improve the training efficiency and segmentation accuracy of the pre-built image segmentation model.

[0028] A pre-built image segmentation model is trained using a labeled image set to obtain a teacher model. The sample image set is then divided into K training and validation sets using K-fold cross-validation. Multiple student models are obtained by training the pre-built image segmentation model using the training and validation sets from these K combinations. In one embodiment of this application, training the pre-built image segmentation model using the training and validation sets from the K combinations allows the student models to approximate the teacher model, improving their accuracy and precision. Simultaneously, to ensure the accuracy of the student models, they are periodically updated and optimized to enhance their ability to capture key features in tooth images, thus ensuring the stability of the student models' segmentation of tooth images.

[0029] The unlabeled images in the unlabeled image set are enhanced, and the enhanced unlabeled images are input into the teacher model to obtain multiple first pseudo-labeled images. In one embodiment of this application, the enhancement operation includes: vertical flipping operation, histogram equalization, and also rotation, scaling, etc. By enhancing the unlabeled images in the unlabeled image set, the quality of the unlabeled images in the unlabeled image set is improved.

[0030] Multiple first pseudo-label images are input into each student model to obtain multiple second pseudo-label images; these second pseudo-label images are then fused to obtain a third pseudo-label image. In one embodiment of this application, the fusion operation includes pixel averaging, binarization, and morphological operations. The process of fusing multiple second pseudo-label images includes: performing pixel-by-pixel averaging on the multiple second pseudo-label images to obtain a fused image with pseudo-labels; performing binarization on the fused image with pseudo-labels to obtain a binarized fused image; and performing morphological operations (e.g., dilation, erosion, opening, closing, etc.) on the binarized fused image to obtain the third pseudo-label image. These operations improve the quality of the third pseudo-label image and achieve accurate segmentation of the pseudo-label image.

[0031] Multiple first pseudo-labeled images, third pseudo-labeled images, and labeled images from the labeled image set are combined to obtain an updated image set. The pre-built image segmentation model is then trained using this updated image set to obtain a tooth image segmentation model. In one embodiment of this application, with only a small number of labeled images, multiple first pseudo-labeled images are obtained through inference on the enhanced unlabeled images using a teacher model. Multiple second pseudo-labeled images are obtained through cross-validation of these first pseudo-labeled images using a student model. Finally, a third pseudo-labeled image is obtained by fusing these second pseudo-labeled images, thus achieving the transformation from unlabeled images to pseudo-labeled images. This not only improves the quality of the pseudo-labeled images but also continuously enhances the generalization ability of the pre-built image segmentation model during training using the updated image set.

[0032] In one embodiment of this application, before obtaining a sample image set for training a pre-built image segmentation model, the tooth image segmentation method based on semi-supervised learning and nnU-Net further includes: A portion of the images in the sample image set are selected as unlabeled images; the remaining images in the sample image set are selected as images to be labeled; the set of unlabeled images is selected as the unlabeled image set. In one embodiment of this application, the ratio of unlabeled images to images to be labeled in the sample image set can be set according to actual conditions. For example, the proportion of unlabeled images in the sample image set is 90%, and the proportion of images to be labeled in the sample image set is 10%.

[0033] The image to be labeled is converted into a preset-bit single-channel grayscale image. In one embodiment of this application, the pixel value of each pixel in the preset-bit single-channel grayscale image is set to a preset pixel value; the preset-bit single-channel grayscale image is an 8-bit single-channel grayscale image, and the preset pixel value is set to 0.

[0034] The process involves obtaining the contour point information and classification category of each tooth in the image to be labeled. Based on the contour point information of each tooth in the image to be labeled, the pixel region of each tooth in a preset single-channel grayscale image is determined. According to the correspondence between classification category and classification label, the pixel region of each tooth in the preset single-channel grayscale image is filled with the corresponding classification label to obtain a labeled image. The set of labeled images is then used as the labeled image set. In one embodiment of this application, the classification label is represented in numerical form, for example, 1, 2, 3. The contour point information and classification category of each tooth in the image to be labeled are extracted from JSON annotations. The JSON annotations store the contour point information and classification category of each tooth in the image to be labeled. The contour point information and classification category of each tooth in the image to be labeled are annotated using the VGG ImageAnnotator (VIA) image annotation tool and saved in JSON format in the JSON annotations. The tooth classification categories include: anterior teeth, posterior teeth, supernumerary teeth, caries, etc. Each classification category is assigned a classification label number, thus obtaining the correspondence between classification categories and classification labels.

[0035] In one embodiment of this application, after inputting the enhanced unlabeled image into the teacher model to obtain multiple first pseudo-labeled images, the tooth image segmentation method based on semi-supervised learning and nnU-Net further includes: Binarization is performed on multiple first pseudo-label images to obtain multiple binarized images. In one embodiment of this application, the process of binarizing multiple first pseudo-label images includes: setting 0.5 as a pixel threshold, treating pixels with a value greater than or equal to 0.5 in each first pseudo-label image as tooth pixels, and treating pixels with a value less than 0.5 in each first pseudo-label image as background pixels, thereby determining the tooth pixels and background pixels in each first pseudo-label image, and setting the pixel value of the tooth pixels to 0 and the pixel value of the background pixels to 1. The above operations help to distinguish the tooth region from the background region.

[0036] Based on multiple binarized images, the connected regions of each binarized image are determined, and the contour of each tooth in each binarized image is determined according to the connected regions of each binarized image. In one embodiment of this application, the number of pixels in the contour of each tooth is greater than or equal to a preset number of pixels; the preset number of pixels is set to 45, and the contour of each tooth has a unique number. The method for determining the connected regions of each binarized image based on multiple binarized images is either the four-connectivity method or the eight-connectivity method. The process of determining the contour of each tooth in each binarized image according to the connected regions of each binarized image is the process of extracting the contour of the connected regions of each binarized image. The process of extracting the contour of the connected regions of each binarized image can be implemented by image edge detection algorithms (e.g., Canny edge detection algorithm, adaptive threshold segmentation algorithm) and boundary tracking algorithms (e.g., Suzuki algorithm, Boundary Tracing).

[0037] Based on the correspondence between the category number and contour number of each tooth in each binarized image, the contour of each tooth in each binarized image is converted into a tooth category number, resulting in multiple fourth pseudo-label images. These fourth pseudo-label images are then used to update multiple first pseudo-label images, which are then input into each student model. In one embodiment of this application, there is a one-to-one correspondence between the tooth category and the tooth contour. After obtaining the contour of each tooth, the corresponding contour number and category number are determined based on the contour. Updating the multiple first pseudo-label images with multiple fourth pseudo-label images and then inputting the updated first pseudo-label images into each student model improves the quality of the pseudo-label images. Compared to multiple first pseudo-label images, multiple fourth pseudo-label images provide more accurate segmentation label information, which is beneficial for improving the segmentation accuracy and reliability of the pre-built image segmentation model during subsequent training.

[0038] In one embodiment of this application, the process of training a pre-built image segmentation model by updating the image set to obtain a tooth image segmentation model includes: The process of inputting sample images from the updated image set into a pre-built image segmentation model to obtain predicted images of the sample images includes the following steps: In one embodiment of this application, if the pre-built image segmentation model includes an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: the input layer receives the sample images and converts them into images of a preset size; the multiple feature extraction branches extract features of different dimensions from the preset size image to obtain feature maps of different dimensions; the feature fusion layer fuses the feature maps of different dimensions to obtain a fused feature map; and the output layer converts the fused feature map into a feature vector of a preset dimension to obtain the predicted image. This process fully utilizes feature information of different dimensions, enabling the pre-built image segmentation model to have stronger feature extraction capabilities when processing complex tooth images. By effectively fusing these features of different dimensions, a more representative and discriminative fused feature map is formed. By converting the fused feature map into a feature vector of a preset dimension, a predicted image is obtained. This predicted image can accurately reflect the segmentation of the teeth, providing a reliable basis for subsequent model training and optimization.

[0039] The goal is to minimize the error between the sample image and the predicted image. The parameters of a pre-built image segmentation model are updated until the error between the predicted image and the sample image output by the updated image segmentation model is less than a preset error threshold, or the number of updates to the parameters of the pre-built image segmentation model is greater than or equal to a preset number of updates threshold. In one embodiment of this application, both the preset error threshold and the preset number of updates threshold are set according to actual conditions. The error between the sample image and the predicted image is represented by a loss function, the expression of which is shown below: Equation (1) in, This represents the error between the sample image and the predicted image. Indicates the number of sample images. Indicates the first One sample image, Indicates the first The predicted image of the nth sample image, the nth The sample image and the first The predicted image for each sample image is represented in the form of a binary vector. Indicates the first Labels for each sample image. Indicates the first The predicted probability of a sample image. This represents the loss weight of the Dice similarity term coefficient. The weights represent the binary cross-entropy loss. .

[0040] In one embodiment of this application, a loss function for training a pre-built image segmentation model is constructed using Dice similarity coefficient loss and binary cross-entropy loss. This is beneficial for enhancing the robustness of training the pre-built image segmentation model and continuously improving the performance and accuracy of the pre-built image segmentation model in tooth segmentation scenarios.

[0041] In one embodiment of this application, a pre-built image segmentation model is trained using a labeled image set to obtain the loss function of the teacher model. The pre-built image segmentation model is trained using the training set and validation set in K combinations to obtain the loss functions of multiple student models. All of these loss functions can be measured using the loss function of formula (1) to measure the error between the sample data and the predicted data.

[0042] In one embodiment of this application, if the pre-built image segmentation model includes an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, then the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: The input layer is used to receive sample images and convert them into images of a preset size. In one embodiment of this application, the preset size can be 360*640, or it can be other sizes.

[0043] Multiple feature extraction branches are used to extract features of different dimensions from an image of a preset size, resulting in feature maps of different dimensions. In one embodiment of this application, the different dimensions include: 16 channels, 32 channels, 64 channels, 128 channels, 256 channels, etc. By extracting features of different dimensions from an image of a preset size, the richness of features is improved.

[0044] The feature fusion layer is used to fuse feature maps of different dimensions to obtain a fused feature map. In one embodiment of this application, the feature fusion layer includes: a feature concatenation layer and two 3*3 convolutional layers.

[0045] The output layer is used to convert the fused feature map into a feature vector of a preset dimension to obtain the predicted image. In one embodiment of this application, the preset dimension feature vector is a one-dimensional feature vector, which can concisely and efficiently express the key information in the fused feature map. The one-dimensional feature vector is transformed according to the spatial dimensions of the sample image to obtain the predicted image. After obtaining the predicted image, the labels of the predicted image and the sample image can be compared and analyzed to evaluate the performance of the pre-built image segmentation model, which facilitates subsequent optimization and adjustment of the pre-built image segmentation model to continuously improve the segmentation accuracy and reliability of the model.

[0046] In one embodiment of this application, the plurality of feature extraction branches include: a first feature extraction branch, a second feature extraction branch, a third feature extraction branch, a fourth feature extraction branch, a fifth feature extraction branch, and a sixth feature extraction branch; The first feature extraction branch is used to extract features of a first preset dimension from an image of a preset size, and to compress these features to obtain a feature map of the first preset dimension. In one embodiment of this application, the first preset dimension has 16 channels. The compression of the first preset dimension features is implemented using the RFEM (Refined Feature Enhancement Module), which helps to enhance the features in the image of the preset size, accurately focus on the key tooth region in the image of the preset size, reduce interference from irrelevant backgrounds, and thus improve the segmentation effect of the pre-built image segmentation model in complex backgrounds.

[0047] The second feature extraction branch is used to extract spatial and channel features from the first preset dimension features to obtain a second preset dimension feature map. In one embodiment of this application, the second preset dimension is equal to the first preset dimension. The process of extracting spatial and channel features from the first preset dimension features is implemented by the CBAM (Convolutional Block Attention Module) module. The CBAM module can automatically assign different attention weights to each convolutional block within the CBAM module, thereby highlighting important features in the preset size image and further mitigating the impact of boundary blurring and uncertainty.

[0048] The third feature extraction branch is used to extract third preset dimension features from the first preset dimension features, and to extract spatial and channel features from the third preset dimension features to obtain first spatial-channel features. The first spatial-channel features are then fused, upsampled, and compressed to obtain a third preset dimension feature map. In one embodiment of this application, the third preset dimension is larger than the second preset dimension, and the third preset dimension has 32 channels. The third feature extraction branch includes: a second feature extraction module, a second CBAM module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module. By setting the second CBAM module and the second RFEM module in the third feature extraction branch, the ability of the pre-built image segmentation model to capture and optimize the third preset dimension features is further enhanced.

[0049] The fourth feature extraction branch is used to extract features of the fourth preset dimension from the features of the third preset dimension, and to extract spatial and channel features from the features of the fourth preset dimension to obtain a second spatial-channel feature map. The second spatial-channel feature map is then subjected to fusion, upsampling, and compression operations to obtain the feature map of the fourth preset dimension. In one embodiment of this application, the fourth preset dimension is larger than the third preset dimension, and the fourth preset dimension has 64 channels. The fourth feature extraction branch includes: a third feature extraction module, a third CBAM module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module. By setting the second CBAM module and the second RFEM module in the third feature extraction branch, the ability of the pre-built image segmentation model to capture and optimize features of the fourth preset dimension is further enhanced.

[0050] The fifth feature extraction branch is used to extract features of the fifth preset dimension from the features of the fourth preset dimension, and to perform fusion, upsampling, and compression operations on the features of the fifth preset dimension to obtain a feature map of the fifth preset dimension. In one embodiment of this application, the fifth preset dimension is larger than the fourth preset dimension, and the fifth preset dimension has 128 channels. The fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module. By setting the fourth RFEM module in the fifth feature extraction branch, the features of the fifth preset dimension can be refined, effectively improving the model's ability to resolve high-resolution features. At the same time, the sixth transposed convolution module ensures the integrity of the information in the spatial dimension of the feature map through progressive upsampling operations.

[0051] The sixth feature extraction branch is used to extract features of the sixth preset dimension from the fifth preset dimension feature map, and to perform upsampling and compression operations on the sixth preset dimension features to obtain the sixth preset dimension feature map. In one embodiment of this application, the sixth preset dimension is larger than the fifth preset dimension, and the sixth preset dimension has 256 channels. The sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module includes two 3*3 convolutional layers. The convolution module is used to extract feature information from the fifth preset dimension feature map, providing a feature basis for subsequent processing.

[0052] The first, second, third, fourth, fifth, and sixth preset dimension feature maps are combined to form feature maps of different dimensions. In one embodiment of this application, the feature maps of different dimensions encompass rich feature information in an image of a preset size. By combining feature maps of different dimensions, the advantages of each dimension can be fully utilized to achieve complementarity of multi-dimensional information. When training a pre-built image segmentation model, the feature maps of different dimensions not only enhance the pre-built image segmentation model's ability to capture detailed features such as tooth edges and textures, but also improve the recognition accuracy of the pre-built image segmentation model for tooth regions in complex backgrounds. This results in the pre-built image segmentation model having higher robustness and accuracy in tooth image segmentation after training.

[0053] In one embodiment of this application, the first feature extraction branch includes a first feature extraction module and a first RFEM module. In another embodiment, the first feature extraction module extracts a first preset dimension feature from an image of a preset size; the first RFEM module compresses the first preset dimension feature to obtain a first preset dimension feature map. The first feature extraction module includes two 3*3 convolutional layers and one feature concatenation layer. The first feature extraction module performs layer-by-layer convolution operations on the image of the preset size through its internal convolutional layers, thereby capturing the first preset dimension feature in the image of the preset size. After processing by the first RFEM module, these features are further compressed and optimized to generate a first preset dimension feature map with stronger expressive power. The combination of the first feature extraction module and the first RFEM module can efficiently extract and compress key features in an image of a preset size, helping to improve the accuracy of subsequent image segmentation.

[0054] The second feature extraction branch includes a first CBAM module. The first CBAM module is used to extract spatial and channel features from the first preset dimension features to obtain a second preset dimension feature map. In one embodiment of this application, the first CBAM module is composed of an attention mechanism, which internally includes a channel attention submodule and a spatial attention submodule. The channel attention submodule mainly focuses on the importance of features in different dimensions of the image, while the spatial attention submodule focuses on features at different spatial locations in the image. By assigning different attention weights to each spatial location, it highlights the features of key locations such as the area where the teeth are located and the edges of the teeth. The first CBAM module organically combines the channel attention submodule and the spatial attention submodule to perform comprehensive feature extraction of the first preset dimension features, thereby obtaining a second preset dimension feature map that accurately reflects the features of the teeth.

[0055] The third feature extraction branch includes: a second feature extraction module, a second CBAM module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module. The second feature extraction module is used to extract a third preset dimension feature from a first preset dimension feature. The second CBAM module is used to extract spatial and channel features from the third preset dimension feature to obtain a first spatial-channel feature. The first feature fusion module is used to fuse the first spatial-channel feature and the fourth preset dimension feature map to obtain a first fused feature. The first transposed convolution module is used to upsample the first fused feature to obtain a first upsampled feature. The second RFEM module is used to compress the first upsampled feature to obtain a first compressed feature. The second transposed convolution module is used to upsample the first compressed feature to obtain a third preset dimension feature map. The feature fusion layer is used to fuse the first preset dimension feature map, the second preset dimension feature map, and the third preset dimension feature map. In one embodiment of this application, a 2*2 pooling layer is set between the second feature extraction module and the first feature extraction module. By combining the pooling layer, the second feature extraction module, the second CBAM module, the first feature fusion module, the first transposed convolution module, the second RFEM module, and the second transposed convolution module, not only is the model's ability to capture detailed features such as tooth edges and textures enhanced, but the recognition accuracy of the pre-built image segmentation model for tooth regions in complex backgrounds is also improved, making the pre-built image segmentation model more robust and accurate for tooth image segmentation tasks.

[0056] The fourth feature extraction branch includes: a third feature extraction module, a third CBAM module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module. The third feature extraction module extracts features from the third preset dimension features into the fourth preset dimension features. The third CBAM module extracts spatial and channel features from the fourth preset dimension features to obtain a second spatial-channel feature map. The second feature fusion module fuses the second spatial-channel feature map and the fifth preset dimension feature map to obtain a second fused feature map. The third transposed convolution module upsamples the second fused features to obtain... The second upsampled feature map is obtained; the third RFEM module is used to compress the second upsampled feature map to obtain the second compressed feature map; the fourth transposed convolution module is used to upsample the second compressed feature map to obtain the fourth preset dimension feature map. In one embodiment of this application, a 2*2 pooling layer is set between the third feature extraction module and the second feature extraction module. By combining the pooling layer, the third feature extraction module, the third CBAM module, the second feature fusion module, the third transposed convolution module, the third RFEM module and the fourth transposed convolution module, the performance and accuracy of the pre-built image segmentation model on the tooth image segmentation task are significantly improved.

[0057] The fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module. The fourth feature extraction module extracts features of a fifth preset dimension from features of a fourth preset dimension, which are used as pooling layer feature maps. The third feature fusion module fuses the pooling layer feature maps and the sixth preset dimension feature maps to obtain a third fused feature map. The fifth transposed convolution module upsamples the third fused feature map to obtain a third upsampled feature map. The fourth RFEM module compresses the third upsampled feature map to obtain a third compressed feature map. The sixth transposed convolution module upsamples the third compressed feature map to obtain a fifth preset dimension feature map. In one embodiment of this application, a 2*2 pooling layer is set between the fourth and third feature extraction modules. Through the combination of the pooling layer, the fourth feature extraction module, the third feature fusion module, the fifth transposed convolution module, the fourth RFEM module, and the sixth transposed convolution module, the adaptability and segmentation accuracy of the pre-built image segmentation model in complex dental image scenarios are greatly improved, providing more reliable and efficient technical support for clinical dental image analysis.

[0058] The sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module is used to extract the sixth preset dimension features from the fifth preset dimension feature map; the seventh transposed convolution module is used to upsample the sixth preset dimension features to obtain the fourth upsampled feature map; the fifth RFEM module is used to compress the fourth upsampled feature map to obtain the fourth compressed feature map; and the eighth transposed convolution module is used to upsample the fourth compressed feature map to obtain the sixth preset dimension feature map. In one embodiment of this application, the convolution module includes two 3*3 convolutional layers, and a 2*2 pooling layer is set between the convolution module and the fourth feature extraction module. By combining the pooling layer, the convolution module, the seventh transposed convolution module, the fifth RFEM module, and the eighth transposed convolution module, the pre-built image segmentation model's ability to perceive the fine structures in tooth images is further enhanced. When processing images with complex tooth shapes and blurred boundaries between adjacent teeth, it can more accurately delineate the tooth contours and effectively distinguish different individual teeth. At the same time, it also enhances the pre-built image segmentation model's resistance to image noise and interference factors. Even under adverse conditions such as a certain degree of image blurring and uneven lighting, it can still maintain high segmentation accuracy and stability. Furthermore, through the collaborative work of multiple modules, tooth image features are extracted, fused, and optimized from different angles and levels, so that the final sixth preset dimension feature map can comprehensively and accurately reflect various key information of the tooth image, providing a solid and reliable feature foundation for subsequent tooth image segmentation tasks. This significantly improves the performance and effect of tooth image segmentation, enabling it to better meet actual needs in clinical applications and provide stronger support for the diagnosis and treatment of dental diseases.

[0059] In one embodiment of this application, all feature extraction modules include two 3*3 convolutional layers and one splicing layer, and all feature fusion modules have the same structure as the feature fusion layer, including one splicing layer and two 3*3 convolutional layers.

[0060] Figure 3 This is a flowchart illustrating another exemplary embodiment of the present application of a tooth image segmentation method based on semi-supervised learning and nnU-Net. Figure 3In this paper, the tooth image segmentation method based on semi-supervised learning and nnU-Net includes: (1) training a pre-built image segmentation model with a set of labeled images to obtain a teacher model, where Ground Truth represents the real label data in the labeled images; (2) performing enhancement operations on the unlabeled images in the unlabeled image set, and inputting the enhanced unlabeled images into the teacher model to obtain multiple first pseudo-labeled images; (3) dividing the sample image set into K training and validation sets according to K-fold cross-validation; and training the pre-built image segmentation model with the training and validation sets in the K combinations to obtain multiple student models (N student models); (4) inputting multiple first pseudo-labeled images into each student model to obtain multiple second pseudo-labeled images (Pred 1-Pred 2). K); and perform a fusion operation on multiple second pseudo-label images to obtain a third pseudo-label image; the fusion operation includes: pixel averaging operation, binarization operation and morphological operation; (5) combine multiple first pseudo-label images, third pseudo-label images and labeled images in the labeled image set to obtain an updated image set; train the pre-built image segmentation model through the updated image set (final training) to obtain a tooth image segmentation model.

[0061] Figure 4 This is a schematic diagram illustrating the structure of a pre-built image segmentation model as shown in an exemplary embodiment of this application. Figure 4 The pre-built image segmentation model includes: an input layer, a feature fusion layer, an output layer, a first feature extraction branch, a second feature extraction branch, a third feature extraction branch, a fourth feature extraction branch, a fifth feature extraction branch, and a sixth feature extraction branch. The first, second, third, fourth, fifth, and sixth feature extraction branches are arranged sequentially from top to bottom. The first feature extraction branch includes: a first feature extraction module and a first RFEM module; the second feature extraction branch includes: a first CBAM module; the third feature extraction branch includes: a second... The system comprises: a feature extraction module, a second CBAM module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module; a fourth feature extraction branch includes: a third feature extraction module, a third CBAM module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module; a fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module; and a sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module.

[0062] In one embodiment of this application, a first feature extraction module extracts channel feature information from an image of a preset size according to a 16-channel dimension, a second feature extraction module extracts channel feature information from an image of a preset size according to a 32-channel dimension, and the 16-channel and 32-channel feature information are finally fused at a feature fusion layer; a third feature extraction module extracts channel feature information from an image of a preset size according to a 64-channel dimension; the 32-channel and 64-channel feature information are fused at a first feature fusion module; a fourth feature extraction module extracts channel feature information from an image of a preset size according to a 128-channel dimension; the 128-channel and 64-channel feature information are fused at a second feature fusion module; and a convolution module extracts channel feature information from an image of a preset size according to a 256-channel dimension, and the 256-channel and 128-channel feature information are fused at a third feature fusion module. The above methods achieve the fusion of 16-channel, 32-channel, 64-channel, 128-channel, and 256-channel feature information.

[0063] This application uses a computer with 64GB of solid-state drive memory, an NVIDIA RTX 4090 graphics card, and an Intel(R) Core(TM) i9-10900K @ 3.70GHz processor. The software environment used includes a Python 3.9.0 operating system, a PyTorch 2.0.0 deep learning framework, and CUDA version 11.8 as the experimental environment for training the pre-built image segmentation model. This application selects training, validation, and test sets from the STS-2D-Tooth dataset to train, validate, and test the pre-built image segmentation model. The training set contains 2380 panoramic dental X-ray images (including 30 labeled images and 2350 unlabeled images), the validation set contains 100 panoramic dental X-ray images (including 50 labeled images and 50 unlabeled images), and the test set contains 20 panoramic dental X-ray images (including 0 labeled images and 20 unlabeled images).

[0064] Figure 5 This is a schematic diagram illustrating the change curve of the loss function value during the training process of a pre-built image segmentation model, as shown in an exemplary embodiment of this application. Figure 5 In the diagram, the horizontal axis represents the training epochs of the pre-built image segmentation model, and the vertical axis represents the loss function value calculated using formula (1). Figure 5As can be seen, with the increase of training rounds, the training loss gradually decreases from about 0.3804 to 0.0082, while the validation loss decreases from about 0.4204 to 0.01. Throughout the training process, both the training loss curve and the validation loss curve show a smooth downward trend with the increase of training rounds, and the downward trends of the training loss curve and the validation loss curve are consistent, indicating that the pre-built image segmentation model has been effectively optimized and overfitting has been avoided during the training process.

[0065] Figure 6 This is a schematic diagram illustrating the change curve of the Dice similarity coefficient during the training of a pre-built image segmentation model, as shown in an exemplary embodiment of this application. Figure 6 In the diagram, the horizontal axis represents the training epochs of the pre-built image segmentation model, and the vertical axis represents the Dice similarity coefficient value. Figure 6 As can be seen, the Dice similarity coefficient value steadily increases with the increase of training rounds. The Dice similarity coefficient value of the training set increases from about 0.3 to 0.92, while the Dice similarity coefficient value of the validation set eventually reaches about 0.89. Moreover, the gap between the Dice similarity coefficient value change curve of the training set and the Dice similarity coefficient value of the validation set remains within the predetermined range, which further verifies the good generalization ability of the image segmentation model during training and the image segmentation model after training on unseen data.

[0066] Combination Figure 5 and Figure 6 It can be seen that the trained image segmentation model has good training effect and can resist the decline in validation performance while maintaining high segmentation accuracy. Therefore, it can have higher segmentation accuracy when facing complex anatomical structures and overlapping areas in dental panoramic X-ray images.

[0067] Figure 7 This is a schematic diagram illustrating the effect of tooth image segmentation in an exemplary embodiment of this application. Figure 7The results of segmenting the same image using the FixMatch method, CPS (Cross Pseudo Supervision), MT (Mersenne Twister) method, and DeepLab V3+ method in related technologies are compared with those of the image segmentation method in this application. The image segmentation method in this application achieves high-precision segmentation on most tooth instances, with complete contours and clear instance boundaries, accurately distinguishing adjacent individual teeth. This demonstrates that the image segmentation method in this application has strong segmentation capabilities in densely structured regions and complex morphological backgrounds. However, the FixMatch method, CPS (Cross Pseudo Supervision), MT (Mersenne Twister) method, and DeepLab V3+ method in related technologies... In some cases, the V3+ method still has certain segmentation errors (as marked by the red circle), mainly including the following situations: (1) Due to the high degree of overlap between teeth or the low contrast of X-ray images, the determination of the boundary of individual teeth is blurred, causing instance fusion or edge contour shift; (2) Some root areas have incomplete instance separation due to bone overlap or image artifact interference; (3) Individual small teeth or abnormal teeth have limited recognition ability due to the scarcity of training samples.

[0068] Figure 8 This is a schematic diagram illustrating the effect of tooth image segmentation, shown in another exemplary embodiment of this application. Figure 8 The paper demonstrates the segmentation performance of the tooth image segmentation model in this application for difficult cases. Compared with the mask image of the image to be segmented, the segmented image output by the tooth image segmentation model in this application shows segmentation deviation at only a few locations. Furthermore, the Dice similarity coefficient between the segmented image output by the tooth image segmentation model in this application and the mask image of the image to be segmented reaches 92.05%, the Normalized Surface Distance (NSD) reaches 94.72%, the Intersection over Union (IoU) reaches 86.27%, and the Recognition Accuracy (IA) reaches 85.48%.

[0069] The following describes an apparatus embodiment of this application, which can be used to execute the tooth image segmentation method based on semi-supervised learning and nnU-Net described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the tooth image segmentation method based on semi-supervised learning and nnU-Net described above.

[0070] Figure 9This is a block diagram illustrating a tooth image segmentation apparatus based on semi-supervised learning and nnU-Net, as shown in an exemplary embodiment of this application.

[0071] like Figure 9 As shown, this exemplary tooth image segmentation device 900 based on semi-supervised learning and nnU-Net includes: The image acquisition module 901 is used to acquire images of teeth to be segmented and to preprocess these images to obtain preprocessed images.

[0072] The image segmentation module 902 is used to input the preprocessed image into the tooth image segmentation model to obtain the tooth segmentation result.

[0073] In one embodiment of this application, the tooth image to be segmented is an X-ray image. Preprocessing methods include image resizing and resolution adjustment, which improve the quality of the tooth image to be segmented and adapt it to the size requirements of the input image for the tooth image segmentation model.

[0074] In one embodiment of this application, the tooth image segmentation model is trained on a pre-built image segmentation model using a semi-supervised learning mechanism. The pre-built image segmentation model is constructed based on the nnU-Net structure and attention mechanism, which reduces the dependence of the pre-built image segmentation model on labeled data during training. That is, the pre-built image segmentation model can be trained using a combination of labeled and unlabeled data, reducing data labeling costs and improving the performance of tooth image segmentation in cases of complex backgrounds, overlapping structures, and unclear boundaries.

[0075] It should be noted that the tooth image segmentation device based on semi-supervised learning and nnU-Net provided in the above embodiments and the tooth image segmentation method based on semi-supervised learning and nnU-Net provided in the above embodiments belong to the same concept. The specific way in which each module and unit performs its operation has been described in detail in the method embodiments, and will not be repeated here. In practical applications, the tooth image segmentation device based on semi-supervised learning and nnU-Net provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0076] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A tooth image segmentation method based on semi-supervised learning and nnU-Net, characterized in that, The method includes: Acquire an image of the tooth to be segmented, and preprocess the image to obtain a preprocessed image; the preprocessing method includes: image size conversion; The preprocessed image is input into the tooth image segmentation model to obtain the tooth segmentation result; the tooth image segmentation model is trained on a pre-constructed image segmentation model using a semi-supervised learning mechanism; the pre-constructed image segmentation model is built based on the nnU-Net structure and attention mechanism. The process of training a pre-built image segmentation model using a semi-supervised learning mechanism to obtain the tooth image segmentation model includes: Obtain a set of sample images for training the pre-constructed image segmentation model; the set of sample images includes: a set of labeled images and a set of unlabeled images; The pre-built image segmentation model is trained using the labeled image set to obtain a teacher model; the sample image set is divided into K training and validation sets using K-fold cross-validation; and the pre-built image segmentation model is trained using the training and validation sets in the K combinations to obtain multiple student models. The unlabeled images in the unlabeled image set are enhanced, and the enhanced unlabeled images are input into the teacher model to obtain multiple first pseudo-labeled images; the enhancement operation includes: vertical flip operation and histogram equalization; Multiple first pseudo-label images are input into each student model to obtain multiple second pseudo-label images; and the multiple second pseudo-label images are fused to obtain a third pseudo-label image; the fusion operation includes: pixel averaging, binarization and morphological operations; Multiple first pseudo-labeled images, the third pseudo-labeled image, and labeled images from the labeled image set are combined to obtain an updated image set; the pre-built image segmentation model is trained using the updated image set to obtain the tooth image segmentation model.

2. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 1, characterized in that, Before obtaining the sample image set for training the pre-built image segmentation model, the method further includes: A subset of images from the sample image set is selected as unlabeled images; the remaining images from the sample image set are selected as images to be labeled; the set of unlabeled images is selected as the unlabeled image set. The image to be labeled is converted into a preset-bit single-channel grayscale image; the pixel value of each pixel in the preset-bit single-channel grayscale image is set to a preset pixel value. The contour point information and classification category of each tooth in the image to be labeled are obtained. Based on the contour point information of each tooth in the image to be labeled, the pixel region of each tooth in the preset single-channel grayscale image is determined. According to the correspondence between the classification category and the classification label, the pixel region of each tooth in the preset single-channel grayscale image is filled with the corresponding classification label to obtain a labeled image. The set of labeled images is used as the labeled image set. The classification label is represented in numerical form.

3. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 1, characterized in that, After inputting the enhanced unlabeled image into the teacher model to obtain multiple first pseudo-labeled images, the method further includes: Binarization is performed on multiple first pseudo-label images to obtain multiple binarized images; Based on multiple binarized images, the connected regions of each binarized image are determined, and the contour of each tooth in each binarized image is determined according to the connected regions of each binarized image; the number of pixels in the contour of each tooth is greater than or equal to the preset number of pixels; the contour of each tooth has a unique number. Based on the correspondence between the category number and the contour number of each tooth in each binarized image, the contour of each tooth in each binarized image is converted into the category number of the tooth, resulting in multiple fourth pseudo-label images. Multiple first pseudo-label images are then updated using these multiple fourth pseudo-label images, and the updated multiple first pseudo-label images are input into each student model.

4. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 1, characterized in that, The process of training the pre-built image segmentation model using the updated image set to obtain the tooth image segmentation model includes: The sample images from the updated image set are input into the pre-built image segmentation model to obtain the predicted images of the sample images; With the goal of minimizing the error between the sample image and the predicted image, the parameters in the pre-built image segmentation model are updated until the error between the predicted image output by the updated image segmentation model and the sample image is less than a preset error threshold, or the number of times the parameters in the pre-built image segmentation model are updated is greater than or equal to a preset number threshold.

5. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 4, characterized in that, The error between the sample image and the predicted image is represented by a loss function, the expression of which is as follows: ， in, This represents the error between the sample image and the predicted image. Indicates the number of sample images. Indicates the first One sample image, Indicates the first The predicted image of the nth sample image, the nth The sample image and the first The predicted image for each sample image is represented in the form of a binary vector. Indicates the first Labels for each sample image. Indicates the first The predicted probability of a sample image. This represents the loss weight of the Dice similarity term coefficient. The weights represent the binary cross-entropy loss. .

6. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 4, characterized in that, If the pre-built image segmentation model includes an input layer, multiple feature extraction branches, a feature fusion layer, and an output layer, then the process of inputting sample images from the updated image set into the pre-built image segmentation model to obtain predicted images of the sample images includes: The input layer is used to receive the sample image and convert the sample image into an image of a preset size; Multiple feature extraction branches are used to extract features of different dimensions from the image of the preset size, resulting in feature maps of different dimensions; The feature fusion layer is used to fuse feature maps of different dimensions to obtain a fused feature map; The output layer is used to convert the fused feature map into a feature vector of a preset dimension to obtain the predicted image.

7. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 6, characterized in that, The multiple feature extraction branches include: the first feature extraction branch, the second feature extraction branch, the third feature extraction branch, the fourth feature extraction branch, the fifth feature extraction branch, and the sixth feature extraction branch; The first feature extraction branch is used to extract the first preset dimension features in the preset size image and compress the first preset dimension features to obtain the first preset dimension feature map; The second feature extraction branch is used to extract spatial and channel features from the first preset dimension features to obtain a second preset dimension feature map; the second preset dimension is equal to the first preset dimension. The third feature extraction branch is used to extract a third preset dimension feature from the first preset dimension feature, and to extract spatial and channel features from the third preset dimension feature to obtain a first spatial-channel feature. The first spatial-channel feature is then subjected to fusion, upsampling, and compression operations to obtain a third preset dimension feature map; the third preset dimension is greater than the second preset dimension. The fourth feature extraction branch is used to extract the fourth preset dimension feature from the third preset dimension feature, and to extract the spatial feature and channel feature of the fourth preset dimension feature to obtain the second spatial-channel feature map. The second spatial-channel feature map is then subjected to fusion operation, upsampling operation and compression operation to obtain the fourth preset dimension feature map; the fourth preset dimension is greater than the third preset dimension. The fifth feature extraction branch is used to extract the fifth preset dimension feature from the fourth preset dimension feature, and to perform fusion, upsampling and compression operations on the fifth preset dimension feature to obtain the fifth preset dimension feature map; the fifth preset dimension is greater than the fourth preset dimension. The sixth feature extraction branch is used to extract the sixth preset dimension feature from the fifth preset dimension feature map, and to perform upsampling and compression operations on the sixth preset dimension feature to obtain the sixth preset dimension feature map; the sixth preset dimension is greater than the fifth preset dimension; The first preset dimension feature map, the second preset dimension feature map, the third preset dimension feature map, the fourth preset dimension feature map, the fifth preset dimension feature map, and the sixth preset dimension feature map are combined to form different dimension feature maps.

8. The tooth image segmentation method based on semi-supervised learning and nnU-Net according to claim 7, characterized in that, The first feature extraction branch includes: a first feature extraction module and a first RFEM module; the first feature extraction module is used to extract a first preset dimension feature from the preset size image; the first RFEM module is used to compress the first preset dimension feature to obtain the first preset dimension feature map; The second feature extraction branch includes: a first CBAM module; the first CBAM module is used to extract spatial features and channel features from the first preset dimension features to obtain the second preset dimension feature map; The third feature extraction branch includes: a second feature extraction module, a second CBAM module, a first feature fusion module, a first transposed convolution module, a second RFEM module, and a second transposed convolution module; the second feature extraction module is used to extract a third preset dimension feature from the first preset dimension feature; the second CBAM module is used to extract spatial and channel features from the third preset dimension feature to obtain the first spatial-channel feature; the first feature fusion module is used to fuse the first spatial-channel feature and the fourth preset dimension feature map to obtain a first fused feature; the first transposed convolution module is used to upsample the first fused feature to obtain a first upsampled feature; the second RFEM module is used to compress the first upsampled feature to obtain a first compressed feature; the second transposed convolution module is used to upsample the first compressed feature to obtain the third preset dimension feature map; the feature fusion layer is used to fuse the first preset dimension feature map, the second preset dimension feature map, and the third preset dimension feature map. The fourth feature extraction branch includes: a third feature extraction module, a third CBAM module, a second feature fusion module, a third transposed convolution module, a third RFEM module, and a fourth transposed convolution module; the third feature extraction module is used to extract a fourth preset dimension feature from the third preset dimension feature; the third CBAM module is used to extract spatial and channel features from the fourth preset dimension feature to obtain a second spatial-channel feature map; the second feature fusion module is used to fuse the second spatial-channel feature map and the fifth preset dimension feature map to obtain a second fused feature map; the third transposed convolution module is used to upsample the second fused feature to obtain a second upsampled feature map; the third RFEM module is used to compress the second upsampled feature to obtain a second compressed feature map; the fourth transposed convolution module is used to upsample the second compressed feature to obtain the fourth preset dimension feature map; The fifth feature extraction branch includes: a fourth feature extraction module, a third feature fusion module, a fifth transposed convolution module, a fourth RFEM module, and a sixth transposed convolution module; the fourth feature extraction module is used to extract a fifth preset dimension feature from the fourth preset dimension feature, as a pooling layer feature map; the third feature fusion module is used to fuse the pooling layer feature map and the sixth preset dimension feature map to obtain a third fused feature map; the fifth transposed convolution module is used to upsample the third fused feature map to obtain a third upsampled feature map; the fourth RFEM module is used to compress the third upsampled feature map to obtain a third compressed feature map; the sixth transposed convolution module is used to upsample the third compressed feature map to obtain the fifth preset dimension feature map; The sixth feature extraction branch includes: a convolution module, a seventh transposed convolution module, a fifth RFEM module, and an eighth transposed convolution module. The convolution module is used to extract a sixth preset dimension feature from the fifth preset dimension feature map. The seventh transposed convolution module is used to upsample the sixth preset dimension feature to obtain a fourth upsampled feature map. The fifth RFEM module is used to compress the fourth upsampled feature map to obtain a fourth compressed feature map. The eighth transposed convolution module is used to upsample the fourth compressed feature map to obtain the sixth preset dimension feature map.

9. A tooth image segmentation device based on semi-supervised learning and nnU-Net, characterized in that, include: The image acquisition module is used to acquire images of teeth to be segmented and to preprocess the images of teeth to be segmented to obtain preprocessed images; Preprocessing methods include: image size conversion; The image segmentation module is used to input the preprocessed image into the tooth image segmentation model to obtain the tooth segmentation result; the tooth image segmentation model is trained on the pre-constructed image segmentation model according to the semi-supervised learning mechanism; the pre-constructed image segmentation model is constructed based on the nnU-Net structure and attention mechanism; The process of training a pre-built image segmentation model using a semi-supervised learning mechanism to obtain the tooth image segmentation model includes: Obtain a set of sample images for training the pre-constructed image segmentation model; the set of sample images includes: a set of labeled images and a set of unlabeled images; The pre-built image segmentation model is trained using the labeled image set to obtain a teacher model; the sample image set is divided into K training and validation sets using K-fold cross-validation; and the pre-built image segmentation model is trained using the training and validation sets in the K combinations to obtain multiple student models. The unlabeled images in the unlabeled image set are enhanced, and the enhanced unlabeled images are input into the teacher model to obtain multiple first pseudo-labeled images; the enhancement operation includes: vertical flip operation and histogram equalization; Multiple first pseudo-label images are input into each student model to obtain multiple second pseudo-label images; and the multiple second pseudo-label images are fused to obtain a third pseudo-label image; the fusion operation includes: pixel averaging, binarization and morphological operations; Multiple first pseudo-labeled images, the third pseudo-labeled image, and labeled images from the labeled image set are combined to obtain an updated image set; the pre-built image segmentation model is trained using the updated image set to obtain the tooth image segmentation model.

Images