Semi-supervised image segmentation method and system based on geometric features

Abstract

The invention provides a semi-supervised image segmentation method and system based on geometric features, and relates to the field of deep learning and image processing, and the method comprises the steps: obtaining a to-be-segmented tooth CBCT image; inputting the tooth CBCT image into the segmentation model after semi-supervised learning, and segmenting and identifying each tooth to obtain a final segmentation result; wherein the semi-supervised learning is to carry out supervised training through a pseudo tag with a weight, and the pseudo tag with the weight is to learn voxel-level segmentation confidence and slice-by-slice tagged edge lines of the tooth by using a segmentation model of a single encoder and double decoders, and to carry out semi-supervised learning on the basis of the confidence and an intersection-to-union ratio IoU representing geometric feature consistency. And assigning a pseudo label to the unlabeled sample and calculating the weight of the pseudo label, and applying the pseudo label with the weight to supervised training of the segmentation model. According to the method, the priori geometric features are deeply fused into pseudo label generation of semi-supervised learning, and finally, the precision, robustness and practicability are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image processing, specifically to a semi-supervised image segmentation method and system based on geometric features. Background Technology

[0002] Accurate CBCT image instance segmentation plays an important role in assisting doctors in analyzing and treating teeth. However, dental CBCT data with voxel-level real labels is expensive and scarce. Manual annotation requires professional doctors to spend a lot of time annotating, which has extremely high requirements for both time and expertise.

[0003] With the rapid development of semi-supervised learning algorithms and computer hardware, semi-supervised learning has become an important way to solve the problem of scarce labeled data in the field of medical imaging. Semi-supervised learning algorithms can achieve performance similar to fully supervised learning by utilizing a small amount of labeled data and a large amount of unlabeled data. Existing semi-supervised learning methods are mainly divided into two categories: threshold-based methods and consistency regularization-based methods, both of which have achieved certain results in clinical practice of medical image segmentation.

[0004] However, unlike other medical images, teeth in CBCT images possess certain inherent prior geometric features. Teeth vary significantly in shape, size, and number of roots depending on their location (maxillary or mandibular) and functional classification (incisors, canines, premolars, molars). Existing semi-supervised image segmentation learning methods do not consider using these prior geometric features in their design phase, thus limiting the upper limit of semi-supervised learning algorithms in dental CBCT image segmentation. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a semi-supervised image segmentation method and system based on geometric features. This method deeply integrates prior geometric features into the pseudo-label generation of semi-supervised learning, ultimately achieving significant improvements in accuracy, robustness, and practicality.

[0006] According to some embodiments, the present invention adopts the following technical solution: A semi-supervised image segmentation method based on geometric features includes: Obtain CBCT images of the teeth to be segmented; The CBCT images of the teeth are input into a semi-supervised learning segmentation model to segment and identify each tooth, thus obtaining the final segmentation result. The semi-supervised learning is conducted through supervised training using weighted pseudo-labels. The weighted pseudo-labels are obtained by using a segmentation model with a single encoder and dual decoder to learn the voxel-level segmentation confidence of teeth and the labeled edge lines of each slice. Based on the intersection-union ratio (IoU) of the confidence and the consistency of the geometric features, pseudo-labels are assigned to unlabeled samples and the weights of the pseudo-labels are calculated. The weighted pseudo-labels are then used for supervised training of the segmentation model.

[0007] According to some embodiments, the present invention adopts the following technical solution: A semi-supervised image segmentation system based on geometric features, comprising: The image acquisition module is configured to acquire CBCT images of the teeth to be segmented; The image segmentation module is configured to input the CBCT images of teeth into a semi-supervised learning segmentation model, segment and identify each tooth, and obtain the final segmentation result. The semi-supervised learning is conducted through supervised training using weighted pseudo-labels. The weighted pseudo-labels are obtained by using a segmentation model with a single encoder and dual decoder to learn the voxel-level segmentation confidence of teeth and the labeled edge lines of each slice. Based on the intersection-union ratio (IoU) of the confidence and the consistency of the geometric features, pseudo-labels are assigned to unlabeled samples and the weights of the pseudo-labels are calculated. The weighted pseudo-labels are then used for supervised training of the segmentation model.

[0008] According to some embodiments, the present invention adopts the following technical solution: A computer program product includes a computer program that, when executed by a processor, implements the aforementioned semi-supervised image segmentation method based on geometric features.

[0009] According to some embodiments, the present invention adopts the following technical solution: A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned semi-supervised image segmentation method based on geometric features.

[0010] According to some embodiments, the present invention adopts the following technical solution: An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned semi-supervised image segmentation method based on geometric features.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention effectively improves the performance of semi-supervised segmentation models on dental CBCT images by introducing prior geometric features of teeth and combining them with an innovative dynamic thresholding and weighting strategy. This method utilizes multi-task learning to simultaneously acquire segmentation masks and tooth edge features. The generation and selection of pseudo-labels are guided by confidence scores and Intersection over Union (IoU), which alleviates class imbalance while ensuring the quality of pseudo-labels. This reduces reliance on labeled data while achieving segmentation accuracy close to that of fully supervised learning, providing a reliable solution for clinical applications. Attached Figure Description

[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention. Figure 1 This is a schematic diagram of the segmentation module and the extraction module in Embodiment 1; Figure 2 This is a schematic diagram of the method flow provided in Embodiment 1; Figure 3 This refers to the tooth categories and corresponding labels provided in Example 1. Figure 4 This is a schematic diagram of the edge line of Embodiment 1; Figure 5 This is a schematic diagram of the pseudo-label generation in Example 1 of this embodiment; Figure 6 This is a schematic diagram of the weight calculation in Example 1 of this embodiment. Detailed Implementation

[0013] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0014] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0015] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0016] Example 1 One embodiment of the present invention provides a semi-supervised image segmentation method based on geometric features, comprising: Step 1: Obtain CBCT images of the teeth to be segmented; Step 2: Input the CBCT images of the teeth into the semi-supervised learning segmentation model to segment and identify each tooth, and obtain the final segmentation result; The semi-supervised learning is conducted through supervised training using weighted pseudo-labels. The weighted pseudo-labels are obtained by using a segmentation model with a single encoder and dual decoder to learn the voxel-level segmentation confidence of teeth and the labeled edge lines of each slice. Based on the intersection-union ratio (IoU) of the confidence and the consistency of the geometric features, pseudo-labels are assigned to unlabeled samples and the weights of the pseudo-labels are calculated. The weighted pseudo-labels are then used for supervised training of the segmentation model.

[0017] As one embodiment, this invention provides a semi-supervised image segmentation method based on geometric features. It uses a single encoder-dual decoder segmentation model to learn geometric features and calculate the Intersection over Union (IoU), which represents the consistency of geometric features. A dynamic thresholding strategy, employing both global and local approaches, reduces the problems of class imbalance in pseudo-labels and the learning difficulty of different classes. Simultaneously, it adjusts existing dynamic thresholding strategies. Building upon existing strategies that only dynamically adjust thresholds using model prediction confidence, it introduces the IoU obtained from the segmentation model. This strategy ensures that the model can learn sufficiently and effectively for all classes, especially the hard examples. The generation process of weighted pseudo-labels is described in detail below: The segmentation model employs a single encoder and dual decoder structure, such as... Figure 1 As shown, a dual-branch 3D-Unet network is used. The first branch consists of an encoder and a first decoder, and the second branch consists of an encoder and a second decoder. The CBCT image of the tooth to be segmented is input into the encoder. The first decoder predicts the confidence of each class on a voxel-by-voxel basis and selects the class with the highest confidence to convert into a mask label. The second decoder extracts the edge lines of the tooth slice by slice. Each edge line contains class information.

[0018] In the training phase, the first and second branches work together to iteratively generate pseudo-labels for unlabeled samples. In the inference phase after training, only the trained first branch is used to segment the CBCT images of teeth.

[0019] The training process for generating pseudo-labels involves first performing supervised training on the segmentation model using labeled samples from the sample set, then using the trained segmentation model to generate initial pseudo-labels for unlabeled samples in the sample set, and finally, iteratively training the segmentation model a second time based on the pseudo-labeled samples and the labeled samples. In each iteration, the threshold, pseudo-labels, and weights are regenerated until the training loss meets the preset conditions.

[0020] The second training iteration is explained below, taking the t-th iteration as an example, as follows: Figure 2 As shown: 1. Generate voxel-level segmentation confidence scores and labeled edge lines for teeth on a slice-by-slice basis using a segmentation model, and calculate the Intersection over Union (IoU).

[0021] 1. The identification information of each tooth is obtained through two methods: "voxel-level segmentation" in the first branch and "edge extraction" in the second branch. Voxel-level segmentation obtains the mask, category, and confidence score of each tooth, while edge extraction obtains the edge line and category of each tooth. Specifically: 1.1 Using the original CBCT images of the teeth as input, a two-branch 3D-Unet network is used. The first branch classifies each voxel and predicts the confidence score, while the second branch predicts the tooth edge line containing category information, denoted as... .

[0022] In this embodiment, the CBCT image is segmented into 33 categories, corresponding to the background portion and the categories corresponding to the 32 teeth, such as... Figure 2 As shown, the 32 teeth are divided into four groups according to their tooth numbers: the upper right side is 11-18, the upper left side is 21-28, the lower right side is 31-38, and the lower left side is 41-48. In each group: number 1 is the incisor, number 2 is the lateral incisor, number 3 is the canine, numbers 4 and 5 are the first and second premolars, and numbers 6, 7, and 8 are the first, second, and third molars. Wisdom teeth are not considered.

[0023] 1.2 Assign the class with the highest probability to each voxel based on the voxel-level segmentation confidence and generate a tooth mask.

[0024] Specifically, for each voxel, the class with the highest confidence predicted by the model is selected as the current class, and the tooth mask of the image is obtained in this way; only the confidence of the highest class is retained as the confidence of the instance (each identified tooth is called an instance), which is used for the following calculation.

[0025] 1.3 Extract the 2D tooth edge lines layer by layer from the generated tooth mask, denoted as... .

[0026] Specifically, using OpenCV's edge detection algorithm, an edge detection algorithm is used to generate an edge detection image for the mask. Figure 4 The edge line shown.

[0027] 2. Edge lines of the same type of teeth obtained through two methods and , for and The Middle Teeth (i.e., numbered) The edge lines of the teeth are used to calculate the intersection-union ratio (IoU) of the region enclosed by the edge lines, which is used to characterize the geometric feature consistency between the two methods.

[0028] IoU, short for Intersection over Union, is an index that measures the degree of overlap between two regions. It is widely used in fields such as image segmentation and object detection to evaluate the similarity between predicted results and true results (or benchmark results).

[0029] In this embodiment, the calculation of the first... The specific steps for IoU in dental-like structures are as follows: 1) First, convert the edge lines into regions: convert the edge lines... and The interior is filled to form a closed region with a defined area, thus creating two regions: region 1. and region .

[0030] 2) Calculate the intersection: Calculate the region and region The total number of pixels in the overlapping area.

[0031] 3) Calculate the union: calculate the region and region The total number of all non-repeating pixels covered is expressed by the formula: Union = Region Pixel count + area Number of pixels - Number of pixels at intersection.

[0032] 4) Calculate IoU: = Number of pixels in the intersection / Number of pixels in the union, where, For the t-th iteration, the... Teeth The intersection and comparison of IoU.

[0033] The Intersection over Union (IoU) obtained in this step, together with the confidence score obtained from voxel-level segmentation, will be used as a standard to measure the model's prediction and learning capabilities. This is the theoretical basis for setting the threshold update strategy in the future.

[0034] II. Pseudo-label generation based on dynamic threshold The original CBCT image of the teeth, after being processed by the first branch to obtain its segmented tooth mask, is as follows: Figure 5 As shown, based on the obtained first... Teeth The confidence level and Intersection over Union (IoU) are used to dynamically adjust the threshold, and pseudo-labels are assigned to unlabeled data based on the threshold.

[0035] This embodiment proposes two dynamic threshold strategies: global and local. The global threshold strategy ensures that the final threshold maintains a low-to-high trend throughout the training process, allowing more samples to participate in training in the early stages and improving the quality of pseudo-labels in the later stages. The local threshold strategy ensures that the model appropriately reduces the threshold for high-difficulty categories when facing different classes, so that the model can learn more samples of that category. The final threshold is calculated based on the global and local thresholds. In the current iteration t, all instances with confidence exceeding the threshold (each identified tooth is considered an instance) will be assigned a pseudo-label. The specific calculation method for the dynamic threshold is as follows: 1. The global threshold of the current iteration t The setup method is as follows:

[0036] in, This represents the total number of categories. In this embodiment, , The hyperparameter is set to 0.99. Set the batch size for unlabeled data to 8. For the previous iteration The global threshold, For the current iteration t, the th iteration The maximum confidence among all voxels of a sample.

[0037] 2. Local threshold The setup method is as follows:

[0038] in, Indicates the t-th iteration. Teeth Local threshold, This is a hyperparameter, set to 0.9.

[0039] 3. The final threshold is calculated by multiplying the local threshold (after maximum normalization) by the global threshold, as expressed by the formula:

[0040]

[0041] In this embodiment, the confidence level of the instance exceeds the final threshold of the same category in the current iteration. At that time, a pseudo-label is assigned to the instance, using the previously obtained tooth mask as the pseudo-label.

[0042] 3. Assign weights to the generated pseudo-tags After obtaining pseudo-labels for the samples, pseudo-labels of different quality are selected and assigned different weights. This embodiment proposes a weighting strategy for pseudo-labels. In this strategy, samples with better geometric features and samples with higher confidence are regarded as high-quality pseudo-label samples. When calculating the unlabeled learning loss, the loss is calculated based on different samples and their weights. like Figure 6 As shown, the weights of pseudo-labels are calculated based on the intersection-union ratio (IoU) of confidence level and the consistency of geometric features. Specifically: 1. For instances that obtain pseudo-labels, calculate the first weight based on the confidence level:

[0043] Where 'a' represents the confidence level obtained from the first branch. The average confidence level for all categories. As a hyperparameter, it is set to 0.1; 2. Calculate the second weight based on the intersection-union ratio:

[0044] Where b represents the intersection-union ratio of the categories corresponding to the pseudo-labels. The average of the crossover ratios for all categories. It is a hyperparameter, set to 0.5.

[0045] 3. Final weights The weighted sum of confidence score and IoU is calculated using the following formula:

[0046] in, It is a hyperparameter, set to 1.2.

[0047] Fourth, the pseudo-labeled samples with obtained weights are re-involved in the training process.

[0048] After obtaining the pseudo-labels and weights, when this example re-engages in the training process, the new predictions of the segmentation model will be used to calculate the loss along with the pseudo-labels, taking into account the weights of the example. The loss function formula is as follows:

[0049] in, The cross-entropy loss is calculated for pseudo-labels and predicted tooth mask labels during retraining.

[0050] Example 2 One embodiment of the present invention provides a semi-supervised image segmentation system based on geometric features, comprising: The image acquisition module is configured to acquire CBCT images of the teeth to be segmented; The image segmentation module is configured to input the CBCT images of teeth into a semi-supervised learning segmentation model, segment and identify each tooth, and obtain the final segmentation result. The semi-supervised learning is conducted through supervised training using weighted pseudo-labels. The weighted pseudo-labels are obtained by using a segmentation model with a single encoder and dual decoder to learn the voxel-level segmentation confidence of teeth and the labeled edge lines of each slice. Based on the intersection-union ratio (IoU) of the confidence and the consistency of the geometric features, pseudo-labels are assigned to unlabeled samples and the weights of the pseudo-labels are calculated. The weighted pseudo-labels are then used for supervised training of the segmentation model.

[0051] Example 3 One embodiment of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned semi-supervised image segmentation method based on geometric features.

[0052] Example 4 In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned semi-supervised image segmentation method based on geometric features.

[0053] Example 5 One embodiment of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned semi-supervised image segmentation method based on geometric features.

[0054] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A semi-supervised image segmentation method based on geometric features, characterized in that, include: Obtain CBCT images of the teeth to be segmented; The CBCT images of the teeth are input into a semi-supervised learning segmentation model to segment and identify each tooth, thus obtaining the final segmentation result. The semi-supervised learning is conducted through supervised training using weighted pseudo-labels. The weighted pseudo-labels are obtained by using a segmentation model with a single encoder and dual decoder to learn the voxel-level segmentation confidence of teeth and the labeled edge lines of each slice. Based on the intersection-union ratio (IoU) of the confidence and the consistency of the geometric features, pseudo-labels are assigned to unlabeled samples and the weights of the pseudo-labels are calculated. The weighted pseudo-labels are then used for supervised training of the segmentation model.

2. The semi-supervised image segmentation method based on geometric features as described in claim 1, characterized in that, The segmentation model with a single encoder and dual decoder uses a dual-branch 3D-Unet network. Unlabeled CBCT images of teeth are input into the encoder. The first decoder predicts the confidence of each class on a voxel-by-voxel basis and selects the class with the highest confidence to convert into a mask label. The second decoder extracts the edge lines of the teeth slice by slice, and each edge line contains class information.

3. The semi-supervised image segmentation method based on geometric features as described in claim 2, characterized in that, The method of assigning pseudo-labels to unlabeled samples uses a dynamic thresholding strategy that combines global and local approaches. The mask label is used as the pseudo-label for the unlabeled sample by comparing the confidence level with the dynamic threshold.

4. The semi-supervised image segmentation method based on geometric features as described in claim 1, characterized in that, The dynamic thresholding strategy that uses both global and local thresholds can be expressed as follows: global threshold The setup method is as follows: Where C represents the total number of categories, For hyperparameters, For unlabeled data, This refers to the t-th iteration during the training process. Local threshold The setup method is as follows: in, Indicates the t-th iteration. Teeth Local threshold, For hyperparameters, For the t-th iteration, the... Teeth The intersection and union ratio of IoU; The final dynamic threshold is calculated by multiplying the local threshold (after maximum normalization) by the global threshold: in, For maximum normalization.

5. The semi-supervised image segmentation method based on geometric features as described in claim 3, characterized in that, The process of using the comparison between confidence level and dynamic threshold to assign pseudo-labels to unlabeled samples means that all samples with confidence levels exceeding the dynamic threshold are given pseudo-labels.

6. The semi-supervised image segmentation method based on geometric features as described in claim 1, characterized in that, The weights of the pseudo-labels are calculated by using confidence level and intersection-over-union ratio (IoU) to calculate two weights, and then weighted summing the two weights to obtain the final weights.

7. A semi-supervised image segmentation system based on geometric features, characterized in that, include: The image acquisition module is configured to acquire CBCT images of the teeth to be segmented; The image segmentation module is configured to input the CBCT images of teeth into a semi-supervised learning segmentation model, segment and identify each tooth, and obtain the final segmentation result. The semi-supervised learning is conducted through supervised training using weighted pseudo-labels. The weighted pseudo-labels are obtained by using a segmentation model with a single encoder and dual decoder to learn the voxel-level segmentation confidence of teeth and the labeled edge lines of each slice. Based on the intersection-union ratio (IoU) of the confidence and the consistency of the geometric features, pseudo-labels are assigned to unlabeled samples and the weights of the pseudo-labels are calculated. The weighted pseudo-labels are then used for supervised training of the segmentation model.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a semi-supervised image segmentation method based on geometric features as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement a semi-supervised image segmentation method based on geometric features as described in any one of claims 1-6.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform a semi-supervised image segmentation method based on geometric features as described in any one of claims 1-6.

Images