A tooth image segmentation method, device and computer readable storage medium

Abstract

This application provides a method for dental image segmentation, including acquiring an initial medical image dataset. While ensuring the uncertainty and diversity of the acquired samples, a subset of the most valuable image data is selected and manually labeled. The segmentation network model then undergoes domain incremental learning, incorporating newly annotated samples in each iteration for further training and fine-tuning, thereby continuously improving the segmentation performance of the network model. By continuously and actively selecting valuable samples to incrementally update the segmentation network model, a segmentation network model with a certain degree of evolutionary capability can be trained with a limited amount of labeled image data.

Description

Technical Field

[0001] This application relates to the field of digital oral technology, specifically providing a tooth image segmentation method, device, and computer-readable storage medium. Background Technology

[0002] In recent years, the rapid development of digital information acquisition methods such as cone-beam computed tomography (CBCT) and intraoral scanning in the dental industry has assisted dentists in diagnosis, surgery, and treatment planning, making digital dentistry the future direction and mainstream.

[0003] Dental image segmentation is a fundamental task in digital dentistry. Many deep learning-based methods have achieved good results in this task. However, most of them are based on fixed datasets with the same data distribution. They assume that the data distribution is fixed or stationary, the training samples are independent and identically distributed, and all data needs to be provided to the model in a single training step. Once the model is deployed, the parameters are fixed, and it can only recognize images with the same data distribution. This often ignores the potential need for model evolution and updates.

[0004] In real-world clinical settings, new patients emerge daily, leading to an ever-increasing number of new data samples across institutions over time. The traditional approach is to mix new and historical data and then retrain the system, but repeatedly learning from large-scale historical data results in a significant waste of computational resources.

[0005] At the same time, although deep learning technology has achieved great success in medical image diagnosis, there is a contradiction in the data annotation of deep learning-based segmentation methods, namely the contradiction between the high dependence on a large amount of high-quality annotated data and the expensive pixel-level annotation cost of medical images.

[0006] Active learning is considered one of the feasible solutions to alleviate this contradiction. During training, it incrementally and proactively selects the most valuable or informative samples for manual annotation, thereby achieving optimal model performance with minimal labeled data. However, current active learning-based segmentation methods still have some problems, such as unreliable uncertainty assessment, inability to effectively learn the boundaries of data samples, and often neglecting the information content of the acquired samples.

[0007] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0008] In order to overcome the above-mentioned shortcomings, this application is proposed to solve, or at least partially solve, the technical problem of how to improve image segmentation performance in the case of a small amount of labeled image data.

[0009] In a first aspect, this application provides a method for segmenting tooth images, the method comprising the following steps S101-S108: Step S101: Obtain an initial medical image dataset, and select a first image dataset and a second image dataset; Step S102: Label the first image dataset, and obtain a first pre-trained segmentation network model through fully supervised training, the first pre-trained segmentation network model comprising a first basic segmentation module and a first accuracy prediction module; S103: The first basic segmentation module filters out a portion of information-rich and diverse image datasets from the labeled first image dataset to form a labeled replay image dataset; Step S104: Using an active learning strategy, combined with a hybrid strategy of accuracy and diversity, sample the second image dataset to filter out a certain number of high-value image data. Step S105: Manually annotate the high-value image dataset to obtain an annotated first incremental image dataset; Step S106: Merge the annotated first incremental image dataset into the annotated replay image dataset to obtain an annotated second updated image dataset; Step S107: Use the annotated second updated image dataset to perform domain incremental learning on the first pre-trained segmentation network model, and train an optimized second pre-trained segmentation network model using a knowledge distillation strategy. The second pre-trained segmentation network model includes a second basic segmentation module and a second precision prediction module; Step S108: Repeat steps S104-S107 for other unannotated datasets until preset conditions are met; Step S109: Use the finally updated segmentation network model to obtain the image segmentation result.

[0010] In a second aspect, this application provides a method for segmenting tooth images, the method comprising the following steps S201-S209: Step S201: Obtain an initial medical image dataset and construct a training data queue Q of {D1, D2, D3, ..., D...} NThe training proceeds sequentially, selecting D1 as the first image dataset and D2 as the second image dataset. Step S202: The first image dataset D1 is labeled, and a first pre-trained segmentation network model M1 is obtained through fully supervised training. The first pre-trained segmentation network model M1 includes a first basic segmentation module and a first accuracy prediction module. Step S203: The first basic segmentation module selects a portion of the labeled first image dataset D1 that is rich in information and diverse, forming a labeled first replay image dataset R1, which is stored in the replay image data queue P1, forming P1{R1}. Step S204: An active learning strategy, combining a hybrid strategy of accuracy and diversity, is used to sample the second image dataset D2. Step S205: Select a certain number of high-value image datasets; Step S206: Manually annotate the high-value image datasets to obtain an annotated first incremental image dataset I1; Step S207: Merge the annotated first incremental image dataset I1 into the annotated replay image data queue P1{R1} to obtain an annotated second updated image dataset; Step S208: Use the annotated second updated image dataset to perform domain incremental learning on the first pre-trained segmentation network model M1, and train it using a knowledge distillation strategy to obtain an optimized second pre-trained segmentation network model M2, wherein the second pre-trained segmentation network model M2 includes a second basic segmentation module and a second precision prediction module; Step S209: For the unannotated dataset {D3,…,D…}, perform domain incremental learning on the first pre-trained segmentation network model M1, and train it using a knowledge distillation strategy to obtain an optimized second pre-trained segmentation network model M2, wherein the second pre-trained segmentation network model M2 includes a second basic segmentation module and a second precision prediction module; Step S2000: Select a certain number of high-value image datasets; Step S201: Manually annotate the high-value image datasets to obtain an annotated first incremental image dataset I1; Step S202: Merge the annotated first incremental image dataset I1 into the annotated replay image data queue P1{R1} to obtain an annotated second updated image dataset; Step S201: Perform domain incremental learning on the first pre-trained segmentation network model M1 using the first pre-trained segmentation network model M1, and train it using a knowledge distillation strategy to obtain an optimized second pre-trained segmentation network model M2, wherein the second pre-trained segmentation network model M2 includes a second basic segmentation module and a second precision prediction module; Step S201: Perform domain incremental learning on the unannotated dataset {D3,…,D…} N The process involves repeatedly executing steps S204-S207, along with a queue of labeled replay image data that needs to be iteratively replayed, until the preset conditions are met; step S209: using the finally updated segmentation network model, the image segmentation result is obtained.

[0011] In a third aspect, a computer device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the tooth image segmentation method described in any of the above-described technical solutions.

[0012] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the tooth image segmentation method described in any of the above-described technical solutions.

[0013] This application employs an efficient active learning strategy to manually annotate a subset of the most valuable image data while ensuring the uncertainty and diversity of the acquired samples. The segmentation network model then undergoes domain incremental learning, incorporating newly annotated samples in each iteration for further training and fine-tuning, continuously improving its segmentation performance. By proactively selecting valuable samples to incrementally update the segmentation network model, a segmentation model with a certain degree of evolutionary capability can be trained with a limited amount of labeled image data. Since datasets from different institutions are typically heterogeneous and exhibit domain differences, the updated model needs to avoid catastrophic forgetting while effectively generalizing to data flows with changing distributions. Therefore, domain incremental learning not only maintains stability with older data but also enhances its adaptability to new data. Attached Figure Description The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Wherein:

[0014] Figure 1 This is a flowchart of the main steps of a tooth image segmentation method according to an embodiment of this application;

[0015] Figure 2 This is a schematic diagram of a tooth image segmentation method framework based on domain incremental learning according to an embodiment of this application;

[0016] Figure 3 This is a schematic diagram of a model training framework according to an embodiment of this application;

[0017] Figure 4 This is a schematic diagram of a framework for domain incremental learning according to an embodiment of this application;

[0018] Figure 5 This is a flowchart of the main steps of a tooth image segmentation method according to another embodiment of this application;

[0019] Figure 6 This is a schematic diagram of a data flow-based domain incremental learning framework according to another embodiment of this application;

[0020] Figure 7 This is a schematic diagram of the computer device structure provided in this application. Detailed Implementation

[0021] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0022] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0023] In the description of this invention, "processor" can include hardware, software, or a combination of both. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc.

[0024] Reference Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating the main steps of a tooth image segmentation method according to an embodiment of this application. Figure 2 This is a schematic diagram of a framework for a tooth image segmentation method based on domain incremental learning according to an embodiment of this application. Figure 1 As shown, the tooth image segmentation method in Embodiment 1 of this application mainly includes the following steps S101-S109. Steps S101 to S109 will be further explained below.

[0025] Step S101: Obtain the initial medical image dataset. The initial medical image dataset refers to an unlabeled image dataset, which is divided into different data distributions based on information such as different institutions or CBCT machines. The dataset with the largest number of samples is selected as the first image dataset, and the dataset with the second largest number of samples is selected as the second image dataset for subsequent steps.

[0026] Specifically, the initial medical image dataset is typically collected from multiple medical sites or hospitals. Due to the significant heterogeneity of data from different sites using different imaging devices or protocols, data from different sites arrive sequentially to simulate real-world clinical scenarios. The image datasets can be divided into different sites based on information such as institution, manufacturer, device field strength, and resolution. The dataset with the largest number of samples is selected as the first image dataset, and the dataset with the second largest number of samples is selected as the second image dataset.

[0027] Step S102: Label the first image dataset and obtain a first pre-trained segmentation network model through fully supervised training. The first pre-trained segmentation network model includes a first basic segmentation module and a first accuracy prediction module.

[0028] In this step, the first image dataset is manually labeled to obtain corresponding labels. Through fully supervised learning, the initial neural network model is trained using the labeled first image dataset to obtain a first pre-trained segmentation network model with certain segmentation capabilities. This model includes a first basic segmentation module and a first accuracy prediction module. The first basic segmentation module predicts the segmentation of subsequent unlabeled image data, and the first accuracy prediction module predicts the segmentation accuracy of the unlabeled image data.

[0029] Specifically, refer to Figure 3 The model training in step S102 above can be achieved through steps S1021 to S1024 below.

[0030] Step S1021: The first pre-trained segmentation network model is composed of the first basic segmentation module and the first accuracy prediction module. During the training process, the two modules can be trained simultaneously, while their optimization processes are decoupled and embedded into an end-to-end unified training framework in a cascade manner.

[0031] Step S1022: For the first basic segmentation module, using the first image dataset as input, the optimization process utilizes cross-entropy loss to minimize the difference between the predicted segmentation result and the true label of the first image dataset. The cross-entropy loss calculation formula is as follows:

[0032]

[0033] Where CE represents the cross-entropy loss, p h,w and y h,w Let H and W represent the predicted label and the ground truth label for the (h,w)th pixel, respectively, where H and W represent the height and width of the image, respectively.

[0034] Step S1023: For the first accuracy prediction module, the images in the first image dataset and the corresponding posterior probabilities predicted by the first basic segmentation module are stitched together as input. The segmentation prediction result obtained by the first basic segmentation module and the multi-class cross-entropy loss with one-hot encoded labels in the first image dataset are used as the objective. The aim is to use the posterior probability of the basic segmentation module to guide accuracy prediction. The optimization process uses mean squared error loss to minimize the difference between the predicted accuracy and the actual accuracy. Wherein, the mean squared error loss L... msr The calculation formula is as follows:

[0035]

[0036] Where n is the number of samples, y i This represents the true value of the i-th sample. It is the prediction value of the accuracy prediction module for the i-th sample.

[0037] Step S1024: In the early stage of training, a short silent period is set for the accuracy prediction module. This silent period can be 5 epochs. This means that only the basic segmentation module is trained during this period. The purpose is to avoid the posterior probability of the basic segmentation module mentioned in the early stage from bringing too much noise to the accuracy prediction module.

[0038] Step S103: The first basic segmentation module filters out a portion of the labeled first image dataset that is rich in information and diverse, forming a labeled replay image dataset.

[0039] During the inference and prediction process of the first basic segmentation module on the second image dataset, it is found that the segmentation effect is poor and the accuracy is not high on this data distribution. Therefore, an active learning strategy can be adopted to sample this dataset for subsequent domain incremental learning. The data replay strategy of domain incremental learning involves selecting a portion of the labeled first image dataset that is rich in information and diverse, thus forming a labeled replay image dataset.

[0040] Specifically, to ensure the diversity of replay data, based on the hidden features of the segmentation model, the KMeans algorithm is used to perform unsupervised clustering on the labeled first image dataset samples to obtain K clusters. Adaptive global average pooling is then used to compress the original representation. Next, iterative queries select the same number of image data from each cluster to form a certain number of replay image datasets, which can be set to 128.

[0041] Step S104: Using an active learning strategy, combined with a hybrid strategy of accuracy and diversity, the second image dataset is sampled to select a certain number of high-value image datasets.

[0042] The active learning strategy combines the accuracy and diversity of the samples, that is, while utilizing the accuracy of the precision prediction module, it also uses the feature representation of the samples to balance the uncertainty and diversity of the acquired samples.

[0043] For the diversity strategy, when actively learning iteratively selects N samples, N can be set to 64. The basic segmentation module obtains the feature maps of the labeled first image dataset and the unlabeled second image dataset in the penultimate layer space of the segmentation model decoder.

[0044] Given the latent space of the labeled first image dataset feature map, the nearest neighbor distance between the second image dataset feature map and the labeled first image dataset feature map is calculated and recorded using the Euclidean distance formula. The sample with the largest nearest neighbor distance is selected, and sampling is repeated until the first N*P unlabeled image data are selected as the diversity-based partial unlabeled image dataset, where P can be set to 4. Specifically, it is essential to ensure that the amount of data selected based on diversity is greater than the value of N. The formula is as follows:

[0045]

[0046] Where h(x) represents the feature map of the penultimate layer space of the model decoder, D l D represents the first labeled image dataset. u This represents the second image dataset that is not labeled.

[0047] Meanwhile, the second image dataset, after being processed by the first pre-trained segmentation network model, yields corresponding prediction accuracies. Based on these prediction accuracies, all image data in the partially unlabeled second image dataset, according to their increasing accuracy, are sorted. The N image datasets with the lowest prediction accuracies are then selected to form the high-value image dataset.

[0048] Step S105: Manually annotate the high-value image dataset to obtain an annotated first incremental image dataset, and remove the first incremental image dataset from the second image dataset to form an updated second unannotated image dataset.

[0049] In this process, the high-value image dataset is manually annotated to obtain a first labeled incremental image dataset, and after obtaining the corresponding labels, the second image dataset needs to be updated by removing the first incremental image dataset and replacing it with other datasets. The other datasets are unlabeled to ensure that there are no labels in the unlabeled image dataset.

[0050] Step S106: Merge the labeled first incremental image dataset into the labeled replay image dataset to obtain the labeled second updated image dataset;

[0051] The labeled first incremental image dataset is used as new data for the next iteration of training, and the labeled replay image dataset is used as old data for the next iteration of training. They need to be merged and updated into a labeled second updated image dataset, and then sent together to the model for training.

[0052] Step S107: Use the labeled second updated image dataset to perform domain incremental learning on the first pre-trained segmentation network model, and use a knowledge distillation strategy to train and obtain an optimized second pre-trained segmentation network model. The second pre-trained model includes a second basic segmentation module and a second precision prediction module.

[0053] Specifically, using the labeled second image dataset, which contains 128 old data points and 64 new data points, domain incremental learning through data replay and knowledge distillation strategies aims to overcome catastrophic forgetting and train an optimized second basic segmentation module and a second precision prediction module.

[0054] See attached document Figure 4 The framework for domain incremental learning in step S107 above is implemented as follows.

[0055] In the Knowledge Distillation (KD) setup, the base segmentation module of the pre-trained segmentation network model serves as the original segmentation model (also known as the teacher network), and the structure of its new segmentation model (also known as the student network) is consistent with the former, obtaining relevant weights through exponentially weighted average (EMA).

[0056] During the training phase of domain incremental learning, only the parameters of the new segmentation network model are updated. The replay image dataset needs to pass through the original segmentation network model to obtain the logit feature soft label of the last layer and the feature maps of each layer of the decoder.

[0057] The labeled second updated image dataset is passed to a new segmentation model for training, which obtains the logit feature soft labels of the replay image dataset in the labeled second updated image dataset, the feature maps of each layer of the decoder, and the hard labels of the last segmentation of the labeled second updated image dataset.

[0058] Then, pixel-by-pixel distillation is used to constrain the original segmentation network model and the new segmentation network model, and the soft label output based on logit features (i.e., distillation loss L) of the representative replay image data is applied. kd1 To transfer prior knowledge and preserve previously learned knowledge, the pixel-by-pixel distillation loss calculation formula is as follows:

[0059]

[0060] in, and Let represent the logit soft target corresponding to the (h,w)th pixel generated by the new segmentation network model and the original segmentation network model, respectively, and KL represent the Kullback Leibler divergence. The L2 distance is represented by T, the temperature is T, and H and W represent the height and width of the image, respectively.

[0061] Based on the feature maps of each layer of the decoder obtained from the original segmentation network model and the new segmentation network model respectively, each image is used to construct paired graph structure data at each layer, including node-level features and graph-level features. A graph feature interaction matrix is ​​constructed using the graph-level features. Then, the graph similarity of the node feature interaction matrix of the paired graph structure data for each image in the replay image dataset is calculated to obtain the distillation loss of paired pixel similarity. The distillation loss of the paired graph structure data at a certain layer is calculated using the squared difference, as shown in the following formula:

[0062]

[0063] Where s = norm(f i T f i ) represents the graph feature interaction matrix, norm represents the normalization process, the superscripts s and t represent the student model and teacher model respectively, and W′ and H′ represent the width and height of the feature map respectively.

[0064] The pairwise graph structure data distillation loss for the feature maps of each layer of the decoder is as follows: Where N is the number of layers in the decoder.

[0065] Guided by prior knowledge, the new segmentation network model aims to reduce the difference between the hard-label output and the true label of the second updated image dataset (i.e., the segmentation loss L). ce This is used to correct and optimize the parameters of the new segmentation network model to adapt to the new data and further consolidate previously learned knowledge. The segmentation loss formula is as follows:

[0066]

[0067] Where CE represents the cross-entropy loss, p h,w and y h,w Let H and W represent the predicted label and the ground truth label for the (h,w)th pixel, respectively, where H and W represent the height and width of the image, respectively.

[0068] Based on the above principles, the total loss function of knowledge distillation can be expressed as L ce and L fea_kd and L p_kd The weighting, i.e.

[0069] L=L_ce+γ(L fe_kd +L p_kd )

[0070] Here, γ is used to balance the plasticity and stability of the model. When γ = 0, it is equivalent to fine-tuning.

[0071] Step S108: Repeat steps S104-S107 using other unlabeled datasets until the preset conditions are met.

[0072] The preset conditions are that the amount of labeled image data reaches the labeling budget, or that the performance of the evolved segmentation network model has reached the standard. The labeling budget is determined based on the actual amount of unlabeled data; specifically, it is less than the total number of unlabeled image data samples.

[0073] Step S109: Use the final updated segmentation network model to obtain the image segmentation result.

[0074] In the above embodiments, the initial medical image dataset obtained only selected two image datasets with different data distributions. The final updated segmentation model can maintain the segmentation accuracy of the first image dataset while gradually learning and adapting to the second image dataset with a small amount of labeled image data and reduced training costs.

[0075] Furthermore, real-world clinical scenarios involve continuous data flows, far exceeding the finite dataset. Example 2 below describes the construction of a data column during the initial medical image dataset partitioning process.

[0076] Reference Figure 5 and Figure 6 , Figure 5 This is a flowchart of the main steps of a tooth image segmentation method according to another embodiment of this application. Figure 6 This is a schematic diagram of a data flow-based domain incremental learning framework according to another embodiment of this application. The tooth image segmentation method in Embodiment 2 of this application mainly includes the following steps S201-S209. Steps S201 to S209 will be further described below.

[0077] Step S201: Obtain the initial medical image dataset, divide it into multiple domain image datasets, and then construct a training data queue containing N datasets according to the number of samples in each domain dataset from largest to smallest. The training data queue Q is {D1, D2, D3, ..., D...} N The training is then carried out one by one, with D1 selected as the first image dataset and D2 as the second image dataset for subsequent steps.

[0078] Taking dental images in the medical field as an example, in real clinical scenarios, new patients appear every day, inducing a continuous increase in new data samples across institutions over time. To simulate this situation, an initial medical image dataset is obtained and divided into multiple domain image datasets based on information such as different institutions or CBCT machines. Then, a training data queue is constructed based on the number of samples in each domain dataset from largest to smallest, forming a data stream.

[0079] Step S202: Label the first image dataset D1, and obtain the first pre-trained segmentation network model M1 through fully supervised training. The first pre-trained segmentation network model M1 includes a first basic segmentation module and a first accuracy prediction module.

[0080] The training of the model is the same as step S102 described in Example 1. Finally, a high-performance first pre-trained segmentation network model M1 is obtained based on dataset D1, which includes a first basic segmentation module and a first precision prediction module.

[0081] Step S203: The first basic segmentation module filters out some information-rich and diverse image datasets from the labeled first image dataset D1 to form the labeled first replay image dataset R1, and stores it in the replay image data queue P1 to form P1{R1}.

[0082] Before learning from the second image dataset D2 in the training queue Q, the first pre-trained segmentation network model M1 needs to use a domain incremental learning data replay strategy. That is, it selects a portion of the image datasets that are rich in information and diverse from the labeled dataset D1 to form the first labeled replay image dataset R1, and stores it in the replay image data queue P1 to form P1{R1}.

[0083] To ensure the diversity of replay data, based on the hidden features obtained by the basic segmentation module, the KMeans algorithm is used to perform unsupervised clustering on the labeled dataset D1 to obtain K clusters. Adaptive global average pooling is then used to compress the original representation. Next, iterative queries select the same number of image data from each cluster to form the first replay image dataset R1, which is stored in the replay image data queue P1, forming P1{R1}. The sample size of the first replay image dataset R1 can be set to 128.

[0084] Step S204: Using an active learning strategy, combined with a hybrid strategy of accuracy and diversity, the second image dataset D2 is sampled to select a certain number of high-value image datasets.

[0085] The active learning strategy is the same as step S104 in Example 1, which uses a hybrid strategy based on accuracy and diversity to query and sample the second image dataset D2. The selection of image data samples can be divided into two stages.

[0086] Phase 1: Using a diversity-based query strategy, the first image dataset D1 and the second image dataset D2 are processed by the first pre-trained segmentation network model to obtain the feature maps of the penultimate layer space of the corresponding decoder. Then, a certain proportion of sample data from the second image dataset D2 is selected by iterative comparison and comparison using Euclidean distance. This proportion can be set to 50%.

[0087] Phase Two: The 50% of the second image dataset D2 can obtain the segmentation prediction accuracy value of the unlabeled images through the first accuracy prediction module. Then, the segmentation prediction accuracy values ​​are sorted in ascending order, and the top 25% of the data samples are selected as the high-value image dataset.

[0088] Step S205: Manually annotate the high-value image dataset to obtain the first annotated incremental image dataset I1.

[0089] Step S206: Merge the labeled first incremental image dataset I1 into the labeled replay image data queue P1{R1} to obtain the labeled second updated image dataset.

[0090] Step S207: Using the labeled second updated image dataset, perform domain incremental learning on the first pre-trained segmentation network model M1, and use a knowledge distillation strategy to obtain an optimized second pre-trained segmentation network model M2. The second pre-trained model M2 includes a second basic segmentation module and a second precision prediction module.

[0091] The domain incremental learning process is the same as the specific operation in step S107 of Example 1, see [link / reference]. Figure 6 By utilizing data replay and knowledge distillation, the labeled replay data queue P1{R1} is used as the old data, and the labeled first incremental image data set I1 is used as the new data. Together, they are used to perform domain incremental learning on the first pre-trained segmentation network model. While maintaining the memory of the old data, the model learns to adapt to the new data, thereby enabling the new model to perform well on both old and new data while reducing training time and labeling costs.

[0092] Step S208: Repeat steps S204-S207 for the unlabeled other datasets and the labeled replay image data queue that needs to be iteratively replayed, until the preset conditions are met. The unlabeled other datasets are {D3,…,D} in the training queue Q. NThe subsequent data for}, the preset condition is that the training queue Q is empty.

[0093] For the training queue Q, {D3,…,D…} N For the dataset}, the repeated loop execution of steps S204-S207 continues until the training queue Q is empty, specifically through the following steps:

[0094] An active learning strategy, combined with a hybrid strategy of accuracy and diversity, is employed for the t-th image dataset D. t Sampling is carried out to select a certain number of high-value image datasets. Specifically, 25% of the high-value image datasets can be selected.

[0095] The high-value image dataset is manually annotated to obtain the annotated (t-1)th incremental image dataset I. t-1 The labeled (t-1)th incremental image dataset I t-1 Merge the labeled replay image data queue P t-1 {R1,R2,…,R t-1}, thus obtaining the labeled updated image dataset of the tth generation;

[0096] Using the labeled t-th updated image dataset, the pre-trained segmentation network model M at the (t-1)-th ... t-1 Domain incremental learning is performed, and a knowledge distillation strategy is used to obtain the optimized t-th pre-trained segmentation network model M. t The t-th pre-trained model M t It includes the t-th basic segmentation module and the t-th precision prediction module.

[0097] Specifically, the well-trained model M at iteration time t-1... t-1 Distillation is performed on the teacher model, where the student model to be learned has the same structure as the teacher model, and its parameters are obtained via EMA. The replay image data queue P is used. t-1 and the labeled image dataset D t 25% of the high-value image data is used together to perform domain incremental learning on the student model, training it to obtain model M at iteration time t. t The replay image data queue P t-1 {R1,R2,…,R t-1} contains information about {D1,D2,…,D t-1 The dataset contains labeled samples designed to prevent catastrophic forgetting when new models learn new data.

[0098] When model M t During convergence, the KMeans algorithm is used to select a certain number of image datasets D based on diversity. tThe data is stored in the replay data queue P for subsequent domain incremental learning data replay.

[0099] Among them, the labeled replay image data queue P t-1 {R1,R2,…,R t-1}, using the (t-1)th pre-trained segmentation network model M t-1 The (t-1)th basic segmentation module performs segmentation on the (t-2)th incremental image dataset I that has been labeled. t-2 A subset of information-rich and diverse image datasets were selected to form the labeled (t-1)th replay image dataset R. t-1 And stored in the replay data queue P t-1 In the middle, updated to P t-1 {R1,R2,…,R t-1}

[0100] For example, for the third image dataset D3, an active learning strategy is employed, combining a hybrid strategy of accuracy and diversity, to sample the third image dataset D3, selecting a certain number of high-value image datasets. Then, these high-value image datasets are manually labeled to obtain a labeled second incremental image dataset I2. This labeled second incremental image dataset I2 is then merged with the labeled replay image data queue P2{R1,R2} to obtain a labeled third updated image dataset. The labeled third updated image dataset is then used to perform domain incremental learning on the second pre-trained model M2, employing a knowledge distillation strategy to obtain an optimized third pre-trained segmentation network model. This third pre-trained model includes a third basic segmentation module and a third accuracy prediction module. In order to ensure the diversity of replay data, the specific operation is the same as in step S203. Based on the hidden features obtained by the second basic segmentation module, the labeled replay image data queue P2{R1,R2} uses the second basic segmentation module of the second pre-trained segmentation network model M2 to filter out some information-rich and diverse image datasets from the labeled first incremental image dataset I1, forming the labeled second replay image dataset R2, which is stored in the replay data queue P2 and updated to P2{R1,R2}.

[0101] The sample size of the second replay image dataset R2 can be set to 5% of the first incremental image dataset O1.

[0102] Step S209: Use the final updated segmentation network model to obtain the image segmentation result.

[0103] This application employs an efficient active learning strategy to manually annotate a subset of the most valuable image data while ensuring the uncertainty and diversity of the acquired samples. The segmentation network model then undergoes domain incremental learning, incorporating newly annotated samples in each iteration for further training and fine-tuning, continuously improving its segmentation performance. By proactively selecting valuable samples to incrementally update the segmentation network model, a segmentation model with a certain degree of evolutionary capability can be trained with a limited amount of labeled image data. Since datasets from different institutions are typically heterogeneous and exhibit domain differences, the updated model needs to avoid catastrophic forgetting while effectively generalizing to data flows with changing distributions. Therefore, domain incremental learning not only maintains stability with older data but also enhances its adaptability to new data.

[0104] Those skilled in the art will understand that all or part of the processes in the methods of any of the above embodiments of this application can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0105] Another aspect of this application provides a computer device. In one embodiment of the computer device according to this application, the computer device mainly includes a storage device and a processor. The storage device can be configured to store a program for executing the tooth image segmentation method of the above-described method embodiments, and the processor can be configured to execute the program in the storage device, which includes, but is not limited to, a program for executing the tooth image segmentation method of the above-described method embodiments. See also... Figure 7 , Figure 7 The illustration exemplarily shows a storage device 11 and a processor 12 connected via a bus. For ease of explanation, only the parts relevant to this embodiment are shown; for specific technical details not disclosed, please refer to the method section of this application.

[0106] In some possible implementations, the computer device may include multiple storage devices and multiple processors. These multiple processors may be processors deployed on the same device; for example, the computer device may be a high-performance device composed of multiple processors, and the multiple processors may be processors configured on that high-performance device. Alternatively, the multiple processors may be processors deployed on different devices; for example, the computer device may be a server cluster, and the multiple processors may be processors on different servers within the server cluster.

[0107] Another aspect of this application provides a computer-readable storage medium. In one embodiment of a computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program for performing the tooth image segmentation method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described tooth image segmentation method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

Claims

1. A method for segmenting tooth images, characterized in that, The method includes the following steps S101-S108: Step S101: Obtain an initial medical image dataset, selecting a first image dataset and a second image dataset; Step S102: Label the first image dataset, and obtain a first pre-trained segmentation network model through fully supervised training. The first pre-trained segmentation network model includes a first basic segmentation module and a first accuracy prediction module; S103: Using a domain incremental learning data replay strategy, the first basic segmentation module filters out some information-rich and diverse image datasets from the labeled first image dataset to form a labeled replay image dataset; Step S104: Using an active learning strategy, combined with a hybrid strategy of accuracy and diversity, sample the second image dataset to filter out a certain number of high-value image datasets; S105: Manually annotate the high-value image dataset to obtain an annotated first incremental image dataset; Step S106: Merge the annotated first incremental image dataset into the annotated replay image dataset to obtain an annotated second updated image dataset; Step S107: Use the annotated second updated image dataset to perform domain incremental learning on the first pre-trained segmentation network model, and train an optimized second pre-trained segmentation network model using a knowledge distillation strategy. The second pre-trained segmentation network model includes a second basic segmentation module and a second precision prediction module; Step S108: Repeat steps S104-S107 for other unannotated datasets until preset conditions are met; Step S109: Use the finally updated segmentation network model to obtain the image segmentation result.

2. The tooth image segmentation method according to claim 1, characterized in that, The initial medical image datasets are all unlabeled image datasets. The dataset with the largest number of samples is selected as the first image dataset, and the dataset with the second largest number of samples is selected as the second image dataset for subsequent steps.

3. The tooth image segmentation method according to claim 1, characterized in that, The other dataset is the second image dataset after removing the first incremental image dataset to form the updated second unlabeled image dataset. Steps S104-S107 are repeatedly executed using this dataset.

4. The tooth image segmentation method according to claim 1, characterized in that, The preset conditions are: the amount of labeled image data reaches the labeling budget, or the performance of the updated segmentation network model has reached the standard.

5. A method for segmenting a tooth image, characterized in that, The method includes the following steps S201-S209: Step S201: Obtain an initial medical image dataset and construct a training data queue Q as { , , ,..., }, and then train them one by one, selecting As the first image dataset, As the second image dataset; Step S202: For the first image dataset The first pre-trained segmentation network model is obtained through annotation and fully supervised training. The first pre-trained segmentation network model Includes a first basic segmentation module and a first precision prediction module; S203: Through a data replay strategy learned by domain incremental learning, the first basic segmentation module performs data replay on the labeled first image dataset. A subset of information-rich and diverse image datasets were selected to form the first labeled replay image dataset. And stored in the replay image data queue. In, constitute Step S204: Employ an active learning strategy, combining a hybrid strategy of accuracy and diversity, on the second image dataset. Sampling is performed to select a certain number of high-value image datasets; Step S205: The high-value image datasets are manually annotated to obtain the first annotated incremental image dataset. Step S206: Transfer the labeled first incremental image dataset And to the labeled replay image data queue Step S207: Obtain the labeled second updated image dataset; Step S208: Use the labeled second updated image dataset to train the first pre-trained segmentation network model. Domain incremental learning is performed, and an optimized second pre-trained segmentation network model is obtained by training using a knowledge distillation strategy. The second pre-trained segmentation network model Includes a second basic segmentation module and a second precision prediction module; Step S208: For the unlabeled dataset And the labeled replay image data queue that needs to be iteratively replayed, repeat steps S204-S207 until the preset conditions are met; Step S209: Use the finally updated segmentation network model to obtain the image segmentation result.

6. The tooth image segmentation method according to claim 5, characterized in that, In step S204, the second image dataset is processed. The second image dataset was selected through iterative comparison using Euclidean distance. A certain proportion of sample data is used, and then the first accuracy prediction module can obtain the segmentation prediction accuracy value of the unlabeled image. The segmentation prediction accuracy values ​​are then sorted in ascending order, and a certain proportion of the data samples are selected as the high-value image dataset.

7. The tooth image segmentation method according to claim 5, characterized in that, In step S208, the replay image data array utilizes the (t-1)th pre-trained segmentation network model. The (t-1)th basic segmentation module on the labeled (t-2)th incremental image dataset A subset of information-rich and diverse image datasets were selected to form the labeled (t-1)th replay image dataset. And stored in the replay data queue. In the middle, updated to .

8. The tooth image segmentation method according to claim 5, characterized in that, The repeated loop execution steps S204-S207 are as follows: An active learning strategy is adopted, combining a hybrid strategy of accuracy and diversity, to process the t-th image dataset. Sampling is performed to select a certain number of high-value image datasets; these high-value image datasets are then manually labeled to obtain the labeled (t-1)th incremental image dataset. The labeled (t-1)th incremental image dataset And to the already labeled replay image data queue Obtain the labeled updated image dataset of the tth generation; use the labeled updated image dataset of the tth generation to train the pre-trained segmentation network model of the (t-1)th generation. Domain incremental learning is performed, and an optimized pre-trained segmentation network model of the t-th generation is obtained using a knowledge distillation strategy. The t-th pre-trained model It includes the t-th basic segmentation module and the t-th precision prediction module.

9. A computer device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the tooth image segmentation method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the tooth image segmentation method according to any one of claims 1 to 8.

Images