Multi-dimensional periodontitis identification and detection method based on incomplete supervision information

Abstract

The invention discloses a multi-dimensional periodontitis identification and detection method based on incomplete supervision information, and the method comprises the steps: firstly, obtaining a marked periodontal disease image, eliminating an image which cannot be identified, cutting out a part which needs to be identified in a remaining image, and carrying out the classification of the image according to two dimensions, namely whether the gingiva is inflamed or not and the periodontal retraction degree; thirdly, estimating label distribution of all samples based on a multi-dimensional classification model, and modifying training loss of each category of data; then, based on a multi-dimensional classification model, generating a pseudo label for the non-labeled periodontitis image data, obtaining a more accurate pseudo label by estimating the category deviation of existing parameters, and retraining the model by using a correction label; and finally, generating a final label corresponding to the image by using the trained model, and obtaining a final recognition result. According to the method, a semi-supervised learning framework is utilized, unbiased loss is obtained by estimating label distribution under the scene that only a small amount of annotated data exists and data categories are unbalanced, false labels are corrected by estimating category deviation, the two problems that the categories are unbalanced and the annotated data are limited are solved, and the model can achieve high recognition accuracy.

Description

Technical Field

[0001] The invention relates to a multi-dimensional periodontitis recognition and detection method based on incomplete supervision information, and belongs to the technical field of image data processing. Background Art

[0002] In current scientific research and clinical applications, the use of image data processing to judge periodontitis is mainly divided into two steps: the first step is to use target detection models such as Fast-RCNN to determine the area in the patient's periodontium that may have inflammation; the second step is to process the potential inflammation area in the previous step based on image classification models such as ResNet for classification judgment. However, this type of method has many problems in real scenarios. In clinical applications, medical imaging data must be obtained from actual patients, which is difficult to obtain on a large scale; secondly, the label distribution of the obtained data samples depends on the prevalence, resulting in an extremely imbalanced relationship between categories; finally, the annotation of medical imaging data must be performed by professional physicians, and the manpower and material resources required are much higher than those of ordinary image data. Therefore, medical imaging data often has many problems such as small total amount, imbalance, and few annotations. In the methods mentioned above, the small number of training samples will make it difficult for the model to learn enough information for judgment, and the imbalanced label distribution will cause the model to be more inclined to learn the majority class, while it is difficult to identify the minority class, and the problem of few annotations makes effective information extraction more difficult. In summary, although the current periodontitis classification method has achieved certain results, the problems of small quantity, imbalance, and few annotations of medical imaging data in practice make it difficult to be used in actual clinical applications. In view of the above defects, the present invention proposes a multi-dimensional periodontitis recognition and detection method based on incomplete supervision information. This method uses labeled samples to accurately estimate the label distribution and unlabeled samples, so that in the clinical application environment where the training data is small, unbalanced, and few labeled, the model can still achieve good classification and discrimination results. Summary of the invention

[0003] Purpose of the invention: Medical imaging data has many problems such as small quantity, imbalance, and few annotations. It is difficult for existing deep learning algorithms to train models with good generalization performance on them. To address this problem, the present invention adopts a semi-supervised learning framework to accurately estimate the label distribution and unlabeled samples using labeled samples during the training process of the neural network, thereby improving the final prediction accuracy of the model. Semi-supervised learning is a learning framework for scenarios with scarce labeled samples, which perfectly adapts to the scenario of medical image recognition. In actual medical scenarios, the semi-supervised learning framework can effectively estimate the true distribution, obtain high-quality pseudo-labels, and improve the accuracy of the model with a small number of labeled samples. In typical application scenarios, the algorithm system based on semi-supervised learning can help the neural network model get rid of its dependence on a large amount of labeled data, while also helping the machine learning task to collect more high-quality data and corresponding high-confidence pseudo-labels, providing a guarantee for the development of other research in the future.

[0004] The method of the present invention can help the deep learning model to effectively learn the current task using the information of unsupervised data when there is limited periodontitis data, while ensuring the stability of the accuracy, greatly reducing the workload of doctors in the data annotation stage.

[0005] Technical solution: A multidimensional periodontitis identification and detection method based on incomplete supervised information, which includes four parts: preprocessing of periodontal image data, initialization and supervised training of the model, pseudo-label generation and model update based on a semi-supervised learning framework, and generation of periodontal image recognition results.

[0006] In the preprocessing of periodontal images: obtain the annotated periodontitis images, exclude the unrecognizable images, and crop the parts that need to be recognized in the remaining images. Then classify the images according to whether the gums are inflamed (+: inflamed, -: not inflamed) and the degree of periodontal recession (0: no black triangle, 1: black triangle exists, 2: severe gum recession). There are 5 categories in total: periodontal health (gingival-, periodontal 0), gingival health (gingival-, periodontal 1), periodontitis (gingival+, periodontal 1), gingivitis (gingival+, periodontal 0), severe periodontitis (gingival+ / -, periodontal 2); During the initialization and supervised training of the model: prepare a multi-dimensional SEResNet pre-trained model; use the labeled periodontal image data to perform multiple rounds of gradient descent and back propagation to update the parameters of the pre-trained model to form the initial classification model M_0; In the pseudo-label generation and model update based on the semi-supervised learning framework: first collect unlabeled periodontal image data, denoted as data set S_0; then, use the initial classification model M_0 to generate pseudo-labels for the unlabeled data in S_0, and denoted the pseudo-labeled data set as S'_0; use the training sample label distribution in the most recent T rounds of updates as an estimate of the training set label distribution, use this estimate and the current model parameters to calculate the pseudo-label correction amount D_t, and add this value to the softmax output of the model to obtain the corrected pseudo-label; denoted the data set after the pseudo-label correction as S''_0, and synthesize it with the initial labeled periodontal image data into data set S_1, and use S_1 to perform a round of gradient descent and back propagation to update the parameters of the initial classification model M_0, and denoted the final model as M_1; use the model M_1 to generate pseudo-labels for the unlabeled periodontal image data in S_0 and correct them, repeat the above operation for k rounds until the model converges, and denoted the final classification model as M; In the generation of periodontal image recognition results: collect the periodontal image test set P; use the classification model M to generate recognition results for the test set P, record the final output logit of each sample, and The category with the highest confidence is taken as the periodontitis classification and recognition result of the patient with serial number i.

[0007] Among them, the periodontal image samples can be further preprocessed in the training and recognition stages, including scaling, adding noise, random perturbation, random cropping, normalization and other operations. The purpose of this is to increase the diversity of the sample space, while improving the generalization ability of the model, it can also improve the prediction performance of the model through integration.

[0008] The gradient descent refers to the stochastic gradient descent method, which iteratively updates the model parameters by calculating the gradient of the loss function on a small batch of data to help the model converge to a (local) optimal solution.

[0009] Beneficial effects: Compared with the existing technology, the multi-dimensional periodontitis recognition and detection method based on incomplete supervision information provided by the present invention proposes a feasible solution to the problems of small amount of image data and few annotations in the field of intelligent assisted medicine, and has been successfully applied in the auxiliary diagnosis of periodontal image diseases. While improving the model prediction accuracy and model training efficiency, it greatly reduces the workload of doctors and provides the possibility for the collection of large amounts of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a flow chart of preprocessing of periodontal image data in an embodiment of the present invention; Figure 2 Flow chart of the initialization and supervised training of the model in an embodiment of the present invention.

[0011] Figure 3 This is a flowchart of pseudo-label generation and model updating based on a semi-supervised learning framework in an embodiment of the present invention.

[0012] Figure 4 This is a flow chart of generating periodontitis prediction results in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0014] A multidimensional periodontitis identification and detection method based on incomplete supervised information, which includes four major processes: preprocessing of periodontal image data, initialization and supervised training of the model, pseudo-label generation and model update based on a semi-supervised learning framework, and generation of periodontal image recognition results.

[0015] The preprocessing process of periodontal image data is as follows: Figure 1 As shown. First, obtain the labeled periodontitis images and exclude the unrecognizable images (step 10); crop the parts that need to be recognized from the remaining images (step 11); classify the images according to whether the gums are inflamed (+: inflamed, -: not inflamed) and the degree of periodontal recession (0: no black triangle, 1: black triangle exists, 2: severe gum recession) (step 12), and obtain the data set S_0, which is divided into 5 categories in total: periodontal health (gingival-, periodontal 0), gingival health (gingival-, periodontal 1), periodontitis (gingival+, periodontal 1), gingivitis (gingival+, periodontal 0), severe periodontitis (gingival+ / -, periodontal 2); The process of model initialization and supervised training is as follows Figure 2 As shown. Prepare a multi-dimensional SEResNet pre-trained model (step 20); use the labeled periodontal image data to perform multiple rounds of gradient descent and back propagation to update the parameters of the pre-trained model to form an initial classification model M_0 (step 21); The process of pseudo-label generation and model updating based on the semi-supervised learning framework is as follows: Figure 3As shown. First, use the initial classification model M_0 to generate pseudo labels for the unlabeled data in S_0, and record the pseudo-labeled data set as S'_0 (step 30); use the training sample label distribution in the most recent T rounds of updates as an estimate of the training set label distribution, use the estimate and the current model parameters to calculate the pseudo-label correction amount D_t, and add this value to the softmax output of the model to obtain the corrected pseudo-label (step 31); record the data set after the pseudo-label correction as S''_0, and synthesize it with the initial labeled periodontal image data to form a data set S_1, and use S_1 to perform a round of gradient descent and back propagation to update the parameters of the initial classification model M_0, and record the final model as M_1 (step 32); if the training process meets the stopping requirements at this time, jump to step 34, otherwise jump to step 31 (step 33); output the classifier M obtained by the semi-supervised training process (step 34); The generation of periodontal image recognition results, the process of prediction and diagnosis of periodontal diseases Figure 4 First, collect the periodontal image test set P (step 40); use the classification model M to generate recognition results for the test set P, and record the final logit output of each sample (step 41); take the category with the highest confidence in logit_i as the periodontitis classification recognition result of the patient with serial number i (step 42).

[0016] Obviously, those skilled in the art should understand that the various steps of the multidimensional periodontitis identification and detection method based on incomplete supervision information of the above-mentioned embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiment of the present invention is not limited to any specific hardware and software combination.

Claims

1. A method for identifying and detecting periodontitis based on incomplete supervision information, characterized in that: It includes preprocessing of periodontal image data, initialization and supervised training of the model, pseudo-label generation and model update based on a semi-supervised learning framework, and generation of periodontal image recognition results; In the preprocessing of periodontal images: obtain the annotated periodontitis images, exclude the unrecognizable images, and crop the parts that need to be recognized in the remaining images. Then classify the images according to whether the gums are inflamed (+: inflamed, -: not inflamed) and the degree of periodontal recession (0: no black triangle, 1: black triangle exists, 2: severe gum recession). There are 5 categories in total: periodontal health (gingival-, periodontal 0), gingival health (gingival-, periodontal 1), periodontitis (gingival+, periodontal 1), gingivitis (gingival+, periodontal 0), severe periodontitis (gingival+ / -, periodontal 2); During the initialization and supervised training of the model: prepare a multi-dimensional SEResNet pre-trained model; use the labeled periodontal image data to perform multiple rounds of gradient descent and back propagation to update the parameters of the pre-trained model to form the initial classification model M_0; In the pseudo-label generation and model update based on the semi-supervised learning framework: first collect unlabeled periodontal image data, denoted as data set S_0; then, use the initial classification model M_0 to generate pseudo-labels for the unlabeled data in S_0, and denoted the pseudo-labeled data set as S'_0; use the training sample label distribution in the most recent T rounds of updates as an estimate of the training set label distribution, use this estimate and the current model parameters to calculate the pseudo-label correction amount D_t, and add this value to the softmax output of the model to obtain the corrected pseudo-label; denoted the data set after the pseudo-label correction as S''_0, and synthesize it with the initial labeled periodontal image data into data set S_1, and use S_1 to perform a round of gradient descent and back propagation to update the parameters of the initial classification model M_0, and denoted the final model as M_1; use the model M_1 to generate pseudo-labels for the unlabeled periodontal image data in S_0 and correct them, repeat the above operation for k rounds until the model converges, and denoted the final classification model as M; In the generation of periodontal image recognition results: collect the periodontal image test set P; use the classification model M to generate recognition results for the test set P, record the final output logit of each sample, and The category with the highest confidence is taken as the periodontitis classification and recognition result of the patient with serial number i.

2. The multi-dimensional periodontitis identification and detection method based on incomplete supervision information according to claim 1 is characterized in that: The specific implementation process of preprocessing periodontal image data is as follows: Step 100, collecting multi-dimensional periodontal images, excluding unrecognizable images, and cropping the parts of the remaining images that need to be recognized; Step 101, for the annotated multi-dimensional periodontal images, classify the images according to two dimensions: whether the gums are inflamed (+: inflamed, -: not inflamed) and the degree of periodontal recession (0: no black triangle, 1: black triangle exists, 2: severe gum recession), and divide them into five categories in total: periodontal health (gingiva -, periodontal 0), gingival health (gingiva -, periodontal 1), periodontitis (gingiva +, periodontal 1), gingivitis (gingiva +, periodontal 0), severe periodontitis (gingiva + / -, periodontal 2), and obtain the data set S_0.

3. The multi-dimensional periodontitis identification and detection method based on incomplete supervision information according to claim 1 is characterized in that: The specific implementation process of model initialization and supervised training is as follows: Step 200, prepare a multi-dimensional SEResNet pre-training model; Step 201, use the label distribution of the training samples in the most recent T rounds to estimate the true label distribution, use this distribution to adjust the cross entropy loss function and use the labeled data in S_0 to perform gradient descent and back propagation to update the parameters of the pre-trained model until the model is overfitted on the training set and the accuracy on the validation set no longer increases significantly. At this time, the pre-training is considered to be completed and the current model is recorded as the initial classification model M_0.

4. The multi-dimensional periodontitis identification and detection method based on incomplete supervision information according to claim 1 is characterized in that: The pseudo-label generation and model update based on the semi-supervised learning framework are as follows: Step 300, using the initial classification model M_0 to generate pseudo labels for the unlabeled data in S_0, and the dataset with pseudo labels is recorded as S'_0; Step 301: Use the training sample label distribution in the most recent T rounds of updates as an estimate of the training set label distribution, use the estimate and the current model parameters to calculate the pseudo-label correction value D_t, and add the value to the softmax output of the model to obtain the corrected pseudo-label; Step 302, the dataset after pseudo-label correction is recorded as S''_0, and it is synthesized with the initial labeled periodontal image data into a dataset S_1, and S_1 is used to perform a round of gradient descent and back propagation to update parameters for the initial classification model M_0, and the final model is recorded as M_1; Step 303, if the training process meets the stop requirement at this time, jump to step 304, otherwise jump to step 301; Step 304: output the classifier M finally obtained in the semi-supervised training process.

5. The multi-dimensional periodontitis identification and detection method based on incomplete supervision information according to claim 4 is characterized in that: The generation of periodontal image recognition results is as follows: Step 400, collecting an unlabeled multimodal periodontal image test set P; Step 401: for each sample p in the test set P, use the classification model M to generate its classification recognition result, and record the final output logit of each sample; Step 402: The category with the highest confidence is taken as the periodontitis classification and recognition result of the patient with serial number i.

6. The multi-dimensional periodontitis identification and detection method based on incomplete supervision information according to claim 4 is characterized in that: In the pseudo-label generation and model updating process based on the semi-supervised learning framework, the method for determining that the training process is stopped is to reach a preset number of training rounds or the model reaches overfitting.

7. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the multi-dimensional periodontitis identification and detection methods based on incomplete supervision information is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing any one of the multi-dimensional periodontitis identification and detection methods based on incomplete supervision information.

Images