Periodontitis identification system based on periodontal picture

Abstract

The invention relates to a periodontitis recognition system based on periodontal photos, which is mainly used for automatically recognizing periodontitis through the periodontal photos. The system comprises five modules: an image acquisition and preprocessing module, a multi-instance learning model training module, a classification and diagnosis module, a system common user side and a system expert user side. According to the system, through an image processing and target detection technology, firstly, collected periodontal pictures are preprocessed, low-quality pictures are eliminated, and tooth areas are cut out; and then classifying the images by using a multi-instance learning framework in combination with the comparison loss, and identifying the risk level of periodontitis. The user side provides a visual diagnosis result, and the expert side provides more detailed analysis information and suggestions for experts. The periodontitis diagnosis precision can be effectively improved, the clinical workload can be reduced, early-stage automatic identification of periodontitis can be realized, and timely and effective treatment suggestions can be provided for patients.

Description

Technical Field

[0001] The present invention relates to medical image processing technology, in particular to periodontitis diagnosis in the dental field, and specifically to a periodontitis recognition system based on periodontal photographs, which uses machine learning and image processing technology to achieve early automatic recognition of periodontitis. Background Art

[0002] Periodontitis is one of the common oral diseases, and its early diagnosis is crucial for the prevention and treatment of the disease. Traditional periodontitis diagnosis usually relies on the experience of clinicians and often requires manual analysis of a large number of X-rays or other medical images, which is time-consuming and laborious. With the development of artificial intelligence technology, automatic recognition based on medical images has gradually become an effective diagnostic auxiliary tool. Existing periodontitis diagnostic methods still have problems such as low recognition accuracy and low degree of automation. Therefore, there is an urgent need for a more efficient and accurate periodontitis recognition system. Summary of the invention

[0003] Purpose of the invention: Medical image data in the dental field has problems such as small quantity, scarce annotations, and imbalanced categories. It is difficult for existing deep learning algorithms to train models with good generalization ability under these conditions, especially in the early identification task of periodontitis, where the scarcity of data and the lack of annotations are particularly prominent. In order to solve these problems, the present invention proposes an automatic periodontitis recognition system based on periodontal photos, aiming to improve the accuracy and efficiency of periodontitis diagnosis and reduce manual intervention through intelligent image processing and multi-instance learning algorithms. Multi-instance learning is a learning method suitable for scenarios with scarce labels and imbalanced categories. It can achieve accurate classification of the entire sample by extracting features and weighted fusion of multiple instances in each sample. Unlike traditional single-label classification methods, multi-instance learning can handle situations where each sample contains multiple instances, thereby effectively improving the classification accuracy of the model, especially when the tooth area is partially occluded or the noise interference is large. Multi-instance learning can focus on key areas by weighting the importance of instances, thereby reducing interference and improving recognition effects. The system also hopes to set up different user terminals for different types of users. The general user terminal simplifies the operation process, allowing non-professional users to easily upload periodontal photos and obtain risk assessments; the expert user terminal provides more professional functions, such as detailed diagnostic reports and analysis charts, to facilitate experts to make further diagnosis and treatment decisions. This design greatly improves the ease of use and universality of the system, allowing it to be widely used in clinical and daily health management.

[0004] Technical solution: A periodontitis recognition system based on periodontal photos, including five parts: image acquisition and preprocessing module, multi-instance learning model training module, classification diagnosis module, system general user end and system expert user end; Image acquisition and preprocessing module: This module is used to collect periodontal photos and preprocess the images through rules and algorithms. The preprocessing part includes image quality screening, eliminating low-quality images, and ensuring data validity; combining rules and YOLO algorithm to detect and crop the tooth area, and extract the effective area required for periodontitis identification.

[0005] Multi-instance learning model training module: This module is responsible for multi-instance learning training of processed images and classifies periodontitis based on feature extraction and weighted fusion mechanism. The multi-instance learning framework effectively identifies the risk of periodontitis by extracting the features of each instance (tooth area) and weightedly fusing these instance features through the attention mechanism.

[0006] Classification and diagnosis module: The classification and diagnosis module uses the trained model to automatically identify the input periodontal photos and output the classification results (healthy periodontium, mild periodontitis, moderate periodontitis, severe periodontitis). This module not only provides classification results, but also combines corresponding diagnostic suggestions and displays the results to users or experts through the interface of the general user terminal or expert user terminal.

[0007] System general user terminal: The general user terminal is mainly used for daily periodontitis risk assessment. Through the user interface, general users can upload periodontal photos, and the system automatically processes and diagnoses the images and outputs the corresponding periodontitis risk level. The user system interface is simple and easy to use, providing intuitive diagnostic results, including periodontitis risk assessment reports and recommendations.

[0008] System expert user terminal: The expert user terminal is used by professional dentists or experts to conduct more in-depth analysis and diagnosis. Expert users can view the diagnosis results in detail through the expert system and propose personalized treatment plans based on specific circumstances. The expert terminal interface includes detailed analysis charts, periodontal photos uploaded by users, and comparative analysis results, etc., providing information to support experts' decision-making.

[0009] In the generation of periodontal image recognition results: collect the periodontal image test set T; use the classification model M to generate recognition results for the test set T, and record the final output logit of each sample; collect the recognition results of multimodal images from the same patient, and use the multimodal fusion integration method to generate the patient's final periodontal image recognition results.

[0010] The implementation of the multi-instance learning model training module is as follows: Step 101, feature extraction and instance segmentation, the input data contains multiple instances (one instance for each tooth region), which constitute a complete sample. The multiple instances of each sample are unpacked in the batch dimension through the correlation function and are sent to the model separately for processing in the subsequent processing. Through the ResNet50 network, each instance (tooth region) will be extracted through the deep convolution layer to output a multi-dimensional feature vector.

[0011] Step 102, introduce the compression and excitation module to weight the feature vector of each instance. First, generate an attention weight with the same dimension as the input feature vector through the fully connected layer, and use these weights to enhance the features of key instances and suppress the features of irrelevant instances. This process can effectively highlight the most representative instances and reduce the impact of noise on the final classification.

[0012] Step 103, generate an attention weight for each instance through the multi-instance attention mechanism module. The module will calculate the contribution of each instance to the entire sample classification result based on its features. Instances with higher weights have a greater impact on the final decision, and finally the features of all instances are fused by weighted summation.

[0013] Step 104, each instance feature is fused through the attention weight to calculate a weighted global feature vector. The fused feature vector is output through the fully connected layer to the final classification result. The classification result will determine the risk level of periodontitis (such as healthy, mild, moderate, severe periodontitis).

[0014] Step 105, based on the multi-instance learning model module described above, use the cross entropy loss function to perform category classification, and optionally apply contrast loss to strengthen the model's learning of the similarity between instances, and further optimize the model's discrimination. When processing multi-class imbalanced data, the model will also adjust the loss of each category through the category reweighting mechanism to enhance the model's learning ability for rare categories. The model is trained using the stochastic gradient descent method, and the learning rate is gradually adjusted during the training process through the learning rate scheduler to ensure the stability and effect of the training. The model will periodically save the training status and automatically save the weights of the current model when a specific training round is reached (such as every 30 rounds).

[0015] Among them, the periodontal image samples can be further preprocessed in the training and recognition stages, including scaling, adding noise, random perturbation, random cropping, normalization and other operations. The purpose of this is to increase the diversity of the sample space, while improving the generalization ability of the model, it can also improve the prediction performance of the model through integration.

[0016] The gradient descent refers to the stochastic gradient descent method, which iteratively updates the model parameters by calculating the gradient of the loss function on a small batch of data to help the model converge to a (local) optimal solution.

[0017] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the periodontitis recognition system based on periodontal photographs is provided.

[0018] A computer-readable storage medium stores a computer program for executing the periodontitis identification system based on periodontal photographs as described above.

[0019] Beneficial effects: Compared with the prior art, the present invention can effectively improve the recognition accuracy of periodontitis by combining image processing, target detection and multi-instance learning algorithms. Through intelligent image processing and classification algorithms, the system can automatically screen and crop tooth areas, greatly reducing manual intervention. At the same time, using the multi-instance learning algorithm, the system can process multiple tooth instances, further improving the classification accuracy. After adding contrast loss, the classification accuracy of the model has been significantly improved, which can better identify the early signs of periodontitis and provide strong support for clinical diagnosis. In addition, the system is designed with two user ends, one for ordinary users and one for expert users, to facilitate different users to conduct risk assessment and further diagnostic analysis of periodontitis. Overall, the present invention provides effective support for the early diagnosis of periodontitis and provides clinicians with an intelligent decision-making support tool. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a framework diagram of a periodontitis identification system based on periodontal photographs referred to in the present invention; Figure 2 This is a structural diagram of the image preprocessing and cropping module in the present invention; Figure 3 It is a module structure diagram of the multi-instance learning model in the present invention; Figure 4 This is a common user terminal operation flow chart in the present invention; Figure 5 This is an operation flow chart of the expert user terminal in the present invention. DETAILED DESCRIPTION

[0021] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0022] A periodontitis recognition system based on periodontal images includes five parts: image acquisition and preprocessing module, multi-instance learning model training module, system general user terminal, system expert user terminal and classification and diagnosis module.

[0023] Image acquisition and preprocessing module: This module is used to collect periodontal photos and preprocess the images through rules and algorithms. The preprocessing part includes image quality screening, eliminating low-quality images, and ensuring data validity; combining rules and YOLO algorithm to detect and crop the tooth area, and extract the effective area required for periodontitis identification.

[0024] Multi-instance learning model training module: This module is responsible for multi-instance learning training of processed images and classifies periodontitis based on feature extraction and weighted fusion mechanism. The multi-instance learning framework extracts the features of each instance (tooth area) and weightedly fuses these instance features through the attention mechanism, thereby effectively identifying the risk of periodontitis. The specific implementation is: Step 1: Feature extraction and instance segmentation. The input data contains multiple instances (one instance for each tooth region), which constitute a complete sample. The multiple instances of each sample are unpacked in the batch dimension through the correlation function and are fed into the model separately for processing in the subsequent processing. Through the ResNet50 network, each instance (tooth region) is extracted through a deep convolutional layer and outputs a multi-dimensional feature vector.

[0025] Step 2: Introduce the compression and excitation module to weight the feature vector of each instance. First, generate an attention weight with the same dimension as the input feature vector through the fully connected layer, and use these weights to enhance the features of key instances and suppress the features of irrelevant instances. This process can effectively highlight the most representative instances and reduce the impact of noise on the final classification.

[0026] Step 3: Generate an attention weight for each instance through the multi-instance attention mechanism module. This module will calculate the contribution of each instance to the entire sample classification result based on its features. Instances with higher weights have a greater impact on the final decision, and finally the features of all instances are fused through weighted summation.

[0027] Step 4: Each instance feature is fused through the attention weight to calculate a weighted global feature vector. The fused feature vector is output through the fully connected layer to obtain the final classification result. The classification result will determine the risk level of periodontitis (such as healthy, mild, moderate, and severe periodontitis).

[0028] Step 5, based on the multi-instance learning model module described above, use the cross entropy loss function to perform category classification, and optionally apply contrast loss to strengthen the model's learning of the similarity between instances, and further optimize the model's discrimination. When processing multi-class imbalanced data, the model also adjusts the loss of each category through the category reweighting mechanism to enhance the model's learning ability for rare categories. The model is trained using the stochastic gradient descent method, and the learning rate is gradually adjusted during the training process through the learning rate scheduler to ensure the stability and effect of the training. The model will periodically save the training status and automatically save the current model M when a specific training round is reached (such as every 30 rounds).

[0029] Classification and diagnosis module: The classification and diagnosis module uses the trained model M to automatically identify the input periodontal photo T and output the classification results (healthy periodontium, mild periodontitis, moderate periodontitis, severe periodontitis). This module not only provides classification results, but also combines corresponding diagnostic suggestions and displays the results to users or experts through the interface of the general user terminal or expert user terminal.

[0030] System general user terminal: The general user terminal is mainly used for daily periodontitis risk assessment. Through the user interface, general users can upload periodontal photos, and the system automatically processes and diagnoses the images using model M, and outputs the corresponding periodontitis risk level. The user system interface is simple and easy to use, providing intuitive diagnostic results, including periodontitis risk assessment reports and recommendations.

[0031] System expert user terminal: The expert user terminal is used by professional dentists or experts to conduct more in-depth analysis and diagnosis. Expert users can view the diagnosis results in detail through the expert system and propose personalized treatment plans based on specific circumstances. The expert terminal interface includes detailed analysis charts, periodontal photos uploaded by users, and comparative analysis results, etc., providing information to support experts' decision-making.

[0032] Obviously, those skilled in the art should understand that the modules of the periodontitis identification system based on periodontal photographs of the above-mentioned embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

Claims

1. A periodontitis recognition system based on periodontal photographs, characterized in that: It includes image acquisition and preprocessing module, multi-instance learning model training module, classification and diagnosis module, system general user terminal and system expert user terminal; Image acquisition and preprocessing module: This module is used to collect periodontal photos and preprocess the images through rules and algorithms. The preprocessing part includes image quality screening, eliminating low-quality images, and ensuring data validity; combining rules and YOLO algorithm to detect and crop the tooth area, and extract the effective area required for periodontitis identification; Multi-instance learning model training module: This module is responsible for multi-instance learning training of processed images and classifies periodontitis based on feature extraction and weighted fusion mechanism. The multi-instance learning framework extracts the features of each instance (tooth area) and weightedly fuses these instance features through the attention mechanism, thereby effectively identifying the risk of periodontitis; Classification and diagnosis module: The classification and diagnosis module uses the trained model to automatically identify the input periodontal photos and output the classification results (healthy periodontium, mild periodontitis, moderate periodontitis, severe periodontitis). This module not only provides classification results, but also combines corresponding diagnostic suggestions and displays the results to users or experts through the interface of the general user terminal or expert user terminal; System general user terminal: The general user terminal is mainly used for daily periodontitis risk assessment. Through the user interface, general users can upload periodontal photos, and the system automatically processes and diagnoses the images and outputs the corresponding periodontitis risk level. The user system interface is simple and easy to use, providing intuitive diagnostic results, including periodontitis risk assessment reports and recommendations; System expert user terminal: The expert user terminal is used by professional dentists or experts to conduct more in-depth analysis and diagnosis. Expert users can view the diagnosis results in detail through the expert system and propose personalized treatment plans based on specific circumstances. The expert terminal interface includes detailed analysis charts, periodontal photos uploaded by users, and comparative analysis results, etc., providing information to support experts' decision-making.

2. The periodontitis identification system based on periodontal photographs according to claim 1, characterized in that: The implementation of the multi-instance learning model training module is as follows: Step 101, feature extraction and instance segmentation, the input data contains multiple instances (one instance for each tooth region), which constitute a complete sample. The multiple instances of each sample are unpacked in the batch dimension through the correlation function and are sent to the model separately for processing in the subsequent processing. Through the ResNet50 network, each instance (tooth region) will be extracted through the deep convolution layer to output a multi-dimensional feature vector; Step 102, introduce the compression and excitation module to weight the feature vector of each instance. First, generate an attention weight with the same dimension as the input feature vector through the fully connected layer, and use these weights to enhance the features of key instances and suppress the features of irrelevant instances. This process can effectively highlight the most representative instances and reduce the impact of noise on the final classification; Step 103, generate an attention weight for each instance through the multi-instance attention mechanism module. The module will calculate the contribution of each instance to the entire sample classification result based on its features. Instances with higher weights have a greater impact on the final decision, and finally the features of all instances are fused by weighted summation; Step 104, each instance feature is fused through the attention weight to calculate a weighted global feature vector. The fused feature vector is output through the fully connected layer to the final classification result. The classification result will determine the risk level of periodontitis (such as healthy, mild, moderate, severe periodontitis); Step 105, based on the multi-instance learning model module described above, use the cross entropy loss function to perform category classification, and optionally apply contrast loss to strengthen the model's learning of the similarity between instances, and further optimize the model's discrimination. When processing multi-class imbalanced data, the model will also adjust the loss of each category through the category reweighting mechanism to enhance the model's learning ability for rare categories. The model is trained using the stochastic gradient descent method, and the learning rate is gradually adjusted during the training process through the learning rate scheduler to ensure the stability and effect of the training. The model will periodically save the training status and automatically save the weights of the current model when a specific training round is reached (such as every 30 rounds).

3. A computer device, characterized in that: The computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the periodontitis identification system based on periodontal photographs as claimed in claims 1 and 2 is implemented.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the periodontitis recognition system based on periodontal photographs as claimed in claims 1 and 2 .

Images