A method and a system for the detection of one or more oral diseases using artificial intelligence

Abstract

The present technology relates to a method and a system for detection of oral diseases using artificial intelligence. The method includes capturing a plurality of images of one or more areas of the oral cavity including at least one of a cheek, a tongue, a floor of a mouth, a palate, and an inner lip by a user of the oral disease detection platform, via a mobile computing device of the user. The method further includes evaluating the captured images for generating a probability risk score for automated identification of oral lesions requiring clinical attention, where the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

Description

[0001] A METHOD AND A SYSTEM FOR THE DETECTION OF ONE OR MORE ORAL DISEASES USING ARTIFICIAL INTELLIGENCE

[0002] Field of Invention

[0003] The present technology relates to oral lesion detection. More particularly, the technology relates to a method and a system for detecting oral diseases using artificial intelligence.

[0004] Background of the invention

[0005] Currently, oral cancer affects more than 350,000 individuals annually and is one of the most common cancers in Asia, and nearly 74% of oral cancer-related deaths occur in Asians. Early detection and timely treatment can improve survival. However, more than 70% of oral cancers are detected at a later stage. Therefore, 50% of patients do not survive beyond 5 years and those who do, suffer significant morbidity, and have a poor quality of life. A major challenge for early diagnosis is access to accurate and timely diagnosis, particularly in remote areas.

[0006] Notably, oral cancer can be preceded by oral potentially malignant disorders (OPMD). Although early detection can be photographed easily, diagnosis requires examination by a specialist in the clinic. The accurate and timely diagnosis is a major challenge, particularly in remote areas. This causes poor survival due to the limited number of specialists and geographical barriers.

[0007] Hence, there is a need for an efficient digital health tool for screening and early detection of oral lesions.

[0008] Summary of the Invention

[0009] The present technology relates to a technology that enables clinicians to remotely provide clinical recommendations for patients with suspicious lesions and subsequently identify those who are most at risk of oral cancer for appropriate management. A critical component of the present technology and a success factor for large-scale implementation is the ability to acquire good-quality images and automatically triage patients, identifying those who require intervention particularly those at risk of oral cancer. The system of the present technology incorporates Al models for image classification that facilitate better quality control of the images submitted by the user and facilitate the automated identification of oral lesions that require clinical attention.

[0010] According to an aspect of the present technology, a method of early detection of oral lesions including oral cancer is provided. The method includes capturing a plurality of images of one or more areas of the oral cavity including at least one of a cheek, a tongue, a floor of a mouth, a palate, and an inner lip by a user of an oral disease detection platform, via a mobile computing device of the user. The method further includes evaluating the captured images using an artificial intelligence (Al) engine associated with the oral disease detection platform for automated feedback on image quality and generating a probability risk score for automated identification of oral lesions requiring clinical attention, where the probability risk score is indicative of the likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

[0011] In an embodiment, the method further includes storing the captured images in a database and sending the captured images to a healthcare practitioner for manual evaluation of the individual case collecting information by the oral disease detection platform (108) for each user, and saving the collected information in a database.

[0012] In an embodiment, evaluating the captured images further includes classifying the captured images into of: good quality and poor quality, classifying the captured images into one of: “Referral” or “No Referral,” passing the images classified as “Referral” for the classification of either “Refer for Risk” or “Refer for Other Reasons.” The resulting classification output presents the risk displayed as a probability score for each category.

[0013] In an embodiment, if the captured image is a classification of “Good” the user is prompted to submit the image and if the captured image is classified as “Poor,” the user is prompted to retake the image.

[0014] In an embodiment, passing the image classified as “Referral” for the classification of “Refer for Risk” or “Refer for Other Reasons” further includes referring the user to a dentist in case of images classified as “Refer for Other Reasons” which are benign lesions needing medical intervention and referring the user to a specialist if there is at least one lesion classified as “Refer for Risk” whereby an oral potentially malignant disorder (OPMD) or oral cancer is present requiring treatment from the specialist.

[0015] In an embodiment, based on the images submitted, if the dentist decides the images need to be referred to a specialist for confirmation, the images are sent to a specialist and the specialist reviews the images to reach a referral decision.

[0016] In an embodiment, once the specialist makes a referral decision, the oral disease detection platform notifies the dentist and the user for further action to be taken.

[0017] In an embodiment, the oral disease detection platform facilitates the rescheduling of an appointment and informs a healthcare practitioner if a user of the oral disease detection platform cancels the appointment.

[0018] In an embodiment, the oral disease detection platform records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment.

[0019] In an embodiment, the oral disease detection platform includes a data vault as a database for storing and securing at least one of screened individual information, one or more captured images, one or more annotations, related data, and machine learning models. The oral disease detection platform further includes a workbench as a web-based platform that hosts the data vault and facilitates the development and improvement of the oral disease detection platform. The oral disease detection platform further includes an Al engine for evaluating the captured images for generating a probability risk score for automated identification of oral lesions requiring clinical attention, where the probability risk score is indicative of the likelihood of the oral lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

[0020] According to another aspect of the present technology, a system of early detection of oral lesions is provided. The system includes a memory including one or more executable modules and a processor configured to execute the one or more executable modules for early detection of oral lesions using an oral disease detection platform accessible via a mobile computing device of a user. One or more executable modules include an image module for receiving a plurality of images of one or more areas of the oral cavity including at least one of the cheek, tongue, floor of the mouth, palate, and inner lip by the user and an oral disease detection platform including the Al engine for evaluating the captured images, evaluating the captured images using Al for generating a probability risk score for automated identification of oral lesions requiring clinical attention, where the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases and a database module for storing the captured images on a database and sending the captured image to a healthcare practitioner for manual evaluation of the individual case and collecting information for each user and saving the collected information in the database.

[0021] In an embodiment, the system further includes storing the captured images in a database and allowing the user to send the captured images to a healthcare practitioner for manual evaluation of the individual cases and collecting information by the oral disease detection platform for each user and saving the collected information in a database.

[0022] In an embodiment, the oral disease detection platform is further configured for classifying the captured images into of good quality and poor quality, classifying the captured images into one “Referral” or “No Referral”, passing the images classified as “Referral” for the classification of either “Refer for Risk” or “Refer for Other Reasons” and computing a probability risk score for providing information to a healthcare practitioner on the likelihood of the Al-predicted referral decision by computing a plurality of matrix operations for extracting relevant information by passing through an activation function.

[0023] In an embodiment, if the captured image is a classification of “Good” the user is prompted to submit the image and if the captured image is classified as “Poor,” the user is prompted to retake the image.

[0024] In an embodiment, passing the image classified as “Referral” for the classification of “Refer for Risk” or “Refer for Other Reasons” further includes referring the user to a dentist in case of benign lesions needing medical intervention and referring the user to a specialist if the lesions are at least one of an oral potentially malignant disorder (OPMD), oral cancer or other lesions requiring treatment from the specialist. In an embodiment, based on the images submitted, if the dentist decides that the images need to be referred to a specialist for confirmation, the images are sent to a specialist and the specialist reviews the images to reach a referral decision.

[0025] In an embodiment, once the specialist makes a referral decision, the oral disease detection platform notifies the dentist and the user for further action to be taken.

[0026] In an embodiment, if a visit to the dentist or specialist is required, appointments are scheduled on the oral disease detection platform and the user is notified via the oral disease detection platform.

[0027] In an embodiment, the oral disease detection platform facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment.

[0028] In an embodiment, the oral disease detection platform records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment.

[0029] In an embodiment, the oral disease detection platform includes a workbench as a web-based platform for facilitating the development and improvement of the oral disease detection platform and an Al engine for evaluating the captured images using an artificial intelligence (Al) engine associated with the oral disease detection platform for generating a probability risk score for automated identification of oral lesions requiring clinical attention. The probability risk score is indicative of the likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases. The probability risk score indicates a likelihood that the images present a case where the screened individual a)does not need to be referred for oral lesions, bjneeds to be referred for risk of oral cancer or an (oral potentially malignant disorder) OPMD, or c)needs to be referred but for oral lesions other than cancer or OPMD.

[0030] In an embodiment, the workbench includes at least one of an “upload” module for enabling data providers to upload oral images and metadata, an “annotate” module for enabling approved and appropriately trained annotators to annotate images and metadata, a “train” module for enabling approved Al partners to upload new Al models to be integrated into the oral disease detection platform and download images and metadata from the database for Al model training and a share module for enabling guests and researchers to view and download images and metadata from the database.

[0031] In an embodiment, the Al-powered oral disease detection platform enables users to record specified clinical data and capture images of the oral cavity and allows repeated documentation and recording of the information and communication between the users.

[0032] The present technology provides an Al-powered digital health tool for screening and early detection of oral lesions. The present technology increases reach particularly to remote areas, reducing reliance on on-site specialists. The present technology expands services on screening and diagnosis for oral lesions facilitating early detection and enabling equal access to healthcare, particularly for rural areas. The present technology increases survival and productivity through early detection and screened individual compliance. An important component of the oral disease detection platform and a success factor for large-scale implementation is the ability to acquire good-quality images and be able to automatically triage patients, identifying those who require intervention particularly those who are at risk of oral cancer. The present technology incorporates artificial intelligence (Al) models for image classification that facilitate better quality control of the images submitted by the user and facilitate the automated identification of oral lesions that require clinical attention. If a visit to the dentist or specialist is required, appointments can be scheduled on the application (App) / oral disease detection platform and the user would be notified via the App. The oral disease detection platform facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment. The oral disease detection platform records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment. All the information collected by the oral disease detection platform for each user is saved in the database.

[0033] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.

[0034] Brief Description of the Drawings

[0035] Other objects, features, and advantages of the invention will be apparent from the following description when read concerning the accompanying drawings. In the drawings, wherein like reference numerals denote corresponding parts throughout the several views:

[0036] FIGURE 1 depicts a system architecture of a system of early detection of oral lesions, in accordance with an embodiment.

[0037] FIGURE 2 depicts an Al-powered digital system of early detection of oral lesions, in accordance with an embodiment.

[0038] FIGURE 3A depicts images captured for early detection of oral lesions.

[0039] FIGURE 3B depicts a user interface of an exemplary scenario of the usage of the mechanism of the system of early detection of oral lesions.

[0040] FIGURE 4 depicts a flow process for early detection of oral cancer using the system of the present technology, in accordance with an embodiment.

[0041] FIGURE 5A-5C depicts the evaluation results of the image by the Al model, in accordance with an embodiment.

[0042] FIGURE 6 depicts the overall architecture of an oral lesion-detecting system of the present technology, in accordance with an embodiment.

[0043] FIGURE 7 depicts a flowchart of a method of early detection of oral lesions using an oral lesions detection system, in accordance with an embodiment.

[0044] FIGURE 8 depicts a flowchart of a method of using an oral lesion-detecting system, in accordance with an embodiment. Detailed Description of the Preferred Embodiments

[0045] In the following detailed description, numerous specific details are outlined to provide a thorough understanding of the invention. However, it will be understood by those of ordinary skill in the art that the invention may be practiced without these specific details. In other instances, well-known methods, procedures, and / or components have not been described in detail so as not to obscure the invention. Reference will now be made in detail to the preferred embodiments of the present technology, examples of which are illustrated in the accompanying drawings.

[0046] The present technology provides a method and system for the early detection of one or more oral diseases, such as oral lesions and other potentially malignant disorders. Various embodiments of the present technology provide an artificial intelligence (Al)-powered digital health tool for screening and early detection of oral diseases. The present technology increases reach particularly to remote areas, thereby reducing reliance on on-site specialists. The present technology expands the services on screening and diagnosis for oral diseases facilitating early detection and enabling equal access to healthcare, particularly for rural areas. The present technology increases survival and productivity for the nation through early detection and screened individual compliance. An important component of the present technology and a success factor for large-scale implementation is the ability to acquire goodquality images and automatically triage patients, by identifying those who require intervention, particularly those at risk of oral diseases. The present technology incorporates artificial intelligence (Al) models for image classification that facilitate better quality control of the images submitted by the user and facilitate the automated identification of oral lesions that require clinical attention. In several embodiments, if a visit to the dentist or specialist is required, appointments will be scheduled on the present system implemented as an application (App)on a mobile device of a user (such as a health care volunteer)and the user would be notified via the App. The App facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment. The App records the journey of the screened individuals enabling easy tracking of whether the screened individuals comply with their treatment. All the information collected by the App for each user is saved in the database. According to an embodiment, a registered user (for example, a healthcare volunteer, HCV) captures images of different areas of the oral cavity including the cheek, the tongue, the floor of the mouth, the palate, and the inner lip. As each of the images are taken, they are evaluated by the first Al model (Stage 1) where the Al model classifies the images into good or poor quality. Following a classification of “Good” the user is prompted to submit the image. On the other hand, when an image is classified as “Poor,” the user is prompted to retake the image. Once the required number of images are taken, the images (and / or the user’s associated data including demographics and risk habits) are captured in the database and sent to a healthcare practitioner (such as a dentist) for manual evaluation of the individual case. This review will culminate into a referral decision given by the dentist such as whether the screened individual does not need a referral, should be referred to a dentist in the case of benign lesions needing medical intervention or should be referred to a specialist if the lesions are an oral potentially malignant disorder (OPMD), oral lesions or any other lesions or oral disease requiring treatment from a specialist. This decision will be sent to the registered user so that the appropriate action can be taken. Based on the images submitted, if the dentist decides that the images need to be referred to a specialist for confirmation, those images will be sent to a specialist. The specialist will review the images to reach a referral decision. Once the specialist makes a referral decision, the App will notify the dentist and the HCV so that further action can be taken. In parallel to the referral decision made by the healthcare practitioner (dentists and specialists), inference is also made by the Al models (Stage 2 and 3). All images are passed through Stage 2 where the Al model classifies the image into “Referral” or “No Referral” and images classified as “Referral” would be passed to Stage 3 for the classification of “Refer for Risk” (for oral lesions or OPMD) or “Refer for Other Reasons”(for other lesions). The images are represented in tensor and when passed into the Al models, various matrix operations are computed to extract relevant information which is then passed through an activation function that computes a probability risk score. The probability risk score is indicative of the likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases. The probability risk score is a percentage, indicating the likelihood that the images present a case where the screened individual:

[0047] A. Does not need to be referred for oral lesions;

[0048] B. Needs to be referred for risk of oral cancer or OPMD; or

[0049] C. Needs to be referred but for oral lesions other than cancer or OPMD.

[0050] Each image will have a similarity measure to indicate which of the referral decisions it is most similar to. The output referral decision is the referral decision that has the highest similarity score. This score provides further information to the healthcare practitioner on the likelihood of the Al-predicted referral decision. The “no referral” shows an example such as “it is 89% likely that the referral decision is No Referral”). If a visit to the dentist or specialist is required, appointments will be scheduled on the App and the user would be notified via the system implemented or accessible via a mobile device of a user. The system facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment. The system records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment. All the information collected by the system for each user is saved in the database.

[0051] FIGURE 1 depicts a system architecture of a system 100 for detection of one or more oral diseases, in accordance with an embodiment. The system 100 comprises a memory 104 including one or more executable modules and a processor 102 configured to execute the one or more executable modules for early detection of oral lesions using a mobile computing device of a user. The one or more executable modules comprises an image module 106, an oral disease detection platform 108, and a database module 110 (used interchangeably with the term “database 110”). The image module 106 is configured for receiving a plurality of images of one or more areas of the oral cavity including at least one of a cheek, a tongue, a floor of the mouth, a palate, and an inner lip, by the user for example from a camera. The oral disease detection platform 108 comprising an Al engine is configured for evaluating the captured images for facilitating quality control of the images submitted by the user and performing an automated identification of oral lesions requiring clinical attention. The database module 110 is configured for storing captured images and sending the captured image to a healthcare practitioner for manual evaluation of the individual case and collecting information for each user and saving the collected information in a database.

[0052] In an embodiment, the system 100 is further configured for storing the captured images in a database and sending the captured images to a healthcare practitioner for manual evaluation of the individual case and collecting information by the oral disease detection platform 108 for each user and saving the collected information in a database.

[0053] In an embodiment, the oral disease detection platform 108 is further configured for classifying the captured images into one of good quality and poor quality, classifying the captured images into one of “Referral” or “No Referral”, passing the images classified as “Referral” for the classification of either “Refer for Risk” or “Refer for Other Reasons” and computing a probability risk score by computing a plurality of matrix operations for extracting relevant information by passing through an activation function.

[0054] In an embodiment, if the captured image is a classification of “Good” the user is prompted to submit the image and if the captured image is classified as “Poor,” the user is prompted to retake the image.

[0055] In an embodiment, passing the image classified as “Referral” for the classification of “Refer for Risk” or “Refer for Other Reasons” further includes referring the user to a dentist in case of benign lesions needing medical intervention and referring the user to a specialist if the lesions are at least one of an oral potentially malignant disorder (OPMD), oral lesions or other lesions requiring treatment from the specialist.

[0056] In an embodiment, based on the images submitted, if the dentist decides the images need to be referred to a specialist for confirmation, the images are sent to a specialist and the specialist reviews the images to reach a referral decision.

[0057] In an embodiment, once the specialist makes a referral decision, the oral disease detection platform 108 notifies the dentist and the user for further action to be taken.

[0058] In an embodiment, if a visit to the dentist or specialist is required, appointments are scheduled on the oral disease detection platform 108 and the user is notified via the oral disease detection platform 108.

[0059] In an embodiment, the oral disease detection platform 108 facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment.

[0060] In an embodiment, the oral disease detection platform 108 records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment.

[0061] In an embodiment, the oral disease detection platform 108 further includes a workbench (explained further in Figure 6) as a web-based platform for facilitating the development and improvement of the oral disease detection platform 108 and an oral disease detection platform 108 including the Al engine for evaluating the captured images for generating a probability risk score for automated identification of oral lesions requiring clinical attention, where the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

[0062] In an embodiment, the system further includes at least one an upload module for enabling data providers to upload oral images and metadata, an annotate module for enabling approved and appropriately trained annotators to annotate images and metadata, a train module for enabling approved Al partners to upload new Al models to be integrated into the oral disease detection platform 108 and download images and metadata from the database for Al model training and a share module for enabling guests and researchers to view and download images and metadata from the database .

[0063] In an embodiment, the oral disease detection platform 108 enables users to record specified clinical data and capture images of the oral cavity and allows repeated documentation and recording of the information and communication between users.

[0064] FIGURE 2 depicts an Al-powered digital system of early detection of oral lesions, in accordance with an embodiment. The healthcare workers 204 capture images and data of the screened individuals 202 using the Al-based system 206. The Al-based system 206 evaluates the image quality and prompts healthcare workers 204 to retake images if they are of poor quality. Once the images are captured, the image and data of the screened individuals 202 are stored in backend services in a cloud 208. The screened individuals 202 are advised to see clinicians 210 based on the referral decision using Al-based system 206.

[0065] FIGURE 3A depicts images captured for early detection of oral lesions, in accordance with an exemplary scenario. FIGURE 3B depicts a user interface 304 of an exemplary scenario of the usage of the mechanism of the system for early detection of oral lesions. More particularly, FIGURE 3B shows system 100 being used to capture an image for the detection of oral lesions in an exemplary scenario and the demonstrated teleconsultation has a 94.5% accuracy compared to the gold standard of a clinical oral examination (COE). FIGURE 4 depicts a flow process for early detection of oral lesions using the system of the present technology, in accordance with an embodiment. A registered user (for example, a healthcare volunteer, user) captures images 404 of different areas of the oral cavity including the cheek, tongue, the floor of the mouth, the palate, and the inner lip, and an image acquisition 402 takes place. As each of the images are taken, they are evaluated by the first Al model 406 (Stage 1 408) where the Al model 406 classifies the images into good quality 412 or poor quality 410. Following a classification of “Good quality” 412 the user is prompted to submit the image. On the other hand, when an image is classified as “Poor quality,” 410 the user is prompted to retake the image. Once the required number of images are taken, the images (and the screened individual’s associated data including demographics and risk habits) are captured in database 434 and sent to a healthcare practitioner (such as a dentist) for manual evaluation of the individual case. This review will culminate into a referral decision given by a dentist, including a) the screened individual does not need a referral, b) the screened individual should be referred to a dentist in the case of benign lesions needing medical intervention or, c) the screened individual should be referred to a specialist if the lesions are an oral potentially malignant disorder (OPMD), oral lesions or any other lesions requiring treatment from a specialist. This decision will be sent to the user so that the appropriate action can be taken. Based on the images submitted, if the dentist decides that the images need to be referred to a specialist for confirmation, those images will be sent to a specialist. The specialist will review the images to reach a referral decision. Once the specialist makes a referral decision, the App would notify the dentist and the user so that further action can be taken. The information stored in the database can be shared with researchers 436, it can be annotated 438 and it can be used to train 440 Al models 406. In parallel to the referral decision made by the healthcare practitioner (dentists and specialists), inference is also made by the Al models 406 (Stage 2 414 and Stage 3 422). All images are passed through Stage 2 414 where the Al model 406 classifies the image into “Referral” 420 or “No Referral” 416 and images classified as “Referral” 420 would be passed to Stage 3 422. The images in stage 3 422 are classified as benign 424 and potentially malignant 428. The images are passed to Stage 3 422 for the classification of “Refer for Risk” (for oral lesions or OPMD) 430 or “Refer for Other Reasons” (for other lesions) 426. The images in Stage 3 422 are classified as benign 424 and potentially malignant 428. The images are represented in tensor and when passed into the Al models 406, various matrix operations are computed to extract relevant information which is then passed through an activation function that computes the probability risk score 432. Each image will have a similarity measure to indicate which of the referral decisions it is most similar to. The output referral decision 432 is the referral decision that has the highest similarity score. This score of 432 provides further information to the healthcare practitioner on the likelihood of the Al-predicted referral decision. The output of no referral 416 shows as an example of “it is 89% likely that the referral decision is No Referral”. If a visit to the dentist or specialist is required, appointments will be scheduled on the App and the user will be notified via the App.

[0066] FIGURE 5A-5C depicts the evaluation of the image by the Al model, in accordance with an embodiment. FIGURE 5A shows stage 1 408 evaluation of Al model 406. As each of the images are taken, they are evaluated by the first Al model 406 (Stage 1 408) where the Al model 406 classifies the images into good 412 or poor quality 410. Following a classification of “Good” 412 the user is prompted to submit the image. On the other hand, when an image is classified as “Poor” 410, the user is prompted to retake the image. All images are passed through Stage 2 414 where the Al model 406 classifies the image into “Referral” 420 or “No Referral” 416 as shown in FIGURE 5B and the images classified as “Referral” 420 would be passed to Stage 3 422 for the classification of “Refer for Risk” (for oral cancer or OPMD) 430 or “Refer for Other Reasons” (for other lesions) 426 as shown in FIGURE 5C. FIGURES 5A- 5C also shows an output screenshot 442.

[0067] The present technology computer vision model aims to classify oral lesions including oral lesions from photographic images. The proposed model leverages state-of-the-art (SOTA) computer vision techniques and advanced machine learning algorithms to achieve high accuracy. Empirically, various techniques have been investigated to identify those with the highest performance. The classification of lesions is done in three separate stages first ensuring high image quality, identifying screened individuals who require medical attention, and finally, and importantly those who are at risk of developing oral lesions. The training dataset consists of white light images of the oral cavity that were labeled with ground truth for image quality and referral decision. All the image quality and referral decision ground truth were hand-labelled based on clinical diagnosis provided by collaborators that were further verified by two clinicians from CRMY. The dataset used for training and validation consists of a total of 7864 images, where 5821 are labelled as ‘good’ quality images while the remaining 2043 are labelled as ‘poor’ quality images 410. The 7864 images consist of 3267 ‘refer for risks’ 430, 1408 ‘refer for other reasons’ 426 and 3189 ‘no referral’ images 416. Considering that the deep learning model must be lightweight and portable to be deployed into a mobile device, the present technology uses the efficient net architecture as the backbone of the technology feature extractor as it is lightweight yet provides better performance when compared to other backbones (refer to Table 1). The backbone was inspired by developing a baseline network by leveraging a multi-objective neural architecture search that optimizes accuracy and floating-point operations per second (FLOPS). This produced an efficient network designated EfficientNet-BO as shown in Table 2. Branching off from EfficientNet- B0, 2019 used the compound scaling method to scale the model into six other larger variants, EfficientNet-Bl to B7.

[0068] Table 1

[0069] Table 1 shows Efficient Net Performance Results on ImageNet compared with other model backbones. Reference Tan, M., Le, Q. (2019b). Efficient Net: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:6105-6114, available from https: / / proceedings.mlr. press / v97 / tanl 9a.html. Table 2

[0070] Table 2 shows the architecture details of an EfficientNet-BO where each row describes stage i with Li layers. Reference Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019a). Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition (pp. 2820-2828).

[0071] The present technology designs the Al model 406 pipeline using Efficient Net architecture 602 in stages comprising Stage 1 408, Stage 2 414, and Stage 3 422. Stage 1 408 controls the quality of the image input into the pipeline. Images will be classified into two different qualities: ‘good’ quality or ‘poor’ quality 410. Stages 2 and 3 414 and 422 will classify the images based on the referral decision. For instance, Stage 2 414 is trained to classify the images into ‘no referral’ 416 or ‘referral’ 420. Meanwhile, in Stage 3 422, the model is trained to further classify those ‘referral’ 420 images into ‘refer for risk’ 430 or ‘refer for other reasons’ 426. In summary, Stage 1 408 is to classify the image quality into good 412 and poor 410 categories and Stage 2 414 is to classify the image into referral 420 and no referral 416 categories and Stage 3 422 is to classify the image into refer for risk 430 and refer for other reasons 426 categories. Accordingly, the present technology deploys deep learning for 3 stages (image quality checking up till referral decision classification) and is trained in an end-to-end manner. FIGURE 6 depicts overall architecture of the oral disease detection platform 108 of the present technology, in accordance with an embodiment. The present technology is a solution for oral lesions recognition using mobile application (App) and machine learning (ML) technologies. The oral disease detection platform detection 108 is a cloud-based platform and includes a workbench 614 and an Al engine 606. The workbench 614 drives the Al engine 606, and the Al engine 606 sits on the workbench 614. The workbench 614 is a web-based platform that operates along with four modules comprising an upload module 602, an annotate module 616, a train module 620 and a share module 610. The upload module 602 enables data providers such as collaborators 604 to upload oral images and metadata. The annotate module 616 enables approved and appropriately trained annotators 618 to annotate images and metadata. The train module 620 enables the approved Al partners 622 to upload new Al models 406 to be integrated into the system 100 and download images and metadata from the database 110 for Al model 406 training. The share module 610 enables guests and researchers 612 to view and download images and metadata from the database 110. The Al engine 606 enables users 608 to record specified clinical data and capture images of the oral cavity. It allows repeated documentation and recording of this information and communication between users. The Al engine 606 also performs Al powered classification of the images of the oral cavity.

[0072] In several exemplary scenarios, the present system 100 may be used as a screening tool, where the primary user of the system 100 is not the screened individuals themselves. This would occur in settings where the photos are captured by a healthcare volunteer / medical doctor / nurse / dentist and sent to a dentist and / or specialist. Once the primary user gets feedback from the dentist / specialist on the app, they will have to tell the screened individual verbally / by writing of the decision.

[0073] In several other exemplary scenarios, the present system 100 may be used as a teleconsultation tool. The primary user may be the screened individual themselves and they have routine appointments with their specialist because they have a known oral lesion. This would occur in settings where the photos are captured by the screened individual themselves and sent to their specialist via the app, either at the specialist’s request, or on the screened individual’s own initiative. The system 100 also allows the screened individual to have video call appointments with their specialist. FIGURE 7 depicts a flowchart of a method of early detection of oral lesions using an oral lesions detection system, in accordance with an embodiment. At step 702, a plurality of images of one or more areas of oral cavity including at least one of: a cheek, a tongue, a floor of a mouth, a palate, and an inner lip is captured by a user of an oral disease detection platform, via a mobile computing device of the user, the oral disease detection platform is accessible via the mobile computing device. At step 704, the captured images are evaluated using an artificial intelligence (Al) engine associated with the oral disease detection platform for generating a probability risk score for automated identification of oral lesions requiring clinical attention, where the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

[0074] In an embodiment, the method further includes storing the captured images in a database and sending the captured images to a healthcare practitioner for manual evaluation of the individual case and collecting information by the oral disease detection platform for each user and saving the collected information in a database.

[0075] In an embodiment, evaluating the captured images further includes classifying the captured images into one of: good quality and poor quality, classifying the captured images into one of: “Referral” or “No Referral”, passing the images classified as “Referral” for the classification of either “Refer for Risk” or “Refer for Other Reasons” and computing a probability risk score for providing information to a healthcare practitioner on the likelihood of the AI- predicted referral decision by computing a plurality of matrix operations for extracting relevant information by passing through an activation function.

[0076] In an embodiment, if the captured image is classified as “Good” the user is prompted to submit the image and if the captured image is classified as “Poor,” the user is prompted to retake the image.

[0077] In an embodiment, passing the image classified as “Referral” for the classification of “Refer for Risk” or “Refer for Other Reasons” further includes referring the user to a dentist in case of benign lesions needing medical intervention and referring the user to a specialist if the lesions are at least one of an oral potentially malignant disorder (OPMD), oral lesions or other lesions requiring treatment from the specialist. In an embodiment, based on the images submitted, if the dentist decides the images need to be referred to a specialist for confirmation, the images are sent to a specialist and the specialist reviews the images to reach a referral decision.

[0078] In an embodiment, once the specialist makes a referral decision, the oral disease detection platform notifies the dentist and the user for taking further actions.

[0079] In an embodiment, if a visit to the dentist or specialist is required, appointments are scheduled on the oral disease detection platform and the user is notified via the oral disease detection platform.

[0080] In an embodiment, the oral disease detection platform facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment.

[0081] In an embodiment, the oral disease detection platform records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment.

[0082] In an embodiment, the oral disease detection platform further includes a workbench as a webbased platform for facilitating the development and improvement of the oral disease detection platform, and an Al engine for evaluating the captured images for facilitating quality control of the images submitted by the user and performing an automated identification of oral lesions requiring clinical attention.

[0083] FIGURE 8 depicts a flowchart of a method of using an oral lesion-detecting system of the present technology, in accordance with an embodiment. At step 802, one or more images of the oral cavity are captured. In some embodiments, if the image quality is poor as determined by the Al engine, the images are captured again. At step 804, the system for early detection of oral lesions generates a probability risk score using Al. At step 806, a human clinician & clinician review the images and can view the risk score for referral decision. At step 808, the human clinician decides on the referral decision and sets up an appointment if needed. At step 810, the user is notified of the appointment. At step 812, the user acknowledges the appointment or reschedules or cancels. At step 814, the clinician is notified of the user’s action and acts accordingly - rescheduling if needed. At step 816, the patient attends an appointment if any, and if not cancelled. At step 818, the clinician captures images of screened individuals at an appointment and submits clinical data as a record using the platform. At step 820, the clinician can set up the next appointment (for a patient-specialist scenario where the patient is under their care).

[0084] Various embodiments of the present technology provide Al-powered digital health tools for screening and early detection of oral diseases. The oral disease detection platform of the present technology is lightweight and can be implemented for use via a mobile device. The present technology increases reaching particularly to remote areas, reducing reliance on onsite specialists. The present technology expands services on screening and diagnosis for oral diseases facilitating early detection and enabling equal access to healthcare, particularly for rural areas. The present technology increases survival and productivity for the nation through early detection and patient compliance. The present technology can acquire good-quality images and being able to automatically triage patients, identifying those who require intervention particularly those who are at risk of oral diseases. The latest iteration of the present technology incorporated artificial intelligence (Al) models for image classification that facilitate better quality control of the images submitted by the user and facilitate the automated identification of oral lesions that require clinical attention. If a visit to the dentist or specialist is required, appointments will be scheduled on the system and the user will be notified via the system. The system facilitates the rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment. The system records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment. All the information collected by the App for each user is saved in the database.

[0085] The systems described above may provide multiple ones of any or each of those components and these components may be provided on either a standalone machine or, in some embodiments, on multiple machines in a distributed system. The systems and methods described above may be implemented as a method, apparatus, or article of manufacture using programming and / or engineering techniques to produce software, firmware, hardware, or any combination thereof. In addition, the systems and methods described above may be provided as one or more computer-readable programs embodied on or in one or more articles of manufacture. The term “article of manufacture” as used herein is intended to encompass code or logic accessible from and embedded in one or more computer-readable devices, firmware, programmable logic, memory devices (e.g., EEPROMs, ROMs, PROMS, RAMS, SRAMs, etc.), hardware (e.g., integrated circuit chip, Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), etc.), electronic devices, a computer-readable non-volatile storage unit (e.g., CD-ROM, floppy disk, hard disk drive, etc.). The article of manufacture may be accessible from a file server providing access to the computer-readable programs via a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc. The article of manufacture may be a flash memory card or a magnetic tape. The article of manufacture includes hardware logic as well as software or programmable code embedded in a computer-readable medium that is executed by a processor. In general, computer-readable programs may be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. The software programs may be stored on or in one or more articles of manufacture as object code.

[0086] As will be readily apparent to those skilled in the art, the present technology may easily be produced in other specific forms without departing from its essential characteristics. The present embodiments are, therefore, to be considered as merely illustrative and not restrictive, the scope of the invention being indicated by the claims rather than the foregoing description, and all changes which come within therefore intended to be embraced therein.

Claims

CLAIMS1. A system (100) for the detection of one or more oral diseases, the system (100) comprising: a memory (104) comprising one or more executable modules; and a processor (102) configured to execute the one or more executable modules for early detection of oral lesions, the one or more executable modules comprising: an image module (106) for receiving a plurality of images of one or more areas of the oral cavity comprising at least one of: cheek, tongue, floor of the mouth, palate, and inner lip by the user; and an oral disease detection platform (108) comprising an Al engine for evaluating the captured images for generating a probability risk score for automated identification of oral lesions requiring clinical attention, wherein the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases. a database module (110) for storing the captured images on a database (434) and sending the captured image to a healthcare practitioner for manual evaluation of the individual case and collecting information for each user and saving the collected information in a database (434).

2. The system (100) of claim 1, further comprises storing the captured images in a database (434) and sending the captured images to a healthcare practitioner for manual evaluation of the individual case and collecting information by the oral disease detection platform for each user and saving the collected information in a database (434).

3. The system (100) of claim 1, wherein evaluating the captured images further comprises: classifying the captured images into one of: good quality (412) and poor quality (410); classifying the captured images into one of: “Referral” (420) or “No Referral” (416); and passing the images classified as “Referral” (420) for the classification of either “Refer for Risk” (430) or “Refer for Other Reasons” (426); andcomputing a probability risk score for providing information to a healthcare practitioner on the likelihood of the Al-predicted referral decision by computing a plurality of matrix operations for extracting relevant information by passing through an activation function.

4. The system (100) of claim 3, wherein if the captured image is a classification of “Good” (412) the user is prompted to submit the image and if the captured image is classified as “Poor” (410), the user is prompted to retake the image.

5. The system (100) of claim 3, wherein passing the image classified as “Referral” (420) for the classification of “Refer for Risk” (430) or “Refer for Other Reasons” (426) further comprises: referring the user to a dentist in case of benign lesions needing medical intervention; and referring the user to a specialist if the lesions is at least one of: an oral potentially malignant disorder (OPMD), oral lesions or other lesions requiring treatment from the specialist.

6. The system (100) of claim 5, wherein based on the images submitted, if the dentist decides the images need to be referred to a specialist for confirmation, the images are sent to a specialist and the specialist reviews the images to reach a referral decision.

7. The system (100) of claim 6, wherein once the specialist makes a referral decision, the oral disease detection platform (108) notifies the dentist and the user for further action to be taken.

8. The system (100) of claim 7, wherein if a visit to the dentist or specialist is required, appointments are scheduled on the oral disease detection platform (108)and the user is notified via the oral disease detection platform (108).

9. The system (100) of claim 1, wherein the oral disease detection platform (108) facilitates rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment.10 The system (100) of claim 1, wherein the oral disease detection platform (108) records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment.

11. The system (100) of claim 1, wherein the oral disease detection platform (108) further comprises: a workbench (614) as a web-based platform for facilitating a development and improvement of the oral disease detection platform; and an Al engine (606) for evaluating the captured images for generating a probability risk score for automated identification of oral lesions requiring clinical attention, wherein the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

12. The system (100) of claim 1, wherein the oral disease detection platform (108) further comprises at least one of: an upload module (602) for enabling data providers to upload oral images and metadata; an annotate module (616) for enabling approved and appropriately trained annotators to annotate images and metadata; a train module (620) for enabling approved Al partners to upload new Al models to be integrated into the oral disease detection platform (108) and download images and metadata from the database (110) for Al model training; and a share module (610) for enabling guests and researchers to view and download images and metadata from the database (110).

13. The system (100) of claim 1, wherein the oral disease detection platform (108) enables users to record specified clinical data and capture images of the oral cavity and allows repeated documentation and recording of the information and communication between users.

14. A method of early detection of one or more oral diseases, the method comprising: capturing (702) a plurality of images of one or more areas of oral cavity comprising at least one of: a cheek, a tongue, a floor of a mouth, a palate, and an innerlip by a user of an oral disease detection platform (108), via a mobile computing device of the user, wherein the oral disease detection platform (108) is accessible via the mobile computing device; and evaluating (704) the captured images using an artificial intelligence (Al) engine (806) associated with the oral disease detection platform (108)for generating a probability risk score for automated identification of oral lesions requiring clinical attention, wherein the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases.

15. The method of claim 14, further comprises storing the captured images in a database (110) and sending the captured images to a healthcare practitioner for manual evaluation of the individual case and collecting information by the oral disease detection platform (108)for each user and saving the collected information in the database (110).

16. The method of claim 14, wherein evaluating the captured images further comprises: classifying the captured images into one of: good quality (412) and poor quality (410); classifying the captured images into one of: “Referral” (420) or “No Referral” (416); passing the images classified as “Referral” (420) for the classification of either “Refer for Risk” (430) or “Refer for Other Reasons” (426); and computing a probability risk score for providing information to a healthcare practitioner on the likelihood of the Al-predicted referral decision by computing a plurality of matrix operations for extracting relevant information by passing through an activation function.

17. The method of claim 14, wherein if the captured image is a classification of “Good” (412) the user is prompted to submit the image and if the captured image is classified as “Poor” (410), the user is prompted to retake the image.

18. The method of claim 14, wherein passing the image classified as “Referral” (420) for the classification of “Refer for Risk” (430) or “Refer for Other Reasons” (426) further comprises: referring the user to a dentist in case of benign lesions needing medical intervention; and referring the user to a specialist if the lesions are at least one of: an oral potentially malignant disorder (OPMD), oral lesions or other lesions requiring treatment from the specialist.

19. The method of claim 18, wherein based on the images submitted, if the dentist decides the images need to be referred to a specialist for confirmation, the images are sent to a specialist and the specialist reviews the images to reach a referral decision.

20. The method of claim 18, wherein once the specialist makes a referral decision, the oral disease detection platform (108) notifies the dentist and the user for further action to be taken.

21. The method of claim 18, wherein if a visit to the dentist or specialist is required, appointments are scheduled on the oral disease detection platform (108)and the user notifies via the oral disease detection platform (108).

22. The method of claim 18, wherein the oral disease detection platform (108) facilitates rescheduling of an appointment and informs the healthcare practitioner if the user cancels the appointment.

23. The method of claim 18, wherein the oral disease detection platform (108) records the journey of the screened individuals enabling easy tracking of whether screened individuals comply with their treatment.

24. The method of claim 14, wherein the oral disease detection platform (108) comprises: a workbench (614) as a web-based platform for facilitating a development and improvement of the oral disease detection platform; andwherein the workbench (614) is communi cab ly coupled to a database(l 10)(l 10) for storing and securing at least one of: patient information, one or more captured images, one or more annotations, related data, and machine learning models and an Al engine(606) for evaluating the captured images for generating a probability risk score for automated identification of oral lesions requiring clinical attention, wherein the probability risk score is indicative of likelihood of the lesions requiring clinical attention by way of referral for risk of cancer or other oral diseases, and wherein the probability risk score indicates a likelihood that the images presents a case where the screened individual a)does not need to be referred for oral lesions, b)needs to be referred for risk of oral cancer or an (oral potentially malignant disorder) OPMD, or c)needs to be referred but for oral lesions other than cancer or OPMD.

Images