Dental image analysis system and method

Abstract

A system and method for analyzing dental radiographs utilizes deep neural network architectures, model training procedures, and data processing approaches for automating dental mapping and disease detection. The system and method generates detailed output that constitutes a comprehensive analysis of the dental radiograph, relating detected conditions to specific teeth.

Description

[Background technology]

[0001] In the field of dentistry, dental radiographs are a tool for identifying various conditions that are not easily detected during a clinical exam. These x-rays provide detailed information about a patient's oral health and can detect a variety of issues, from cavities to more complex conditions such as bone loss and hidden tooth structure. However, the process of analyzing these x-rays and recording the results can be tedious and time-consuming for dental professionals. This is especially true when the analysis involves numerous x-rays or complex cases with multiple dental conditions.

[0002] The challenge lies not only in interpreting radiographs, but also in recording and documenting findings accurately. Manual record-keeping is prone to errors and inconsistencies and can be a major waste of resources in dental practices. Furthermore, the complexity of tooth structure, which often exhibits irregular shapes and curves, makes the task even more difficult.

[0003] In recent years, technological advances have facilitated the development of automated systems for analyzing dental images. These systems typically use image recognition techniques to identify and locate various objects within an image. However, traditional image recognition systems typically use bounding boxes to indicate the location of detected objects. While this approach can identify the presence and approximate location of an object, it cannot accurately delineate its boundaries, especially for non-rectangular objects such as teeth.

[0004] Automating dental image analysis requires more sophisticated image detection and processing methods. Specifically, methods are needed that not only identify and locate objects in images, but also accurately delineate their boundaries. Such methods would allow for a more accurate representation of object shape and size, which is particularly important in the case of dental radiographs.

[0005] Improved methods and systems for automating dental image analysis remain desirable. Summary of the Invention

[0006] The present invention relates to a system and method for automated analysis of dental images. The system and method leverage advanced techniques such as object segmentation to provide more accurate and efficient analysis of dental radiographs. The goal is to address the limitations of existing methods, improve the accuracy of dental charts, and reduce the time and effort required for this task.

[0007] In a first embodiment, a system for analyzing dental radiographs includes a dental numbering module that receives at least one dental radiograph and locates and labels teeth contained therein; a condition detection module that is configured to receive the dental radiograph and identify and locate conditions present therein; and a fusion module that is configured to receive a first output from the tooth numbering module and a second output from the condition detection module and fuse both outputs to generate a report that includes labeled teeth with conditions identified and located for the particular teeth.

[0008] In another embodiment, the system further includes an image type classifier for determining the type of dental x-ray.

[0009] In a first alternative configuration, the condition detector locates the conditions identified with bounding boxes and masks. In a second alternative configuration, the condition detection module uses multiple condition detection models, each configured and developed for a particular type of dental radiograph. In a third alternative configuration, the condition detection module uses an object detection model to identify conditions present in at least one dental radiograph. In a fourth alternative configuration, the condition detection module uses an object segmentation model to locate conditions present in at least one dental radiograph.

[0010] In another alternative configuration, the fusion module is configured to fuse the first output and the second output such that the teeth and conditions are independently observable by the display device. In another alternative configuration, the fusion module is configured to fuse the first output and the second output such that individual teeth and associated conditions are displayed on the display device.

[0011] In another alternative configuration, the system includes a queue manager for managing operations on the tooth numbering module, the condition detection module, and the fusion module.

[0012] In another alternative configuration, the tooth numbering module uses an object segmentation model to identify and label teeth present in at least one dental radiograph. In another configuration, the dental number module is configured with multiple dental number models, each developed for a particular type of dental radiograph.

[0013] In another embodiment, a computer method for analyzing dental radiographs includes receiving at least one dental radiograph, the dental radiograph having a type; analyzing the radiograph to locate and label teeth contained therein to generate a first output; analyzing the radiograph to locate and identify conditions present therein to generate a second output; and fusing the first output from the tooth numbering module and the second output from the condition detection module to generate a report including the teeth labeled with conditions identified and located on the particular teeth.

[0014] In an alternative configuration, the method further includes identifying a type of dental radiograph. In another configuration, the type of dental radiograph is bitewing, periapical, or panoramic. In another alternative configuration, the method includes analyzing the at least one dental radiograph using a neural network to locate and label the teeth. In another alternative configuration, the neural network performs the analysis in a single pass. In another alternative embodiment, the method further includes generating a report configured to display individual teeth and their associated conditions on a display device.

[0015] In another embodiment, a non-transitory computer-readable storage medium having program instructions for analyzing dental radiographs, when executed on a processor, implements a method including receiving at least one dental radiograph having a type; analyzing the radiograph to locate and label teeth contained therein to generate a first output; analyzing the radiograph to locate and identify conditions present therein to generate a second output; and fusing the outputs of the tooth numbering module and the condition detection module to generate a report including labeled teeth with specific tooth conditions identified and their locations indicated.

[0016] The present invention, together with these and other advantages, will be better understood from the following detailed description of the embodiments thereof as illustrated in the drawings. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 shows a block diagram of an embodiment of an automated radiograph analysis system in accordance with the principles of the present invention.

[0018] [Figure 2] FIG. 2 shows a block diagram of the state detection module of FIG.

[0019] [Figure 3] FIG. 3 is a block diagram showing the data flow of the fusion module of FIG.

[0020] [Figure 4] FIG. 4 shows a block diagram of a portion of the system shown in FIG. 1, including the communication between the user interface module and the image analysis system.

[0021] [Figure 5A] 5A, 5B, and 5C are examples of dental radiographs suitable for use with the system of FIG. [Figure 5B] 5A, 5B, and 5C are examples of dental radiographs suitable for use with the system of FIG. [Figure 5C] 5A, 5B, and 5C are examples of dental radiographs suitable for use with the system of FIG.

[0022] [Figure 6] FIG. 6 shows an example of a user interface for the user interface module of FIG.

[0023] [Figure 7A] 7A, 7B, and 7C show examples of visual results from the system of FIG. [Figure 7B] 7A, 7B, and 7C show examples of visual results from the system of FIG. [Figure 7C] 7A, 7B, and 7C show examples of visual results from the system of FIG.

[0024] [Figure 8] FIG. 8 shows an example of the user interface of the system of FIG.

[0025] [Figure 9] FIG. 9 shows a flow diagram of the operation of the system of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0026] The implementations described in this document include methods and systems that utilize deep neural network architectures, model training procedures, and data processing methods for automating dental mapping and condition detection. The systems and methods generate detailed output consisting of a comprehensive analysis of dental radiographs that associates detected conditions with specific teeth. The systems and methods improve the performance of subsequent operations. The systems and methods generate reports that consolidate the results of the radiographic analysis and can be saved in a variety of formats compatible with other medical software systems and applications.

[0027] Deep learning, a subset of artificial intelligence (AI), enables computer algorithms to autonomously learn and extract features from input data to model specific phenomena. This ability to learn relevant features from raw input data distinguishes deep learning techniques from traditional image processing techniques, which typically require identifying, creating, or designing descriptive features.

[0028] The models are trained with deep learning to identify and analyze various types of dental radiographs, including bitewing, periapical, and panoramic images. The goal is to detect a wide range of dental conditions, anatomical structures, restorations, and abnormalities. The analysis system uses the output of these trained models to create labeled images. These images, which display useful diagnostic information, can be displayed to up to 20 dental professionals through an intuitive user interface.

[0029] In the example implementation, the system leverages machine learning capabilities. The goal is to consistently and automatically detect a variety of conditions using input X-ray data. Experts mark different types of X-ray images and radiographs during the training phase of the machine learning models. These trained models are used synergistically to automatically tag or label incoming radiographs in real time or near real time via an application programming interface (API).

[0030] Additionally, feedback from active users, such as dentists, is used to improve the system's performance in identifying various conditions. In certain settings, physicians can use a medical image viewing application to retrieve patient x-rays stored on a server in their office. The application displays an interactive interface on a networked computer system, such as a desktop or laptop computer.

[0031] The application programming interface (API) manages the bridge for interaction and data exchange between the various components of the analysis system and external components. The queue management system is also used to send messages from the API server to the machine learning engine. The API server and queue manager perform the processing functions of the X-ray files received in digital format. The API server and queue manager ensure seamless data transfer to and from the various machine learning models, some of which are specifically designed to recognize various dental diseases. The image tags generated as a result of these interactions are sent to the computer system used by the medical device.

[0032] Typically, a patient's x-ray images are taken, sent to a cloud-based medical image analysis system, evaluated and annotated using machine learning, and submitted for review within a medical image viewing application. Identified conditions or abnormalities are displayed as overlays on the original x-ray image within the user interface. These overlays guide the specialist to the area of the x-ray that contains the detected condition or pathology. The system allows the physician to edit the visualization using selections in the user interface, for example, highlighting specific conditions to view more details.

[0033] FIG. 1 illustrates an implementation diagram of an automated radiograph analysis system 100 implemented in a network computing environment. The system 100 includes an image analysis system 130 with a machine learning engine 140 and a user application for image analysis, which in this implementation is included in a medical device 120. The diagram provides an overview of the architecture and interactions between the various components of an implementation of the automated radiograph analysis system. Those skilled in the art will appreciate that the system architecture can be implemented in a variety of configurations and should not be considered limited to the specific configuration shown. The automated radiograph analysis system is designed to be Health Insurance Portability and Accountability Act (HIPAA) compliant, ensuring the privacy and security of patient data at all stages of processing.

[0034] Within system 100, medical device 120 includes an image viewer 124. The viewer includes image manipulation elements 125 that allow a user to view an image and adjust various parameters such as brightness and contrast, invert colors, zoom in on specific areas, and rotate the image. Image viewer 124 also includes a detection display 126 that can display output generated by machine learning engine 140. The specific features and functionality of image viewer 124 provided herein are merely exemplary; other features and functionality are possible within the scope of the invention.

[0035] The medical device 120 also includes a first data storage device 122, which may be, for example, a hard drive on which x-ray images are stored and from which the user can upload them to the application. The medical device 120 communicates with the network 104. In this embodiment, communication is via a web browser; however, other embodiments are possible within the scope of the present invention.

[0036] The medical device 120 communicates with a database 150, an image analysis system 130, and a remote data storage component 102 via a network 104. The database 150 stores application data related to user logins, organizational and practice information, patient visits, user findings, and patient treatment information. The database 150 also stores the results of X-ray analysis performed by the image analysis system 130 and physician information. The database 150 is the central data repository for all information stored in the network computing environment 100. This component 150 connects to the API server 132 and provides the information needed to fulfill user requests.

[0037] The medical device 120 also communicates with one or more remote data storage units 102, such as Amazon® Web Services (AWS) Remote Disk (S3) or similar services, and with the image analysis system 130 via an API server 132. The API server 132, with the help of a queue manager 134, coordinates the various machine learning algorithms in the machine learning engine 140.

[0038] The queue manager 134 coordinates the queues for sending messages from the API server 132 to the machine learning engine 140. The various types of messages sent relate, for example, to image analysis by various machine learning modules, detection to generate a final image, and image manipulation commands. Messages are held in the queue until they are read. Thus, if a model server, such as a dental identification module, is unresponsive, the message remains in the queue and is read later when the model server becomes available again. One advantage of a queue manager is asynchronous processing, allowing various components of the system to operate independently. As explained above, a first component can initiate its own task without waiting for another component to complete it. This is particularly useful in systems where tasks take a long time, such as processing and analyzing dental x-rays. Another advantage of a queue manager is load balancing. A queue manager distributes tasks approximately evenly across components or servers, preventing a single component from becoming a bottleneck. This is particularly advantageous in systems that must process large amounts of data or requests. Another advantage of a queue manager is fault tolerance. If a particular component fails or crashes, the queue manager tends to retain tasks and redirect those tasks to another component. This makes the system more robust and reduces the chance of data loss. Another advantage of the queue manager is scalability. As the X-ray analysis system expands and the amount of data increases, the queue manager maintains system efficiency by distributing tasks across the maximum number of components or servers. Another advantage of the queue manager is that it can manage the order and priority of tasks. In other words, the queue manager improves the efficiency, robustness, and scalability of the analysis system.

[0039] In various alternative configurations, the queue can be hosted, for example, on Amazon® Simple Queue Service (SQS). The queue manager 134 receives messages written by the API server and sends these messages to the machine learning engine 140. The machine learning engine 140 reads the messages, retrieves the images from the remote data store 102, and processes them as needed.

[0040] The remote data storage 102 and image analysis system 130 may be hosted on various cloud platforms or on local servers. Communications are encrypted and secure, ensuring HIPAA compliance.

[0041] The machine learning engine 140 includes several modules, including an image type classifier 142, a tooth numbering module 144, and a condition detection module 145. The condition detection module 145 includes at least one condition detection model, and typically includes multiple condition detection models. The tooth numbering module 144 and the condition detection module 145 operate independently. Each module 144 and 145 is coordinated by the API server 132 and the queue manager 134.

[0042] The machine learning engine 140 also includes a fusion module 146 that combines the results of different machine learning tasks. It receives the results from the tooth numbering module 144 and the various condition detectors 145 and performs processing to associate each condition with a corresponding tooth. This functionality provides an intuitive format for processing the detections in the 124 image viewer.

[0043] The image viewer 124 communicates with the feedback module 160 using the API server 132 over the network 104 to further improve the performance of the model in the machine learning engine 140. When the results generated by the machine learning are received from the image analysis system 130, the user can reject the analysis, which triggers a feedback loop to an external system for manual annotation. A dental professional reviews the output, applies any necessary changes, and then imports the annotated analysis into the training dataset. This feedback improves the performance of the model after retraining.

[0044] Networked computer system 100 also incorporates bridge 170, which is an interface module between third-party services and the dental practice management system (PMS). This bridge includes connections to dental imaging / QMS systems, data aggregation services such as Sikka®, and third-party platforms such as GuideMia®. Bridge 170 acts as a conduit to facilitate communication between APT server 132 and external dental software solutions 180.

[0045] In addition to the above, network computing environment 100 is designed with the principle of "security by design" in mind. Security considerations are incorporated into all phases of the system's design and operation. This includes measures to prevent unauthorized access to user information, maintain system integrity and reliability, and ensure availability. Network computing environment 100 also includes measures to adapt to new threats and vulnerabilities as they emerge and to ensure ongoing compliance with HIPAA and other relevant regulations.

[0046] Figure 2 shows a block diagram of condition detection module 145. This module contains multiple condition detection models 201, 202, and 220, also known as condition detectors. Module 145 uses machine learning techniques to identify and classify various dental conditions and symptoms. Each model 201, 202, and 220 in module 145 is developed for a specific set of tasks and operates to detect a specific condition or condition.

[0047] Models 201, 202, and 220 in module 145 are trained and optimized to detect and classify various tooth conditions on radiographs. Models 201, 202, and 220 provide the exact location of detected conditions using a mask that covers the detected results or a bounding box, which is a rectangle that surrounds the detected condition. Masks and search boxes can be provided together, allowing users to flexibly switch between the two views. Object segmentation techniques enable detailed and precise analysis, improving the accuracy and usability of the system. Object segmentation detects pixels that make up an object and groups them together as an object of interest.

[0048] The condition detection module 145 uses object detection and segmentation models, such as the YOLO (You Only Look Once)® series, to identify and identify tooth conditions on radiographs. These conditions include both pathological conditions such as cavities, pulp erosion, and wear, as well as non-pathological conditions such as implants and restorations. Note that the conditions detected by this system are not limited to those described herein.

[0049] The object detection model used in this implementation is YOLOv8, for example. YOLOv8 is a model that can be used for object detection, image classification, and instance segmentation tasks. It was developed and open-sourced by Ultralytics, Inc. YOLOv8 generally outperforms other known models in both accuracy and runtime for large-scale computer vision tasks. YOLOv8 uses a single neural network to classify and predict bounding boxes and masks for detected objects, optimizing detection performance. As a segmentation model, YOLOv8 generates pixel-level masks for each detected object, providing more accurate localization of the object in the image.

[0050] The YOLOv8 model has three main components for making predictions: the backbone, the neck, and the head. The backbone is a deep learning architecture based on a convolutional neural network (CNN) and is responsible for extracting features from the input image. The neck acts as a feature aggregator, collecting characteristics from the various stages of the backbone. The head, also known as an object detector, takes the characteristics from the neck and performs localization and classification of various conditions. Each condition is located in the image by a mask, a rectangular hunting hockey, and its class.

[0051] In this implementation, several YOLOv8 models are trained, each based on several criteria. For each condition detected in the input radiograph, each model generates a pixel-level mask along with a class label and bounding box indicating the condition. These models operate asynchronously, and their resulting outputs are passed to a fusion module 146 along with the output of the tooth numbering model to obtain the final output.

[0052] Using an object segmentation model, the tooth numbering module 144 uses advanced deep learning techniques to identify the location of each tooth on a dental x-ray. The tooth numbering module 144 receives the input tooth images and generates a mask that groups all pixels for each tooth. The tooth numbering module 144 generates a corresponding tooth numbering. While a universal numbering system is used in the illustrated diagram, other numbering systems may be employed within the scope of the present invention.

[0053] In this embodiment, Mask R-CNN and YOLOv8 segmentation models are used, however, the systems and methodologies of the present invention are not limited to these particular models or object detectors.

[0054] Mask R-CNN is a type of deep learning architecture for image analysis called a convolutional neural network (CNN). It generates a class label, a bounding box, and a corresponding mask (a set of pixels) for each candidate object. Mask R-CNN consists of three components: i) a deep fully convolutional network called a Region Proposal Network (RPN) that proposes regions; ii) a detector that extracts features from each candidate box using Region of Interest Pooling (RoIPool) and performs classification and bounding box regression; and iii) a segmentation mask prediction for each region of interest (ROI) at the pixel level.

[0055] The specific design and implementation of these models can be adapted and extended depending on the application requirements. The use of object segmentation techniques allows for detailed and precise analysis, improving the accuracy and usability of the system. These models work together to provide a complete and accurate analysis of dental radiographs.

[0056] Image type classifier 142, also known as image type classification module, is a component of machine learning engine 140 shown in FIG. 1. This module uses a convolutional neural network (CNN), a type of deep learning model, to classify dental x-ray types. CNNs are designed to automatically and adaptively learn a spatial feature hierarchy from input dental x-rays.

[0057] The CNN model used in the image type classification module 142 is trained to recognize and classify different types of dental radiographs, such as bitewing images, periapical images, panoramic images, etc. The model takes a dental radiograph as input and generates an x-ray type. This radiograph type classification is used in the subsequent steps of numbering and tooth condition detection, as strategies for these tasks may differ depending on the radiograph type.

[0058] The CNN model used for image type classification consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers. The convolutional layers extract features from the input image, the pooling layers reduce the spatial size of the extracted features to reduce the computational complexity, and the fully connected layers perform the final classification based on the extracted and pooled features.

[0059] The output of the image type classification module 142 drives the appropriate models that are run in the tooth numbering module 144 and the condition detection module 145. The appropriate model depends on the type of input radiograph. For example, if the input radiograph is classified as a panoramic, then panoramic tooth numbering and panoramic condition detector are run. This output is also used by the fusion module to indicate the appropriate type of image viewer 124.

[0060] The architecture and specific parameters of the CNN model can be customized and optimized depending on the application requirements. The model is trained and refined using a large dataset of labeled dental x-rays to achieve high classification accuracy.

[0061] Although this embodiment uses CNN for image type classification, the systems and methods of the present invention are not limited to this particular model or any particular type of deep learning model, and other types of machine learning algorithms are possible within the scope of the present invention.

[0062] Collecting training data In one embodiment of the present invention, the data collection and annotation process is meticulously performed to ensure the accuracy and reliability of the training data. This process is part of the development of the machine learning models used in the 100 system, as the quality and accuracy of the training data directly impacts the performance of the models.

[0063] The training data is generated through a series of reviews by multiple dental professionals, including dentists and radiologists, who have specialized annotation tools that facilitate the process of marking and labeling dental radiographs for relevant conditions.

[0064] The annotation tool is designed with an intuitive interface, allowing annotators to draw bounding boxes or polygons around each tooth present and each identified condition or anatomical structure. The tool also includes the ability to add any type of metadata needed to identify tooth conditions. To allow annotators to focus on one type of annotation at a time, the tool provides options to show or hide annotations.

[0065] Annotators work with a list of anonymized X-rays and process each image individually. To speed up the annotation process, one or more models can be trained on a subset of already annotated images. These models can then be used to process other unannotated images and generate initial annotations in a format compatible with the annotation tool. Annotators can start with these initial annotations and make any necessary updates, speeding up the process.

[0066] The annotation tool can generate the resulting annotations in several formats, including polygon coordinates, bounding boxes, and all relevant metadata information. The specific design and implementation of the annotation tool can be tailored and extended depending on the application requirements.

[0067] This detailed and rigorous data collection and annotation process enables the creation of a high-quality training dataset that is essential for training machine learning models to accurately classify image types, count teeth, and detect dental conditions, thereby improving the overall system performance and usability.

[0068] Training and evaluating the model The process of training the machine learning engine 140 models involves using the collected data described above: each annotated image is used to train a model tailored to a specific task set.

[0069] During the training process, a series of tests can be run to select the best hyperparameters for each model. Hyperparameters are parameters that are set before the training process, rather than learned from data during it. They control the learning process of a model and can have a significant impact on its performance. Examples of hyperparameters include the learning rate, the number of layers in a neural network, the number of units in each layer, and the type of optimizer used for training.

[0070] Once a model is trained, it is evaluated using a subset of images that were not used during the training process. This subset of images is independent of the training process and is used to test the performance of the model. Model performance is measured by a set of metrics such as precision, recall, accuracy, F1 score, sensitivity, and specificity using the following formulas:

[0071] Precision rate=VP / (VP+FP)

[0072] Recall rate=VP / (VP+FN)

[0073] Accuracy = (VP+TN) / (VP+FP+TN+FN)

[0074] F1=(2×precision×recall) / (precision+recall)

[0075] Sensitivity=Recall=VP / (VP+FN)

[0076] Specificity=TN / (TN+FP)

[0077] Here, a true positive (TP) is a case where the model correctly identifies the presence of a condition. A false positive (FP) is a case where the model incorrectly identifies the presence of a condition. A true negative (TN) is a case where the model correctly identifies the absence of a condition. A false negative (FN) is a case where the model incorrectly identifies the absence of a condition.

[0078] In the context of segmentation and object detection, Intersection over Union (IoU) is used to calculate TP, FP, VN, and FN. IoU is the overlap area between the predicted segmentation and the actual value divided by the overlap area between the predicted segmentation and the actual value. If IoU is above a certain threshold, the prediction is considered VN; if it is lower, it is considered FP. VN and FN are calculated in a similar way.

[0079] Precision measures the proportion of positive identifications that are actually correct. Recall measures the proportion of true positives that are correctly identified. Accuracy measures the proportion of all classifications that are correct. The Fl score is the harmonic mean of precision and recall, providing a balance between these two metrics. Sensitivity is another name for recall. Specificity measures the proportion of true negatives that are correctly identified.

[0080] Together, these metrics provide a complete view of a model's performance, taking into account both successes (TP and TN) and failures (FP and FN).

[0081] Figure 3 is a block diagram illustrating the data flow of the fusion module 146 within the machine learning engine 140 of Figure 1. The fusion module 146 is designed to integrate the outputs of the tooth numbering module 144 and the ten condition detectors 201, 202, ..., 220 with the image type classifier 142.

[0082] As described above, the tooth numbering module 144 uses object segmentation models to identify and classify teeth on dental radiographs, providing detailed masks or polygons covering each tooth and the correct tooth number according to a recognized notation system.

[0083] The models 201, 202, ..., 220 of the condition detection module 145 provide detailed masks or polygons that cover the detected findings and / or bounding boxes surrounding each detected condition.

[0084] The fusion module 146 takes this detailed data and combines it to create a complete analysis of the dental x-ray, attributing each condition to each tooth, if applicable. The final detection output 304 is communicated to the image viewer 124. This output includes the type of x-ray processed, the location and classification of each tooth, and the location and classification of any detected conditions.

[0085] The fusion module 146 accurately combines the outputs of the different models and presents the output of the combination process in an intuitive format. This integrated approach enables comprehensive and detailed analysis of dental radiographs, improving the functionality and usability of the system.

[0086] Figure 4 is a block diagram of a portion of the system shown in Figure 1. Figure 4 shows communication between a user interface module 402 and the image analysis system 130. This communication begins with a first step 404 of loading an x-ray and ends with a final step 406 of saving a report.

[0087] The user interface 402 includes a login section 403 that allows the user to securely access the system. Once logged in, the user can interact with the patient management system 420, which provides the option to select an existing patient 421 or create a new patient 422. This information is communicated to the database 150 via the API server 132.

[0088] The user interface 402 also facilitates uploading 404 one or more x-ray images. These x-rays can be loaded, for example, from a local storage device. The 100 system is designed to handle a variety of image formats, ensuring compatibility with a variety of x-ray imaging technologies.

[0089] Post-processing 405 is another feature of the user interface 402. This feature provides the user with a set of tools to add, delete, or edit one or more teeth and one or more findings. These tools allow the analysis results to be refined so that the final output accurately represents the patient's dental condition.

[0090] The final step 406 is to save the report. This report consolidates all results generated for all loaded radiographs and also includes documentation of various possible findings for educational purposes. This report can be saved in various formats and shared with patients and other healthcare professionals to facilitate informed decision-making regarding the patient's dental health.

[0091] Figures 5A, 5B, and 5C are examples of three types of dental radiographs that are compatible with this system. These radiographs include a bitewing radiograph (Figure 5A), a periapical radiograph (Figure 5B), and a panoramic radiograph (Figure 5C).

[0092] A bitewing radiograph (Figure 5A) is a type of dental radiograph commonly used to detect cavities between teeth. Because it clearly shows both the upper and lower teeth in a single image, it is a useful tool for detecting cavities and changes in bone density.

[0093] On the other hand, a periapical radiograph (Figure 5B) is used to examine the entire tooth from the crown to the root and surrounding bone structure. This is particularly useful for identifying abnormalities in the root structure or surrounding bone tissue.

[0094] Finally, the panoramic radiograph (Figure 5C) provides a wide view of the entire mouth, capturing all the teeth, upper and lower jaws, and surrounding structures and tissues in a single image. This type of x-ray is often used to plan treatments such as implants, tooth extractions, orthodontics, and dentures.

[0095] The 100 system is designed to manage and analyze these different types of radiographs, demonstrating its versatility and wide applicability in a variety of dental diagnostic scenarios.

[0096] Figure 6 illustrates the user interface module 402 with a particular focus on the ability to upload an X-ray. This diagram provides a visual representation of the user experience during the initial steps of the image analysis process.

[0097] The interface displays the patient name 601. There is a dedicated button 602 for the user to upload one or more x-rays. The functionality is designed to be intuitive and efficient, allowing for easy navigation and operation.

[0098] Additionally, the user can manually select the x-ray type from a predefined list; however, the system 100 is equipped with machine learning algorithms 142 that automatically process uploaded x-rays to accurately determine their type. This automation improves the system's usability and efficiency, reducing the need for manual input and the potential for error.

[0099] 7A illustrates the interactive capabilities of the system 100, focusing specifically on the display of tooth numbers. This figure shows that the user can review and approve the tooth numbers before proceeding to the next step in pathology visualization and post-processing.

[0100] Once the image analysis system 130 processes the radiograph, a list of detected teeth is communicated over the network 104 and displayed to the user in panel 701. Hovering the mouse 702 over a tooth number in panel 701 draws a bounding box 703 around the corresponding tooth on the x-ray displayed in image viewer 124. Detected teeth are also labeled with tooth numbers 704, providing a clear and intuitive display.

[0101] The user interface module 402 allows the user to edit (720) or accept (710) the tooth numbers. The editing feature allows the user to add, remove, or change teeth by drawing a polygon around the tooth and specifying the number. This interactive feature aligns the system information with the user's understanding and expectations, improving the accuracy and ease of use of the system.

[0102] Additionally, the user interface module 402 includes navigation arrows 705 and 706 that the user can use to move to the next or previous radiograph. This functionality allows for efficient navigation between multiple radiographs, enhancing the user experience.

[0103] 7B and 7C illustrate the system's ability to display detected pathologies in two different formats.

[0104] In Figure 7B, the system displays detected pathologies in the form of bounding boxes 740. Each bounding box is a rectangle that surrounds a detected finding, providing a clear and concise visual representation of the location and extent of the pathology. This display format is particularly useful for providing a quick overview of the detected pathologies.

[0105] In Figure 7C, the system displays the detected pathologies as 760 masks. Each mask precisely covers the detected findings, providing a more detailed and accurate visualization of the pathology. This format is particularly useful for more detailed analysis of the detected pathologies.

[0106] The user interface includes a switch 750 that allows the user to easily toggle between these two display formats. This feature gives the user flexibility to choose the display format that best suits their needs and preferences. If the user does not move the mouse over any tooth number in panel 701, the system will display the status in the default display format.

[0107] Figure 8 illustrates the interactive capabilities of the system, allowing users to view and manage the pathologies detected for selected teeth once the final numbering has been approved. This figure demonstrates that the system provides a personalized and accurate analysis, allowing users to add, delete, or modify the list of findings.

[0108] Once the final tooth numbering is approved, the system displays a list of the findings detected for each tooth in a dedicated panel 805. Another panel 801 displays the selected teeth and allows the user to navigate between the different teeth using arrows.

[0109] For each detected finding in panel 805, the user can edit it using the edit button 803 or reject it using the reject button 804. Editing a finding allows the user to change the finding's name or adjust its position on the radiograph.

[0110] Additionally, the user can add a new finding to the current tooth using the add button 806. This feature allows for a more complete and personalized analysis, as the user can include additional findings that the system may not have initially detected.

[0111] Detected findings associated with the selected tooth are displayed in the image viewer in the form of a mask or bounding box, depending on the user's preference set with button 750. When the user moves the mouse over a particular finding in panel 805, only that finding is displayed in the image viewer, providing a focused view of the selected pathology.

[0112] Figure 9 shows a flow diagram of the operation of the system of Figure 1. In step 905, the image analysis system receives at least one radiograph for analysis. As previously mentioned, images for analysis are stored on one or more storage devices, and a user typically selects the radiographs to analyze through a user interface.

[0113] In step 910, the image analysis system's image type classifier determines the type of the received radiograph (e.g., high-wing, apical, panoramic, etc.). This module uses a convolutional neural network (CNN), a type of deep learning model, to classify the dental x-ray type. CNNs are designed to automatically and adaptively learn a spatial hierarchy of features from input dental x-rays. The model takes a dental x-ray as input and generates an x-ray type. This radiograph type classification is used in subsequent steps of tooth numbering and condition detection.

[0114] In step 915, the dental numbering module receives information about the x-ray and its classification. This module uses object segmentation models and advanced deep learning techniques to identify the location of each tooth on the x-ray. The specific model used depends on the type of x-ray. The module analyzes the x-ray, identifies the location of the teeth, and generates a location with a corresponding number. Numbering systems include, for example, the global numbering system, also known as the "American system."

[0115] In step 920, the condition detector also receives information about the x-ray and its classification from the classification module. The condition detection module uses object detection and segmentation models to identify and locate tooth conditions on the radiograph. These conditions can be both pathological, such as caries, pulp erosion, and wear, or non-pathological, such as implants and restorations. The condition detector creates condition masks and bounding boxes.

[0116] In step 925, the fusion module receives as input the x-ray analyses generated by the tooth numbering module and the condition detection module. Both modules operate independently and in parallel. The fusion module takes the output from both modules and combines them to create a complete analysis of the dental x-ray. In this analysis, each condition is attributed to its respective tooth if it belongs to a tooth, or to both teeth if it is between two teeth. Conditions not found on any teeth are typically considered false positives.

[0117] In step 930, the analysis of the x-ray is saved for use by the image viewer on the medical device.

[0118] In step 935, the user accesses the report or other results including the analyzed radiographs.

[0119] In step 940, the user can accept or reject the analyzed radiograph. The user can also annotate rejected analyzed radiographs to indicate the reason for the rejection. Rejected x-rays are sent to the feedback module.

[0120] In step 945, the feedback module receives the rejected radiographs and additional information from the user. The feedback module processes the x-rays and information in various ways. The results are used to update the models used in the classifier, tooth count module, and condition detection module. This maintains the accuracy and improves the performance of the analysis system.

[0121] It is understood that the above-described embodiments are merely illustrative of the principles of the present invention, and those skilled in the art can make various modifications and variations which incorporate the principles of the present invention and are consistent with the spirit and scope of the present invention.

Claims

1. 1. A system for analyzing dental radiographs, comprising: a tooth numbering module configured to receive at least one dental radiograph and to locate and label teeth present in the at least one dental radiograph; a condition detection module configured to receive the at least one dental radiograph and identify and locate a condition in the at least one dental radiograph; a fusion module configured to receive a first output from the tooth numbering module and a second output from the condition detection module, and to fuse the first output and the second output to generate a report including labeled teeth with identified conditions and locations for particular teeth; A system including:

2. The system of claim 1 , further comprising an image type classifier for determining a type of the dental radiograph.

3. The state detector of claim 1 , wherein the identified states are located using bounding boxes and masks.

4. The fusion module of claim 1 , further configured to fuse the first output and the second output such that the teeth and conditions are independently viewable by a display device.

5. The fusion module of claim 1 , further configured to fuse the first output and the second output such that individual teeth and associated conditions are observable by a display device.

6. The system of claim 1 , further comprising a queue manager for managing operations in the tooth numbering module, the condition detection module, and the fusion module.

7. The system of claim 1 , wherein the tooth numbering module uses an object segmentation model to locate and label teeth present in the at least one dental radiograph.

8. The system of claim 2 , wherein the tooth numbering module is configured to include a plurality of tooth numbering models, each developed for a particular type of dental radiograph.

9. The system of claim 2 , wherein the condition detection module is configured to include a plurality of condition detection models, each developed for a particular type of dental radiograph.

10. The system of claim 1 , wherein the condition detection module uses an object detection model to identify conditions present in the at least one dental radiograph.

11. The system of claim 1 , wherein the condition detection module uses an object segmentation model to locate conditions present in the at least one dental radiograph.

12. 1. A computer-implemented method for analyzing dental radiographs, comprising: receiving at least one dental x-ray, the at least one dental x-ray having a type; analyzing the at least one dental radiograph to identify and label teeth present in the at least one dental radiograph and generate a first output; analyzing the at least one dental radiograph to locate and identify conditions present in the at least one dental radiograph and generate a second output; fusing the first output from the tooth numbering module and the second output from the condition detection module to generate a report including labeled teeth with identified and located conditions for specific teeth; 11. A computer-implemented method comprising:

13. The computer-implemented method of claim 12 , further comprising identifying a type of the at least one dental radiograph.

14. 14. The computer-implemented method of claim 13, further comprising identifying at least one dental radiograph type from the group consisting of bitewing, periapical, and panoramic.

15. 13. The computer-implemented method of claim 12, wherein analyzing the at least one dental radiograph to identify and label tooth locations further comprises using a neural network for one-time analysis.

16. 13. The computer-implemented method of claim 12, wherein fusing further comprises generating a report configured to make each tooth and associated condition viewable on a display device.

17. A non-transitory computer readable storage medium having stored thereon computer program instructions for analyzing dental radiographs, the instructions, when executed on a processor, comprising: receiving at least one dental radiograph, the at least one dental radiograph having a type; analyzing the at least one dental radiograph to identify and label teeth present in the at least one dental radiograph and generate a first output; analyzing the at least one dental radiograph to locate and identify conditions present in the at least one dental radiograph and generate a second output; fusing the first output from the tooth numbering module and the second output from the condition detection module to generate a report including labeled teeth with identified and located conditions for specific teeth; A non-transitory computer-readable storage medium implementing a method comprising:

Images