Dentist artificial intelligence assisted operating system

Abstract

This invention discloses a dental AI-assisted operating system, belonging to the field of dental assisted operation technology. It includes a three-dimensional visual perception module, a data transmission and interaction module, an AI data processing and fusion center, an intelligent display module, and a central control and early warning module. The AI ​​data processing and fusion center includes a safety boundary early warning unit, a surgical instrument tracking and loss early warning unit, a bleeding monitoring unit, and a surgical step guidance and verification unit. Through the safety boundary early warning unit, this invention fuses static preoperative CBCT data with dynamic real-time three-dimensional vision, achieving dynamic, high-precision, real-time distance monitoring and early warning of key anatomical structures such as the inferior alveolar nerve canal and maxillary sinus. This system can issue an alarm before surgical instruments are about to touch dangerous areas, transforming traditional postoperative remediation into preoperative prevention, fundamentally and significantly reducing the probability of serious complications such as permanent nerve damage and maxillary sinus perforation.

Description

Technical Field

[0001] This invention relates to the field of dental assistive technology, and more particularly to an artificial intelligence-assisted operating system for dentists. Background Technology

[0002] With the deep integration of artificial intelligence technology into the medical field, its application in dentistry has gradually covered aspects such as diagnostic planning, surgical assistance, laboratory design, and professional training. Currently, professional surgical navigation systems such as NeocisYomi have emerged in the industry, effectively improving the accuracy and efficiency of dental surgery through robot-assisted methods.

[0003] However, the dental surgical environment is extremely complex and fraught with risks. The oral cavity has a dense network of anatomical structures, with vital nerves and blood vessels distributed within the jawbone. In procedures such as implant surgery and complex tooth extractions, the space for instrument manipulation is extremely limited. If instruments are misaligned or improperly operated during the procedure, these critical anatomical structures can easily be damaged, leading to serious complications such as nerve numbness and massive bleeding, causing permanent harm to the patient. Furthermore, dental surgical procedures are intricate, involving numerous small instruments and foreign bodies, including burs, implants, sutures, and root canal files. During the procedure, these items may accidentally slip out. Given that patients are often under local anesthesia and their gag reflex is weakened, there is a significant risk that dislodged instruments or tooth fragments may be swallowed or aspirated. The latter can instantly cause airway obstruction, constituting a life-threatening medical emergency.

[0004] Existing auxiliary systems mostly focus on preoperative planning and static navigation, leaving a significant gap in the perception, early warning, and proactive prevention of real-time dynamic risks during surgery. Based on this, we propose an artificial intelligence-assisted operating system for dentists. Summary of the Invention

[0005] The purpose of this invention is to solve the problems existing in the prior art and to propose an artificial intelligence-assisted operating system for dentists.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The dental AI-assisted operating system includes a 3D vision perception module, a data transmission and interaction module, an AI data processing and fusion center, an intelligent display module, and a central control and early warning module. The AI ​​data processing and fusion center includes a safety boundary early warning unit, a surgical instrument tracking and loss early warning unit, a bleeding monitoring unit, and a surgical procedure guidance and verification unit; The three-dimensional visual perception module is used to acquire RGB images and depth information of the surgical area in real time and construct a dynamic three-dimensional point cloud model. The data transmission interaction module is used to receive the patient's preoperative multidimensional data, which includes at least CBCT images, intraoral scan digital models, and a predetermined surgical plan. The AI ​​data processing and fusion center is configured to register and fuse the dynamic 3D point cloud model with the preoperative multidimensional data to generate an augmented reality surgical scene. The intelligent display module is used to overlay the AI ​​data processing and fusion center output information and display it in the surgeon's field of vision. The central control and early warning module is used to connect and coordinate data interaction among various modules and manage the output of early warning signals.

[0007] As a preferred embodiment, the safety boundary early warning unit: calculates the Euclidean distance between the tip of the surgical instrument and the preset critical anatomical structure in real time based on the registered and fused augmented reality surgical scene; When the Euclidean distance is lower than a first threshold, a level one visual warning signal is generated; when the Euclidean distance is lower than a more stringent second threshold, a level two audiovisual composite alarm signal is generated; the formula for calculating the Euclidean distance d is:

[0008] in The three-dimensional coordinates of the tip of the surgical instrument. The coordinates are the three-dimensional coordinates of the nearest point on the surface of the critical anatomical structure.

[0009] As a preferred embodiment, the surgical instrument tracking and loss warning unit uses a target detection model based on the YOLOv7 deep learning architecture to perform real-time identification and two-dimensional localization of surgical instruments in the RGB image; Combining the dynamic 3D point cloud model, the KLT optical flow tracking algorithm is used to continuously track the trajectory of the identified surgical instruments in 3D space. The KLT optical flow tracing algorithm tracks feature points by minimizing the following error function:

[0010] Where I(x,y,t) is the image intensity at pixel (x,y) at time t; u,v: The displacements of the feature point in the x and y directions; W: Window surrounding the feature point; The optimal displacement (u,v) is found by summing the squares of the intensity differences of all pixels within window W and minimizing this sum.

[0011] As a preferred embodiment, the surgical instrument tracking and loss warning unit defines a virtual three-dimensional surgical work area based on the actual surgical area. When the tracked surgical instrument disappears from the virtual three-dimensional surgical work area and no confirmation signal indicating that the surgical instrument has been returned to the instrument tray is received, a surgical instrument loss alarm is immediately triggered, and the last three-dimensional position of the surgical instrument is highlighted in the intelligent display module to remind the surgeon.

[0012] As a preferred embodiment, the bleeding detection unit: uses a semantic segmentation model based on the U-Net architecture to perform pixel-level segmentation of the bleeding region in each frame of RGB image, maps the segmented two-dimensional bleeding region onto the dynamic three-dimensional point cloud model, and estimates the approximate three-dimensional area and volume of the bleeding surface by calculating the depth information of pixels in the connected domain. Establish a dynamic blood loss baseline model based on the surgical type and the current surgical step. When the real-time estimated blood loss volume or the blood loss rate per unit time continuously exceeds the preset percentage of the dynamic blood loss baseline model, generate an abnormal blood loss alarm. The formula for estimating the bleeding volume is as follows:

[0013] in, Let i be the actual area corresponding to the i-th pixel. is the depth value at that pixel, and n is the total number of pixels in the bleeding area.

[0014] As a preferred embodiment, the surgical procedure guidance and verification unit uses a Google MediaPipeHolistic model based on time-series modeling to identify the surgeon's hand gestures and the instruments used during the operation. The identified surgical actions and instruments are then matched with a predefined sequence of standard surgical procedures to determine the current surgical progress.

[0015] As a preferred embodiment, the surgical procedure guidance is dynamically displayed in the intelligent display module, showing the current step description, the next step preview, and key parameters. Key verification points are designed in advance. At these points, the current scenario is compared with the predetermined surgical plan. When the parameters are inconsistent, a verification alarm is issued through the central control and early warning module.

[0016] As a preferred embodiment, it also includes an automatic medical record recording engine, wherein the automatic medical record recording engine: The system continuously records the step sequence, timestamps, and status output by the surgical step guidance and verification unit; records the blood loss estimation data sequence output by the bleeding monitoring unit; and records all warning events triggered by the safety boundary warning unit and their contextual scene screenshots. At the end of the surgery, the records are automatically compiled and a surgical report document is generated.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention, through a safety boundary warning unit, fuses static preoperative CBCT data with dynamic real-time three-dimensional vision, achieving dynamic, high-precision, real-time distance monitoring and early warning of key anatomical structures such as the inferior alveolar nerve canal and maxillary sinus. This system can issue an alarm before surgical instruments are about to touch the danger zone, transforming traditional postoperative remediation into preoperative prevention, fundamentally and significantly reducing the probability of serious complications such as permanent nerve damage and maxillary sinus perforation.

[0018] 2. This invention utilizes YOLOv7 and KLT optical flow algorithms in the surgical instrument tracking and loss warning unit to continuously track surgical instruments throughout the entire process in three-dimensional space. Once an instrument abnormally leaves the monitoring area, the system immediately alarms and locates its last position, greatly reducing the extreme risk of small instruments (such as suture needles and root canal files) being swallowed or aspirated into the airway by the patient, thus ensuring the patient's life safety. Attached Figure Description

[0019] Figure 1 This is a framework diagram of the dental AI-assisted operating system proposed in this invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "top / bottom," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0022] Example, refer to Figure 1 The dental AI-assisted operating system includes a 3D vision perception module, a data transmission and interaction module, an AI data processing and fusion center, an intelligent display module, and a central control and early warning module. The AI ​​data processing and fusion center includes a safety boundary early warning unit, a surgical instrument tracking and loss early warning unit, a bleeding monitoring unit, and a surgical procedure guidance and verification unit; 3D Vision Perception Module: Used to acquire RGB images and depth information of the surgical area in real time and build a dynamic 3D point cloud model. It mainly consists of a depth camera or multi-view camera integrated on the surgical light or microscope. It is responsible for acquiring RGB color images and depth information of the surgical field in real time and building a dynamic 3D point cloud model. 3D data processing is implemented using PointNet++, a classic neural network used to process 3D point cloud data for point cloud classification and segmentation (more fine segmentation of soft tissue and bone tissue).

[0023] Data transmission and interaction module: used to receive the patient's preoperative multidimensional data, which includes at least CBCT images, intraoral scan digital model and predetermined surgical plan. The preoperative multidimensional data is based on existing dental technology and will not be described in detail here. AI Data Processing and Fusion Center: Its configuration is used to register and fuse dynamic 3D point cloud models with preoperative multidimensional data to generate augmented reality surgical scenes; Intelligent display module: It is used to overlay the output information of AI data processing and fusion center and display it in the surgeon's field of vision. It can be displayed by overlaying the AI ​​analysis results (such as danger boundaries, virtual implant position, guidance information) on the real surgical field of vision through a head-mounted display or surgical display screen. Central control and early warning module: used to connect and coordinate data interaction between modules and manage the output of early warning signals. The early warning module provides visual, audible and tactile alarms.

[0024] Safety boundary early warning unit: Based on the registered and fused augmented reality surgical scene, calculates the Euclidean distance between the tip of the surgical instrument and the preset critical anatomical structure in real time; When the Euclidean distance is below the first threshold, a first-level visual warning signal is generated; when the Euclidean distance is below the more stringent second threshold, a second-level audiovisual composite alarm signal is generated, in which the dangerous area is marked in real time with a conspicuous colored outline in the system's intelligent display module interface.

[0025] When the instrument tip enters the warning zone, the outline turns yellow and flashes; when it enters the danger zone, it turns red and emits a buzzer alarm. By fusing static preoperative CBCT data with dynamic real-time three-dimensional vision, the registration error caused by intraoperative tissue movement and deformation is solved, and dynamic, high-precision safety warnings are achieved.

[0026] Among them, the critical anatomical structures were determined by segmenting CBCT images, including at least the inferior alveolar nerve canal and the floor of the maxillary sinus. The formula for calculating the Euclidean distance d is:

[0027] in The three-dimensional coordinates of the tip of the surgical instrument. The coordinates are the three-dimensional coordinates of the nearest point on the surface of the critical anatomical structure.

[0028] Surgical instrument tracking and loss warning unit: Utilizes a target detection model based on the YOLOv7 deep learning architecture to perform real-time identification and two-dimensional localization of surgical instruments in RGB images; By combining a dynamic 3D point cloud model, the KLT optical flow tracking algorithm is used to continuously track the trajectory of the identified surgical instruments in 3D space. The KLT optical flow tracing algorithm tracks feature points by minimizing the following error function:

[0029] Where I(x,y,t) is the image intensity at pixel (x,y) at time t; u,v: The displacements of the feature point in the x and y directions; W: A window surrounding the feature point (used to define the computation area and improve the robustness of displacement estimation by utilizing pixel information within the window); The optimal displacement (u,v) is found by summing the squares of the intensity differences of all pixels within window W and minimizing this sum.

[0030] Surgical instrument tracking and loss warning unit: Define a virtual three-dimensional surgical work area based on the actual surgical area. When the tracked surgical instrument disappears from the virtual three-dimensional surgical work area and no confirmation signal is received that the surgical instrument has been returned to the instrument tray, the surgical instrument loss alarm is immediately triggered, and the last three-dimensional position of the surgical instrument is highlighted in the intelligent display module to remind the surgeon.

[0031] This system automates and digitally registers all surgical instruments at the start of the surgery, and tracks the position of each instrument in real time throughout the procedure. When an instrument (such as a suture) disappears from the 3D vision monitoring area and is not recorded as "returned to the instrument tray" by the system, the system will immediately lock onto its last known location and highlight it, while simultaneously issuing a voice alarm stating "Surgical instrument may be lost, please check."

[0032] Bleeding detection unit: Using a semantic segmentation model based on the U-Net architecture, the bleeding region in each frame of RGB image is segmented at the pixel level. The segmented two-dimensional bleeding region is mapped onto a dynamic three-dimensional point cloud model. By calculating the depth information of pixels in the connected domain, the approximate three-dimensional area and volume of the bleeding surface are estimated. U-Net is a benchmark architecture in the field of biomedical image segmentation. Its encoder-decoder structure has skip connections, which can accurately segment irregularly shaped bleeding areas and perform well even when the image contrast is not ideal.

[0033] Establish a dynamic blood loss baseline model based on the surgical type and current surgical step. When the real-time estimated blood loss volume or blood loss rate per unit time continuously exceeds the preset percentage of the dynamic blood loss baseline model, generate an abnormal blood loss alarm. Formula for estimating blood loss volume:

[0034] in, Let i be the actual area corresponding to the i-th pixel. is the depth value at that pixel, and n is the total number of pixels in the bleeding area.

[0035] Surgical procedure guidance and verification unit: Using the Google MediaPipeHolistic model based on time-series modeling, it identifies the surgeon's hand movements and the instruments used during surgery. It then matches the identified surgical movements and instrument combinations with a predefined sequence of standard surgical procedures to determine the current surgical progress.

[0036] The surgical procedure guide dynamically displays the current step description, the next step preview, and key parameters in the intelligent display module; Key verification points are designed in advance. At these points, the current scenario is compared with the predetermined surgical plan. When the parameters are inconsistent, a verification alarm is issued through the central control and early warning module.

[0037] The system displays the current dental procedure steps, the next step, and precautions on the screen. In critical steps, such as before implant placement, the system checks whether the implant type and placement depth are consistent with the preoperative plan. If they are inconsistent, a reminder will be issued.

[0038] The matching process uses a dynamic time warping algorithm to calculate the similarity between the current action sequence and the standard step sequence. If the similarity is lower than a threshold, the step is considered abnormal. The loss function of dynamic time warping is:

[0039] Where Q and C are the feature vectors of the current action sequence and the standard step sequence, respectively, and K is the length of the normalized path; For each pair of matched elements and First calculate the square of the Euclidean distance. Then, sum and take the square root of all K pairs of matching elements to obtain the distance under the regular path.

[0040] The dental AI-assisted operating system also includes an automatic medical record engine. Continuously record the step sequence, timestamp, and status output by the surgical step guidance and verification unit; record the blood loss estimation data sequence output by the bleeding monitoring unit; record all warning events triggered by the safety boundary warning unit and their contextual scene screenshots; At the end of the surgery, a surgical report document is automatically recorded and integrated. The generation of the surgical report document is based on each module of the system. The system automatically records the time from the start to the end of the surgery, the time spent on each step, the list of instruments used, the estimated blood loss, safety boundary alarm records, and screenshots or short videos of key steps. After the surgery, a standardized structured surgical report can be generated with one click, realizing automatic recording of surgical details. This invention provides surgeons with clear step guidance and key parameter verification by identifying the surgeon's actions and instrument usage in the surgical procedure guidance and verification unit and matching them with the standard surgical procedure (SOP) through dynamic time warping (DTW). This effectively reduces operational deviations caused by differences in individual surgeon experience and habits, promotes the standardization and normalization of surgical procedures, and ensures the uniformity and high standard of surgical quality. The bleeding monitoring unit uses the U-Net model and three-dimensional volume estimation formula to transform the previous subjective bleeding observation into objective and quantitative monitoring of bleeding volume and rate. The system can promptly alarm when the bleeding volume increases abnormally, reminding doctors to check and treat potentially damaged large blood vessels, thus avoiding circulatory problems caused by excessive blood loss during surgery. The system intuitively overlays guidance and warning information through an AR interface, allowing doctors to focus more on the current operation without having to frequently switch between the surgical field of vision, the screen, and memory, thus shortening the operation time and improving concentration and operational fluency.

[0041] Through a closed-loop architecture that integrates perception, fusion, decision-making, and display, multiple intelligent modules such as safety warning, instrument management, bleeding monitoring, procedure guidance, and automatic recording are organically integrated. Data exchange and synergy among the modules generate a systemic effect, jointly constructing a comprehensive and seamless intelligent auxiliary environment for dental surgery. The high degree of technological integration and innovation has formed a solid technological barrier.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A dental AI-assisted operating system, characterized in that, It includes a 3D vision perception module, a data transmission and interaction module, an AI data processing and fusion center, an intelligent display module, and a central control and early warning module; The AI ​​data processing and fusion center includes a safety boundary early warning unit, a surgical instrument tracking and loss early warning unit, a bleeding monitoring unit, and a surgical procedure guidance and verification unit; The three-dimensional visual perception module is used to acquire RGB images and depth information of the surgical area in real time and construct a dynamic three-dimensional point cloud model. The data transmission interaction module is used to receive the patient's preoperative multidimensional data, which includes at least CBCT images, intraoral scan digital models, and a predetermined surgical plan. The AI ​​data processing and fusion center is configured to register and fuse the dynamic 3D point cloud model with the preoperative multidimensional data to generate an augmented reality surgical scene. The intelligent display module is used to overlay the AI ​​data processing and fusion center output information and display it in the surgeon's field of vision. The central control and early warning module is used to connect and coordinate data interaction among various modules and manage the output of early warning signals.

2. The dental AI-assisted operating system according to claim 1, characterized in that, The safety boundary early warning unit: based on the registered and fused augmented reality surgical scene, calculates in real time the Euclidean distance between the tip of the surgical instrument and the preset critical anatomical structure; When the Euclidean distance is below a first threshold, a level one visual warning signal is generated; when the Euclidean distance is below a more stringent second threshold, a level two audiovisual composite alarm signal is generated; wherein, the critical anatomical structures are determined by segmenting CBCT images, and at least include the inferior alveolar nerve canal and the floor of the maxillary sinus. The formula for calculating the Euclidean distance d is: in The three-dimensional coordinates of the tip of the surgical instrument. The coordinates are the three-dimensional coordinates of the nearest point on the surface of the critical anatomical structure.

3. The dental AI-assisted operating system according to claim 1, characterized in that, The surgical instrument tracking and loss warning unit uses a target detection model based on the YOLOv7 deep learning architecture to perform real-time identification and two-dimensional localization of surgical instruments in the RGB image; Combining the dynamic 3D point cloud model, the KLT optical flow tracking algorithm is used to continuously track the trajectory of the identified surgical instruments in 3D space. The KLT optical flow tracing algorithm tracks feature points by minimizing the following error function: in, The image intensity at pixel (x,y) at time t; u,v: The displacements of the feature point in the x and y directions; W: Window surrounding the feature point; The optimal displacement (u,v) is found by summing the squares of the intensity differences of all pixels within window W and minimizing this sum.

4. The dental AI-assisted operating system according to claim 3, characterized in that, The surgical instrument tracking and loss warning unit defines a virtual three-dimensional surgical work area based on the actual surgical area. When the tracked surgical instrument disappears from the virtual three-dimensional surgical work area and no confirmation signal indicating that the surgical instrument has been returned to the instrument tray is received, a surgical instrument loss alarm is immediately triggered, and the last three-dimensional position of the surgical instrument is highlighted in the intelligent display module to remind the surgeon.

5. The dental AI-assisted operating system according to claim 1, characterized in that, The bleeding detection unit uses a semantic segmentation model based on the U-Net architecture to perform pixel-level segmentation of the bleeding region in each frame of RGB image, maps the segmented two-dimensional bleeding region onto the dynamic three-dimensional point cloud model, and estimates the approximate three-dimensional area and volume of the bleeding surface by calculating the depth information of pixels in the connected domain. Establish a dynamic blood loss baseline model based on the surgical type and the current surgical step. When the real-time estimated blood loss volume or the blood loss rate per unit time continuously exceeds the preset percentage of the dynamic blood loss baseline model, generate an abnormal blood loss alarm. The formula for estimating the bleeding volume is as follows: in, Let i be the actual area corresponding to the i-th pixel. is the depth value at that pixel, and n is the total number of pixels in the bleeding area.

6. The dental AI-assisted operating system according to claim 1, characterized in that, The surgical procedure guidance and verification unit uses a Google MediaPipeHolistic model based on time-series modeling to identify the surgeon's hand gestures and the instruments used during surgery. It then matches the identified surgical actions and instruments with a predefined sequence of standard surgical procedures to determine the current surgical progress.

7. The dental AI-assisted operating system according to claim 1, characterized in that, The surgical steps are guided to dynamically display the current step description, the next step preview, and key parameters in the intelligent display module; Key verification points are designed in advance. At these points, the current scenario is compared with the predetermined surgical plan. When the parameters are inconsistent, a verification alarm is issued through the central control and early warning module.

8. The dental AI-assisted operating system according to claim 1, characterized in that, It also includes an automated medical record engine, which: The system continuously records the step sequence, timestamps, and status output by the surgical step guidance and verification unit; records the blood loss estimation data sequence output by the bleeding monitoring unit; and records all warning events triggered by the safety boundary warning unit and their contextual scene screenshots. At the end of the surgery, the records are automatically compiled and a surgical report document is generated.

Images