Oral cavity intelligent robot positioning system and method based on real world recognition

Abstract

The invention relates to the technical field of dental medical robots, and provides an oral cavity intelligent robot positioning system based on real world recognition, which comprises a real scene data acquisition and labeling module, a real environment data acquisition and labeling module and a real scene data processing module, establishing an identification rule system containing a dangerous structure, an operation-related structure and a safety attribute through structured labeling; the oral cavity internal positioning and planting path matching module is used for constructing an implant position model based on CT and oral scanning data and realizing rapid matching of planting paths; the dangerous area real-time judgment and avoidance module integrates AI image recognition and spatial distance calculation functions and is used for monitoring the operation environment in real time and triggering graded safety control in the operation process; and the system hardware and technology fusion optimization module adopts a lightweight hardware architecture of a humanoid robot and a dexterous hand. The robot can sense and recognize a real operation scene, a dangerous structure is automatically avoided, the operation process is simplified, the labor cost is reduced, and the operation safety and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the technical field of dental medical robots, and more particularly to an intelligent oral robot positioning system and method based on real-world recognition. It can perceive real surgical scenarios, achieve precise positioning, and avoid dangerous structures, making it suitable for various dental surgical scenarios such as implant surgery and endodontic surgery. Background Technology

[0002] With the continuous development of oral medical technology and the popularization of intelligent equipment, the field of dental surgery has gradually introduced automated equipment such as implant robots and endodontic robots. The aim is to improve surgical precision, reduce the workload of doctors, and promote the standardization and minimally invasive development of dental surgery through technological means. The core working principle of existing dental surgical robots revolves around "visual recognition + coordinate registration": First, specific markers (such as metal positioning elements or optical positioning points) are set on the surface of the patient's teeth or adjacent areas. Then, the robot's visual recognition system (such as a CCD camera or optical sensor) captures the spatial position information of the markers. The coordinates of the acquired markers are aligned and registered with a pre-generated three-dimensional virtual model of the patient's jawbone and teeth in the computer. Finally, the surgical path is planned based on the registered virtual model, driving the robotic arm or surgical instruments to complete core surgical operations such as implant preparation and pulp cleaning.

[0003] From the perspective of current clinical applications, although such robots have improved the stability of surgical procedures to some extent, they still have many technical limitations and cannot fully meet the core clinical needs for surgical safety and efficiency. Specific problems are as follows: (1) Significant safety hazards and lack of real-world perception: Existing robots' "perception range" is limited to preset marker coordinates and virtual model data, making them unable to actively identify the real physical environment and dynamic changes in the surgical scene. On the one hand, they cannot distinguish between non-surgical target structures such as the patient's skin, facial soft tissues (e.g., cheeks, lips), and gums, nor can they perceive unexpected objects in the surgical environment (e.g., surgical instruments, medical staff's hands). On the other hand, because key anatomical structures such as the jawbone and alveolar bone are located below soft tissues and cannot be directly observed with the naked eye, robots can only rely on pre-registered coordinates to determine their position, lacking the ability to actively identify and avoid high-risk structures near the oral cavity (e.g., eyes, nasal cavity, mandibular canal), making them highly susceptible to accidental contact during operation. More importantly, if medical staff briefly lower their heads or leave their sight during surgery due to actions such as operating foot pedals or passing instruments, the robot will move "unimpeded" within the surgical space according to a preset program—unable to identify dangerous structures in the real-time environment, it will continue to advance along the planned path, potentially directly impacting the patient's eyes, scratching facial soft tissues, or damaging the neurovascular bundles within the jawbone, thus leading to medical accidents and disputes. As robots gradually become more common in primary clinics and community hospitals, the varying levels of expertise among operators will further increase the probability of such safety accidents, severely hindering their clinical application.

[0004] (2) High labor costs and lower-than-expected surgical efficiency: From a clinical collaboration perspective, traditional manual dental surgeries (such as single-tooth implantation) typically require only one surgeon and one nurse—the nurse handles instrument delivery and intraoperative suction, while the surgeon focuses on the operation, with the average surgery time kept to 1-1.5 hours. However, the collaboration process for existing robotic surgeries is more complex: in addition to the surgeon and nurse, 1-2 professional engineers are needed to handle preoperative equipment setup, intraoperative coordinate calibration, and troubleshooting, forming a 3-4 person collaboration model of "surgeon + nurse + engineer." Limited by the complex preoperative preparation process and intraoperative monitoring requirements, current robots can typically complete only 2 simple implant surgeries per morning (based on a 4-hour workday). This not only fails to achieve the technical design goal of "replacing humans with simple tasks," but also results in lower overall surgical efficiency than traditional manual operations due to increased personnel collaboration costs and cumbersome procedures, thus increasing the human and time cost burden on medical institutions.

[0005] In summary, existing dental surgical robots lack the ability to perceive real surgical scenarios, posing significant safety hazards. They also require multiple personnel to operate, resulting in low surgical efficiency and failing to meet the core clinical needs for safety, efficiency, and convenience. There is an urgent need for a new positioning system that can overcome the aforementioned technical bottlenecks to promote the development of dental surgical robots towards greater safety and efficiency, and to meet the needs of widespread application in primary healthcare settings. Summary of the Invention

[0006] To address the problems of existing dental surgical robots being unable to perceive the real world, having low safety, low efficiency, and complex systems, the present invention aims to provide an oral intelligent robot positioning system and method based on real-world recognition, enabling the robot to perceive and recognize real surgical scenarios, automatically avoid dangerous structures, simplify operating procedures, reduce labor costs, and improve surgical safety and efficiency.

[0007] The above-mentioned objective of this invention is achieved through the following technical solutions: A dental intelligent robot positioning system based on real-world recognition, comprising: The system comprises several modules: a real-scene data acquisition and annotation module, and a system of identification rules including hazardous structures, surgical-related structures, and safety attributes. The intraoral positioning and implant path matching module constructs an implant position model based on CT and intraoral scan data and utilizes a multi-angle view mapping algorithm to achieve rapid implant path matching without complex 3D reconstruction and coordinate registration processes. The real-time hazardous area judgment and avoidance module integrates AI image recognition and spatial distance calculation functions to monitor the operating environment in real time during surgery and trigger graded safety controls to prevent the robot from accidentally touching hazardous structures. Finally, the system hardware and technology integration and optimization module adopts a lightweight hardware architecture combining a humanoid robot and a dexterous hand. By embedding real-scene recognition algorithms and view mapping matching logic, it achieves technological integration with mature robot control systems, improving system practicality and stability.

[0008] Furthermore, the real-scene data acquisition and annotation module includes an acquisition unit and an annotation unit; The acquisition unit uses a 3D depth camera or a 2D camera in conjunction with a depth calculation algorithm to capture the panoramic environment of the dental clinic, the complete shape of the patient's head and face while lying on the dental chair, and the surrounding area of ​​the oral cavity. It acquires depth image data containing spatial location information and structural morphology information, simulating the robot's eyes to achieve comprehensive perception of the real surgical scene. The annotation unit manually classifies and annotates the acquired depth image data, clearly marking important physiological structures adjacent to the oral cavity, including the eyes and nasal cavity, as dangerous structures; marking structures directly related to the surgery, including the gum margin and tooth surface, as surgery-related structures; and marking the permissible range of surgical instruments as safe zones. At the same time, it defines the safety attributes and avoidance rules for each structure and inputs the annotated structure-attribute-avoidance rule data into the robot control system to establish the correspondence between the real scene and the robot's operating rules.

[0009] Furthermore, the intraoral positioning and implantation path matching module includes a raw data unit, a perspective mapping learning unit, and a real-time perspective matching unit; The original data unit uses CT data and intraoral scan data as core original data. It simulates the tooth arrangement in the computer through virtual tooth arrangement technology. Based on the patient's physiological characteristics, including jawbone structure and interdental spaces, it designs the optimal position of the implant and establishes a three-dimensional correspondence between CT coordinates and implant position to ensure the medical rationality of the implantation plan. The viewpoint mapping learning unit utilizes the characteristic of adjusting the viewpoint of the oral scan data in computer software in 360 degrees without blind spots to record the position of the oral scan image and the virtual model of the implant under each viewpoint, generating a massive number of corresponding samples of oral scan viewpoint and implant position to form a viewpoint mapping sample library. That is, every time the viewpoint of the oral scan data is adjusted, the relative positional relationship between the tooth structure and the virtual model of the implant in the oral scan image under that viewpoint is recorded, so that the robot system learns the positional rules of the implant under different viewpoints. The real-time view matching unit is equipped with a small camera on the robot's implant handpiece or surgical instrument. During the operation, the small camera captures one or more real views of the patient's oral cavity. The robot control system calls the view mapping sample library to match the real view captured in real time with the view in the sample library, quickly determine the actual position of the implant and the surgical path under the current view, and directly output the implant operation parameters including the implantation depth and angle, without the need for traditional three-dimensional reconstruction and coordinate registration processes.

[0010] Furthermore, the real-time danger zone determination and avoidance module includes an AI recognition unit; The AI ​​recognition unit integrates an AI image recognition algorithm. During the operation, the three-dimensional depth camera in the real scene data acquisition and annotation module captures image data of the surgical area in real time. The AI ​​algorithm automatically identifies the marked dangerous structures in the image and converts the recognition results into electrical signals that are transmitted to the robot control system.

[0011] Furthermore, the real-time danger zone determination and avoidance module also includes a safety triggering unit; The safety triggering unit presets a safety red line in the robot control system, which is the safe distance range around the dangerous structure. When the AI ​​recognition unit detects that the distance between the surgical instrument and the dangerous structure is close to the safety red line, the safety triggering unit automatically triggers a deceleration command. If the instrument continues to approach and reaches the boundary of the safety red line, the safety triggering unit immediately triggers a stop command, forcing the surgical instrument to stop moving, ensuring that the instrument always operates within the safe area and avoids the dangerous structure.

[0012] Furthermore, the system hardware and technology integration optimization module adopts a lightweight hardware design scheme; The lightweight hardware design scheme specifically adopts a hardware form of humanoid robot + dexterous hand. The humanoid robot is equipped with a dexterous hand that can simulate human hand movements. The end of the dexterous hand holds an implantation mobile phone or other surgical instruments with a small camera attached, so that the robot can sit next to the patient and operate at close range like a human doctor. This replaces the complex hardware of traditional robots, including large host and independent camera bracket, reducing the space occupied by the equipment and reducing the complexity of the system.

[0013] Furthermore, the system hardware and technology integration optimization module adopts a technology integration and upgrade scheme; The specific technology integration and upgrade solution is as follows: combining mature robot technology, embedding real scene recognition algorithms and viewpoint mapping matching logic into the control system of mature robots, and improving the operational accuracy of robots by leveraging the motion precision and stability of mature hardware, while reducing R&D costs and technical difficulties.

[0014] A method for locating an intelligent oral robot based on real-world recognition, employing the aforementioned real-world recognition-based positioning system, includes the following steps: S1: The acquisition unit of the real-scene data acquisition and annotation module acquires depth image data of the surgical scene, which is then categorized and annotated by the annotation unit and input into the robot control system to establish a correspondence between the real scene and the operating rules; S2: The raw data unit of the oral cavity positioning and implant path matching module designs the optimal implant position based on CT data and intraoral scan data, establishing a three-dimensional correspondence between CT coordinates and implant position; S3: The viewpoint mapping learning unit adjusts the viewpoint of the intraoral scan data, generates a sample library of intraoral scan viewpoint-implant position, and stores it in the robot control system; S4: During surgery, the real-time viewpoint matching unit captures the real viewpoint of the oral cavity through a small camera, matches it with the sample library to determine the implant position and surgical path, and outputs the implant operation parameters; S5: The AI ​​recognition unit of the real-time danger area judgment and avoidance module identifies dangerous structures in real time, and the safety triggering unit triggers deceleration or stop commands based on the distance between the instrument and the dangerous structure to ensure surgical safety; S6: The surgical operation is completed, achieving precise positioning of the oral intelligent robot based on real-world recognition.

[0015] A computer device includes a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method described above.

[0016] A computer-readable storage medium storing computer code that, when executed, performs the method described above.

[0017] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) Significantly improved safety: Relying on the three-dimensional depth imaging and structured annotation technology of the real scene data acquisition and annotation module, as well as the AI ​​image recognition and spatial distance calculation function of the dangerous area real-time judgment and avoidance module, the robot can accurately identify dangerous structures such as eyes and nasal cavity. Combined with graded safety control (deceleration / stop command triggering) to achieve active avoidance, the robot can eliminate the risk of accidental contact caused by "unobstructed" movement from a technical perspective, greatly reduce the incidence of surgical accidents and the probability of medical disputes, and is especially suitable for the safety needs of non-professional engineer on-duty scenarios such as primary clinics.

[0018] (2) Efficiency and cost optimization: The intraoral positioning and implant path matching module uses a multi-angle perspective mapping algorithm to directly match the implant position and path based on CT and intraoral scan data, eliminating the three-dimensional reconstruction and coordinate registration process that traditional robots rely on; at the same time, no engineer is required to participate in intraoperative monitoring, and only 1-2 medical staff are needed to complete the surgery. This not only shortens the preparation time of a single surgery by more than 30%, but also breaks through the efficiency limit of only 2 surgeries that can be completed in a morning, significantly reducing the cost of human collaboration and getting closer to the technical goal of "replacing humans to complete simple tasks".

[0019] (3) Improved ease of operation: The system achieves full-process automation of "real environment perception - planting path generation - danger avoidance" through real scene recognition algorithm and autonomous path matching logic. Doctors do not need to take care of the collaborative operation of multiple systems and only need to focus on the core operation of the surgery. Moreover, the lightweight hardware design (humanoid robot + dexterous hand architecture) simulates the operating posture of human doctors, reduces the learning threshold of doctors for the equipment, reduces the risk of operational errors, and improves the clinical adaptability.

[0020] (4) Strong compatibility and universality: The system hardware and technology integration and optimization module supports embedding real scene recognition algorithms and view mapping matching logic into existing mature brand robot control systems. It can be directly adapted to existing dexterous hands, implant mobile phones and other hardware, without medical institutions having to repurchase a complete set of equipment. At the same time, the lightweight hardware architecture reduces the requirements for clinic space, making it easy to deploy quickly in primary medical scenarios such as community hospitals and chain clinics, and promoting the large-scale popularization and application of dental robot technology. Attached Figure Description

[0021] Figure 1 This is an overall structural diagram of the oral cavity intelligent robot positioning system based on real-world recognition according to the present invention; Figure 2 This is an overall flowchart of the oral cavity intelligent robot localization method based on real-world recognition according to the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0024] First Embodiment like Figure 1 As shown, this embodiment provides an oral intelligent robot positioning system based on real-world recognition, including: The real-scene data acquisition and annotation module uses 3D depth imaging technology to acquire real-world environmental data of the surgical scene. It establishes a recognition rule system including hazardous structures, surgically relevant structures, and safety attributes through structured annotation. This module is used to acquire real-world data of the surgical scene and establish recognition rules, including acquisition and annotation units. The intraoral cavity positioning and implant path matching module constructs an implant position model based on CT and intraoral scan data. It uses a multi-angle view mapping algorithm to achieve rapid implant path matching without complex 3D reconstruction and coordinate registration processes. This module is used to determine implant position and surgical path without complex 3D reconstruction and coordinate registration processes, including raw data units, view mapping learning units, and real-time view matching units. The real-time hazardous area judgment and avoidance module integrates AI image recognition and spatial distance calculation functions. It is used to monitor the operating environment in real time during surgery and trigger graded safety controls to prevent the robot from accidentally touching hazardous structures. This module is used to monitor the surgical process in real time and prevent the robot from accidentally touching hazardous structures, including AI recognition units and safety trigger units. The system hardware and technology integration and optimization module adopts a lightweight hardware architecture of humanoid robots and dexterous hands. By embedding real scene recognition algorithms and perspective mapping matching logic, it achieves technical integration with mature robot control systems, thereby improving the system's practicality and stability.

[0025] The following provides a detailed description of each module: (1) The real-scene data acquisition and annotation module includes an acquisition unit and an annotation unit; The acquisition unit uses a 3D depth camera or a 2D camera in conjunction with a depth calculation algorithm to capture the panoramic environment of the dental clinic, the complete shape of the patient's head and face while lying in the dental chair, and the surrounding area of ​​the oral cavity. It acquires depth image data containing spatial location information and structural morphology information, simulating the robot's eyes to achieve comprehensive perception of the real surgical scene.

[0026] Taking the actual application of the acquisition unit in a clinical implant surgery scenario as an example, we can further clarify the details of its technical implementation: In the preoperative preparation stage, staff will fix a 3D depth camera (such as the Intel RealSense D455 model, which has a resolution of 1280×720, a frame rate of 30fps and an effective detection distance of 10 meters, suitable for a typical dental clinic space of 20-30㎡) on the center of the clinic ceiling or on the side bracket of the dental chair, to ensure that the field of view can completely cover the panoramic environment of the dental clinic (including the position of the dental chair, the layout of the instrument table, and the area where the surrounding equipment is placed), the complete shape of the patient's head and face while lying on the dental chair (from the hairline to the lower edge of the mandible, covering key areas such as the eyes, nose, and cheeks), and the space within 1 meter around the oral cavity (including the doctor's operating position and the nurse's auxiliary area). During filming, the 3D depth camera simultaneously acquires RGB color images and depth information, generating depth image data containing spatial coordinates (accurate to the millimeter level) and structural features (such as facial contour curvature and dental chair armrest height). If a 2D camera (such as a high-definition industrial camera) is used due to budget constraints, it must be combined with a binocular vision algorithm or a structured light depth calculation model to reconstruct spatial depth information from 2D images captured from multiple perspectives, thus achieving 3D data acquisition of the surgical scene. This acquired depth image data can accurately restore the positional relationships and structural details of objects in the real surgical scene, as if equipping the robot with "eyes that can perceive distance"—for example, it can clearly distinguish the spatial distance between the patient's left eye and the edge of the dental chair, and the relative position of the implant handpiece and the gingival surface. This provides a high-precision data foundation for the subsequent structural recognition and avoidance rules of the labeled units, preventing the robot from misjudging dangerous areas due to missing or insufficient scene data.

[0027] The annotation unit manually categorizes and annotates the acquired depth image data, clearly marking important physiological structures adjacent to the oral cavity, including the eyes and nasal cavity, as dangerous structures; marking structures directly related to the surgery, including the gum margin and tooth surface, as surgery-related structures; and marking the permissible range of surgical instruments as safe zones. Simultaneously, it defines the safety attributes (e.g., setting the safe distance for the eyes to 10 cm and the safe distance for other sensitive structures to 5 cm) and avoidance rules for each structure. The annotated structure-attribute-avoidance rule data is then input into the robot control system to establish a correspondence between the real-world scenario and the robot's operating rules.

[0028] Taking the practical process of annotation unit before clinical implantation surgery as an example, its implementation logic can be further clarified through specific scenarios and technical details: After obtaining panoramic view of the clinic and depth image data of the patient's head and face, the annotators will use professional image annotation software (such as LabelMe or Medical Labeler) to perform frame-by-frame and layer-by-layer classification annotation of the images. For the patient's head and face area, the eyes (labeled "Dangerous Structure - Eye - L" for the left and "Dangerous Structure - Eye - R" for the right) and the nasal cavity (labeled "Dangerous Structure - Nasal Cavity") are selected using red dedicated labels. The label attributes clearly define the safe distance rules: a 10 cm radius around the eyes is set as the "Level 1 Safety Red Line" (deceleration is triggered when the instrument approaches), and a 5 cm radius is set as the "Level 2 Safety Red Line" (stop is triggered when the instrument approaches). The 5 cm radius around the nasal cavity is uniformly set as the "Level 1 Safety Red Line". For the oral cavity area, the gingival margin (labeled "Surgery-Related Structure - Gingival Margin") and tooth surface (labeled "Surgery-Related Structure - Tooth Surface") are selected using green labels. Key parameters such as the vertical distance from the gingival margin to the alveolar bone and the gap between the tooth surface and adjacent teeth are marked to provide a reference for subsequent implant path design. For the examination room environment area, a 15 cm radius outside the dental chair armrest and a 20 cm radius around the instrument table are selected using blue labels (labeled "Safe Area - Permitted Operation Area"). It is clear that instruments can move normally within this area, and exceeding this area will trigger an environmental obstacle warning. During the annotation process, structured values ​​are assigned to the "avoidance rules"—for example, when the implanted mobile phone enters the 10 cm red line of the "dangerous structure-eye-L" boundary, the system needs to output the instruction "reduce the moving speed from 5 mm / s to 2 mm / s"; when it enters the 5 cm red line, it outputs the instruction "stop moving immediately" and simultaneously triggers an audible and visual alarm. After all annotations are completed, the associated data of "structure label-safety attribute parameter-avoidance rule instruction" is exported in XML format and imported into the robot control system (such as a control platform built on NVIDIA Jetson AGX Xavier). Through data mapping algorithms, the correspondence between each pixel in the depth image and the robot's operation rules is established, so that when the robot captures the corresponding structure during surgery, it can directly call the preset rules to achieve accurate response to the real scene.

[0029] (2) The oral cavity positioning and implantation path matching module includes a raw data unit, a perspective mapping learning unit, and a real-time perspective matching unit; The original data unit uses CT data and intraoral scan data as core raw data. It simulates the tooth arrangement in a computer using virtual tooth arrangement technology. Based on the patient's physiological characteristics, including jawbone structure and interdental spaces, it designs the optimal position of the implant and establishes a three-dimensional correspondence between CT coordinates and implant position to ensure the medical rationality of the implantation plan.

[0030] In clinical practice, the operation of the raw data unit can be achieved through the following specific process: First, high-precision CT data of the patient's oral cavity is acquired (slice thickness set to 0.2mm, covering the maxilla, mandible and surrounding soft tissues, ensuring clear presentation of key anatomical information such as alveolar bone density and mandibular nerve canal orientation) and intraoral scan data (using an intraoral scanner to acquire surface information such as crown morphology and gingival margin position with a precision of 50μm). After importing the two types of data into oral implant planning software (such as 3ShapeImplant Studio), the system uses an automatic registration algorithm to fuse the three-dimensional bone structure of the CT scan with the surface morphology data of the intraoral scan to generate a complete digital oral model. Using a virtual tooth alignment tool, dentists simulate the arrangement angles and occlusal relationships of natural teeth in software. For example, in cases of missing mandibular posterior teeth, based on the remaining alveolar bone height (if measured to be 8mm), width (6mm), and interdental space (7mm) shown on CT scans, they select implants with a diameter of 4.5mm and a length of 6mm. The three-dimensional position of the implant is then determined through virtual adjustments—the axial tilt angle is controlled within 5° to avoid the nerve canal, a 2mm safety distance is maintained between the root apex and the upper edge of the mandibular nerve canal, and a 0.5mm gap is maintained between the crown and the contact point with adjacent teeth. During this process, the software calculates in real time the contact area between the implant and bone tissue (≥50% of the implant surface area) and the initial stability coefficient (ISQ value estimate ≥65) to ensure that the design conforms to the biological principles of implant dentistry. Finally, the system automatically records the three-dimensional coordinate parameters of the implant in the CT coordinate system (such as X=25.3mm, Y=18.7mm, Z=-42.1mm, with the right mandibular angle as the origin), establishes a precise mapping relationship between CT data and implant position, and provides a standardized original positioning benchmark for subsequent view mapping learning units.

[0031] The viewpoint mapping learning unit utilizes the characteristic of 360-degree angle-free adjustment of oral scan data in computer software to record the position of the oral scan image and the virtual implant model under each viewpoint, generating a massive number of corresponding samples of oral scan viewpoint and implant position, forming a viewpoint mapping sample library. That is, every time the viewpoint of the oral scan data is adjusted, the relative positional relationship between the tooth structure and the virtual implant model in the oral scan image under that viewpoint is recorded, enabling the robot system to learn the positional rules of the implant under different viewpoints.

[0032] Taking a single mandibular molar implant as an example, the specific operation process of the perspective mapping learning unit can be detailed as follows: After acquiring the patient's intraoral scan data and completing the virtual positioning of the implant, the system calls a dedicated perspective control module. Using the center point of the implant tip as the reference point, it performs 360° omnidirectional perspective adjustments in three-dimensional space at 5° intervals (i.e., recording the perspective every 5° along the horizontal axis, generating a total of 72 basic perspectives; simultaneously, adjusting the pitch angle from -30° to +30° along the vertical axis, ultimately forming a dataset containing 5184 different perspectives). For each perspective, the system automatically extracts the tooth structure of the corresponding field of view from the intraoral scan image (such as the alveolar ridge morphology of the mandibular second molar missing area, the crown margins of adjacent teeth (first and third molars), gingival curves, etc.), and simultaneously records the spatial coordinate parameters of the implant virtual model under that perspective (including the angle between the implant central axis and the long axis of the adjacent teeth, the vertical distance between the implant tip and the gingival plane, etc.). For example, when the viewing angle is "45° directly above the missing area," the system records the pixel distance between the distal margin ridge of the first molar and the mesial margin of the implant (equivalent to an actual distance of 2.3 mm), as well as the relative position of the implant shoulder and the gingival margin (covering 1.2 mm). When the viewing angle is switched to "30° buccal to the missing area," the system focuses on recording the tilt angle (8°) between the buccal alveolar bone cleft and the implant axis, and the buccal-lingual distance between the implant and the second premolar (5.7 mm). These correspondences between "intraoral scan image features and implant position parameters" are stored as structured samples (each sample contains a 2048-dimensional image feature vector and 6-DOF position parameters), ultimately constructing a viewing angle mapping sample library containing millions of samples. By training the sample database with deep learning algorithms, the robotic system can master the correlation between tooth anatomy features and implant position from different perspectives. For example, by identifying the "specific relative position of the triangular ridge of the adjacent tooth crown and the tip of the implant", the depth parameters of the implant can be calculated, providing an accurate basis for real-time matching during surgery.

[0033] The real-time view matching unit is equipped with a small camera on the robot's implant handpiece or surgical instrument. During the operation, the small camera captures one or more real views of the patient's oral cavity. The robot control system calls the view mapping sample library to match the real view captured in real time with the view in the sample library, quickly determine the actual position of the implant and the surgical path under the current view, and directly output the implant operation parameters including the implantation depth and angle, without the need for traditional three-dimensional reconstruction and coordinate registration processes.

[0034] In clinical implant surgery, the workflow of the real-time perspective matching unit can be illustrated through the following scenario: Before surgery, technicians attach a miniature wide-angle camera (e.g., using a 1 / 3-inch CMOS sensor with a resolution of 1280×720 and a field of view of 120° to ensure complete capture of the implant area and 2-3 adjacent teeth) to the end of the robot's implant handpiece (approximately 3cm from the instrument's working end) and use a calibration algorithm to eliminate camera distortion and ensure image measurement accuracy. During the surgery, when the robot delivers the implant handpiece to the vicinity of the patient's mouth, the small camera captures the real-time perspective of the inside of the mouth at a frame rate of 30fps. For example, in the case of maxillary anterior tooth implantation, the camera simultaneously acquires two key perspectives: one is a "45° perspective directly in front of the implant site," clearly showing the gingival morphology of the central incisor missing area and the mesial marginal ridge of the lateral incisor; the other is a "30° perspective on the palatal side," capturing the palatal bone plate contour and the root projection of adjacent teeth.

[0035] At this point, the robot control system immediately calls upon a pre-set viewpoint mapping sample library and extracts key anatomical landmarks (such as the incisal edge apex of the lateral incisor and the curvature change of the gingival inflection line) from the real-time image using feature extraction algorithms (such as SIFT feature point matching). These landmarks are then compared with the oral scan viewpoint features of similar cases in the sample library. When the system detects feature combinations in the real-time image such as "the distance between the mesial marginal ridge of the lateral incisor and the gingiva at the implant site is 3.2 mm" and "the angle between the palatal bone plate and the root of the adjacent tooth is 15°", it quickly matches the most similar viewpoint sample in the sample library (matching time < 200ms). Based on the positional parameters associated with the sample, it then calculates the actual three-dimensional coordinates of the implant under the current viewpoint: for example, the implant depth must reach 10 mm (from the alveolar ridge apex), the angle with the midline must be 8°, and the labial-palatal tilt must be controlled within 3°. These parameters are directly converted into electrical signals to drive the robotic arm, eliminating the need for time-consuming 3D reconstruction or coordinate registration as in traditional systems. Traditional processes typically require 5-10 minutes to complete registration, while this unit directly matches the viewpoint, requiring only 0.5 seconds from image acquisition to parameter output, and the positioning error can be controlled within 0.1mm, significantly improving surgical efficiency and accuracy.

[0036] (3) The real-time danger zone determination and avoidance module includes an AI recognition unit and a safety triggering unit; The AI ​​recognition unit integrates an AI image recognition algorithm. During the operation, the three-dimensional depth camera in the real scene data acquisition and annotation module captures image data of the surgical area in real time. The AI ​​algorithm automatically identifies the marked dangerous structures in the image (such as whether the eyes have entered the surgical operation range) and converts the recognition results into electrical signals and transmits them to the robot control system.

[0037] During implant surgery, the real-time operation of the AI ​​recognition unit can be specifically demonstrated in the following clinical scenarios: The AI ​​image recognition algorithm on the system adopts an improved YOLOv8 model (trained and optimized with data from 5,000 oral surgery scenarios, focusing on improving the recognition accuracy of dangerous structures in the head and face), and works in conjunction with the 3D depth camera in the real-scene data acquisition module. This camera continuously captures the surgical area at a frame rate of 30fps, covering the area from the patient's head and face to the shoulders, and simultaneously outputs color images and depth information. When the robot operates with the implant handpiece on the left side of the patient's mouth, the 3D depth camera captures dynamic images of areas such as the patient's left eyelid and left nasal ala in real time. The AI ​​algorithm uses multi-scale feature fusion technology to accurately extract the contour features of dangerous structures from the images: for example, when recognizing the left eye, the algorithm focuses on capturing key feature points such as the arc-shaped edges of the upper and lower eyelids and the eyelash distribution area, and calculates the spatial straight-line distance between the center of the eyeball and the implant handpiece by combining depth data; when recognizing the nasal cavity, it locates it by the triangular shape of the nasal ala and the circular outline of the nostrils, and simultaneously calculates the 3D coordinate difference between the anterior nasal spine and the tip of the instrument.

[0038] During the identification process, the algorithm generates a confidence score for each hazardous structure (e.g., 99.2% confidence score for left eye identification and 98.7% confidence score for nasal cavity identification). When the score exceeds a preset threshold (95%), the system determines it as a valid identification and then converts the identification result into a standardized electrical signal (e.g., "Hazardous structure - left eye, distance 12cm, coordinates (X:45.2, Y:18.7, Z:-32.1)") via TCP / IP protocol, transmitting it to the central processor of the robot control system at a frequency of 50ms / time. Even if there is slight head movement (amplitude <5cm) or brief entry of medical staff's hands into the field of vision during surgery, the algorithm can still maintain stable identification of hazardous structures through the dynamic tracking module, ensuring that the position information transmitted to the control system always reflects the real spatial relationship, providing a reliable basis for the subsequent safety trigger unit's decision-making.

[0039] The safety triggering unit presets a safety red line in the robot control system, which is a safe distance range around dangerous structures (e.g., 10 cm around the eyes, 5 cm around other sensitive structures). When the AI ​​recognition unit detects that the surgical instrument is close to the safety red line, the safety triggering unit automatically triggers a deceleration command. If the instrument continues to approach and reaches the boundary of the safety red line, the safety triggering unit immediately triggers a stop command, forcing the surgical instrument to stop moving, ensuring that the instrument always operates within the safe area and avoids dangerous structures.

[0040] In clinical surgical scenarios, the graded protection mechanism of the safety trigger unit can be implemented through the following specific process: The system presets dual safety red line parameters in the real-time monitoring module of the robot control system—for high-risk structures such as the eyes, 10 cm is set as the first-level red line (warning threshold) and 5 cm as the second-level red line (emergency stop threshold); for other sensitive structures such as the nasal cavity, 8 cm is set as the first-level red line and 3 cm as the second-level red line. These parameters are stored in the safety configuration module of the control system through a structured data table. During the operation, when the AI ​​recognition unit continuously tracks the real-time distance between the implanted handpiece and the patient's left eye, which gradually decreases from 15 cm to 11 cm, the safety trigger unit, through real-time data interaction with the AI ​​recognition unit (communication latency <50ms), detects that the distance has approached the first-level red line (10 cm) of the left eye, and immediately sends a PWM (pulse width modulation) signal to the robot's drive system, linearly reducing the moving speed of the implanted handpiece from the normal operation of 8 mm / s to 3 mm / s. At the same time, a yellow warning light illuminates on the doctor's operating table display screen, indicating "Approaching danger zone, deceleration has been slowed down".

[0041] If, during the surgical procedure, slight head movements by the patient cause the distance to shorten further to 5.5 cm (still between the first and second level red lines), the safety trigger unit will maintain a deceleration state and emit intermittent beeps (once every 2 seconds). When the distance continues to approach 5 cm (reaching the second level red line boundary), the system immediately triggers an emergency braking command—by cutting off the power output of the robotic arm's drive motor, it forces the implanted handpiece to completely stop moving within 0.3 seconds. Simultaneously, the control panel display switches to a red alarm interface, displaying the text "Safety red line triggered: 5 cm from left eye," and broadcasting "Instrument has stopped, please confirm safety" via the voice module. At this point, even if the doctor continues to operate the foot switch to attempt to advance the instrument, the hardware-level interlocking mechanism of the safety trigger unit will prioritize blocking the control signal until medical personnel confirm the danger has passed and press the "reset" button, at which point the system will restore normal operating permissions. This tiered response mechanism avoids the frequent start-stop problems that may result from a single threshold, while providing absolute protection when truly approaching danger, ensuring that the surgical instruments always operate within the safety boundary.

[0042] (4) The system hardware and technology integration optimization module adopts a hardware lightweight design scheme and an integration upgrade scheme; The lightweight hardware design scheme specifically adopts a hardware form of humanoid robot + dexterous hand. The humanoid robot is equipped with a dexterous hand that can simulate human hand movements. The end of the dexterous hand holds an implantation mobile phone or other surgical instruments with a small camera attached, so that the robot can sit next to the patient and operate at close range like a human doctor. This replaces the complex hardware of traditional robots, including large host and independent camera bracket, reducing the space occupied by the equipment and reducing the complexity of the system.

[0043] The implementation of lightweight hardware design in clinical scenarios can be illustrated by the following configuration example: The system uses a compact humanoid robot body with a height of 1.5 meters (weighing approximately 45 kg, equivalent to 70% of an adult's body size). Its torso integrates the core control system and drive module. The shoulders employ 3-DOF rotary joints (range of motion ±90°), and the elbows and wrists are each equipped with 2-DOF drive axes, simulating the flexion, extension, and rotation movements of a human surgeon's arm. The robot's end effector uses a 12-DOF dexterous hand (3 joints per finger, driven by a harmonic reducer, achieving a fingertip force control accuracy of 0.1N). Its palm has a built-in adaptive gripping mechanism, compatible with various implant handpieces and surgical instruments (such as ultrasonic bone scalpels and periodontal curettes) with diameters of 3-10mm. Grip stability is monitored in real-time by pressure sensors (the gripping force is maintained at 5-8N, preventing instrument slippage and excessive clamping leading to deformation).

[0044] In terms of instrument integration, for example, the implant handpiece held by the dexterous hand has a pre-installed miniature binocular camera (weighing only 15g, which does not affect operational balance). It is connected to the image processing module of the robot's torso via a flexible cable, eliminating the need for an additional independent camera stand. During actual surgery, the humanoid robot moves to the dental chair (approximately 50cm from the patient's head, consistent with the human surgeon's operating position) using braked omnidirectional wheels. The shoulder joint drives the arm to extend naturally forward, and the wrist flexibly adjusts the instrument angle, allowing the implant handpiece to reach the oral operating area with the precision of a surgeon's hand. Compared to traditional implantation robots (which typically require 2-3 square meters of space, including a 1.2m×0.8m×1.5m main unit cabinet, a robotic arm track spanning the examination room, and 3-4 independent positioning cameras), this lightweight design reduces the equipment's footprint to less than 0.5 square meters and eliminates the need for complex cable connections (requiring only one power cord and one data transmission cable). This not only makes it suitable for the typical 20-30 square meter examination room space in primary care clinics but also reduces the difficulty of equipment installation and debugging. Traditional systems require professional engineers to complete calibration for 2-3 hours, while this humanoid robot, through its built-in spatial positioning sensor, can automatically complete the relative position calibration with the dental chair within 5 minutes of powering on, significantly improving the convenience of clinical applications.

[0045] The specific technology integration and upgrade solution is as follows: combining mature robot technology, embedding real scene recognition algorithms and viewpoint mapping matching logic into the control system of mature robots, and improving the operational accuracy of robots by leveraging the motion precision and stability of mature hardware, while reducing R&D costs and technical difficulties.

[0046] For example, the practical implementation of the technology integration and upgrade solution can be illustrated by adapting it to mature hardware such as the "Walker X Humanoid Robot" and the "Dexterous Hand G1 Dexterous Hand": First, for the control system of the Walker X robot (based on the ROS 2 architecture, supporting real-time data transmission and multi-module collaboration), an algorithm interface compatible with its communication protocol is developed—the "real scene recognition algorithm" of this solution is encapsulated as an independent function package, and data interaction is achieved with the robot's vision processing module through the UDP protocol. The information such as the coordinates of dangerous structures and safe distances output by the algorithm can be directly converted into "obstacle position data format" that the Walker X system can recognize; at the same time, the "view mapping matching logic" is integrated into the robot's path planning module, and its native motion control API (application programming interface) is used to convert the planting depth and angle parameters obtained by view matching into motion control commands for the dexterous hand's end effector (such as "move 10mm downward along the Z-axis and rotate 5° around the X-axis").

[0047] In terms of hardware adaptation, the Dexterous Hand G1 itself has a motion accuracy of 0.1mm and a force control range of 10N. When integrating the technology, there is no need to modify its mechanical structure. Only a "view-motion mapping" driver needs to be implanted into its control system. For example, when the view matching unit outputs the instruction "the implanted phone needs to be slightly adjusted 3mm towards the cheek", the driver can directly call the dexterous hand's knuckle motor control function to achieve precise displacement of the implanted phone by adjusting the gripping angle of the thumb and index finger. This fusion approach not only leverages the motion precision (avoiding errors caused by self-developed mechanical structures) and stability of mature hardware (the Walker X robot has undergone 100,000 surgical scenario simulation tests with a failure rate of less than 0.5%), but also significantly reduces R&D costs. Developing a humanoid robot and dexterous hand with equivalent performance from scratch would require an investment of over 5 million yuan and a two-year development cycle. However, through the fusion solution of algorithm embedding and protocol adaptation, R&D costs can be reduced to less than 800,000 yuan, and the cycle shortened to 6 months. At the same time, it avoids hardware compatibility issues (such as motor drive conflicts and data transmission delays), ensuring that the robot can stably output a full-process response of "recognition-decision-action" in clinical surgery, with operational accuracy improved by more than 30% compared to traditional self-developed hardware.

[0048] Second Embodiment This embodiment provides a specific example of the system's application in implant surgery, as detailed below: (1) Real-world data collection and annotation process Scene data acquisition and model training were completed 24 hours before the implant surgery: An Intel RealSense D455 3D depth camera (equipped with a 1280×720 resolution CMOS sensor, 30fps frame rate) was used, fixed to the center of the examination room ceiling (2.5 meters above the ground) using a tripod. The shooting range covered a 30㎡ panoramic view of the examination room (including the coordinates of the dental chair, the layout of the instrument table, medical staff passage, and other environmental elements) and the complete morphology of the patient's head and face (from 2cm below the hairline to 3cm below the mandibular margin, ensuring that key structures such as the eyes and nasal cavity were fully included). During the acquisition process, RGB images and depth information were acquired simultaneously, and the generated point cloud data had an accuracy of ±2% (within a 2-meter range), which could accurately distinguish the spatial relationship between the patient's eyeballs and eyelids.

[0049] During the annotation phase, the LabelImg professional annotation tool was used, and manual annotation was completed by dentists with more than 5 years of clinical experience in dentistry: the bilateral area was marked as "dangerous structure 1" (coordinate range of X:150-200, Y:80-120 for the left eye, and X:300-350, Y:80-120 for the right eye), with a safety distance of 10cm (three-dimensional straight-line distance); the nasal cavity area was marked as "dangerous structure 2" (coordinate range of X:220-280, Y:140-180), with a safety distance of 5cm; the gingival margin was marked as "surgery-related structure" (dynamically selected according to the position of the teeth); the dental chair armrests (X:50-100, Y:400-500), the instrument table edge (X:400-500, Y:300-400), etc. were marked as "environmental obstacles", with a safety distance of 8cm. Multi-level avoidance rules are defined synchronously: when the device approaches 120% of the safe distance threshold (e.g., 12cm from the eyes), a first-level warning is triggered; when it approaches 100% (e.g., 10cm from the eyes), a second-level braking is triggered.

[0050] The 1,000 labeled datasets (including head and face data from 200 patients of different genders and ages, and 800 combinations of clinic environments) were imported into a robot control system equipped with an NVIDIA Jetson AGX Xavier processor. The YOLOv8 model was fine-tuned through transfer learning (100 epochs, learning rate 0.001). The final model achieved a structural recognition accuracy of 99.1% on the validation set, laying the foundation for real-time intraoperative recognition.

[0051] (2) Implementation of intraoral positioning and implant pathway matching One hour before surgery, patient data processing and pathway planning were completed: Siemens SOMATOM go.Now CT scanners were used to acquire jawbone data (0.5mm slice thickness, 0.3mm slice spacing, 512×512 resolution). Simultaneously, a 3Shape Trios 5 intraoral scanner (5μm precision) was used to acquire oral soft tissue morphology. The two types of data were automatically registered using ICAT Link software (registration error <0.1mm). In Crown & Bridge Design planning software, based on the alveolar bone conditions (bone height 12mm, bone width 8mm) of the patient's left mandibular first molar malformation area, the dentist selected a Straumann BLX implant (4.5mm diameter, 10mm length). The optimal implantation position was determined using a virtual tooth alignment tool: the implant shoulder was located 1mm below the gingiva, the axis was parallel to the long axis of the adjacent teeth (second premolar and second molar), and the root apex was 2mm from the upper edge of the mandibular nerve canal.

[0052] During the construction phase of the perspective mapping sample library, the system establishes a local coordinate system in the software with the implant tip as the origin. The intraoral scan data perspective is adjusted every 5° along the horizontal axis (a total of 72 angles). At each angle, the following are recorded simultaneously: ① Visible crown features of adjacent teeth in the intraoral scan image (such as the mesial marginal ridge of the second premolar and the distal cusp of the second molar); ② Spatial coordinate parameters of the implant virtual model (X / Y / Z axis offsets and rotation angles). The generated 72 sets of samples are transformed into structured data containing 1024-dimensional feature vectors through a feature extraction algorithm and stored in a MongoDB database, forming a personalized perspective mapping sample library for this patient.

[0053] During the surgery, the NSK VarioSurg 3 implant handpiece on the robot is pre-installed with a micro USB camera (640×480 resolution, 120° field of view, weighing only 18g). When the instrument enters the left cheek area of ​​the patient's mouth (about 5cm from the implant site), the camera captures real images of the surgical area at 30fps. The control system calls the FLANN matching algorithm to compare the real-time image features with the sample database data (matching time <150ms). Under the condition of confidence >95%, the implant position parameters corresponding to the current view are determined: 3mm from the distal surface of the left second premolar, 10mm deep, and at an angle of 85° with the horizontal plane. The parameters are transmitted to the robotic arm controller in real time via the CAN bus, driving the implant handpiece to move along the planned path.

[0054] (3) Real-time identification and avoidance mechanism for dangerous areas During the surgery, an Intel RealSense D455 camera continuously acquired dynamic images of the patient's head and face. These images were processed in real-time using the YOLOv8 algorithm (25fps), maintaining a recognition confidence level of over 98.5% for the eye region in each frame. A 3D coordinate transformation algorithm was used to calculate the real-time distance between the tip of the implant handpiece and the left eye (refresh rate 100Hz). When the surgery reached the implant preparation stage, as the instruments advanced along the planned path towards the bone surface: When the detected distance drops to 12cm (120% of the safe distance), the AI ​​recognition unit sends a warning signal to the control system via TCP / IP protocol (data packet format: [dangerous structure ID, current distance, coordinate information]). The safety trigger unit responds immediately by adjusting the duty cycle of the PWM wave to linearly reduce the movement speed of the implantation mobile phone from 5mm / s to 2mm / s. At the same time, the LED indicator on the doctor's operating table turns yellow, and the buzzer emits an intermittent beep (1Hz frequency).

[0055] If the distance continuously shortens to 10cm (the safety red line threshold) due to slight head movement by the patient, the system immediately triggers secondary braking: ① The power supply to the robotic arm drive motor is cut off via a relay, causing the instrument to stop completely within 0.2 seconds (braking distance <0.5mm); ② A red alarm interface pops up on the control panel display, showing "Danger structure 1 triggers safety red line, distance 10cm"; ③ The voice module simultaneously announces "Instrument has stopped, please confirm safety." At this time, even if the doctor operates the foot switch, the hardware interlock circuit of the safety trigger unit will block the control signal until the doctor confirms that the patient's head has returned to its original position (distance restored to more than 15cm) and presses the "Reset" button. Only then will the system release the braking and resume normal operation.

[0056] Through the above three-level protection mechanism, the instruments always maintain a safe distance from dangerous structures throughout the entire surgical process. The deviation between the implant placement and the planned scheme is controlled within 0.3mm. The operation time is shortened by 40% compared with traditional robots, and no engineer is required to be on duty throughout the entire process. Only one doctor and one nurse are needed to complete the entire operation.

[0057] Third Embodiment This embodiment provides a specific implementation method for integrating the system with existing robot hardware, specifically including: 1. Hardware selection and adaptation design This embodiment uses the "Walker X" humanoid robot body as the core hardware carrier. This model adopts a lightweight aluminum alloy frame, is 1.45m high, weighs 42kg, and has 7 degrees of freedom upper limb joints (3 axes for the shoulder, 1 axis for the elbow, and 3 axes for the wrist). The range of motion of the joints covers ±90° (shoulder) and 0-150° (elbow), which can simulate the adjustment of surgical operation postures of human doctors from standing to sitting. Its torso has an Intel Core i7-12700H processor and 16GB DDR5 memory built in, providing basic computing power support for algorithm operation.

[0058] The end effector uses the "Dexterous Hand G1" dexterous hand, which has 12 degrees of freedom (3 axes for the thumb, 2 axes each for the index and little fingers, plus 1 palm rotation axis). The fingertips are wrapped in silicone (Shore A 50 hardness) and equipped with a 6-axis force sensor (range 0-50N, accuracy ±0.01N), enabling micro-displacement control at the 0.05mm level, adapting to the instrument clamping and angle fine-tuning needs in implant surgery. The end effector of the dexterous hand holds the implant handpiece (same as the NSK VarioSurg 3 in Example 1) through a customized quick-change interface (compatible with ISO 13485 medical standards), and a micro USB camera (same as the 640×480 resolution model in Example 1) is pre-installed 3cm from the working end of the implant hand. The camera data cable is connected to the image processing module of the robot torso through the internal wiring of the dexterous hand, avoiding interference from external cables during surgical operations.

[0059] 2. Path to Technology Integration (1) Algorithm embedding and protocol adaptation For the ROS 2 Humble control system of the Walker X robot, a dedicated algorithm adaptation layer was developed: the "real scene recognition algorithm" of this invention (based on the improved YOLOv8 model) is encapsulated into a ROS function package, and data interaction is established with the robot's native visual perception module through the UDP protocol. The scene images collected by the 3D depth camera (Intel RealSense D455), after being processed by the algorithm, output information such as the coordinates of dangerous structures and safe distances, which can be directly converted into the "sensor_msgs / PointCloud2" format supported by the Walker X system; the "view mapping matching logic" is integrated into the robot's "moveit!" motion planning module. By calling its API interface, the planting depth and angle parameters obtained by view matching are converted into the "geometry_msgs / PoseStamped" motion instructions of the dexterous hand end effector, realizing seamless connection between the algorithm and the hardware.

[0060] (2) Communication delay optimization To reduce communication latency, an "edge computing + hardware acceleration" solution was adopted: An NVIDIA Jetson Orin NX edge computing module (200 TOPS computing power) was installed at the torso extension interface of the Walker X robot to run the "real scene recognition algorithm" and "view mapping matching logic" to avoid the algorithm consuming the robot's main processor resources; communication between the algorithm and the hardware was achieved through the SDK interface (Walker SDK V2.0 provided by Zhiyuan and Dexterous Hand SDK V1.5 provided by Yusu), and the RTPS (Real-Time Publish-Subscribe) protocol was used to transmit key control signals. After testing, the latency from the algorithm outputting control commands to the dexterous hand executing the action was stable at 85-95ms, which meets the real-time requirements of surgery (latency threshold <100ms).

[0061] (3) Functional integration and calibration After the algorithm embedding was completed, multiple rounds of functional integration and accuracy calibration were performed: ① The distortion of the micro USB camera on the implantation phone was corrected using a "chessboard calibration board" to ensure that the image measurement error was <0.05mm; ② Using a "standard implant model" (including a bone tissue module with preset coordinates), the accuracy of implantation by the dexterous hand according to the angle matching parameters was tested. By adjusting the joint PID parameters, the final implantation position deviation was controlled within 0.1mm; ③ The "approaching dangerous structure" condition in the surgical scenario was simulated (tested using an eye model of a human head model) to verify the linkage response of the AI ​​recognition unit and the safety trigger unit, ensuring that the distance detection error was <0.2cm and the braking response time was <0.3s.

[0062] 3. Verification of clinical application effects In a dental clinic setting, the integrated robot was used to perform 10 single-tooth implant surgeries (patients aged 25-60 years, all cases of missing mandibular posterior teeth). The application results are as follows: Operational precision: The repeatability of the dexterous hand movements reached 0.1mm. Postoperative CT scans showed that the average deviation between the actual implant position and the planned scheme was 0.25mm, which meets the clinical precision requirements for dental implantation (deviation threshold <0.5mm). Labor costs: The surgery requires only one surgeon (responsible for confirming the treatment plan and handling emergencies) and one nurse (responsible for instrument delivery), without the need for on-site monitoring by an engineer, reducing labor costs by 50% compared to traditional robotic surgery (3-4 people working together); Surgical efficiency: The average time for a single surgery (from disinfection and preparation to implant placement) is 45 minutes, which is about 30% shorter than the average time of traditional robotic surgery (65 minutes). Moreover, 5 surgeries can be completed in one morning (4 hours of work), breaking through the efficiency limit of "2 surgeries in one morning" of traditional robots. Safety: In 10 surgeries, the safety trigger unit triggered a total of 3 Level 1 warnings (instrument approaching the eye to within 12cm) and 1 Level 2 braking (instrument approaching the eye to within 10cm). All responses were accurate and there were no false triggers. No dangerous structures were accidentally triggered, and no patients experienced postoperative complications.

[0063] Fourth embodiment like Figure 2 As shown, this embodiment provides a method for locating an intelligent oral robot based on real-world recognition, employing the aforementioned real-world recognition-based positioning system. The method includes the following steps: S1: The acquisition unit of the real-scene data acquisition and annotation module acquires depth image data of the surgical scene. After classification and annotation by the annotation unit, the data is input into the robot control system to establish a correspondence between the real scene and the operating rules. S2: The raw data unit of the intraoral cavity positioning and implant path matching module designs the optimal implant position based on CT data and intraoral scan data, establishing a three-dimensional correspondence between CT coordinates and implant position. S3: The viewpoint mapping learning unit adjusts the intraoral scan data viewpoint, generating a sample library of intraoral scan viewpoint-implant position data, which is stored in the robot control system. S4: During surgery, the real-time viewpoint matching unit captures the real-world viewpoint of the oral cavity using a small camera, matches it with the sample library to determine the implant position and surgical path, and outputs implant operation parameters. S5: The AI ​​recognition unit of the real-time danger zone judgment and avoidance module identifies dangerous structures in real time. The safety triggering unit triggers deceleration or stop commands based on the distance between the instrument and the dangerous structure to ensure surgical safety. S6: The surgical operation is completed, achieving precise positioning of the oral intelligent robot based on real-world recognition.

[0064] The following explanation, from the perspective of clinical operation logic and technical collaboration, details each step of the positioning method of this intelligent oral robot, clarifying the core role of each step and the data flow relationship: This positioning method is designed around the clinical process of "preoperative preparation - intraoperative execution - safety protection". Each step is closely linked and deeply adapted to the system modules: Step S1 serves as the preoperative foundation. The acquisition unit obtains depth images of the examination room environment and the patient's head and face (including spatial location and structural morphology information). Then, the labeling unit classifies and marks dangerous structures (such as eyes and nasal cavity), surgical-related structures (such as gingival margins), and safety attributes (safe distance, avoidance rules). Finally, the data is input into the control system. This step is equivalent to "establishing a cognitive system" for the robot, enabling it to master the recognition standards and operational taboos of the real scene before the operation, laying the data foundation for safe identification during the operation.

[0065] Steps S2 and S3 focus on the precise planning of the implantation path: S2 uses CT data (reflecting the anatomical structure of the jawbone) and intraoral scan data (reflecting the surface morphology of the oral cavity) as the core, and designs the optimal position of the implant (taking into account medical requirements such as bone volume and occlusion) through virtual tooth arrangement technology, and establishes a three-dimensional correlation between CT coordinates and the implant to ensure the clinical rationality of the plan; S3 utilizes the characteristic of intraoral scan data that can flexibly adjust the viewing angle to generate a sample library of "intraoral scan view - implant position" - this step is equivalent to letting the robot "learn the viewpoint rules", and master the correspondence between tooth features and implant positions under different views through a large number of samples, providing data support for rapid path matching during surgery.

[0066] Steps S4 to S6 are the intraoperative execution and safety assurance stages: In S4, a small camera on the surgical instrument captures the real view of the oral cavity, and the robot calls the sample library for rapid matching (without traditional 3D reconstruction), directly outputting operational parameters such as implantation depth and angle, achieving "what you see is what you get" path positioning, greatly improving surgical efficiency; S5 is the core of safety protection, where the AI ​​recognition unit captures images in real time through a 3D depth camera, automatically identifying dangerous structures, and the safety triggering unit responds in stages (deceleration or stopping) according to the distance between the instrument and the dangerous structure, avoiding the risk of accidental contact, which is equivalent to adding a "smart safety valve" to the surgery; S6 serves as the finishing touch, after the implantation operation is completed, the positioning accuracy is verified through postoperative examination to ensure that the surgical effect meets the standards, forming a complete closed loop of "planning-execution-verification".

[0067] Overall, this method, through its workflow design of "preoperative data preparation - intraoperative precise matching - real-time safety protection," leverages the technical advantages of each module (such as real-scene recognition and viewpoint mapping matching) while also meeting the actual needs of clinical surgery, achieving a synergistic improvement in positioning accuracy, surgical efficiency, and operational safety.

[0068] A computer-readable storage medium stores computer code that, when executed, performs the methods described above. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0069] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An oral cavity intelligent robot positioning system based on real world recognition, characterized in that, The system comprises the following modules: A real scene data acquisition and labeling module acquires real environment data of a surgical scene using three-dimensional depth imaging technology, and establishes a recognition rule system including dangerous structures, surgery-related structures, and safety attributes through structured labeling; An intraoral positioning and implant path matching module constructs an implant position model based on CT and oral scan data, and realizes rapid matching of the implant path using a multi-angle perspective mapping algorithm, without the need for complex three-dimensional reconstruction and coordinate registration processes; A dangerous area real-time determination and avoidance module integrates AI image recognition and spatial distance calculation functions to monitor the operating environment in real time during surgery and trigger graded safety controls to avoid the robot touching dangerous structures by mistake; A system hardware and technology fusion optimization module uses a lightweight hardware architecture of a humanoid robot and a dexterous hand, embeds real scene recognition algorithms and perspective mapping matching logic, realizes technology fusion with mature robot control systems, and improves the practicality and stability of the system.

2. The real world recognition based oral cavity intelligent robot positioning system according to claim 1, wherein, The real scene data acquisition and labeling module comprises an acquisition unit and a labeling unit. The acquisition unit uses a three-dimensional depth camera or a two-dimensional camera in combination with a depth calculation algorithm to capture panoramic environments of dental clinics, complete forms of patients lying on dental chairs, and the surrounding areas of the oral cavity, to obtain depth image data containing spatial position information and structural form information, and to simulate the eyes of the robot to achieve comprehensive perception of the real surgical scene. The labeling unit classifies and labels the acquired depth image data manually, clearly marks important physiological structures adjacent to the oral cavity including the eyes and nasal cavity as dangerous structures, marks structures directly related to surgery including the gum edge and tooth surface as surgery-related structures, marks the allowable range of operation of surgical instruments as a safety area, defines the safety attributes and avoidance rules of each structure, and inputs the labeled structure-attribute-avoidance rule data into the robot control system to establish a correspondence between the real scene and the robot operation rules.

3. The real world recognition based oral cavity intelligent robot positioning system according to claim 1, wherein, The intraoral positioning and implant path matching module comprises an original data unit, a perspective mapping learning unit, and a real-time perspective matching unit. The original data unit uses CT data and oral scan data as core original data, simulates tooth arrangement states in a computer through virtual tooth arrangement technology, designs the optimal position of the implant according to the physiological characteristics of the patient including the jaw bone structure and tooth gap, establishes a three-dimensional correspondence between the CT coordinates and the implant position, and ensures the medical rationality of the implant plan; The perspective mapping learning unit uses the 360-degree dead angle-free adjustment of the perspective of the oral scan data in the computer software to record the positions of the oral scan image and the virtual implant model under each perspective, generates a large number of corresponding samples of the oral scan perspective-implant position, forms a perspective mapping sample library, and makes the robot system learn the position rules of the implant under different perspectives every time the perspective of the oral scan data is adjusted. The real-time visual angle matching unit is equipped with a small camera on the planting handset or surgical instrument of the robot. During the surgery, the small camera captures one or more real visual angles of the patient's oral cavity. The robot control system calls the visual angle mapping sample library to match the real-time captured visual angle with the visual angle in the sample library, quickly determines the actual position of the implant and the surgical path under the current visual angle, and directly outputs the implant operation parameters including the implant depth and angle, without the traditional three-dimensional reconstruction and coordinate registration process.

4. The real world recognition based oral cavity intelligent robot positioning system according to claim 1, wherein, The dangerous area real-time judgment and avoidance module includes an AI recognition unit; The AI recognition unit integrates an AI image recognition algorithm. During the surgery, the three-dimensional depth camera in the real scene data acquisition and labeling module captures image data of the surgical area in real time. The AI algorithm automatically identifies the labeled dangerous structure in the image and converts the recognition result into an electrical signal to be transmitted to the robot control system.

5. The real world recognition based oral cavity intelligent robot positioning system according to claim 4, wherein, The dangerous area real-time judgment and avoidance module also includes a safety trigger unit; The safety trigger unit presets a safety red line, i.e., a safety distance range around the dangerous structure, in the robot control system. When the AI recognition unit detects that the distance between the surgical instrument and the dangerous structure approaches the safety red line, the safety trigger unit automatically triggers a deceleration instruction. If the instrument continues to approach and reaches the safety red line boundary, the safety trigger unit immediately triggers a stop instruction to force the surgical instrument to stop moving, ensuring that the instrument always operates within a safe area and avoids dangerous structures.

6. The real world recognition based oral cavity intelligent robot positioning system according to claim 1, wherein, The system hardware and technology fusion optimization module adopts a hardware lightweight design scheme; The hardware lightweight design scheme specifically adopts a hardware form of humanoid robot + dexterous hand. The humanoid robot is equipped with a dexterous hand that can simulate human hand movements. The dexterous hand at the end holds a small camera-equipped implant handset or other surgical instrument, allowing the robot to operate at close range like a human doctor, replacing the complex hardware of traditional robots including large mainframes and independent camera supports, reducing equipment space occupation, and reducing system complexity.

7. The real world recognition based oral cavity intelligent robot positioning system according to claim 1, wherein, The system hardware and technology fusion optimization module adopts a technology fusion upgrade scheme; The technology fusion upgrade scheme specifically combines mature robot technology to embed real scene recognition algorithms and visual angle mapping matching logic into the control system of a mature robot, taking advantage of the motion accuracy and stability of mature hardware to improve the operation accuracy of the robot while reducing research and development costs and technical difficulty.

8. A real world recognition based oral intelligent robot positioning method using the real world recognition based oral intelligent robot positioning system according to any one of claims 1 to 7, characterized by, The steps include: S1: Obtain the depth image data of the surgical scene through the acquisition unit of the real scene data acquisition and labeling module. After classification and labeling by the labeling unit, input the data into the robot control system to establish the corresponding relationship between the real scene and the operation rules; S2: The original data unit of the oral cavity internal positioning and implant path matching module designs the optimal position of the implant based on CT data and oral scan data, establishing a three-dimensional correspondence between the CT coordinates and the implant position; S3: The visual angle mapping learning unit adjusts the oral scan data visual angle to generate a sample library of oral scan visual angle-implant position, which is stored in the robot control system; S4: During surgery, the real-time view matching unit captures the real view of the oral cavity through a small camera, matches it with the sample library to determine the implant position and surgical path, and outputs the implant operation parameters; S5: The AI recognition unit of the dangerous area real-time judgment and avoidance module identifies the dangerous structure in real time, and the safety trigger unit triggers the deceleration or stop instruction according to the distance between the instrument and the dangerous structure to ensure the safety of the surgery; S6: Complete the surgical operation and realize the precise positioning of the oral cavity intelligent robot based on real-world recognition.

9. A computer device comprising a memory and one or more processors, the memory having computer code stored therein, which, when executed by the one or more processors, causes the one or more processors to perform the method of claim 8.

10. A computer-readable storage medium having computer code stored therein, which, when executed, performs the method of claim 8.

Images