Oral bone grafting navigation system based on multi-modal mixed reality interaction

Abstract

The invention relates to the field of oral medicine, and discloses an oral bone grafting navigation system based on multi-modal mixed reality interaction, which comprises a data processing module, a mixed reality display module and a navigation module, the data processing module is used for acquiring CBCT data and oral scanning data of the oral cavity of the patient, generating a composite three-dimensional model fusing bone tissue and dentition morphology according to the acquired data, and obtaining the three-dimensional morphology, the target placement position and the screw fixing position of the bone grafting bone sheet; the mixed reality display module comprises a head-mounted display device capable of displaying the virtual indication unit and the virtual operation area presentation unit; and the navigation module comprises a navigation marker and an optical positioning camera and is used for tracking the spatial positions of the surgical instrument, the bone sheet and the screw in real time. According to the system and the method, multi-modal image data are combined, and the mixed reality display module and the navigation module are combined, so that real-time visual guidance of the bone slice position and the screw implanting path is realized, and the bone grafting operation efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to the field of stomatology, and in particular to an oral bone grafting navigation system based on multimodal mixed reality interaction. Background Art

[0002] In oral bone grafting, precise fixation of the bone fragment is crucial for ensuring postoperative bone fusion. Traditional bone grafting relies on the surgeon's experience and the use of screws, which presents the following challenges: 1. Inadequate screw positioning accuracy: The surgeon relies solely on two-dimensional images to determine the screw placement, depth, and angle, which can easily cause the screw to deviate from the planned path, potentially damaging nerves and blood vessels (such as the inferior alveolar nerve) or penetrating the bone cortex. 2. Low fixation efficiency: The position of the bone fragment and screws must be repeatedly adjusted, requiring multiple intraoperative X-rays for confirmation, which prolongs the surgery. 3. Lack of visual guidance: The lack of real-time, dynamic three-dimensional guidance makes it impossible to intuitively determine the spatial relationship between the screw, the bone fragment, and the host bone, potentially causing the fragment to shift or the screw to loosen.

[0003] While existing navigation systems can track the position of bone fragments, they lack precise guidance for the screw fixation process. Furthermore, they rely on comparing a virtual model displayed on a screen with the actual surgical field, requiring frequent eye switching during operation, which impacts continuity. Therefore, a mixed reality system is urgently needed that can simultaneously navigate both the bone fragment and the fixation screws in real time, addressing the inaccurate fixation positioning and insufficient visualization issues of existing technologies. Summary of the Invention

[0004] In response to the problems of low positioning accuracy and insufficient visualization in the screw fixation of bone fragments in oral bone grafting surgery in the prior art, the present invention provides an oral bone grafting navigation system based on multimodal mixed reality interaction.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is: An oral bone grafting navigation system based on multimodal mixed reality interaction, comprising a data processing module, a mixed reality display module, and a navigation module; Data processing module: includes an image processing unit, a bone graft planning unit, and a data output unit; the image processing unit is used to obtain the patient's oral CBCT data and oral scan data, align the CBCT data with the oral scan data through feature points, and generate a three-dimensional oral model containing the bone defect area through a three-dimensional reconstruction algorithm, annotating the bone structure and bone defect boundary; the bone graft planning unit is used to plan the three-dimensional shape and target placement position of the bone fragment in the three-dimensional model, mark the fitting surface between the bone fragment and the host bone; and design the screw fixing position on the three-dimensional model, annotating the starting point, depth, and angle of the screw implantation path; the data output unit is used to generate navigation data including the three-dimensional model of the bone fragment and the screw planning coordinates, and transmit it to the mixed reality display module and navigation module; Mixed reality display module: includes a head-mounted display device capable of displaying a virtual indication unit and a virtual surgical area presentation unit. The virtual indication unit is used to display the target and real-time positions of bone fragments and screws in real time. The virtual surgical area presentation unit is used to display a three-dimensional model of the patient's oral cavity and a real-time virtual model of the instruments used during surgery. Navigation module: includes navigation markers and optical positioning cameras, which are used to track the spatial position of surgical instruments, bone fragments, and screws in real time; and transmit real-time spatial position data and three-dimensional spatial deviation to the data processing module and mixed reality display module.

[0006] Furthermore, the virtual indication unit includes a bone fragment virtual indicator and a screw virtual indicator; the bone fragment virtual indicator is used to indicate the position, angle and depth of the bone fragment, and the screw virtual indicator is used to indicate the horizontal position, depth and angle accuracy of the screw.

[0007] Furthermore, the bone fragment virtual indicator uses two positioning frames to indicate the position and angle of the bone fragment. The target positioning frame is fixed next to the target position of the patient's bone defect area, and the real-time positioning frame is attached to the edge of the bone fragment. When the two positioning frames completely overlap, the position and angle of the bone fragment meet the planning requirements; the depth of the bone fragment is indicated by a depth progress bar, and the depth accuracy bar is displayed next to the target positioning frame, which displays the vertical distance between the current position of the bone fragment and the target position in real time.

[0008] Furthermore, the screw virtual indicator uses two positioning circles to indicate the horizontal position and depth of the screw. The target positioning circle is displayed at the planned screw implantation point, and the real-time positioning circle is attached to the implant mobile phone drill bit. When the real-time positioning circle and the target positioning circle completely overlap, the horizontal position of the screw meets the planning requirements. A ring-shaped depth indicator bar is set on the outer circle of the target positioning circle to display the implantation depth. When the ring is completely filled, it indicates that the screw has reached the preset depth; the angle accuracy of the screw is indicated by a ring-shaped dotted line indicator bar. The ring-shaped dotted line indicator bar surrounds the implant mobile phone, and the length of the dotted line in the corresponding direction reflects the angle deviation in that direction.

[0009] Furthermore, the virtual surgical area presentation unit displays a reconstructed three-dimensional skeletal model of the patient. The three-dimensional skeletal model is rendered in translucent white, and the bone defect area and the preoperatively planned bone fragments and screw positions are marked on the three-dimensional skeletal model. It also dynamically displays a virtual model of the implant mobile phone synchronized with the real instrument posture and a virtual model of the screw that moves in real time with the drill bit position. Furthermore, the mixed reality display module also includes a spatial matching unit, which includes a positioning plate with marking points on it. The marking points can be recognized by the navigation module and the mixed reality display module; the navigation module and the mixed reality display module respectively identify the spatial coordinates of the positioning plate, and establish a relative transformation relationship between the coordinate systems of the two modules to achieve spatial alignment.

[0010] Furthermore, the mixed reality display module also includes a gesture recognition unit. The system interface adopts a layered focus design, which is divided into a status layer and a guide layer. The status layer continuously displays basic surgical information, and the guide layer dynamically presents navigation instructions.

[0011] Furthermore, the navigation markers include fixed markers and movable markers. The fixed markers are set on the patient's oral teeth and are used to establish the patient coordinate system reference; the movable markers are set at the end of the bone clamp and the tail of the implant handpiece to establish the instrument coordinate system and provide the real-time position and posture of the instrument; the optical positioning camera collects the characteristic points of the navigation markers and establishes the conversion relationship between the instrument coordinate system and the patient coordinate system.

[0012] Furthermore, the fixed marker is a medical-grade ARuco QR code marking plate, the movable marker at the end of the bone clamp is an ARuco QR code, and the movable marker at the tail of the implant handpiece is a ring-shaped ARuco marking band.

[0013] The beneficial effects of the present invention are: The oral bone grafting navigation system of the present invention combines multimodal imaging data, including CBCT data and high-precision oral scan data. By combining mixed reality display technology with the navigation system, it can achieve real-time visual guidance of bone fragment position and screw implantation path, accurately control the bone fragment placement angle and the position, depth, and angle of screw implantation, ensure stable fixation of the bone fragment to the host bone, improve the efficiency of bone grafting surgery, reduce the risk of neurovascular injury, and enhance the postoperative bone fusion effect.

[0014] The oral bone grafting navigation system of the present invention achieves submillimeter-level real-time registration of multimodal imaging data. Through the innovative "marker point-surface" dual registration strategy, the registration error of the traditional navigation system is reduced from 1 to 2 mm to less than 0.3 mm. The system of the present invention uses improved ArUco markers as the basic registration reference, arranges at least three non-coplanar marker points in the patient's mouth, establishes a stable patient coordinate system, and significantly improves the recognition accuracy and anti-occlusion ability of the marker points. Based on the marker point registration, the system further performs feature point-based surface registration, obtains the surface topology data of the surgical area through a high-precision oral scanner, extracts at least 5,000 feature points, and performs ICP (Iterative Closest Point) matching with the three-dimensional model reconstructed by preoperative CBCT. To overcome the problem that traditional ICP algorithms are prone to falling into local optimality, ArUco markers are first used as the basic registration reference, and then surface registration of feature points is used to greatly improve the effect. The dual registration results are fused through a Kalman filter, and a stable spatial transformation matrix is ​​finally output to ensure accurate alignment of the virtual model with the real anatomical structure.

[0015] Obviously, based on the above contents of the present invention, according to common technical knowledge and customary means in this field, without departing from the above basic technical ideas of the present invention, other various forms of modifications, replacements or changes can be made.

[0016] The following is a further detailed description of the present invention through specific embodiments in the form of examples. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-mentioned content of the present invention fall within the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the three-dimensional model.

[0018] Figure 2 It is the status display area of ​​the bone fragment virtual indicator.

[0019] Figure 3 This is a diagram showing the virtual indicator for the screw.

[0020] Figure 4 This is a display diagram of the virtual surgical area.

[0021] Figure 5 A diagram indicating the navigation status. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0023] Example 1: An oral bone grafting navigation system based on multimodal mixed reality interaction includes a data processing module, a mixed reality display module, and a navigation module.

[0024] (1) Data processing module This module is used for data input and three-dimensional reconstruction, and specifically includes an image processing unit, a bone grafting planning unit, and a data output unit.

[0025] The image processing unit is used to obtain the patient's oral CBCT data and oral scan data containing the morphology of the dentition and soft tissue surface, align the CBCT data and the oral scan data through feature points (alignment error ≤ 50μm), and generate an oral 3D model containing the bone defect area through a 3D reconstruction algorithm, marking the bone structure and bone defect boundary, such as Figure 1 shown.

[0026] The bone graft planning unit is used by doctors to plan the source of bone chips, three-dimensional shape (size / radian / thickness) and target placement position (coordinates / angles) in the three-dimensional model, and mark the fitting surface between the bone chips and the host bone, such as Figure 1 As shown in the yellow area in the middle; design the screw fixation position on the three-dimensional model (plan 1 to 3 fixation points for each bone fragment), and mark the starting point, depth, and angle of the screw implantation path (the angle with the normal line of the bone fragment surface is ≤15°) to avoid damaging important structures.

[0027] The data output unit is used to generate navigation data including a three-dimensional model of the bone fragment and screw planning coordinates (X / Y / Z axis position, implantation depth, and angle parameters), and transmits it to the mixed reality display module and navigation module.

[0028] (2) Mixed Reality Display Module The module is based on a mixed reality head-mounted display device and provides visual guidance for bone chip and screw placement through intuitive indicators. Specifically, it includes: 1. Virtual indication unit According to the three-dimensional shape of the bone fragment and the target placement position, and the target position, angle, and depth of the screw designed by the data processing module, the corresponding bone fragment virtual indicator and screw virtual indicator are displayed in real time in the mixed reality display module.

[0029] The bone fragment virtual indicator includes position and angle indication and depth indication functions. Figure 2 As shown, the position and angle of the bone fragment are indicated using two positioning frames, wherein the target positioning frame (purple) is fixed next to the target position of the patient's bone defect area and keeps the relative position unchanged as the patient's head moves. The real-time positioning frame (white) is attached to the edge of the bone fragment and moves synchronously with the movement of the bone fragment, keeping the relative position unchanged with the bone fragment. By adjusting the position and angle of the bone fragment, the real-time positioning frame is completely embedded in the target positioning frame and kept overlapping. At this time, the position and angle of the bone fragment meet the planning requirements. The depth indication of the bone fragment is displayed by displaying a depth progress bar next to the target positioning frame, showing the vertical distance between the current position of the bone fragment and the target position in real time. The progress bar value is dynamically updated to intuitively reflect the depth state of the bone fragment entering the target position. Preferably, the shape of the positioning frame is rectangular. The screw virtual indicator includes horizontal position and depth indication and angle accuracy indication functions. Figure 3As shown, the horizontal position and depth of the screw are indicated using two positioning circles. The target positioning circle (blue) is displayed at the planned screw insertion point and maintains its relative position as the patient moves. The real-time positioning circle (yellow) is attached to the implant handpiece drill bit and moves synchronously with the implant handpiece. When the real-time positioning circle and the target positioning circle completely overlap, the screw's horizontal position meets the planned requirements. A circular depth indicator bar is located outside the target positioning circle to indicate the implant depth. Preferably, the circular depth indicator bar is filled with color (white), starting at 0° and gradually filling to 360°. When the circular depth indicator bar is completely filled, it indicates that the screw has reached the preset depth. The screw's angular accuracy is indicated by a circular dashed line indicator bar. This circular dashed line is a circle of "angle error dashed lines" surrounding the implant handpiece. The length of the dashed line corresponds to the angular deviation in different directions—the greater the deviation, the longer the dashed line in the corresponding direction. The doctor adjusts the handpiece's posture by extending the dashed line in the opposite direction until the angular error is within the threshold. 2. Virtual surgical area presentation unit The virtual surgical area presentation unit is used to display the patient's three-dimensional bone model reconstructed in translucent white rendering, clearly marking the bone defect area and the bone slices and screw positions planned before the operation, such as Figure 4 As shown; on the other hand, the virtual model of the implant mobile phone that is dynamically displayed and synchronized with the real instrument posture, as well as the virtual model of the screw that moves in real time with the drill bit position, intuitively present the surgical operation trajectory. 3. Spatial matching unit To achieve precise alignment between the virtual indicator and the real scene, a multifunctional positioning board is placed in the surgical area as a coordinate system registration medium. The board features markers that can be recognized by both the navigation module and the mixed reality display module. By bidirectionally identifying the board's spatial coordinates, the relative transformation relationship between the two modules' coordinate systems is calculated and established, ensuring that the virtual indicator is accurately mapped to the actual position of the patient and the implant handpiece.

[0030] 4. Gesture recognition unit The system interface adopts a layered focus design, such as Figure 5 As shown in Figure 1, key information is divided into a status layer and a guidance layer, each corresponding to different visual salience. The status layer continuously displays basic surgical information, such as the current instrument position and bone fragment status, while the guidance layer dynamically presents navigation instructions, such as the target path and current position.

[0031] (3) Navigation module The navigation module includes a high-precision optical positioning camera and navigation markers, which use optical positioning technology to track the spatial position of surgical instruments and bone fragments in real time.

[0032] Navigation markers include fixed and movable markers. The fixed markers are medical-grade ARuco QR code markers, affixed to the lingual surfaces of the patient's maxillary and mandibular central incisors, respectively, to establish the oral coordinate system reference. The movable markers include bone fragment clamping device markers and implant handpiece markers, used to establish the instrument coordinate system. The bone fragment clamping device marker is an ARuco QR code marked on the end of the bone clamp, enabling 6-degree-of-freedom (position + posture) tracking of the bone fragment. The implant handpiece marker is a circular ARuco marker band marked on the end of the implant handpiece, ensuring stable recognition by the optical positioning camera even when the handpiece is rotated.

[0033] At least two optical positioning cameras are used to capture images of navigation markers and decode them in real time. They calculate the 3D coordinates and posture parameters of each navigation marker, establish a transformation between the instrument coordinate system and the patient coordinate system, and perform real-time positioning calculations. They also perform deviation analysis for bone fragment positioning and screw placement, calculating the 3D coordinate and angular deviations between the actual fragment position and the preoperatively planned position. They also analyze in real time the distance and angular deviations between the implant handpiece drill tip and the planned screw path. This deviation data is then transmitted to the mixed reality display module via a low-latency wireless transmission protocol, driving the dynamic update of the virtual indicators for the bone fragment and screws.

[0034] The specific implementation of the oral bone grafting navigation system of the present invention is as follows: 1. Preoperative preparation 1.1. Multimodal data acquisition and 3D planning through data processing module The patient's oral CBCT data (0.3mm slice thickness) is collected and input into a data processing module to generate a 3D model of the bone defect area. The doctor then plans the bone fragment's 3D shape (radian matching error ≤ 0.5mm) and target position (coordinate accuracy ±0.1mm) within the model. One to three screw fixation points are marked on the fragment's surface, and the implantation path is designed: starting at the center of the planned points, at a depth of 8-15mm, and with an angle of ≤15° relative to the normal to the fragment's surface.

[0035] Use a high-precision oral scanner to obtain the surface morphology of the dentition and soft tissue (accuracy 20μm, single-frame scanning time ≤ 0.5s) The CBCT data and oral scan data were registered by feature points (registration error ≤ 50 μm) to generate a composite three-dimensional model that integrates bone tissue and dentition morphology.

[0036] 1.2. Navigation marker installation and calibration A medical-grade ARuco marker plate was affixed to the lingual side of the patient's maxillary and mandibular central incisors. A customized marker module was installed at the end of the pliers, and a circular marker band was fixed to the end of the implant handpiece. An optical positioning camera (accuracy ±0.1mm) captured the navigation marker feature points and established a conversion relationship between the instrument coordinate system and the patient coordinate system (error ≤0.3mm).

[0037] (2) Surgical Procedure 2.1 System initialization and spatial registration The doctor wears a mixed reality headset, and the system establishes the patient's coordinate system using fixed markers. A multifunctional positioning plate is placed in the surgical area, and bidirectional coordinate calculations are used to align the virtual model with the real scene (registration error ≤ 0.5mm). The display module then loads the translucent bone model, the target triangle frame for the bone fragment, and the planned screw path.

[0038] 2.2 Bone Slice Placement Navigation Operation Grasp the bone fragment with a bone forceps with a marker, and the mixed reality device displays the real-time positioning frame (white) of the edge of the bone fragment and the target positioning frame (purple) next to the bone defect area in real time. Figure 2 The doctor adjusts the position of the bone fragment. When the white real-time positioning frame is completely embedded in the purple target positioning frame and the depth progress bar shows that the vertical distance is less than 0.3mm and the angle deviation is less than 3°, the indicator turns solid green, indicating that the gap between the bone fragments is ≤0.2mm.

[0039] 2.3 Precise Navigation for Screw Insertion Replace the implant mobile phone with a marker, the display module displays the target positioning circle (blue) at the planned fixed point, and the real-time positioning circle (yellow) is displayed synchronously near the drill bit. Figure 3 When the two circles completely overlap (horizontal deviation ≤ 0.5mm), the outer ring of the target positioning circle fills with color to indicate the implant depth (0° to 360°), while the dotted line indicating the angular error around the implant handpiece dynamically shortens. When the depth reaches the preset value (e.g., 12mm) and the angular deviation is ≤ 3°, the system sounds a warning and the doctor inserts the screw.

[0040] 2.4. Multi-screw collaborative fixing process After the first screw is secured, the display module automatically activates the next screw guide. The virtual surgical field renders the spatial relationship between the implanted screws and the bone fragment in real time to avoid screw interference. Once all screws are secured, the system generates a deviation report (position deviation ≤ 0.5mm, angular deviation ≤ 3°).

[0041] (3) Postoperative treatment 3.1 Data Recording and Equipment Disinfection Remove the markers and disinfect the equipment, and save the surgical data: actual coordinates of the bone fragment (deviation ±0.15mm), measured screw implantation depth / angle, and real-time deviation curve (sampling frequency 10Hz).

[0042] 3.2. Postoperative evaluation report generation The system generates an evaluation report based on intraoperative data, including 3D reconstruction comparison images and neurovascular safety distance detection results (≥2mm), which can be fused with postoperative CBCT images to analyze the fixation effect.

[0043] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. An oral bone grafting navigation system based on multimodal mixed reality interaction, characterized by: Including data processing module, mixed reality display module, and navigation module; Data processing module: includes an image processing unit, a bone graft planning unit, and a data output unit; the image processing unit is used to obtain the patient's oral CBCT data and oral scan data, align the CBCT data with the oral scan data through feature points, and generate a three-dimensional oral model containing the bone defect area through a three-dimensional reconstruction algorithm, annotating the bone structure and bone defect boundary; the bone graft planning unit is used to plan the three-dimensional shape and target placement position of the bone fragment in the three-dimensional model, mark the fitting surface between the bone fragment and the host bone; and design the screw fixing position on the three-dimensional model, annotating the starting point, depth, and angle of the screw implantation path; the data output unit is used to generate navigation data including the three-dimensional model of the bone fragment and the screw planning coordinates, and transmit it to the mixed reality display module and navigation module; Mixed reality display module: includes a head-mounted display device capable of displaying a virtual indication unit and a virtual surgical area presentation unit. The virtual indication unit is used to display the target and real-time positions of bone fragments and screws in real time. The virtual surgical area presentation unit is used to display a three-dimensional model of the patient's oral cavity and a real-time virtual model of the instruments used during surgery. Navigation module: includes navigation markers and optical positioning cameras, used to track the spatial position of surgical instruments, bone fragments, and screws in real time; The real-time spatial position data and three-dimensional spatial deviation are transmitted to the data processing module and the mixed reality display module.

2. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 1, characterized in that: The virtual indicator unit includes a bone fragment virtual indicator and a screw virtual indicator; the bone fragment virtual indicator is used to indicate the position, angle and depth of the bone fragment, and the screw virtual indicator is used to indicate the horizontal position, depth and angle accuracy of the screw.

3. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 2, characterized in that: The bone fragment virtual indicator uses two positioning frames to indicate the position and angle of the bone fragment. The target positioning frame is fixed next to the target position in the patient's bone defect area, and the real-time positioning frame is attached to the edge of the bone fragment. When the two positioning frames completely overlap, the position and angle of the bone fragment meet the planning requirements; the depth of the bone fragment is indicated by a depth progress bar, and the depth accuracy bar is displayed next to the target positioning frame, which displays the vertical distance between the current position of the bone fragment and the target position in real time.

4. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 2, characterized in that: The screw virtual indicator uses two positioning circles to indicate the horizontal position and depth of the screw. The target positioning circle is displayed at the planned screw implantation point, and the real-time positioning circle is attached to the implant mobile phone drill bit. When the real-time positioning circle completely overlaps with the target positioning circle, the horizontal position of the screw meets the planning requirements. A circular depth indicator bar is set on the outer circle of the target positioning circle to display the implantation depth. When the circular depth indicator bar is completely filled, it indicates that the screw has reached the preset depth; the angle accuracy of the screw is indicated by a circular dotted line indicator bar. The circular dotted line indicator bar surrounds the implant mobile phone, and the length of the dotted line in the corresponding direction reflects the angle deviation in that direction.

5. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 1, characterized in that: The virtual surgical area presentation unit displays a reconstructed three-dimensional bone model of the patient. The three-dimensional bone model is rendered in translucent white, and the bone defect area and the bone fragments and screw positions planned before the operation are marked on the three-dimensional bone model. It also dynamically displays a virtual model of the implant mobile phone that is synchronized with the real instrument posture, and a virtual model of the screw that moves in real time with the drill bit position.

6. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 1, characterized in that: The mixed reality display module also includes a spatial matching unit, which includes a positioning plate with marking points on it. The marking points can be recognized by the navigation module and the mixed reality display module; the navigation module and the mixed reality display module respectively identify the spatial coordinates of the positioning plate and establish a relative conversion relationship between the coordinate systems of the two modules to achieve spatial alignment.

7. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 1, characterized in that: The mixed reality display module also includes a gesture recognition unit. The system interface adopts a layered focus design, which is divided into a status layer and a guide layer. The status layer continuously displays basic surgical information, and the guide layer dynamically presents navigation instructions.

8. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 1, characterized in that: The navigation markers include fixed markers and movable markers. The fixed markers are set on the patient's oral teeth and are used to establish the patient's coordinate system reference; Active markers are located at the end of the bone clamp and the tail of the implant handpiece to establish the instrument coordinate system and provide the real-time position and posture of the instrument; The optical positioning camera collects the feature points of the navigation marker and establishes the conversion relationship between the instrument coordinate system and the patient coordinate system.

9. The oral bone grafting navigation system based on multimodal mixed reality interaction according to claim 8, characterized in that: The fixed marker is a medical-grade ARuco QR code marking plate, the movable marker at the end of the bone clamp is an ARuco QR code, and the movable marker at the tail of the implant handpiece is a circular ARuco marking band.

Images