Oral osteotomy navigation system and method based on mixed reality interaction and storage medium

Abstract

The invention provides an oral osteotomy navigation system and method based on mixed reality interaction and a storage medium, and belongs to the field of oral medicine. The oral cavity osteotomy navigation system comprises a data processing module, a mixed reality display module, a navigation module and related navigation markers. According to the system, the mixed reality display technology and the navigation system are combined, and precise navigation of oral cavity osteotomy operation is achieved. The system can visually display the bone cutting line and the virtual operation area, so that a doctor does not need to frequently switch sight lines in the operation process, information such as the bone cutting position, depth and angle is accurately obtained in real time, the precision and efficiency of operation operation are improved, operation wounds are reduced, and the operation effect is improved.

Description

Technical Field

[0001] The present invention belongs to the field of stomatology, and in particular relates to an oral osteotomy navigation system, method and storage medium based on mixed reality interaction. Background Art

[0002] In the field of dentistry, bone cutting is a common and critical step in orthognathic surgery and maxillofacial deformity correction. Traditional surgical methods rely primarily on the surgeon's clinical experience and preoperative two-dimensional CT images or X-rays for surgical planning and execution. However, this approach has many limitations. During surgery, it is difficult for the surgeon to obtain real-time and intuitive three-dimensional anatomical information within the patient's mouth. Furthermore, it is difficult to accurately determine the position of the surgical instruments and the target bone cutting area.

[0003] This difficulty in obtaining information significantly impacts surgical accuracy, making it prone to problems such as deviations in bone cutting position, depth, or angle that do not conform to preoperative planning. These issues not only increase surgical risks, such as potentially damaging nerves and blood vessels and affecting postoperative occlusion, thus compromising surgical outcomes, but also prolong operations and cause greater trauma and pain to patients.

[0004] While some existing navigation systems can provide a certain degree of positioning information, most are based on traditional display technologies, such as screen displays. During surgery, doctors need to frequently switch their line of sight between the patient's mouth and the screen, which is not only inconvenient but also easily distracts the doctor. Furthermore, traditional navigation systems lack the ability to display bone cutting lines in real time and perform dynamic intraoperative calibration, making it difficult to accurately integrate virtual bone cutting plans with the patient's actual oral environment. This limits the precision of complex bone cutting procedures.

[0005] Therefore, it is urgent to develop a navigation system that can assist doctors in performing oral osteotomy operations more intuitively and accurately, and the oral osteotomy navigation system based on mixed reality interaction came into being. Summary of the Invention

[0006] In view of the above problems existing in the prior art, the object of the present invention is to provide an oral osteotomy navigation system, method and storage medium based on mixed reality interaction.

[0007] The present invention provides an oral osteotomy navigation system based on mixed reality interaction, comprising: The input module is configured to input the patient's oral cone-beam CT data; The data processing module is configured to generate a three-dimensional model of the patient's oral cavity through a three-dimensional reconstruction algorithm using the patient's oral cone-beam CT data to obtain bone cutting line design information; A mixed reality display module is configured as a mixed reality head-mounted display device capable of displaying a bone cutting indicator and a virtual surgical field. The bone cutting indicator display displays bone cutting line information in real time within a three-dimensional model of the patient's oral cavity. The virtual surgical field display integrates the three-dimensional model of the patient's oral cavity and real-time position data during surgery. The navigation module is configured to obtain real-time position data of the patient's oral cavity and bone cutting instruments through markers, and transmit the real-time position data and three-dimensional spatial deviation to the data processing module and the mixed reality display module.

[0008] Furthermore, the marker includes a fixed marker and a movable marker. The fixed marker is located in a stable area of ​​the patient's head and is used to determine the position reference of the patient's mouth and provide a unified coordinate system origin; the movable marker is located at the end of the bone cutting instrument and provides the real-time position and posture of the bone cutting instrument.

[0009] Furthermore, the bone cutting line information includes the bone cutting position, shape, depth and angle.

[0010] Furthermore, the calculation formula of the three-dimensional space deviation is as follows: DOI = in, Indicates the Euclidean distance deviation from the instrument tip to the bone cutting line; Indicates the angle between the instrument angle and the normal of the planned bone cutting surface; Indicates the speed of movement of the device; represents the local bone density correction factor; , , represents the weight coefficient, is the distance deviation weight coefficient; is the angle deviation weight coefficient; is the velocity-density deviation weight coefficient, , , It is 0~1.

[0011] Furthermore, is 0.5, is 0.25, is 0.25.

[0012] Furthermore, the three-dimensional reconstruction algorithm is a Marching Cubes algorithm.

[0013] The present invention also provides an oral osteotomy navigation method based on mixed reality interaction, the oral osteotomy navigation method comprising the following steps: S1. Acquire the patient's oral cone-beam CT data; S2. Generate a 3D model of the patient's oral cavity using a 3D reconstruction algorithm based on the patient's oral cone-beam CT data to obtain bone cutting line design information; S3. In combination with a head-mounted display device, the bone cutting line information and real-time position data in the three-dimensional model of the patient's oral cavity are obtained in real time through the bone cutting indicator and the virtual surgical area; S4. Obtain the relative position data of the patient's oral cavity and the bone cutting instrument through markers, and implement navigation based on the relative position data and three-dimensional space deviation.

[0014] Furthermore, the marker includes a fixed marker and a movable marker. The fixed marker is located in a stable area of ​​the patient's head and is used to determine the position reference of the patient's mouth and provide a unified coordinate system origin; the movable marker is located at the end of the bone cutting instrument and provides the real-time position and posture of the bone cutting instrument.

[0015] Furthermore, the bone cutting line information includes the bone cutting position, shape, depth and angle.

[0016] Furthermore, the calculation formula of the three-dimensional space deviation is as follows: DOI = in, Indicates the Euclidean distance deviation from the instrument tip to the bone cutting line; Indicates the angle between the instrument angle and the normal of the planned bone cutting surface; Indicates the speed of movement of the device; represents the local bone density correction factor; , , represents the weight coefficient, is the distance deviation weight coefficient; is the angle deviation weight coefficient; is the velocity-density deviation weight coefficient, , , It is 0~1.

[0017] Furthermore, the , , It is obtained through clinical data training and optimization.

[0018] The present invention also provides a storage medium storing a computer program, wherein the computer program is configured to execute the above-mentioned oral osteotomy navigation system based on mixed reality interaction when running.

[0019] The present invention has achieved the following beneficial effects: This invention provides an oral osteotomy navigation system based on mixed reality interaction. By combining mixed reality display technology with the navigation system, precise navigation for oral osteotomy procedures is achieved. The system intuitively displays the osteotomy line and virtual surgical area, enabling surgeons to accurately obtain information such as the osteotomy position, depth, and angle in real time without frequent visual adjustments during surgery. This improves surgical precision and efficiency, reduces surgical trauma, and enhances surgical outcomes.

[0020] Obviously, based on the above contents of the present invention, according to common technical knowledge and customary means in this field, without departing from the above basic technical ideas of the present invention, other various forms of modifications, replacements or changes can be made.

[0021] The following is a further detailed description of the present invention through specific embodiments in the form of examples. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-mentioned content of the present invention fall within the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the structure of the oral osteotomy navigation system of the present invention.

[0023] Figure 2 Schematic diagram for surgical design.

[0024] Figure 3 Displays a diagram for the bone cutting indicator.

[0025] Figure 4 This is a diagram showing the virtual surgical area. DETAILED DESCRIPTION

[0026] For better understanding and implementation, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0027] In the description of this application, it should be noted that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "far", "near", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limitations on this application.

[0028] It should be noted that the algorithms for data collection, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures, circuit connections, etc. not specifically described can all be implemented through the disclosed content of the prior art.

[0029] Example 1: Oral osteotomy navigation system based on mixed reality interaction of the present invention 1. Overview of the structure of the oral osteotomy navigation system The oral osteotomy navigation system of the present invention includes a data processing module, a mixed reality display module, a navigation module, and related navigation markers and active markers. Figure 1 This is a schematic diagram of the structure of the oral osteotomy navigation system of the present invention. The modules are arranged according to the process logic and can be divided into the following parts: The input module is configured to input the patient's cone beam CT (CBCT) data as a basis for three-dimensional reconstruction of the surgical area; The data processing module is configured to generate a three-dimensional model of the patient's oral cavity through a three-dimensional reconstruction algorithm (a common Marching Cubes algorithm) using the patient's oral CBCT data to obtain bone cutting line design information; A mixed reality display module is configured as a mixed reality head-mounted display device capable of displaying a bone cutting indicator and a virtual surgical field. The bone cutting indicator display displays bone cutting line information in real time within a three-dimensional model of the patient's oral cavity. The virtual surgical field display integrates the three-dimensional model of the patient's oral cavity and real-time position data during surgery. The navigation module is configured to obtain real-time position data of the patient's oral cavity through navigation markers, obtain real-time position data of the bone cutting instrument through active markers, and transmit the real-time position data and three-dimensional spatial deviation of the oral cavity and the bone cutting instrument to the data processing module and the mixed reality display module.

[0030] The core parts are summarized as follows: (1) Data processing module This module acquires CBCT data of the patient's oral cavity and uses an improved deep learning algorithm (using a cascaded 3D convolutional neural network to extract hierarchical features from the mandibular CBCT images and an attention gating mechanism to enhance key anatomical structures during the decoding phase) to reconstruct the mandibular surface. Combined with a super-resolution network, it enhances the resolution of small structures (such as the alveolar nerve canal) to generate a 3D model of the patient's oral cavity. Based on the generated 3D model, the surgeon can use this module to plan the bone cutting line before surgery. Based on the patient's specific condition and surgical requirements, the surgeon can determine the optimal bone cutting position, shape, depth, and angle on the 3D model. The data processing module stores and transmits the bone cutting line design information (including 3D coordinates and geometric parameters) for use in subsequent modules. Furthermore, the data processing module utilizes a deep learning-based 3D model segmentation method to accurately segment the mandibular nerve canal. During navigation, the surgeon can predict the spatial relationship between the 3D position of the mandibular nerve canal and the bone cutting path in real time, providing prompts when necessary.

[0031] Schematic diagram of surgical plan Figure 2 shown.

[0032] (2) Mixed Reality Display Module The module includes a mixed reality head-mounted display device that can integrate virtual information with the real surgical scene. The specific functions are as follows: 1. Bone cutting indicator display: According to the bone cutting line designed by the data processing module, the corresponding bone cutting indicator is displayed in real time on the mixed reality display device. The bone cutting indicator is displayed in a specific color and shape (such as Figure 3 : It is divided into two parts, the angle indicator and the depth indicator) are superimposed on the actual position of the surgical instrument. The doctor can directly see the specific position and direction of the bone cutting line in the patient's mouth through the head-mounted display device, realizing intuitive guidance of the bone cutting operation.

[0033] like Figure 3 As shown, the white progress bar on the left represents the cutting depth indicator, indicating the cutting depth of the bone knife head, and the ring composed of white dotted lines on the right side of the progress bar is the angle indicator, indicating the angle between the knife head and the cutting surface.

[0034] 2. Virtual surgical area display: By integrating a 3D model of the patient's oral cavity with real-time positional data during surgery, a virtual surgical field is displayed on a mixed reality display. This field reflects the progress of the bone cutting procedure in real time and accurately matches the actual surgical field, allowing the surgeon to fully understand the anatomical changes in the surgical area.

[0035] like Figure 4The upper left corner is the virtual operating area, where you can more intuitively see the relative position relationship between the surgical instrument and the patient's jaw; as well as the position of the bone cutting line; you can also see the indicator data.

[0036] (3) Navigation module This module integrates navigation markers placed around the surgical instruments and the patient's mouth. Through the collaborative operation of multiple high-frame-rate optical cameras (200Hz+) and the visual sensors built into the MR headset, a multi-view fusion tracking system is constructed to acquire real-time positional information of the surgical instruments and the patient's mouth. The navigation module transmits this positional information (including 3D coordinates and posture parameters) to the data processing module and the mixed reality display module, enabling the mixed reality display module to dynamically adjust the displayed virtual information based on the actual position, ensuring that the bone cutting indicator and virtual surgical field accurately match the actual position of the patient's mouth. For example, as the surgical instrument moves, the navigation module tracks its position in real time and displays the relative position of the instrument tip and the bone cutting line (such as distance deviation and angle deviation) on the mixed reality display device, assisting the surgeon in making precise adjustments. Specifically, to meet the navigation system's precision requirements, the module integrates weighted parameters such as the 3D spatial deviation (distance and angle) between the surgical instrument and the planned bone cutting line, the instrument's movement speed, and bone density differences. This quantifies the degree of deviation (DOI) of the bone cutting operation in real time and intuitively prompts the surgeon to make adjustments through the mixed reality display module.

[0037] DOI = : Euclidean distance deviation from the instrument tip to the bone cutting line; : The angle between the instrument angle and the normal of the planned bone cutting surface; : Instrument movement speed (dynamic correction of error sensitivity during fast operation); : Local bone density correction factor (through preoperative CT value mapping, high-density areas require greater force feedback compensation); , , is the weight coefficient, which is optimized through clinical data training (experimental test comparison and experience adjustment). In this embodiment, , , The specific settings are 0.5, 0.25, and 0.25.

[0038] (4) Navigation markers Includes fixed markers mounted on the patient's head and movable markers mounted on surgical instruments: 1. Fixed markers: Used to determine the position reference of the patient's mouth, usually attached or clamped to a stable area of ​​the patient's head (such as the zygomatic bone or maxilla) to provide a unified coordinate system origin.

[0039] 2. Activity markers: Integrated into the end of bone-cutting instruments (such as bone drills and osteotome), it tracks the real-time position and posture of the surgical instrument. Its surface features specific identification features (a positioning QR code) to facilitate identification and positioning by the navigation module. This active marker also incorporates a high-precision inertial measurement unit (IMU). Using an adaptive Kalman filter algorithm, it fuses optical positioning data with IMU posture information in real time, effectively compensating for posture drift caused by minor patient movements.

[0040] 2. Specific implementation methods 1. Preoperative preparation 1. Perform a CT scan of the patient's oral cavity to obtain high-resolution CT data, which is then fed into the data processing module. The data processing module uses a 3D reconstruction algorithm to generate a 3D model of the patient's oral cavity (including bone and soft tissue structures).

[0041] 2. In the data processing module, the doctor designs the bone cutting line based on the 3D model, marking the boundaries, depth, and angle of the target bone cutting area to generate a personalized bone cutting plan. Once the design is completed, the bone cutting line information is stored as an interactive 3D data model.

[0042] 3. Install fixed navigation markers in the fixed area of ​​the patient's head to ensure that the markers are stable and unobstructed. At the same time, install movable navigation markers on the osteotomy instrument to calibrate the conversion relationship between the instrument coordinate system and the navigation system coordinate system.

[0043] (2) Surgical Procedure 1. The doctor puts on a mixed reality headset and activates the navigation system. The navigation module uses fixed and movable markers to obtain real-time information about the patient's oral position and the position of the bone cutting instruments, establishing a unified 3D coordinate system.

[0044] 2. The mixed reality display module uses the 3D model and bone cutting line information from the data processing module, as well as the real-time position data from the navigation module, to display a virtual surgical field within the patient's mouth on the head-mounted display. The bone cutting indicator is superimposed at the corresponding real-world location. The doctor directly observes the bone cutting line and the patient's actual bones through the headset, and performs the bone cutting operation according to the indicator's guidance.

[0045] 3. During the bone cutting process, the navigation module continuously tracks the position and posture of the bone cutting instrument, and the mixed reality display module dynamically displays the relative position of the instrument blade and the bone cutting line (presented by the bone cutting indicator), assisting the doctor to adjust the instrument operation trajectory in real time to ensure that the bone cutting operation is strictly carried out according to the preoperative plan.

[0046] 4. The virtual operating area updates the relative position relationship between the surgical instruments and the patient's jaw in real time, making it easier for doctors to observe the current position relationship of surgical instruments in places that the doctor cannot see.

[0047] In summary, the present invention provides an oral osteotomy navigation system, method and storage medium based on mixed reality interaction. The oral osteotomy navigation system of the present invention includes a data processing module, a mixed reality display module, a navigation module and related navigation markers. The system of the present invention realizes precise navigation of oral osteotomy operations by combining mixed reality display technology with a navigation system. The system can intuitively display the osteotomy line and virtual surgical area, so that the doctor does not need to frequently switch his line of sight during the operation, and can obtain information such as the osteotomy position, depth and angle in real time and accurately, thereby improving the accuracy and efficiency of surgical operations, reducing surgical trauma, and improving surgical results.

Claims

1. An oral osteotomy navigation system based on mixed reality interaction, characterized in that: include: The input module is configured to input the patient's oral cone-beam CT data; The data processing module is configured to generate a three-dimensional model of the patient's oral cavity through a three-dimensional reconstruction algorithm using the patient's oral cone-beam CT data to obtain bone cutting line design information; A mixed reality display module is configured as a mixed reality head-mounted display device capable of displaying a bone cutting indicator and a virtual surgical field. The bone cutting indicator display displays bone cutting line information in real time within a three-dimensional model of the patient's oral cavity. The virtual surgical field display integrates the three-dimensional model of the patient's oral cavity and real-time position data during surgery. The navigation module is configured to obtain relative position data of the patient's oral cavity and the bone cutting instrument through markers, and transmit the relative position data and three-dimensional spatial deviation to the data processing module and the mixed reality display module.

2. The oral osteotomy navigation system based on mixed reality interaction according to claim 1, characterized in that: The markers include fixed markers and movable markers. The fixed markers are located in the stable area of ​​the patient's head and are used to determine the position reference of the patient's mouth and provide a unified coordinate system origin; the movable markers are located at the end of the bone cutting instrument and provide the real-time position and posture of the bone cutting instrument.

3. The oral osteotomy navigation system based on mixed reality interaction according to claim 1, characterized in that: The bone cutting line information includes the bone cutting position, shape, depth and angle.

4. The oral osteotomy navigation system based on mixed reality interaction according to claim 1, characterized in that: The calculation formula of the three-dimensional space deviation is as follows: DOI = in, Indicates the Euclidean distance deviation from the instrument tip to the bone cutting line; Indicates the angle between the instrument angle and the normal of the planned bone cutting surface; Indicates the speed of movement of the device; represents the local bone density correction factor; , , represents the weight coefficient, is the distance deviation weight coefficient; is the angle deviation weight coefficient; is the velocity-density deviation weight coefficient, , , It is 0~1.

5. An oral osteotomy navigation method based on mixed reality interaction, characterized in that: The oral osteotomy navigation method comprises the following steps: S1. Acquire the patient's oral cone-beam CT data; S2. Generate a 3D model of the patient's oral cavity using a 3D reconstruction algorithm based on the patient's oral cone-beam CT data to obtain bone cutting line design information; S3. In combination with a head-mounted display device, the bone cutting line information and real-time position data in the three-dimensional model of the patient's oral cavity are obtained in real time through the bone cutting indicator and the virtual surgical area; S4. Obtain the relative position data of the patient's oral cavity and the bone cutting instrument through markers, and implement navigation based on the relative position data and three-dimensional space deviation.

6. The method according to claim 5, characterized in that The markers include fixed markers and movable markers. The fixed markers are located in the stable area of ​​the patient's head and are used to determine the position reference of the patient's mouth and provide a unified coordinate system origin; the movable markers are located at the end of the bone cutting instrument and provide the real-time position and posture of the bone cutting instrument.

7. The method according to claim 5, characterized in that The bone cutting line information includes the bone cutting position, shape, depth and angle.

8. The method according to claim 5, characterized in that The calculation formula of the three-dimensional space deviation is as follows: DOI = in, Indicates the Euclidean distance deviation from the instrument tip to the bone cutting line; Indicates the angle between the instrument angle and the normal of the planned bone cutting surface; Indicates the speed of movement of the device; represents the local bone density correction factor; , , represents the weight coefficient, is the distance deviation weight coefficient; is the angle deviation weight coefficient; is the velocity-density deviation weight coefficient, , , It is 0~1.

9. The method according to claim 8, characterized in that described , , It is obtained through clinical data training and optimization.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the oral osteotomy navigation system based on mixed reality interaction according to any one of claims 1 to 4 when running.

Images