An oral osteotomy navigation system, method and storage medium based on mixed reality interaction

Abstract

The application provides an oral bone cutting navigation system, method and storage medium based on mixed reality interaction, and belongs to the oral medical field. The oral bone cutting navigation system comprises a data processing module, a mixed reality display module, a navigation module and related navigation markers. The system combines the mixed reality display technology with the navigation system to realize accurate navigation of the oral bone cutting operation. The system can intuitively display the bone cutting line and the virtual operation area, so that the doctor does not need to frequently switch the line of sight during the operation, and can obtain the bone cutting position, depth and angle and other information in real time and accurately, thereby improving the operation precision and efficiency, reducing the operation trauma and improving the operation effect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of oral medicine, and particularly relates to an oral bone cutting navigation system, method and storage medium based on mixed reality interaction. BACKGROUND

[0002] In the field of oral medicine, bone cutting operation is a common and key step in orthognathic surgery, correction of maxillofacial deformity and the like. The traditional surgical method mainly relies on the clinical experience of the doctor and the two-dimensional CT image or X-ray film before the operation to perform surgical planning and operation. However, this method has many limitations. During the operation, the doctor is difficult to obtain the three-dimensional anatomical structure information of the patient's oral cavity in real time and intuitively, and at the same time, the positional relationship between the surgical instrument and the target bone cutting area is also difficult to accurately grasp.

[0003] This difficulty in information acquisition greatly affects the operation accuracy, and problems such as deviation of bone cutting position, bone cutting depth or angle not meeting the preoperative planning are likely to occur. These problems not only increase the risk of surgery, such as possible damage to nerves and blood vessels, affect postoperative occlusion, and thus affect the surgical effect, but also prolong the operation time, causing greater trauma and pain to the patient.

[0004] Although some existing navigation systems can provide certain positioning information, they are mostly based on traditional display technologies, such as screen display. The doctor needs to frequently switch his line of sight between the patient's oral cavity and the screen during the operation, which is not only inconvenient, but also easy to distract the doctor's attention. In addition, the traditional navigation system has deficiencies in real-time display of the bone cutting line and intraoperative dynamic calibration capability, and it is difficult to accurately fuse the virtual bone cutting planning with the actual oral environment of the patient, thereby limiting the fine operation of complex bone cutting surgery.

[0005] Therefore, it is urgent to develop a navigation system that can more intuitively and accurately assist the doctor in performing oral bone cutting operation, and an oral bone cutting navigation system based on mixed reality interaction emerges as the times require. SUMMARY

[0006] In view of the above problems existing in the prior art, the purpose of the present application is to provide an oral bone cutting navigation system, method and storage medium based on mixed reality interaction.

[0007] The present application provides an oral bone cutting navigation system based on mixed reality interaction, comprising:

[0008] An input module configured to input the patient's oral cone beam CT data;

[0009] A data processing module configured to generate a three-dimensional model of the patient's oral cavity by a three-dimensional reconstruction algorithm based on the patient's oral cone beam CT data, and obtain bone cutting line design information;

[0010] The mixed reality display module is configured to a mixed reality head-mounted display device capable of mixed reality display of a bone cutting indicator and a virtual surgical area, the bone cutting indicator display is real-time display of bone cutting line information in a three-dimensional model of a patient's oral cavity; the virtual surgical area display is the integration of the three-dimensional model of the patient's oral cavity and the real-time position data in the surgical process;

[0011] The navigation module is configured to obtain real-time position data of the patient's oral cavity and the bone cutting instrument through the markers, and transmit the real-time position data and the three-dimensional space deviation to the data processing module and the mixed reality display module.

[0012] Further, the markers include fixed markers and active markers, the fixed markers are located in the stable area of the patient's head, used to determine the position reference of the patient's oral cavity, and provide a unified coordinate system origin; the active markers are located at the end of the bone cutting instrument, providing real-time position and attitude of the bone cutting instrument.

[0013] Further, the bone cutting line information includes bone cutting position, shape, depth and angle.

[0014] Further, the calculation formula of the three-dimensional space deviation is as follows:

[0015] DOI =

[0016] wherein,

[0017] represents the Euclidean distance deviation of the instrument cutter head to the bone cutting line;

[0018] represents the angle between the instrument angle and the normal of the planned bone cutting surface;

[0019] represents the instrument moving speed;

[0020] represents the local bone density correction coefficient;

[0021] , , represents the weight coefficient, is the distance deviation weight coefficient; is the angle deviation weight coefficient; is the speed-density deviation weight coefficient, , , is 0~1.

[0022] Further, is 0.5, is 0.25, 0.25.

[0023] Further, the three-dimensional reconstruction algorithm is a Marching Cubes algorithm.

[0024] The application also provides an oral bone cutting navigation method based on mixed reality interaction, comprising the following steps:

[0025] S1, collecting cone beam CT data of the patient's oral cavity;

[0026] S2, generating a three-dimensional model of the patient's oral cavity according to the cone beam CT data of the patient's oral cavity through a three-dimensional reconstruction algorithm, and obtaining bone cutting line design information;

[0027] S3, combining a head-mounted display device, acquiring bone cutting line information and real-time position data in the three-dimensional model of the patient's oral cavity through a bone cutting indicator and a virtual surgical area in real time;

[0028] S4, acquiring relative position data of the patient's oral cavity and bone cutting instruments through markers, and performing navigation according to the relative position data and three-dimensional space deviation.

[0029] Further, the markers include fixed markers and movable markers, the fixed markers are located in a stable area of the patient's head, used to determine the position reference of the patient's oral cavity, and provide a unified coordinate system origin; the movable markers are located at the end of the bone cutting instruments, and provide real-time position and attitude of the bone cutting instruments.

[0030] Further, the bone cutting line information includes bone cutting position, shape, depth and angle.

[0031] Further, the calculation formula of the three-dimensional space deviation is as follows:

[0032] DOI =

[0033] wherein,

[0034] represents the Euclidean distance deviation of the instrument cutter head to the bone cutting line;

[0035] represents the angle between the instrument angle and the normal of the planned bone cutting surface;

[0036] represents the instrument moving speed;

[0037] represents the local bone density correction coefficient;

[0038] , , represents the weight coefficient, is a distance deviation weight coefficient; is an angle deviation weight coefficient; is a speed-density deviation weight coefficient, , , is 0~1.

[0039] Further, the system , , is obtained by clinical data training optimization.

[0040] The application further provides a storage medium, which stores a computer program, wherein the computer program is arranged to execute the oral bone cutting navigation system based on mixed reality interaction when running.

[0041] The application has the following beneficial effects:

[0042] The application provides an oral bone cutting navigation system based on mixed reality interaction. By combining mixed reality display technology and a navigation system, precise navigation of oral bone cutting operation is realized. The system can intuitively display a bone cutting line and a virtual surgical area, so that a doctor does not need to frequently switch visual field during surgery, and real-time and accurate bone cutting position, depth and angle information is obtained, the precision and efficiency of surgical operation are improved, surgical trauma is reduced, and surgical effect is improved.

[0043] Obviously, according to the above content of the application, according to the ordinary technical knowledge and common means in the art, other various forms of modifications, replacements or changes can be made without departing from the above technical idea of the application.

[0044] The above content of the application is further described in detail through the specific implementation mode in the form of examples. However, this should not be understood as the range of the above subject matter of the application being limited to the following examples. Any technology realized based on the above content of the application belongs to the range of the application. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a structure schematic diagram of the oral bone cutting navigation system of the application.

[0046] Figure 2 is a surgical design schematic diagram.

[0047] Figure 3 is a bone cutting indicator display diagram.

[0048] Figure 4 is a virtual surgical area display diagram. DETAILED DESCRIPTION

[0049] For better understanding and implementation, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0050] In the description of the present application, it should be noted that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "far", "near" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0051] It should be particularly noted that the algorithms of data acquisition, transmission, storage and processing steps not specifically described in the embodiments, and the hardware structure, circuit connection and the like not specifically described can be realized by the existing technology disclosed.

[0052] Embodiment 1, oral bone cutting navigation system based on mixed reality interaction

[0053] I. Overview of the structure of the oral bone cutting navigation system

[0054] The oral bone cutting navigation system of the present application comprises a data processing module, a mixed reality display module, a navigation module, and related navigation markers and active markers. Figure 1 The structure diagram of the oral bone cutting navigation system of the present application is shown in the figure, and each module is arranged according to the process logic and can be divided into the following parts:

[0055] The input module is configured to input the cone beam CT (Cone Beam CT, CBCT) data of the patient as the basis for three-dimensional reconstruction of the surgical area;

[0056] The data processing module is configured to generate a three-dimensional model of the patient's oral cavity by three-dimensional reconstruction algorithm (common Marching Cubes algorithm) from the patient's oral CBCT data, and obtain the bone cutting line design information;

[0057] The mixed reality display module is configured as a mixed reality head-mounted display device that can display bone cutting indicators and virtual surgical area;

[0058] The navigation module is configured to acquire real-time position data of the patient's oral cavity through the navigation marker, and acquire real-time position data of the osteotomy instrument through the activity marker, and transmit the real-time position data of the oral cavity and the osteotomy instrument and the three-dimensional space deviation to the data processing module and the mixed reality display module.

[0059] The core part is outlined as follows:

[0060] (I) Data processing module

[0061] This module is used to acquire CBCT data of the patient's oral cavity, reconstruct the surface of the jaw bone using an improved deep learning algorithm (extracting hierarchical features of the jaw CBCT image through a cascading three-dimensional convolutional neural network, and introducing an attention gate mechanism in the decoding stage to strengthen key anatomical structures), and combining with a super-resolution network to improve the resolution of small structures (such as the alveolar nerve canal), to generate a three-dimensional model of the patient's oral cavity. Based on the generated three-dimensional model, the doctor can design the osteotomy line before surgery through this module, and plan the best osteotomy position, shape, depth and angle on the three-dimensional model according to the specific condition of the patient and the surgical requirements. The data processing module stores and transmits the osteotomy line design information (including three-dimensional coordinates, geometric parameters, etc.) for subsequent module use. In addition, the data processing module also uses a deep learning-based 3D model segmentation method to accurately segment the mandibular nerve canal, and in the navigation process, it predicts the spatial relationship between the three-dimensional position of the mandibular nerve canal and the osteotomy path in real time, and gives a prompt when necessary.

[0062] The surgical design schematic diagram is shown in Figure 2 .

[0063] (II) Mixed reality display module

[0064] This module includes a mixed reality head-mounted display device, which can display virtual information and real surgical scenes, and the specific functions are as follows:

[0065] 1. Osteotomy indicator display:

[0066] According to the osteotomy line designed by the data processing module, the corresponding osteotomy indicator is displayed in real time in the mixed reality display device. The osteotomy indicator is superimposed on the actual position of the surgical instrument in a specific color and shape (such as Figure 3 : divided into two parts, angle indicator and depth indicator), the doctor can directly see the specific position and trend of the osteotomy line in the patient's oral cavity through the head-mounted display device, and realize intuitive guidance of the osteotomy operation.

[0067] As shown in Figure 3 , the white progress bar on the left represents the depth of the cut, indicating the depth of the bone knife head cut, and the ring composed of white dotted lines on the right of the progress bar is the angle indicator, indicating the angle of the knife head and the cutting surface.

[0068] 2. Virtual surgical zone display:

[0069] By integrating the three-dimensional model of the patient's oral cavity and real-time positional data during the operation, a virtual surgical zone is displayed in the mixed reality display device. The virtual surgical zone can reflect the progress of the bone cutting operation in real time and accurately match the real surgical zone, enabling the surgeon to have a comprehensive understanding of the anatomical structure changes in the surgical area.

[0070] As Figure 4 The upper left corner is the virtual surgical zone, where the relative positional relationship between the surgical instrument and the patient's jawbone can be more intuitively observed, as well as the position of the bone cutting line; at the same time, the data of the indicator can also be observed.

[0071] (Three) Navigation module

[0072] This module combines the navigation markers installed on the surgical instruments and around the patient's oral cavity, and through the cooperative working mode of multiple high-frame-rate optical cameras (200 Hz+) and the visual sensor built-in MR head-mounted display, a multi-view fusion tracking system is constructed to obtain the positional information of the surgical instruments and the patient's oral cavity in real time. The navigation module transmits these positional information (including three-dimensional coordinates, attitude parameters) to the data processing module and the mixed reality display module, so that the mixed reality display module can dynamically adjust the displayed virtual information according to the actual position, ensuring that the bone cutting indicator and the virtual surgical zone are accurately matched with the actual position of the patient's oral cavity. For example, when the surgical instrument moves, the navigation module tracks its position in real time and displays the relative positional relationship between the instrument knife head and the bone cutting line (such as distance deviation, angle deviation) in the mixed reality display device, assisting the surgeon in making precise adjustments. In particular, in view of the accuracy requirements of the navigation system, this module integrates the three-dimensional spatial deviation (distance, angle) of the surgical instrument and the planned bone cutting line, the weighted parameters of the instrument movement speed and the bone density difference. Real-time quantification of the deviation degree of the bone cutting operation (DOI), and intuitive prompt the surgeon to adjust through the mixed reality display module.

[0073] DOI =

[0074] : Euclidean distance deviation of the instrument knife head to the bone cutting line;

[0075] : The normal angle between the instrument angle and the planned bone cutting surface;

[0076] : Instrument movement speed (dynamic correction of error sensitivity during fast operation);

[0077] : Local bone density correction coefficient (mapped by preoperative CT value, high-density area needs more force feedback compensation);

[0078] , , is a weight coefficient, which is optimized by training with clinical data (experimental test comparison and experience parameter adjustment), and in this embodiment, , , Specifically set to 0.5, 0.25, 0.25.

[0079] (Four) Navigation markers

[0080] Including fixed markers installed on the patient's head and active markers installed on surgical instruments:

[0081] 1. Fixed markers:

[0082] Used to determine the position reference of the patient's oral cavity, usually pasted or clamped in the stable area of the patient's head (such as the zygomatic bone, maxilla), providing a unified coordinate system origin.

[0083] 2. Active markers:

[0084] Integrated into the end of bone cutting instruments (such as bone drills, bone knives), used to track the real-time position and attitude of surgical instruments, and its surface has specific identification features (positioning two-dimensional code) for easy identification and positioning by the navigation module. The active marker also integrates a high-precision inertial measurement unit (IMU), which fuses optical positioning data and IMU attitude information in real time through an adaptive Kalman filter algorithm, effectively compensating for the drift of the position and attitude caused by the patient's slight movement.

[0085] II. Specific implementation method

[0086] (I) Preoperative preparation

[0087] 1. Perform CT scanning on the patient's oral cavity to obtain high-resolution CT data, and input the data into the data processing module. The data processing module generates a three-dimensional model of the patient's oral cavity (including bone, soft tissue structure) through a three-dimensional reconstruction algorithm.

[0088] 2. The doctor designs the bone cutting line in the data processing module based on the three-dimensional model, marks the boundary, depth and angle of the target bone cutting area, and generates a personalized bone cutting planning scheme. After the design is completed, the bone cutting line information is stored as an interactive three-dimensional data model.

[0089] 3. Install fixed navigation markers on the fixed area of the patient's head to ensure that the marker position is stable and unobstructed; at the same time, install active navigation markers on the bone cutting instruments to calibrate the conversion relationship between the instrument coordinate system and the navigation system coordinate system.

[0090] (II) Surgical procedure

[0091] 1. The doctor wears a mixed reality head-mounted display device and activates the navigation system. The navigation module acquires the positional reference of the patient's oral cavity and the positional information of the osteotomy instruments in real time through fixed and moving markers, establishing a unified three-dimensional spatial coordinate system.

[0092] 2. The mixed reality display module uses the 3D model and osteotomy line information from the data processing module, along with real-time position data from the navigation module, to display a virtual surgical area of ​​the patient's mouth on the head-mounted display device. An osteotomy indicator is then overlaid on the corresponding real-world location. The doctor directly observes the fusion of the osteotomy line and the patient's actual bone through the head-mounted device and performs the osteotomy operation guided by the indicator.

[0093] 3. During the bone cutting process, the navigation module continuously tracks the position and posture of the bone cutting instrument, and the mixed reality display module dynamically displays the relative position of the instrument tip and the bone cutting line (presented through the bone cutting indicator), assisting the doctor to adjust the instrument operation trajectory in real time and ensuring that the bone cutting operation is strictly performed in accordance with the preoperative plan.

[0094] 4. The virtual surgical area updates the relative positional relationship between surgical instruments and the patient's jawbone in real time, making it easier for doctors to observe the positional relationship of the surgical instruments in places that are not visible to the doctor.

[0095] In summary, this invention provides an oral osteotomy navigation system, method, and storage medium based on mixed reality interaction. The oral osteotomy navigation system of this invention includes a data processing module, a mixed reality display module, a navigation module, and related navigation markers. By combining mixed reality display technology with a navigation system, this invention achieves precise navigation for oral osteotomy operations. The system can intuitively display the osteotomy line and virtual surgical area, allowing surgeons to obtain real-time and accurate information such as the osteotomy position, depth, and angle without frequently switching their line of sight during surgery. This improves the precision and efficiency of surgical operations, reduces surgical trauma, and enhances surgical outcomes.

Claims

1. An oral osteotomy navigation system based on mixed reality interaction, characterized in that, include: The input module is configured to input the patient's oral cone-beam CT data; The data processing module is configured to generate a three-dimensional model of the patient's oral cavity from the patient's cone-beam CT data using a three-dimensional reconstruction algorithm, thereby obtaining osteotomy line design information; The mixed reality display module is configured as a mixed reality head-mounted display device capable of displaying a bone-cutting indicator and a virtual surgical area. The bone-cutting indicator displays bone-cutting line information in real time within a three-dimensional model of the patient's oral cavity. The virtual surgical area display integrates a three-dimensional model of the patient's oral cavity with real-time position data during the surgical process. The navigation module is configured to acquire relative position data of the patient's oral cavity and bone-cutting instruments through markers, and transmit the relative position data and three-dimensional spatial deviation to the data processing module and the mixed reality display module; The markers include fixed markers and movable markers. The fixed markers are located in a stable area of ​​the patient's head and are used to determine the positional reference of the patient's oral cavity, providing a unified coordinate system origin. The movable markers are located at the end of the osteotomy instrument and provide the real-time position and orientation of the osteotomy instrument. The osteotomy line information includes the osteotomy location, shape, depth, and angle; The formula for calculating the three-dimensional spatial deviation is as follows: DOI = in, This indicates the Euclidean distance deviation between the tip of the osteotomy instrument and the osteotomy line; This indicates the angle between the osteotomy instrument angle and the normal of the planned osteotomy surface; Indicates the speed at which the osteotomy instrument moves; This represents the local bone mineral density correction factor; , , Indicates the weighting coefficient. This is the distance deviation weighting coefficient; This is the angle deviation weighting coefficient; The velocity-density deviation weighting coefficient, , , It ranges from 0 to 1.

2. A method for oral osteotomy navigation based on mixed reality interaction, characterized in that, The oral osteotomy navigation method includes the following steps: S1. Acquire cone-beam CT data of the patient's oral cavity; S2. Based on the patient's oral cone-beam CT data, a three-dimensional model of the patient's oral cavity is generated using a three-dimensional reconstruction algorithm to obtain osteotomy line design information; S3. Combined with a head-mounted display device, the osteotomy line information and real-time position data within the three-dimensional model of the patient's oral cavity are obtained in real time through the osteotomy indicator and virtual surgical area; S4. Obtain the relative position data of the patient's oral cavity and bone-cutting instruments through markers, and implement navigation based on the relative position data and three-dimensional spatial deviation; The markers include fixed markers and movable markers. The fixed markers are located in a stable area of ​​the patient's head and are used to determine the positional reference of the patient's oral cavity, providing a unified coordinate system origin. The movable markers are located at the end of the osteotomy instrument and provide the real-time position and orientation of the osteotomy instrument. The osteotomy line information includes the osteotomy location, shape, depth, and angle; The formula for calculating the three-dimensional spatial deviation is as follows: DOI = in, This indicates the Euclidean distance deviation between the tip of the osteotomy instrument and the osteotomy line; This indicates the angle between the osteotomy instrument angle and the normal of the planned osteotomy surface; Indicates the speed at which the osteotomy instrument moves; This represents the local bone mineral density correction factor; , , Indicates the weighting coefficient. This is the distance deviation weighting coefficient; This is the angle deviation weighting coefficient; The velocity-density deviation weighting coefficient, , , It ranges from 0 to 1.

3. The method according to claim 2, characterized in that, The , , It was obtained through training and optimization using clinical data.

4. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the oral osteotomy navigation system based on mixed reality interaction as described in claim 1 at runtime.

Images