Virtual reality and reality image fusion navigation method and system for temporal bone surgery

Abstract

The invention relates to a virtual reality and reality image fusion navigation method and system for temporal bone surgery, and the method comprises the steps: obtaining medical image data of a temporal bone region, and carrying out the three-dimensional reconstruction, thereby obtaining a temporal bone dynamic virtual model; optical tracking data and instrument space pose data are obtained, dynamic registration mapping is conducted on the temporal bone dynamic virtual model, and a real-time space fusion relation is obtained; performing navigation view construction on the instrument space pose data and the real-time space fusion relationship to obtain an augmented reality navigation view; performing structural distance analysis on the augmented reality navigation view to obtain dynamic early warning parameters; and performing display regulation and control according to the augmented reality navigation view and the dynamic early warning parameters to obtain a real-time navigation strategy. The problem of traditional static image registration lag can be effectively solved, and it is ensured that the virtual model and the actual anatomical structure are always kept in high-precision matching.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of air source heat pumps, in particular to a virtual reality and real image fusion navigation method and system for temporal bone surgery. BACKGROUND

[0002] Virtual reality (VR) and augmented reality (AR) navigation systems are gradually becoming important means to improve the safety and accuracy of temporal bone surgery. Existing surgical navigation technologies usually rely on preoperative static image registration, but dynamic changes caused by tissue displacement, instrument operation and other factors during surgery cannot be real-time feedback, which limits the navigation accuracy. In addition, traditional methods lack real-time distance warning function for surgical instruments and anatomical structures, which increases the risk of intraoperative injury. SUMMARY

[0003] The main purpose of the application is to provide a virtual reality and real image fusion navigation method and system for temporal bone surgery, which can effectively solve the problem of lag in traditional static image registration and ensure that the virtual model and the actual anatomical structure always maintain high-precision matching.

[0004] To achieve the above purpose, the application provides a virtual reality and real image fusion navigation method for temporal bone surgery, comprising: Obtaining medical image data of the temporal bone region for three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone; Obtaining optical tracking data and instrument spatial pose data, dynamically registering and mapping the dynamic virtual model of the temporal bone to obtain a real-time spatial fusion relationship; Constructing a navigation view for the instrument spatial pose data and the real-time spatial fusion relationship to obtain an augmented reality navigation view; Analyzing the structure distance of the augmented reality navigation view to obtain a dynamic warning parameter; Displaying and controlling according to the augmented reality navigation view and the dynamic warning parameter to obtain a real-time navigation strategy.

[0005] Further, the obtaining of the medical image data of the temporal bone region for three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone comprises: Obtaining CT image data and MRI image data of the temporal bone region for multi-modal registration and alignment to obtain the medical image data; Segmenting the medical image data to obtain an initial structure mask set; Reconstructing a three-dimensional surface mesh for the initial structure mask set to obtain a target structure mesh model; Assigning spatial attributes and binding dynamic response parameters to the target structure mesh model to obtain the dynamic virtual model of the temporal bone.

[0006]

[0006] Further, the optical tracking data and the instrument spatial pose data are acquired, the dynamic virtual model of the temporal bone is dynamically registered and mapped to obtain a real-time spatial fusion relationship, comprising: Acquiring optical data collected by a preset head marker array, performing optical tracking identification to obtain the optical tracking data; Performing instrument degree of freedom pose analysis through a preset surgical instrument tracking adapter to obtain the instrument spatial pose data; Performing coordinate system rigid alignment on the optical tracking data and the instrument spatial pose data to obtain a registration reference data set; According to the registration reference data set and the dynamic virtual model of the temporal bone, surface iterative registration is performed to obtain the real-time spatial fusion relationship.

[0007] Further, the optical tracking data and the instrument spatial pose data are acquired, the dynamic virtual model of the temporal bone is dynamically registered and mapped to obtain a real-time spatial fusion relationship, comprising: According to the registration reference data set, target point positioning is performed to obtain a target feature point cloud set; Performing differential manifold matching on the target feature point cloud set and the dynamic virtual model of the temporal bone to obtain an initial registration matrix; Performing vibration main frequency separation on the instrument spatial pose data to obtain main vibration mode component parameters; According to the main vibration mode component parameters, spatial registration is performed on the initial registration matrix to generate an anti-vibration registration matrix; Performing tympanic cavity connectivity verification on the anti-vibration registration matrix to obtain the real-time spatial fusion relationship.

[0008] Further, the instrument spatial pose data and the real-time spatial fusion relationship are navigated to construct a navigation view to obtain an augmented reality navigation view, comprising: Performing surgical instrument position identification based on the instrument spatial pose data to obtain surgical instrument position information; Performing spatial coordinate transformation on the surgical instrument position information and the real-time spatial fusion relationship to obtain virtual spatial instrument mapping data; Performing structure perception rendering on the dynamic virtual model of the temporal bone to obtain a multi-layer transparency volume rendering view; Performing depth fusion on the virtual spatial instrument mapping data and the multi-layer transparency volume rendering view to obtain a primary view framework; Performing physical occlusion relationship reconstruction and optical distortion correction on the primary view framework to obtain the augmented reality navigation view.

[0009] Further, the augmented reality navigation view is subjected to structure distance analysis to obtain a dynamic early warning parameter, comprising: spatial trajectory extraction is performed on the augmented reality navigation view to obtain instrument motion vector information; neighbor structure search is performed according to the instrument motion vector information and the temporal bone dynamic virtual model to obtain a target spatial distribution set; dynamic distance field calculation is performed on the target spatial distribution set to obtain a multi-structure real-time distance mapping table; trajectory collision prediction is performed based on the target spatial distribution set and the multi-structure real-time distance mapping table to obtain collision distribution prediction data; spatial correlation integration is performed on the collision distribution prediction data and the multi-structure real-time distance mapping table to obtain dynamic early warning parameters.

[0010] Further, display regulation is performed according to the augmented reality navigation view and the dynamic early warning parameters to obtain a real-time navigation strategy, including: temporal bone sub-region spatial weight analysis is performed on the dynamic early warning parameters to obtain a target area regulation factor set; structure visual correlation optimization is performed based on the target area regulation factor set and the augmented reality navigation view to obtain a temporal bone specialized navigation view; drilling trajectory collision prediction is performed according to the temporal bone specialized navigation view to obtain a safe trajectory obstacle avoidance strategy; operation visual field mapping is performed on the temporal bone specialized navigation view according to the safe trajectory obstacle avoidance strategy to obtain an operation guide view; optical rendering fusion adaptation is performed on the operation guide view to obtain a real-time navigation strategy.

[0011] Further, drilling trajectory collision prediction is performed according to the temporal bone specialized navigation view to obtain a safe trajectory obstacle avoidance strategy, including: stage constraint construction is performed on the temporal bone specialized navigation view to obtain a three-order operation channel; collision detection is performed on the temporal bone dynamic virtual model based on the three-order operation channel to obtain collision interference volume information; plane obstacle avoidance planning is performed according to the collision interference volume information to obtain a collision-free trajectory option set; resolution path mapping and visual conversion are performed on the collision-free trajectory option set to obtain a visual cognitive strategy; virtual drilling verification is performed on the visual cognitive strategy to obtain a safe trajectory obstacle avoidance strategy. The application also provides a virtual reality and real image fusion navigation system for temporal bone surgery, which is applied to the virtual reality and real image fusion navigation method for temporal bone surgery described in any one of the above. The acquisition module is used for acquiring medical image data of a temporal bone region for three-dimensional reconstruction to obtain a temporal bone dynamic virtual model; The analysis module is used for acquiring optical tracking data and instrument spatial pose data, performing dynamic registration mapping on the temporal bone dynamic virtual model to obtain a real-time spatial fusion relationship; The correlation module is used for performing navigation view construction on the instrument spatial pose data and the real-time spatial fusion relationship to obtain an augmented reality navigation view; The processing module is used for performing structural distance analysis on the augmented reality navigation view to obtain a dynamic early warning parameter; The control module is used for performing display regulation and control according to the augmented reality navigation view and the dynamic early warning parameter to obtain a real-time navigation strategy.

[0012] The virtual reality and real image fusion navigation method and system for temporal bone surgery provided by the application have the following beneficial effects: By acquiring optical tracking data and instrument spatial pose data in real time and performing dynamic registration mapping in combination with a temporal bone dynamic virtual model, the problem of lag in traditional static image registration can be effectively solved, high-precision matching between the virtual model and the actual anatomical structure can be ensured, and thus the accuracy of surgical navigation can be improved. The augmented reality navigation view is constructed based on the instrument spatial pose data and the real-time spatial fusion relationship, the relative positions of the surgical instrument and the key anatomical structure can be intuitively observed, the real-time performance and the interactivity of intraoperative operation are enhanced, and the fast-changing operating field environment is adapted. By performing structural distance analysis on the augmented reality navigation view, the distance between the instrument and the key tissue is calculated in real time, and a dynamic early warning parameter is generated, potential risks can be timely prompted, the probability of accidentally injuring important nerves and blood vessels can be reduced, and the safety of surgery can be improved. According to the augmented reality navigation view and the dynamic early warning parameter, real-time display regulation and control can be performed to provide an optimal navigation strategy, assist precise operation, improve surgical efficiency, and reduce operation errors caused by visual errors or delayed feedback. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 It is a virtual reality and real image fusion navigation method flowchart for temporal bone surgery provided by the application. Figure 2 It is a virtual reality and real image fusion navigation system structure diagram for temporal bone surgery provided by the application.

[0014] The implementation of the application, the functional characteristics and the advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0016] The present application is further described below in combination with the drawings and specific embodiments.

[0017] Referring to Figure 1 The present application provides a virtual reality and real image fusion navigation method for temporal bone surgery, comprising: Step S1: Obtain medical image data of the temporal bone region for three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone; Step S2: Obtain optical tracking data and instrument spatial pose data, and perform dynamic registration mapping on the dynamic virtual model of the temporal bone to obtain a real-time spatial fusion relationship; Step S3: Construct a navigation view for the instrument spatial pose data and the real-time spatial fusion relationship to obtain an augmented reality navigation view; Step S4: Analyze the structure distance of the augmented reality navigation view to obtain a dynamic warning parameter; Step S5: Display and control according to the augmented reality navigation view and the dynamic warning parameter to obtain a real-time navigation strategy.

[0018] Based on the above steps, the detailed step process is described as follows: Step S1: The CT image data and MRI image data of the patient's temporal bone region are collected by a medical imaging device. CT scanning focuses on capturing bone structures such as ossicles and semicircular canals, with a layer thickness controlled within 0.5 millimeters to ensure accuracy. MRI is focused on imaging soft tissue structures such as facial nerves and blood vessels. The collected multi-modal image data is processed by registration and fusion to accurately align the bone information of CT with the soft tissue contrast of MRI in space. The fused data set is input into a pre-set deep learning segmentation model to automatically identify and extract 7 key anatomical structures within the temporal bone, including facial nerve tube, cochlea and vestibule, etc. Then, morphological processing is used to optimize the continuity of the segmentation results, especially in fine areas such as cochlear base transition. After completion of the segmentation, a three-dimensional mesh model is generated using surface reconstruction technology, and smoothing optimization is performed at the facial nerve transition to ensure natural curve transition.

[0019] Dynamic response parameters are embedded in the model, including the elastic modulus of bone tissue and the strain threshold of nerve structure, so that it can simulate the deformation effect in real time when interacting with surgical instruments. The model supports intraoperative updating through cone beam CT to ensure synchronization with the actual anatomical state of the patient.

[0020] Step S2: By starting the optical tracking system, the infrared camera captures passive reflective marker points on the patient head holder at a rate of 120 frames per second, calculates their three-dimensional coordinates to establish a reference coordinate system. The tracking adapter installed on the bone drill and other instruments provides real-time feedback on the position and attitude of the instrument, and through the pre-calibrated geometric offset matrix, the precise coordinates of the instrument tip are calculated. Then the tracking coordinate system and the instrument coordinate system are spatially aligned and unified into the same world coordinate system.

[0021] In the exposed area of the temporal bone, the calibration probe is used to collect spatial point cloud data at anatomical landmark points such as the entrance of the antrum and the pyramid, which are the key inputs for registration. Through the point cloud registration algorithm, the physical feature points collected during the operation are matched with the virtual model, while compensating for the pose drift caused by the high-frequency vibration of the bone drill. After registration, verification points are selected in the promontory area for error detection, and the topological connectivity of the scala tympani to the scala vestibuli channel is verified. When the spatial error is less than 0.3 mm and the channel is complete, the final registration result is output.

[0022] Step S3: Apply the spatial relationship obtained by registration to the visualization system. Convert the real-time position data of the instrument to the virtual coordinate system, eliminate operation jitter through motion smoothing processing, and generate stable display of the spatial pose of the instrument. Then render the virtual model with structure perception, set the bone structure to 20% transparency to maintain the perspective effect, and use 40% and 60% transparency for nerves and blood vessels respectively to enhance the distinction, and enable curvature perception rendering in the cochlea spiral area to highlight the complex geometric features. Through the depth buffer algorithm, analyze the spatial occlusion relationship between the instrument and the anatomical structure, and automatically hide the occluded part of the virtual model when the instrument is behind the facial nerve.

[0023] Adapt the rendering result to different display devices: superimpose a 50% transparent virtual outline through an optical coupler in the surgical microscope, and generate a binocular stereoscopic view in the VR headset. Finally, correct the inherent distortion of the optical device in real time, and dynamically adjust the projection parameters according to the interpupillary distance to ensure that the visual information and the actual surgical field are accurately matched in space.

[0024] Step S4: Quantitative analysis of spatial relationship based on real-time rendering navigation view. Extract the motion trajectory vector of the surgical instrument from the view data stream, including the displacement rate and direction change of the instrument tip. Synchronously call the spatial coordinates of the anatomical structures in the dynamic virtual model of the temporal bone, and focus on searching for adjacent structures in high-risk areas such as the tympanic segment of the facial nerve canal, the outer semicircular canal convex, and the vestibular window. Through the construction of a dynamic distance field calculation engine, the Euclidean distance from the instrument tip to each dangerous structure is measured in real time, and the spatial marking mechanism is triggered when the instrument enters the preset warning range.

[0025] According to the instrument motion trajectory, the path envelope in the next 60 milliseconds is predicted, the potential collision risk is predicted in combination with the vibration characteristics of the bone drill, and a multi-dimensional early warning data set including distance value, collision probability and time threshold is generated. Finally, the distance information is bound with the spatial coordinates, and after verifying the channel connectivity in the promontory area, the structured early warning parameters are output, providing a quantitative basis for display control.

[0026] Step S5: The distance and risk level data in the dynamic early warning parameters are analyzed, and the visual control factors are generated according to the sub-regional classification of the facial nerve area (threshold value 0.5mm), bone labyrinth area (threshold value 1.0mm) and other temporal bone sub-regions. According to the factor weight, the transparency distribution of the navigation view is dynamically adjusted: the transparency of the bony area is reduced to 15% to highlight the instrument position, and the surface of the facial nerve canal is superimposed with a red pulsating boundary (frequency 2Hz) to strengthen the warning effect.

[0027] Synchronous path planning decision making, when the early warning parameters indicate high risk, automatically generate an obstacle avoidance path (offset angle 8°-15°) around the vertical segment of the facial nerve and project it to the surgical field with a green arrow. On the microscope eyepiece or head-mounted display end, the spatial distortion is optimized through the distortion correction matrix, and the binocular parallax is calibrated according to the interpupillary distance. Finally, the visual guidance signal and the path parameters are integrated, a safe drilling speed scale is superimposed at the entrance of the antrum, and a comprehensive strategy output consistent with the clinical operation intuition is formed.

[0028] The virtual reality and real image fusion navigation method for temporal bone surgery provided by the application can effectively solve the problem of lagging of traditional static image registration by real-time acquisition of optical tracking data and instrument spatial pose data and dynamic registration mapping combined with a dynamic virtual model of the temporal bone, ensuring that the virtual model and the actual anatomical structure always maintain high-precision matching, thereby improving the accuracy of surgical navigation. Based on the spatial fusion relationship between the instrument spatial pose data and the real-time space, an augmented reality navigation view is constructed, which can intuitively observe the relative position of the surgical instrument and the key anatomical structure, enhance the real-time and interactivity of intraoperative operation, and adapt to the rapidly changing surgical field environment. By analyzing the structure distance of the augmented reality navigation view, the distance between the instrument and the key tissue is calculated in real time, and dynamic early warning parameters are generated, which can timely prompt potential risks, reduce the probability of accidentally injuring important nerves and blood vessels, and improve surgical safety. According to the real-time display and control of the augmented reality navigation view and the dynamic early warning parameters, the optimal navigation strategy can be provided to assist precise operation, improve surgical efficiency, and reduce operation errors caused by visual errors or delayed feedback.

[0029] In one embodiment, medical image data of the temporal bone region is acquired for three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone, including: The temporal bone region image acquisition adopts a standardized scanning process performed by medical imaging equipment. CT image data is obtained through a spiral scanner, with a slice thickness controlled not to exceed a preset threshold voltage value in line with clinical standards. MRI image data uses specific sequence parameters, with an inter-slice interval set to meet spatial resolution requirements. The two types of data are input into a multi-modal registration system to perform coordinate alignment operations: a similarity measurement method based on tissue gray scale distribution characteristics drives optimization of spatial transformation parameters, achieving coordinate system unity of bone structures and soft tissue contrast information.

[0030] The registration process uses a rigid transformation model to adjust the displacement and rotational deviation between the scanning coordinate systems, ultimately generating medical image data. This data set combines the bone window visualization capabilities of CT images with the neurovascular resolution characteristics of MRI images, covering the complete anatomical range from the petrous part of the temporal bone to the mastoid region. The registration accuracy is verified to meet sub-millimeter error standards, ensuring the geometric structure fidelity and spatial consistency of subsequent processing. Image data is transmitted using a verification mechanism to ensure data integrity.

[0031] Medical image data is input into a three-dimensional segmentation processing module, which is divided into multiple target structures based on anatomical features. The segmentation process uses a neural network architecture to extract pixel-level features, capturing global context through down-sampling in the encoding stage and restoring spatial details through up-sampling in the decoding stage. The segmentation results are binarized to generate a mask sequence corresponding to each type of structure. Post-processing stage implements morphological operations to correct segmentation defects: hollow filling algorithm is performed on thin-walled areas to ensure structure continuity; discrete noise areas are removed based on graph theory connectivity analysis. Key transition areas such as tube cavity junctions implement an artificial interactive review mechanism to ensure the spatial topology of tubular structures is complete. The final output of the initial structure mask set preserves the substructure separation information of the bone gap and provides a discrete voxel representation basis for subsequent surface reconstruction. The mask data storage format supports three-dimensional spatial coordinate mapping queries.

[0032] The initial structure mask set is input into the meshing processing unit. Surface reconstruction uses an isosurface extraction algorithm to convert binary voxels into triangular mesh surfaces, with the isosurface threshold set to balance structure integrity requirements and detail preservation requirements. The mesh generation process enables adaptive subdivision strategies for areas with sudden curvature changes to control mesh density and geometric deviation balance. The original mesh is iteratively optimized using a smoothing algorithm to constrain vertex displacement amplitude and maintain the original structure morphology. Parameterized surface reconstruction technology is applied in complex surface intersection areas to improve surface smoothness. The output target structure mesh model passes the tightness detection and self-intersection verification, with the number of triangular facets within a controllable range. The mesh topology in narrow channel areas meets the anatomical structure characteristics, and the turning angles and curvature distribution meet the clinical recognition standards. The mesh model file format includes vertex coordinates and facet index data structures.

[0033] The target structure mesh model loading attribute mapping function. The spatial attribute assignment process parses the original image metadata, converts the physical resolution parameters into vertex spatial coordinate attributes, and assigns the preset transparency level and color coding scheme according to the tissue type label. The dynamic response parameter binding configures the mechanical property simulation component: sets the vibration transfer function for bone structure nodes and defines the strain threshold response mechanism; establishes a kinematic constraint model in the movable joint area; and deploys deformation monitoring rules in the thin-walled area. The model integration real-time update interface supports intraoperative image data input, and the grid vertex position is updated through the displacement field calculation engine. The constructed temporal bone dynamic virtual model integrates spatial attribute data and dynamic response logic, supports real-time interactive simulation and spatial state feedback in the surgical navigation process. The model output format is compatible with mainstream three-dimensional rendering pipeline architecture.

[0034] The embodiment ensures the spatial consistency of bone structure image data and soft tissue image data through multi-modal image registration alignment operation, forms fused medical image data with structure recognition and geometric precision, and effectively supports the anatomical fidelity requirements of subsequent three-dimensional reconstruction. The tissue structure segmentation process implements hierarchical processing based on anatomical features, preserves the spatial topological properties of the thin-walled area through a neural network architecture, and the initial structure mask set fully reflects the connectivity state of the key channels of the temporal bone, significantly improving the structural integrity of subsequent mesh modeling. The three-dimensional surface mesh reconstruction adopts isosurface extraction combined with parameterized surface optimization, controls geometric deviation while maintaining the morphological characteristics of complex surfaces, and the generated target structure mesh model meets the recognition standards of narrow gaps in clinical practice.

[0035] In one embodiment, optical tracking data and instrument spatial pose data are acquired, and a temporal bone dynamic virtual model is dynamically registered and mapped to obtain real-time spatial fusion relationships, including: The infrared optical tracking system is located in the surgical area and is erected with three or more high-resolution cameras, and the angle configuration covers the marker array operation space of the patient's head. The array is composed of passive reflective balls, the reflective balls match the clinical navigation accuracy standard in diameter, and are installed on the head frame in a non-coplanar geometric configuration. The camera group synchronously collects the reflected light spot images, and the frame rate is set to 120Hz to meet the real-time requirements.

[0036] The image processing module performs spot center coordinate extraction and background interference filtering, and calculates the three-dimensional spatial coordinates of each marker point using the spatial triangulation principle. The original coordinate data is processed by filtering and noise reduction to remove motion jitter and environmental vibration noise. The coordinate sequence is marked with a timestamp and encapsulated as a data stream, and the transmission integrity is guaranteed through a digital check code mechanism. The output optical tracking data accuracy is verified to be less than 0.3mm, meeting the temporal bone surgery navigation standard. The coordinate storage format includes spatial displacement vectors and rotation quaternion parameters.

[0037] A surgical instrument handle integrated active tracking adapter is provided with an array of infrared light emitting marker points embedded on the adapter surface. The spatial arrangement of the marker points is established through a pre-calibration process to establish a geometric mapping relationship with the functional end point of the instrument, with a calibration error controlled within 0.1 millimeters. A pose solving engine receives real-time coordinates of the adapter marker points and solves six degrees of freedom pose parameters based on an orthogonal projection model.

[0038] The rotational component is expressed in quaternion form to avoid singularity problems, and the translational component outputs the absolute coordinates of the instrument tip in the tracking coordinate system. The instrument spatial pose data incorporates a kinematic compensation mechanism: for high-speed drill with built-in vibration spectrum analysis module, automatically corrects the axial deviation caused by 12-18 kHz vibration. The data output frequency is strictly synchronized with the optical tracking frame rate, and the time delay verification is less than 2 milliseconds. The pose data is transmitted to the navigation system interface through an encryption protocol, and the verification mechanism prevents data tampering. The format specification includes pose matrix and instrument type identification code.

[0039] The optical tracking data and surgical instrument spatial pose data input coordinate system alignment module. The processing flow defines the head reference frame coordinate system as the spatial registration reference system, and solves the rigid transformation matrix from the tracking world coordinate system to the reference frame coordinate system through the singular value decomposition algorithm. The instrument spatial pose data performs coordinate transformation according to the transformation matrix to generate six degrees of freedom pose parameters in the unified coordinate system. The conversion process includes a residual error checking step: in the instrument static state, detect the spatial coordinate transformation residual error, with a threshold of 0.2 millimeters. The registration reference data set constructs an integrated three-dimensional spatial point cloud and instrument pose mapping relationship, and the point cloud acquisition range covers the temporal mastoid cortex, conical protuberance and anatomic landmarks in the entrance area of the antrum. The data set is labeled with a time synchronization stamp, and the storage format includes coordinate transformation parameters and spatial point topology relationship index. Before data output, perform an encrypted digest verification to ensure the integrity of the subsequent registration process input.

[0040] The registration reference data set is loaded into the surface registration engine, which calls the triangular facet geometry data of the temporal bone dynamic virtual model. The iterative registration process is divided into two stages: the initial coarse registration stage estimates the spatial transformation parameters through principal component analysis; the fine registration stage uses the iterative closest point algorithm to optimize the corresponding point distance error. The registration process introduces an anatomical feature constraint mechanism: limits the length deviation of the horizontal segment of the facial nerve canal to no more than 0.25 millimeters, and maintains the continuity of the curvature change in the tympanic shield plate area.

[0041] The optimization engine integrates a vibration compensation module to generate a dynamic pose correction field based on the spectral characteristics of the instrument acceleration. The registration output includes seven-dimensional transformation parameters (rotation quaternion + translation vector) and a confidence coefficient. A spatial verification process is performed in the promontorium region: five non-coplanar check points are selected, and the root mean square error between the measured coordinates and the mapped points of the virtual model must be less than the 0.3 mm threshold. The final real-time spatial fusion relationship data stream is updated synchronously with the surgical operation through time stamp, and the output interface is compatible with the augmented reality rendering pipeline protocol.

[0042] This embodiment ensures the sub-millimeter spatial accuracy of optical tracking data and the real-time nature of instrument pose solving through the cooperative processing of head marker array optical tracking and instrument adapter pose analysis, establishing a reliable spatial reference for multi-source data. The rigid alignment operation unifies the head reference system and the instrument coordinate system through rigid transformation, eliminating the inherent bias of the system, and the registration reference data set completely covers the key anatomical landmark region of the temporal bone, providing high spatial features for dynamic registration. The surface iterative registration process combines anatomical constraint mechanism and vibration dynamic compensation, effectively suppressing the pose drift caused by high-frequency vibration of the bone drill while ensuring the integrity of the length of the facial nerve canal and the morphology of the tympanic cavity thin-walled structure. The real-time spatial fusion relationship output is executed before the promontorium region multi-point spatial verification, confirming that the spatial mapping error is below the clinical safety threshold.

[0043] In one embodiment, surface iterative registration is performed according to the registration reference data set and the dynamic virtual model of the temporal bone to obtain the real-time spatial fusion relationship, including: The coordinate sequence in the registration reference data set is analyzed, and the points located in the exposed area of the mastoid process of the temporal bone are selected. The mastoid process is the core operation area of the temporal bone surgery, including the anatomic landmarks such as the entrance of the mastoid antrum, the pyramidal eminence, and the posterior superior spine of the external auditory canal. The bony structure shows obvious gray difference in the image, which is convenient for feature recognition. By setting the curvature threshold to identify geometric mutation points, it is ensured that the selected points are the tips or turning points of the bony structure, rather than the soft tissue attachment area.

[0044] To ensure the uniformity and coverage of the point cloud distribution, a spatial constraint strategy is adopted: the minimum distance between points must be greater than 0.5 mm to avoid calculation redundancy caused by dense points; the point density in the key area (such as the perimeter of the mastoid antrum) must be higher than that in other areas to ensure the positioning accuracy of the area. Morphological filtering is performed on the candidate points, and a 3x3x3 cubic kernel is used for closing operation to fill small cavities and smooth the surface, and remove isolated points caused by noise.

[0045] The obtained target feature point cloud set contains 60-120 points, covering 6-8 key anatomical landmarks such as the entrance of the antrum, the apex of the concha, and the posterior superior spine of the external auditory canal, each point being attached with three-dimensional coordinates and curvature attributes. These point clouds are time-stamped and aligned with the operation timeline to ensure real-time processing. The point cloud accuracy is verified to be less than 0.3 mm, meeting the sub-millimeter level navigation requirements.

[0046] Differential manifold matching aims to geometrically align the target feature point cloud set with the surface of the dynamic virtual temporal bone model, and find the optimal spatial transformation matrix. This process needs to consider both global matching accuracy and local anatomical structure preservation, avoiding key structure deformation.

[0047] A spatial index of the target feature point cloud set and the virtual model is constructed. The matching algorithm uses an improved Iterative Closest Point (ICP) algorithm: in the initial stage, the principal component analysis (PCA) is used to estimate the rough rotation and translation parameters, and the point cloud is roughly aligned to the model surface; in the fine optimization stage, anatomical constraints are introduced to rigidly limit the length error of the tympanic segment of the facial nerve canal and the curvature gradient of the tympanic shield area, ensuring the integrity of the key structure form.

[0048] The optimization process uses a constrained nonlinear least squares method to adjust the transformation parameters through gradient descent iteration, minimizing the weighted distance sum of the point cloud and the corresponding points of the model. The weight distribution is tilted towards key areas (such as the facial nerve canal area with twice the weight of other areas), prioritizing the alignment of important structures. After the iteration converges, the initial registration matrix is obtained, containing the rotation quaternion and translation vector, which maps the target point cloud to the virtual model coordinate system.

[0049] The matrix effectiveness is verified by residual error evaluation: calculate the average distance of all matching points, if less than 0.3 mm, the initial registration is successful; otherwise, adjust the constraints and iterate again. The final output of the initial registration matrix provides the initial transformation relationship for subsequent anti-vibration registration, and its numerical stability is verified by orthogonality test to ensure that the transformation is not singular.

[0050] High-frequency vibrations generated during the operation of surgical instruments (such as bone drills and suction devices) can cause pose data drift, which needs to be separated from the pose sequence to extract the main vibration mode component parameters for vibration compensation.

[0051] The instrument pose data is decomposed into a time series, and the acceleration signal is split into multiple intrinsic mode functions (IMF) through empirical mode decomposition (EMD), each IMF corresponding to a vibration component of different frequency. By filtering the termination condition (the number of consecutive IMF extreme points < threshold), the dominant vibration mode (usually the first 3 orders) is extracted, corresponding to the 8000-15000 Hz frequency band (main vibration range of bone drills).

[0052] The frequency domain analysis module performs a fast Fourier transform (FFT) on the extracted IMFs to identify the amplitude, frequency, and phase information of the dominant mode components. The bone drill type instrument focuses on extracting the first three order amplitude attenuation coefficients (describing the vibration energy decay characteristics over time); the suction type instrument captures the phase offset of the low frequency drift (usually <100 Hz).

[0053] The dominant mode component parameters are output in time series form, including amplitude, frequency, phase, and corresponding timestamp, synchronized with the instrument motion state. These parameters are subsequently used to construct a vibration compensation field, which eliminates the pose drift caused by vibration through inverse displacement correction, ensuring the stability of the registration process.

[0054] The dominant mode component parameters reflect the influence of vibration on the pose during the operation of the surgical instrument, and need to be integrated into the optimization process of the initial registration matrix to offset the pose drift caused by vibration. In specific implementation, first, the dominant mode component parameters (including amplitude, frequency, phase, and timestamp) are spatiotemporally aligned with the initial registration matrix to ensure that the time dimension of vibration compensation is consistent with the time axis of pose transformation.

[0055] The core of spatial registration is to construct a vibration compensation field: according to the dynamic changes of the amplitude in the dominant mode parameters, generate a displacement correction vector opposite to the vibration direction; adjust the action time of the correction vector combined with the frequency characteristics, so that the compensation effect is synchronized with the vibration period; use the phase information to calibrate the direction of the correction vector to avoid reverse errors caused by phase offset. This process is realized through transformation synthesis in Lie group space - the quaternion multiplication operation is performed between the rotation component of the initial registration matrix and the rotation correction amount of the vibration compensation, and the vector superposition is performed between the translation component and the translation correction amount of the vibration, finally generating the anti-vibration registration matrix.

[0056] To ensure the effectiveness of the compensation, a dynamic error suppression mechanism is introduced in the registration process: when the instrument vibration amplitude exceeds the preset threshold (such as 0.5 mm), the compensation weight in the corresponding direction is automatically enhanced; if the vibration frequency deviates from the statistical mean of the dominant mode parameters (such as ±10%), the action time of the compensation vector is adjusted. Before the anti-vibration registration matrix is output, a local area verification is performed: three non-coplanar points on the periphery of the eustachian entrance are selected, and the position deviation of these points in the virtual model and the actual space is calculated. If the average deviation is less than 0.2 mm, it is determined that the compensation is effective, otherwise the extraction strategy of the dominant mode parameters is adjusted. The final generated anti-vibration registration matrix integrates the spatial relationship of the initial registration and the dynamic correction of the vibration compensation, providing a stable foundation for subsequent topology verification.

[0057] The tympanic cavity connectivity verification is a key step to verify whether the anti-vibration registration matrix accurately reflects the continuity of the internal anatomical structure of the temporal bone, directly affecting the reliability of the navigation system. In specific implementation, first, the three-dimensional geometric data of the tympanic cavity channel in the dynamic virtual model of the temporal bone (including the spatial distribution of the tympanic step, vestibular step and mastoid air cell) are called, and the virtual point cloud after mapping by the anti-vibration registration matrix is spatially superimposed.

[0058] The verification process focuses on the morphological integrity of the tympanic cavity channel: by calculating the continuity index (such as curvature mutation rate, normal vector angle deviation) of the superimposed point cloud at the tympanic step-vestibular step interface, it is judged whether there is a structure fracture or overlap caused by registration error. If the continuity index exceeds the preset threshold (such as curvature mutation rate > 0.3 / mm), it is determined that there is local distortion in the registration matrix, and the main vibration mode parameters need to be called again for compensation correction; if the index meets the requirements, the connectivity of the channel is further verified - 5 non-coplanar verification points are selected, and the geodesic distance of each point in the virtual model and the actual space is calculated. If the maximum deviation of all geodesic distances is less than 0.3mm, it is determined that the spatial fusion relationship is effective.

[0059] The verified real-time spatial fusion relationship includes the final transformation parameters of the anti-vibration registration matrix, the spatial coordinates of the verification points and the continuity index, and is output with a time stamp and an instrument state code embedded to ensure real-time synchronization with the surgical operation. The real-time spatial fusion relationship is directly used for rendering the augmented reality navigation view, providing accurate virtual-real space mapping to support subsequent obstacle avoidance strategy generation and operation guidance.

[0060] This embodiment establishes a high-discrimination spatial reference for temporal bone surgery navigation through accurate positioning of the target feature point cloud set, and the sub-millimeter level positioning accuracy of key anatomical landmarks (such as the antrum entrance and pyramidal eminence) effectively reduces the initial deviation of subsequent registration. The differential manifold matching process takes into account global geometric alignment and local anatomical constraints, and the morphological integrity of key parameters such as the length of the facial nerve canal and the curvature of the tympanic shield is preserved, avoiding spatial deformation deviation between the virtual model and the actual structure. The vibration main frequency separation and the generation of the anti-vibration registration matrix specifically suppress the pose drift caused by high-frequency vibration of the bone drill, and the dynamic compensation mechanism ensures the real-time and stability of the spatial mapping in high-speed drilling operation.

[0061] In one embodiment, the instrument spatial pose data and the real-time spatial fusion relationship are used to construct a navigation view, obtaining an augmented reality navigation view, including: The spatial pose data of the surgical instrument is collected in real time by a preset tracking adapter, which is usually installed near the handle or functional end of the instrument, and obtains the three-dimensional coordinates and rotational attitude of the instrument in the surgical field through infrared reflective marker points or active light-emitting elements. The pose data includes translation components (X, Y, Z axis coordinates) and rotation components, which reflect the absolute position and direction of the instrument in the tracking coordinate system.

[0062] The core of position recognition is to convert these pose parameters into actual spatial positions in the surgical field. In specific implementation, first, the translation vector in the pose data is extracted, which directly represents the displacement of the instrument tip relative to the tracking origin; the rotation component is used to determine the pointing direction of the instrument, which is converted into a direction cosine matrix through Euler angles to obtain the unit direction vector of the instrument tip. To ensure the accuracy of the position information, double checking is performed on the original data: first, cross-verification through multiple cameras of the optical tracking system is performed to eliminate single-view measurement errors; second, in combination with the kinematics model of the instrument, low-pass filtering processing is performed on the pose jitter caused by high-frequency vibration to retain the low-frequency steady-state position. For example, a bone drill will produce 12-18 kHz vibration when rotating at high speed, and through filtering, high-frequency noise can be removed, and low-frequency steady-state displacement of 0.1-10 Hz can be retained. The final output of the surgical instrument position information includes the three-dimensional coordinates (accuracy ≤0.3 mm) and the pointing direction (error ≤2°) of the instrument tip, and is stored in the form of a time stamp sequence, which is strictly synchronized with the timing of the surgical operation, providing a reliable physical position reference for subsequent spatial mapping.

[0063] The real-time spatial fusion relationship defines the coordinate mapping rule of the virtual model and the real surgical field, including a rotation matrix and a translation vector, which is generated by the registration process of step 4. The goal of spatial coordinate transformation is to convert the actual position of the surgical instrument (from the position information of step 1) into the corresponding coordinates in the virtual model coordinate system, to achieve accurate superposition of virtual and real.

[0064] The transformation parameters (rotation matrix R and translation vector T) in the real-time spatial fusion relationship are called to convert the instrument position information from the tracking coordinate system to the virtual model coordinate system. Among them, the rotation matrix R is determined by the registration process to ensure that the spatial direction of the virtual model and the real surgical field is consistent; the translation vector T compensates for the origin offset of the tracking coordinate system and the virtual model coordinate system.

[0065] Constrained verification is performed on the virtual coordinates: if the projection of the instrument tip in the virtual model exceeds the anatomical boundary of the temporal bone (such as entering the intracranial or mastoid air cell), the position correction mechanism is triggered. The correction strategy includes adjusting the pitch angle of the rotation matrix or the forward displacement of the translation vector, limiting the instrument position within the effective range of the temporal bone bony structure (such as from the mastoid cortex to the tympanic tegmen). The final output of the virtual spatial instrument mapping data includes the coordinates (Xvirtual, Yvirtual, Zvirtual) and the direction vector (Uvirtual, Vvirtual, Wvirtual) of the instrument tip in the virtual model, and the corresponding relationship error of the virtual model surface grid is ≤0.5 mm, providing a spatial alignment basis for subsequent rendering.

[0066] The temporal bone dynamic virtual model contains bone structures (such as mastoid cortex, petrous pyramid), soft tissues (such as facial nerve, blood vessels), and other types of anatomical elements. The core of structure-aware rendering is to set rendering parameters differently according to the anatomical characteristics and clinical importance of the tissues to highlight key structures.

[0067] Material classification of the model: bone structures (such as mastoid cortex) are given high-density materials with a refractive index of 1.5 to simulate their dense characteristics; soft tissues (such as facial nerve sheath) are given low-density materials with a refractive index of 1.3 to highlight the interface differences with bone structures; nerve and blood vessels (such as nerve fibers in the facial nerve canal) are given translucent materials with a transparency of 0.4 to maintain visibility and avoid blocking deep structures.

[0068] The rendering process uses a layered rendering strategy: the bottom layer is the bone structure, which is rendered using a depth-first algorithm to ensure the clarity of the anatomical profile; the middle layer is the soft tissue, which simulates its light transmission characteristics through ray tracing, and the refractive index difference of the light at the soft tissue-bone structure interface enhances the visibility of the interface; the top layer is the nerve and blood vessels, which use edge enhancement algorithms to strengthen their boundaries, with a transparency of 0.6 to balance visibility and penetration.

[0069] To adapt to the dynamic changes of the surgical scene, the rendering parameters support real-time adjustment: when the instrument approaches the facial nerve (such as distance < 2mm), automatically increase the transparency of the facial nerve layer to 0.8, while reducing the transparency of the surrounding bone structure to 0.3, to guide attention to the key area; when the instrument moves away from the key structure, restore the default transparency settings to maintain the integrity of the overall anatomical structure. The final multi-layer transparency volume rendering view contains depth buffer information and material attribute table, providing a visualization basis for subsequent fusion with instrument mapping data.

[0070] The fusion of virtual space instrument mapping data and multi-layer transparency volume rendering view lies in establishing the consistency of virtual instrument and three-dimensional anatomical model in spatial position and visual presentation. In specific implementation, first, based on the coordinate information in the virtual space instrument mapping data, establish the spatial correspondence between the three-dimensional position of the instrument tip and the corresponding anatomical structure in the multi-layer transparency volume rendering view. For example, the coordinates of the instrument tip in the virtual model are mapped to the rendering layer corresponding to the coordinates in the multi-layer view.

[0071] The occlusion relationship between the instrument and the model needs to be handled during the fusion process: if the instrument tip is located behind a certain anatomical structure (such as the facial nerve canal) in the virtual model, adjust the rendering order of the instrument in the multi-layer view so that it is occluded by the structure; if the instrument is in front, increase its transparency to ensure the visibility of the structure behind. At the same time, dynamically adjust the display attributes of the instrument according to its function type (such as bone drill, aspirator) - increase the edge highlight of bone drill type instruments when they approach bone structures, and reduce the transparency of aspirator type instruments when they approach soft tissues to highlight the operation area.

[0072] The output primary view framework includes the spatial position of the instrument, display attributes (transparency, brightness), and occlusion relationship with the model, forming a preliminary visualization result of virtual and real superposition, providing a basis for subsequent optimization.

[0073] The optimization of the primary view framework needs to solve two core problems: the true restoration of physical occlusion and the optimization of optical display.

[0074] The reconstruction of physical occlusion relationship is achieved through depth information: first, calculate the depth value (i.e. the distance to the surgical field camera) of the instrument tip and each structure point of the model, and determine the drawing order according to the depth value - the structure with small depth value (close to the camera) is drawn first, and the structure with large depth value (far from the camera) is drawn later, to ensure that the occluded structure is not displayed incorrectly. For complex occlusion scenes (such as the instrument partially occluding the facial nerve canal), adjust the drawing order or transparency to correct the occlusion relationship, ensuring that the occlusion logic is consistent with the actual surgical field.

[0075] Optical distortion correction is carried out for the inherent characteristics of the display device. After obtaining the distortion parameters through device calibration, adjust each pixel point in the primary view framework to make the display content conform to the geometric shape of the actual surgical field. For example, the barrel distortion caused by a wide-angle lens can be corrected by adjusting the pixel coordinate distribution in reverse, correcting the display deformation of the instrument and the model, and ensuring the accuracy of the spatial relationship.

[0076] The output augmented reality navigation view integrates the true logic of physical occlusion and the geometric optimization of optical display, and the spatial relationship between the instrument and the model is completely consistent with the actual surgical field, providing accurate virtual-real fusion navigation information.

[0077] This embodiment ensures the position error of the surgical instrument in the virtual and real space through accurate instrument position recognition and spatial coordinate transformation, provides a reliable reference for navigation; structure perception rendering highlights key anatomical structures of the temporal bone through layered transparency setting, improves the recognition efficiency of deep structures; the depth fusion of virtual and real views and the reconstruction of physical occlusion eliminate the display misplacement of virtual instruments and models, ensuring that the occlusion relationship is consistent with the actual surgical field; optical distortion correction corrects the inherent deformation of the display device, ensuring the geometric reality of the navigation picture.

[0078] In one embodiment, the augmented reality navigation view is analyzed for structure distance to obtain dynamic warning parameters, including: The augmented reality navigation view combines virtual models with spatial information of the real surgical field. The motion trajectory of the instrument in the view intuitively reflects the real-time position change of the instrument relative to the temporal bone anatomical structure. The core of spatial trajectory extraction is to capture the position change rule of the instrument tip through continuous frame analysis, and to generate vector information describing the motion direction and speed. In specific implementation, first, the view is scanned frame by frame, and image feature matching or optical flow tracking technology is used to identify the pixel coordinates of the instrument tip in each frame and record the corresponding time stamp. Then, by comparing the coordinate changes of adjacent frames, the instantaneous moving direction and speed of the instrument are calculated, and finally the motion vector information of the instrument containing the position sequence and the corresponding speed direction (such as along the positive direction of the X axis, the negative direction of the Y axis, etc.) is generated. The time resolution of this information is synchronized with the view refresh rate, providing dynamic position reference for subsequent neighbor structure search.

[0079] The goal of neighbor structure search is to identify key anatomical structures near the current position of the instrument, providing spatial context for subsequent distance calculation and collision prediction. In specific implementation, first, the current position in the instrument motion vector is taken as the center to define a search range matching the instrument size and anatomical structure density. Then, the triangular mesh data of the dynamic virtual model of the temporal bone is traversed to filter out all structure points whose vertex coordinates fall within the search range. To improve efficiency, spatial indexing technology is used to pre-organize the model vertices, limiting the search range to the local area around the instrument. The filtered structure points need to meet two core conditions: one is to belong to the core anatomical structure of the temporal bone (such as the facial nerve canal, semicircular canal, auditory ossicles, etc.), and the other is to have potential contact risk with the instrument in the virtual model (such as a distance less than a safety threshold). The final target spatial distribution set contains the vertex coordinates of these adjacent structures, the belonging anatomical type (such as bony / soft tissue), and the initial distance from the instrument, providing input data for dynamic distance field calculation.

[0080] The core of dynamic distance field calculation is to quantitatively calculate the spatial distance between the instrument and each adjacent structure in real time, forming a distance mapping relationship that changes over time. In specific implementation, first, for each structure point in the target spatial distribution set, the spatial distance between the structure point and the current frame of the instrument is calculated by comparing their position coordinates (such as the straight-line distance between the structure point coordinates and the instrument coordinates). To avoid repeated calculation, an incremental update strategy is used: if the displacement of the instrument motion vector is small (such as the difference from the previous frame position is less than 0.1 mm), the distance value of the previous frame is reused and a small error is corrected; if the displacement is large (such as more than 0.5 mm), the distances of all adjacent points are recalculated. The calculation results are stored by anatomical structure type, forming a multi-structure real-time distance mapping table containing structure type (such as facial nerve canal, outer semicircular canal), structure point coordinates, current distance value, and time stamp. The mapping table is updated in real time with the instrument motion, providing dynamic distance data support for subsequent trajectory conflict prediction.

[0081] The core of trajectory collision prediction is to predict the risk of collision that may occur in the future by analyzing the spatial relationship between the current motion state of the instrument and the adjacent anatomical structure. In specific implementation, first, a motion trajectory prediction model for a short period of time is constructed based on the speed direction in the instrument motion vector and the historical displacement rule. For example, assuming that the instrument moves at a constant speed (ignoring high-frequency vibration effects), the position after a future time can be represented as a linear superposition of the current position and the speed vector.

[0082] Subsequently, the predicted trajectory is analyzed by spatial superposition with the positions of the adjacent structures in the target spatial distribution set: for each adjacent structure point, the minimum spatial distance between each point on the predicted trajectory and the structure point is calculated; if the predicted distance at a certain time is less than the collision safety threshold of the structure, the structure point is marked as a potential collision risk point, and the time interval and spatial position of the collision are recorded.

[0083] To improve the prediction accuracy, a motion trend correction mechanism is introduced: if the speed or direction of the instrument motion vector changes abruptly (such as when drilling encounters resistance, causing the speed to decrease), the time step of the prediction model is adjusted to shorten the prediction period to improve real-time performance. The final collision distribution prediction data includes the structure type of potential collision, the predicted collision time (within 0.5-2 seconds in the future), and the collision risk level (high / medium / low), which provides risk distribution basis for subsequent warning parameter generation.

[0084] The generation of dynamic warning parameters needs to combine the spatial risk of collision prediction with the dynamic changes of real-time distance to form specific warning information that can guide the operation. In specific implementation, first, each risk point (such as a certain area of the facial nerve canal) in the collision distribution prediction data is associated with the current distance of the corresponding structure point in the multi-structure real-time distance mapping table - for example, the current distance of the structure point corresponding to the risk point is 1.8 mm (close to the safety threshold of 2 mm), and the distance will decrease at a speed of 0.1 mm / s in the next 0.3 seconds, then the dynamic warning parameter of the risk point can be defined as "the front area of the facial nerve canal, high collision risk in the next 0.3 seconds, current distance 1.8 mm".

[0085] Different risk levels of collision points are identified differently: high-risk points (such as predicted collision time <1 second and current distance < safety threshold) are marked with red flashing boundaries; medium-risk points (such as predicted collision time 1-2 seconds and current distance close to safety threshold) are marked with yellow gradient boundaries; low-risk points (such as predicted collision time >2 seconds or current distance much greater than safety threshold) are marked with green static boundaries.

[0086] The output dynamic warning parameters include the spatial coordinates of the risk point, collision time prediction value, current distance value and risk level identification. These parameters are superimposed on the surgical field in real time through the augmented reality navigation view, providing the surgeon with clear obstacle avoidance guidance.

[0087] This embodiment uses spatial trajectory extraction to track instrument motion patterns in real time, providing a dynamic position reference for subsequent analysis. Nearest neighbor structure search accurately locates key anatomical areas, avoiding invalid computational interference. Dynamic distance field calculation quantifies the spatial relationship between the instrument and surrounding structures, making risks perceptible. Trajectory conflict prediction identifies potential collision risks in advance, buying time for operational adjustments. Dynamic warning parameters transform risk information into intuitive guidance, directly assisting the surgeon in decision-making. This overall approach achieves full-link coordination between instrument motion, anatomical structure, and risk warning, reducing the probability of accidental contact and improving the accuracy and safety of surgical operations.

[0088] In one embodiment, display control is performed based on the augmented reality navigation view and dynamic warning parameters to obtain a real-time navigation strategy, including: Dynamic warning parameters include information such as the real-time distance between the instrument and various anatomical structures of the temporal bone, the collision risk level, and the predicted collision time. Spatial weights must be assigned based on the functional zoning characteristics of the temporal bone. Temporal bone subregions can be divided into high-sensitivity, medium-risk, and low-sensitivity areas based on their anatomical function. Weight analysis is guided by clinical risk: high-sensitivity areas are weighted 0.7-1.0 due to their involvement with critical structures such as neurovascular structures; medium-risk areas are weighted 0.4-0.6 due to potential impacts on hearing or balance; and low-sensitivity areas are weighted 0.1-0.3 due to their high anatomical redundancy.

[0089] The risk level (such as high / medium / low) and the corresponding structure type in the dynamic warning parameters are extracted, and the risk level is associated with the sub-region weight through a predefined anatomical partition mapping table. For example, the high-risk warning in the facial nerve canal area corresponds to a weight of 1.0, and the medium-risk warning in the external semicircular canal area corresponds to a weight of 0.5. Subsequently, the local weights of risk points at different locations in the same sub-region are corrected - if the distance between a point and the critical structure is less than a safety threshold (such as 2mm), its local weight is increased by 0.2; if the distance is greater than the threshold but less than the warning threshold (such as 5mm), the basic weight is maintained. The final target area control factor set contains the weight values ​​of each sub-region and sub-region, providing a differentiated control basis for subsequent display optimization.

[0090] Augmented reality navigation view fuses virtual model with spatial information of real surgical field, needs to optimize key areas through structural visual association to reduce cognitive load of surgeons. In specific implementation, first, the target area control factor set is matched with the anatomical structure in the view in space, for example, the high-weight structure in the facial nerve canal area corresponds to a specific color (such as red) and transparency (such as 0.8) in the view, the medium-weight structure in the outer semicircular canal area corresponds to yellow and transparency 0.5, and the low-weight structure corresponds to gray and transparency 0.2.

[0091] The optimization process adopts a layered rendering strategy: the bottom layer is the bony structure, and the outline is ensured to be clear through the depth buffer algorithm; the middle layer is the soft tissue and blood vessels, and the display intensity is adjusted according to the control factor - the high-weight soft tissue enhances the edge contrast, and the medium-weight structure maintains regular display; the top layer is the risk prompt mark, and the attention is strengthened through the flickering animation.

[0092] In order to adapt to the dynamic changes of the operation scene, the optimization parameters support real-time adjustment: when the instrument approaches the high-weight area, the transparency of the area is automatically increased to 0.9 and a highlight frame is superimposed; when the instrument moves away from the risk area, the default display parameters are restored. The final specialized navigation view of the temporal bone presents the key structures and risk information intuitively through differentiated visual expression, helping surgeons quickly locate the anatomical areas that need attention.

[0093] The core of drilling trajectory conflict prediction is to perform spatial superposition analysis on the motion path of the instrument and the key structures in the specialized view, identify potential collision risks, and generate avoidance instructions. In specific implementation, first, the spatial boundary of the high-weight structure in the specialized view is extracted and converted into a forbidden area in three-dimensional space. Then, the real-time motion trajectory of the instrument is obtained, and the minimum spatial distance between the trajectory line and the forbidden area is calculated.

[0094] If the minimum distance is less than the safety threshold (such as 2mm for the facial nerve canal), it is determined as a high-risk conflict, and a first-level obstacle avoidance strategy is generated: prompt "2mm ahead is the facial nerve canal, suggest adjusting the drilling angle by 15°"; if the minimum distance is between the safety threshold and the warning threshold (such as 5mm), it is determined as a medium-risk conflict, and a second-level obstacle avoidance strategy is generated: prompt "5mm ahead approaches the facial nerve canal, the current speed needs to be reduced to 1000rpm"; if the minimum distance is greater than the warning threshold, it is determined as low risk and no adjustment is needed.

[0095] By introducing a motion trend correction mechanism: if the curvature of the instrument motion trajectory suddenly increases (such as drilling encountering resistance), the prediction time step is shortened (from 1 second to 0.5 seconds), and the conflict risk is recalculated. The final safety trajectory obstacle avoidance strategy contains parameters such as risk level, conflict position, adjustment suggestion (angle / speed / pause), etc., which are superimposed in real time to the surgical field through augmented reality view, providing clear operation guidance for surgeons.

[0096] The safety trajectory obstacle avoidance strategy contains key information such as risk level, conflict position, and adjustment suggestion, which needs to be spatially associated with the anatomical structure in the temporal bone specialized navigation view to generate operation guidelines that the operator can directly refer to. In specific implementation, first, the spatial coordinate range of the risk area (such as the facial nerve canal and the outer semicircular canal) in the obstacle avoidance strategy is extracted and matched with the corresponding structure in the specialized view - for example, the conflict position of the facial nerve canal area corresponds to the surface area of its three-dimensional model in the view.

[0097] The mapping process uses a spatial projection technique: the two-dimensional risk prompt in the strategy is converted into a three-dimensional spatial annotation in the view, and the actual projection position in the operating field is determined by the ray projection method. For high-risk areas, a semi-transparent red warning box is superimposed in the view, and the edge of the box body is aligned with the anatomical structure contour of the risk area; for medium-risk areas, a yellow gradient box is superimposed, and the box body range is expanded to a 5mm buffer zone outside the risk area.

[0098] Adjustment suggestions are labeled as text labels near the corresponding risk area, and the label content is dynamically bound to the specific parameters in the strategy. To avoid information overload, a hierarchical display strategy is adopted: high-risk prompts are always on top, and medium and low-risk prompts are displayed dynamically according to the instrument movement state - when the instrument approaches the risk area, it is displayed, and when it moves away, it is hidden. The final operation guide view deeply integrates strategy information and anatomical structure through spatial association, and the operator can directly identify the areas of concern and corresponding operation requirements through the view.

[0099] The operation guide view needs to be integrated with the real-time operating field image to ensure that the information is synchronized with the actual operation scene, and to adapt to the display characteristics of different optical devices. In specific implementation, first, the real-time video stream of the operating field is obtained, and the annotations (such as warning boxes and text labels) in the operation guide view are aligned with the corresponding positions in the video stream through image registration technology - for example, the red warning box in the view needs to accurately cover the actual display area of the facial nerve canal in the operating field.

[0100] The fusion process uses a transparency blending algorithm: the annotation layer of the guide view is superimposed with the operating field video layer to ensure that the annotation information is clear and visible without affecting the overall observation of the operating field. For different optical devices, the rendering parameters are dynamically adjusted: in microscope mode, the contrast is enhanced to highlight the annotations; in head-mounted mode, the stereo parallax is optimized to match the human eye observation habit. By using a double-buffer rendering mechanism: the main buffer stores the fusion result of the current frame, and the secondary buffer pre-renders the guide view of the next frame, and the vertical synchronization signal is used to realize frame rate synchronization. The final output of the real-time navigation strategy contains the fused operating field image and dynamic annotation information.

[0101] This embodiment uses dynamic weight analysis to accurately locate the highly sensitive area of ​​the temporal bone, combined with structural visual association optimization, to differentiate the key structures and risk information in the augmented reality navigation view, significantly improving the surgeon's efficiency in identifying anatomical key points; trajectory conflict prediction uses spatial superposition analysis to identify potential collision risks in advance, generate a graded warning strategy, reserve time for the surgeon to adjust the operation, and reduce the probability of accidentally touching key structures; operation field mapping deeply integrates obstacle avoidance strategies with real-time surgical field images, and uses spatial projection technology to ensure that prompt information accurately corresponds to the actual anatomical position, avoiding decision-making bias caused by information dislocation; optical rendering adaptation optimizes display parameters for different devices to ensure clear visibility of navigation information under complex lighting conditions. The overall realization of full-link collaboration from risk warning to operation guidance improves the reliability of surgical navigation and the surgeon's confidence in operation.

[0102] In one embodiment, drilling trajectory collision prediction is performed based on the temporal bone-specific navigation view to obtain a safe trajectory obstacle avoidance strategy, including: Through structural visual correlation optimization, the specialized temporal bone navigation view has clarified the spatial distribution and display weights of high-, medium-, and low-sensitivity areas. The core of the phase constraint construction is to combine the anatomical exposure patterns and risk evolution characteristics of the surgical process, dividing the drilling operation into three phases with clear spatial boundaries and defining the permitted operation channels for each phase.

[0103] Based on the natural anatomical boundaries of the temporal bone, the surgical field is divided into an initial exposure zone (stage one), a deep approach zone (stage two), and an area adjacent to critical structures (stage three). The initial exposure zone corresponds to the superficial temporal bone. The operating channel is defined as a three-dimensional columnar region extending from the surgical field entrance to the mastoid cortical surface, with a cross-sectional diameter twice the diameter of the drill bit, allowing for free passage of instruments.

[0104] The deep approach zone corresponds to the area where the mastoid air cells and semicircular canals are located. The operating channel is contracted into a conical area extending deep along the original path. The cross-sectional diameter is reduced by 30% compared with the initial stage, limiting the lateral swing of the instrument.

[0105] The areas adjacent to key structures correspond to highly sensitive areas such as the facial nerve canal and vestibular window. The operating channel is further contracted to a linear area that only accommodates the tip of the drill bit, with a width not exceeding 1.2 times the diameter of the drill bit, ensuring that the tip of the instrument maintains a preset safety distance from the key structure.

[0106] The boundary parameters (e.g. shrinkage ratio, linear width) of the operation channel are synchronized with the temporal bone dynamic virtual model in real time - if the mastoid air cell fusion is detected, the cone region angle of the second stage is automatically adjusted to avoid abnormal air cell; through real-time registration data identification, when the facial nerve canal position deviates from the anatomical atlas, the linear channel of the third stage is synchronously offset by the same distance, and the preset safety distance from the key structure is always maintained. The final three-stage operation channel is stored in the form of a three-dimensional spatial grid, each grid cell is labeled with the stage (1 / 2 / 3) and the maximum allowed drilling depth (e.g. first stage ≤2mm, second stage ≤5mm, third stage ≤1mm), providing spatial constraint rules for subsequent collision detection.

[0107] The goal of collision detection is to identify whether the spatial position of the drilling instrument in the current operation stage overlaps with the high-risk structure in the temporal bone dynamic virtual model. The three-dimensional grid data of the three-stage operation channel is called, and the real-time position of the instrument tip (from the pose data of the augmented reality navigation view) is mapped to the virtual model coordinate system to obtain its allowed operation space range in the current stage (e.g. linear channel in the third stage).

[0108] A cylindrical detection region (diameter = drill diameter + 0.2mm, compensating for slight instrument swing) centered on the instrument tip is constructed, and a spatial intersection operation is performed between this region and the triangular mesh of the temporal bone dynamic virtual model.

[0109] Through a layered detection strategy, the detection efficiency is improved to detect whether the instrument tip exceeds the boundary of the operation channel in the current stage (e.g. the linear channel width limit in the third stage), and if it does, it is directly marked as a high-risk collision; if it does not, it is further detected to detect the overlap between the detection region and the high-risk structure mesh. The determination of the overlap region is made by comparing the coordinates of each point in the detection region with the coordinates of the vertices of the virtual model mesh, and calculating the minimum Euclidean distance. When the minimum distance is less than the drill radius, it is determined as a collision interference. The final generated collision interference volume information includes the three-dimensional coordinate range of the interference region, the interference type (e.g. light contact / severe penetration) and the corresponding anatomical structure name, providing specific risk position description for subsequent obstacle avoidance planning.

[0110] The planar obstacle avoidance planning is based on the collision interference volume information, and generates drilling path options that avoid all high-risk regions under the constraints of the three-stage operation channel. The boundary surface of the collision interference volume is extracted (fitted by the triangular facets of the virtual model mesh), and converted into a set of obstacle planes in three-dimensional space. Then, for each obstacle plane, the projection area within the operation channel of the current operation stage is calculated to determine the spatial range that needs to be avoided.

[0111] From the current position of the instrument, an initial path line is generated in the original drilling direction. If the path line intersects with the obstacle plane, two branch paths, the left offset path and the right offset path, are generated at the intersection point, with an offset distance of 1.5 times the diameter of the drill bit, ensuring a safe distance from the obstacle plane. Repeat the above detection for each branch path until the path is completely free of all collision interference volumes. The obtained conflict-free trajectory option set contains multiple alternative paths, each of which is labeled with a starting point, an ending point, a maximum allowed depth, and a minimum safe distance from the key structure. These paths all satisfy the spatial constraints of the third-order operation channel and have no spatial overlap with the high-risk structures in the temporal bone dynamic virtual model.

[0112] Performing path mapping and visual conversion on the conflict-free trajectory option set to obtain a visual cognitive strategy The conflict-free trajectory option set contains multiple alternative paths that satisfy the third-order operation channel constraints and have no spatial overlap with high-risk structures. It needs to be converted into visual information that the operator can intuitively understand to support operation decisions. In specific implementation, first, map the three-dimensional coordinate data of each path to the coordinate system of the augmented reality navigation view, and ensure that the path is consistent with the actual spatial position of the operating field through spatial registration technology, for example, the starting point of the path corresponds to the current tip position of the instrument, and the ending point corresponds to the target depth.

[0113] To distinguish the priority and safety of the paths, assign differentiated visual attributes to each path: the recommended path (with the lowest overall risk and the closest depth to the target) uses high-light green lines, the auxiliary path uses yellow dashed lines, and the dangerous area boundary is marked with a red gradient shadow. The width of the path is dynamically adjusted according to the diameter of the drill bit - if a 2mm diameter drill bit is currently being used, the path width is set to 2.5mm (with a 0.5mm safety redundancy), ensuring that it matches the actual operation size.

[0114] According to the real-time pose data of the operating field camera, calculate the projection coordinates of the path on the retina to ensure that the path form is consistent with the actual spatial trend. At the same time, superimpose key information labels on the path: the starting point is labeled as "Starting point (X1, Y1, Z1)", the turning point is labeled as "Turning point (X2, Y2, Z3)", and the ending point is labeled as "Ending point (X3, Y3, Z3)", and the depth information is displayed beside each label (such as "Current depth: 3mm, Maximum allowed depth: 5mm"). The final generated visual cognitive strategy is superimposed on the operating field through the augmented reality view, and the operator can directly observe the spatial distribution and safety range of the multiple selectable paths, providing intuitive basis for operation decision-making.

[0115] The visual cognitive strategy needs to be verified in a virtual environment for its actual effectiveness, and to ensure that it can accurately guide the operator to avoid the risk area in the real operation. In specific implementation, first, a virtual verification scene consistent with the temporal bone anatomical structure of the patient is constructed: the dynamic virtual model of the temporal bone is imported, and the same initial conditions as in the operation (such as instrument type, initial position, drilling direction) are set. According to the path in the visual cognitive strategy, the virtual drilling operation is performed in sequence, and the instrument position, speed and spatial relationship with the model at each step are recorded.

[0116] Three types of risks are detected during the verification process: first, path deviation risk - if the actual position of the instrument deviates from the planned position by more than 0.3mm (caused by optical tracking system error or model deformation), the path is marked as "needs to be corrected". Second, collision omission risk - through a virtual collision detection algorithm (based on the spatial intersection of the model grid and the instrument cylinder), it is checked whether there is a potential collision area (such as the facial nerve canal branch or semicircular canal protrusion) that is not marked in the visual strategy. Third, depth loss of control risk - compare the actual depth of virtual drilling with the maximum allowed depth set in the strategy, if the error exceeds 0.2mm (caused by drill wear or speed fluctuation), mark the path as "depth unreliable".

[0117] For the problem path found in the verification, the re-planning process is triggered: based on the position of the deviation or collision area, adjust the deviation direction of the path (such as left / right translation 1mm) or shorten the depth (such as from 5mm to 4.5mm), generate the corrected path and verify it again, until all risk indicators meet the standard. The final safe trajectory obstacle avoidance strategy contains the optimized path parameters (such as corrected coordinates, depth, width) and real-time update mechanism - when a small change in the anatomical structure is detected, automatically adjust the boundary parameters of the path to ensure that it always maintains a preset safety distance from the key structure.

[0118] This embodiment divides the drilling operation into three-order space channels through stage constraint construction, and clearly defines the safe operation range of different regions. Collision detection and obstacle avoidance planning identify the interference volume in real time and generate a conflict-free path to avoid potential collision risks in advance. Path mapping and visual conversion convert the abstract three-dimensional path into intuitive annotations, improving the clarity of the operation guide. Virtual drilling verification tests the path reliability by simulating the actual scene, ensuring the effectiveness of the planning.

[0119] Referring to Figure 2 The application also provides a virtual reality and real image fusion navigation system for temporal bone surgery, which is applied to the virtual reality and real image fusion navigation method for temporal bone surgery according to any one of the preceding embodiments, and comprises: A collection module is configured to acquire medical image data of a temporal bone region for three-dimensional reconstruction, and obtain a dynamic virtual model of the temporal bone. The analysis module is used to acquire optical tracking data and instrument spatial pose data, dynamically register and map the temporal bone dynamic virtual model, and obtain a real-time spatial fusion relationship; The correlation module is used to construct a navigation view for the instrument spatial pose data and the real-time spatial fusion relationship, and obtain an augmented reality navigation view; The processing module is used to analyze the structure distance of the augmented reality navigation view, and obtain a dynamic early warning parameter; The control module is used to display and control according to the augmented reality navigation view and the dynamic early warning parameter, and obtain a real-time navigation strategy.

[0120] The virtual reality and real image fusion navigation system for temporal bone surgery provided by the application can effectively solve the problem of lagging of traditional static image registration, ensure that the virtual model and the actual anatomical structure always maintain high-precision matching, and thus improve the accuracy of surgical navigation. The augmented reality navigation view is constructed based on the instrument spatial pose data and the real-time spatial fusion relationship, which can intuitively observe the relative position of the surgical instrument and the key anatomical structure, enhance the real-time and interactivity of intraoperative operation, and adapt to the rapidly changing operating field environment. The structure distance of the augmented reality navigation view is analyzed, the distance between the instrument and the key tissue is calculated in real time, and the dynamic early warning parameter is generated, so as to timely prompt potential risks, reduce the probability of injury to important nerves and blood vessels, and improve the safety of surgery. According to the augmented reality navigation view and the dynamic early warning parameter, real-time display and control can be performed, so as to provide the optimal navigation strategy, assist precise operation, improve the efficiency of surgery, and reduce the operation errors caused by visual errors or delayed feedback.

[0121] It should be noted that, for the convenience and brevity of description, the specific working processes of the system and each module described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0122] The above only describes the preferred embodiments of the application, and does not limit the patent scope of the application, and any equivalent structure or equivalent process transformation based on the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the application.

Claims

1. A virtual reality and real image fusion navigation method for temporal bone surgery, characterized in that: include: Obtain medical imaging data of the temporal bone area for three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone; Acquiring optical tracking data and instrument spatial posture data, performing dynamic registration mapping on the temporal bone dynamic virtual model, and obtaining a real-time spatial fusion relationship; Constructing a navigation view based on the spatial posture data of the instrument and the real-time spatial fusion relationship to obtain an augmented reality navigation view; Performing structural distance analysis on the augmented reality navigation view to obtain dynamic warning parameters; Display control is performed according to the augmented reality navigation view and the dynamic warning parameters to obtain a real-time navigation strategy.

2. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 1, characterized in that: The obtaining of medical imaging data of the temporal bone region and performing three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone includes: Acquiring CT image data and MRI image data of the temporal bone region and performing multimodal registration and alignment to obtain the medical image data; Performing tissue structure segmentation on the medical image data to obtain an initial structure mask set; Reconstructing the three-dimensional surface mesh of the initial structure mask set to obtain a target structure mesh model; The target structure grid model is assigned spatial attributes and bound with dynamic response parameters to obtain the temporal bone dynamic virtual model.

3. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 1, characterized in that: The acquiring of optical tracking data and instrument spatial posture data, and performing dynamic registration mapping on the temporal bone dynamic virtual model to obtain a real-time spatial fusion relationship includes: Acquiring optical data collected by a preset head marker array, performing optical tracking and recognition, and obtaining the optical tracking data; Performing an instrument degree of freedom posture analysis through a preset surgical instrument tracking adapter to obtain the instrument spatial posture data; performing a rigid coordinate system alignment on the optical tracking data and the instrument spatial posture data to obtain a registration reference data set; The real-time spatial fusion relationship is obtained by performing surface iterative registration on the registration reference data set and the temporal bone dynamic virtual model.

4. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 3, characterized in that: The performing surface iterative registration on the registration reference data set and the temporal bone dynamic virtual model to obtain the real-time spatial fusion relationship includes: Positioning the target point according to the registration benchmark data set to obtain a target feature point cloud set; Performing differential manifold matching on the target feature point cloud set and the temporal bone dynamic virtual model to obtain an initial registration matrix; Separating the main vibration frequency of the instrument spatial posture data to obtain main vibration mode component parameters; Performing spatial registration on the initial registration matrix according to the principal vibration mode component parameters to generate an anti-vibration registration matrix; Tympanic connectivity verification is performed on the anti-vibration registration matrix to obtain the real-time spatial fusion relationship.

5. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 1, characterized in that: The step of constructing a navigation view based on the fusion relationship between the spatial posture data of the instrument and the real-time space to obtain an augmented reality navigation view includes: Performing surgical instrument position identification based on the instrument spatial posture data to obtain surgical instrument position information; Performing spatial coordinate transformation on the surgical instrument position information and the real-time spatial fusion relationship to obtain virtual space instrument mapping data; Performing structure-aware rendering on the temporal bone dynamic virtual model to obtain a multi-layer transparency volume rendering view; Deeply fusing the virtual space instrument mapping data with the multi-layer transparency volume rendering view to obtain a primary view frame; Physical occlusion relationship reconstruction and optical distortion correction are performed on the primary view frame to obtain the augmented reality navigation view.

6. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 1, characterized in that: The performing structural distance analysis on the augmented reality navigation view to obtain dynamic warning parameters includes: Extracting a spatial trajectory from the augmented reality navigation view to obtain device motion vector information; Performing a neighbor structure search based on the instrument motion vector information and the temporal bone dynamic virtual model to obtain a target space distribution set; Performing dynamic distance field calculation on the target spatial distribution set to obtain a multi-structure real-time distance mapping table; Performing trajectory collision prediction based on the target spatial distribution set and the multi-structure real-time distance mapping table to obtain collision distribution prediction data; The collision distribution prediction data and the multi-structure real-time distance mapping table are spatially correlated and integrated to obtain dynamic warning parameters.

7. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 1, characterized in that: The display control is performed based on the augmented reality navigation view and the dynamic warning parameters to obtain a real-time navigation strategy, including: Performing temporal bone subregion spatial weight analysis on the dynamic warning parameters to obtain a target region regulatory factor set; Performing structural visual correlation optimization based on the target region regulatory factor set and the augmented reality navigation view to obtain a temporal bone specialized navigation view; Predicting drilling trajectory conflicts based on the temporal bone-specific navigation view to obtain a safe trajectory obstacle avoidance strategy; Performing operation field mapping on the temporal bone specialized navigation view according to the safe trajectory obstacle avoidance strategy to obtain an operation guidance view; Optical rendering fusion adaptation is performed on the operation guidance view to obtain a real-time navigation strategy.

8. The virtual reality and real image fusion navigation method for temporal bone surgery according to claim 7, characterized in that: Predicting drilling trajectory conflicts based on the temporal bone-specific navigation view to obtain a safe trajectory obstacle avoidance strategy includes: Performing phase constraint construction on the temporal bone specialized navigation view to obtain a three-order operation channel; Performing collision detection on the temporal bone dynamic virtual model based on the third-order operation channel to obtain collision interference volume information; Performing planar obstacle avoidance planning based on the collision interference volume information to obtain a set of conflict-free trajectory options; Performing path resolution mapping and visual transformation on the conflict-free trajectory option set to obtain a visual cognitive strategy; The visual recognition strategy is verified by virtual drilling to obtain a safe trajectory obstacle avoidance strategy.

9. A virtual reality and real image fusion navigation system for temporal bone surgery, characterized in that: The virtual reality and real image fusion navigation method for temporal bone surgery according to any one of claims 1 to 8 comprises: An acquisition module is used to acquire medical imaging data of the temporal bone region for three-dimensional reconstruction to obtain a dynamic virtual model of the temporal bone; An analysis module, configured to acquire optical tracking data and instrument spatial posture data, perform dynamic registration mapping on the temporal bone dynamic virtual model, and obtain a real-time spatial fusion relationship; an association module, the association module being configured to construct a navigation view based on the fusion relationship between the spatial posture data of the instrument and the real-time space to obtain an augmented reality navigation view; a processing module, the processing module being configured to perform structural distance analysis on the augmented reality navigation view to obtain dynamic warning parameters; A control module is used to perform display control based on the augmented reality navigation view and the dynamic warning parameters to obtain a real-time navigation strategy.

Images