An Ophthalmic Surgery Real-Time Navigation System and Method Based on Dynamic Visual Field Tracking

Abstract

The present application discloses an ophthalmic surgery real-time navigation system and method based on dynamic visual field tracking, which relates to the technical field of medical systems. It includes matching the operation data closest to the current surgery from the database to generate the standard operation navigation for the current surgery; comparing the real-time image of the surgery process with the image of the standard operation navigation in real time to determine the current step of the surgery, and at the same time displaying the image navigation for the subsequent surgery of the patient; generating a spatial mapping matrix between the eyeball movement trajectory and the target area by combining the target area boundary data segmented from the preoperative OCT image and the intraoperative navigation coordinate system; dynamically correcting the virtual navigation path of the surgical instrument according to the real-time eyeball movement trajectory to align the surgical instrument with the target area. Through the superposition display of the three-dimensional virtual path and the real-time image, the spatial relationship between the instrument and the target area can be intuitively grasped, actively adapting to the unconscious displacement or physiological tremor of the patient during the surgery to ensure the continuous accuracy of the navigation path.

Description

Technical Field

[0001] This application relates to the technical field of medical systems, and particularly to an ophthalmic surgery real-time navigation system and method based on dynamic visual field tracking. Background Art

[0002] With the continuous development of modern medicine, especially the progress in the field of ophthalmic surgery, how to improve the accuracy and safety of ophthalmic surgery has become an important research direction. In ophthalmic surgery, especially delicate surgeries such as retinal surgery and cataract surgery, the patient's eyes may still move slightly after anesthesia. Therefore, extremely high requirements are placed on the doctor's operation accuracy and experience. With the development of technology, surgical navigation technology and surgical assistance systems have improved the success rate of surgeries to a certain extent. However, existing ophthalmic surgery navigation systems mostly rely on preoperative image and video data for assisted operations. Although these technologies can achieve the positioning of the surgical area to a certain extent, their accuracy and dynamic adaptability still have deficiencies. For example, in the prior art, Patent CN202010516777.9 (ophthalmic surgery navigation system) aims to assist in adjusting the positioning of surgical instruments through real-time monitoring of the dynamic movement of the eyeball. However, this patent only proposes dynamic tracking based on feature points on the surface of the eyeball. Although it can provide certain assistance for the surgery, its tracking accuracy and dynamic adaptability are relatively limited. Specific problems include: relying on feature points on the surface of the eyeball for tracking, but lacking comprehensive calculation and processing of the movement trajectory of the eyeball in three-dimensional space, and it is difficult to provide sufficiently detailed eyeball movement data.

[0003] Therefore, the technical problem to be solved by this application is: how to provide an ophthalmic surgery real-time navigation system that can dynamically adjust the path of surgical instruments in real time through the dynamic visual field tracking of the patient's eyeball, based on the three-dimensional dynamic movement trajectory of the patient's eyeball, combined with preoperative image data and the surgical navigation coordinate system, and improve the surgical accuracy through a real-time feedback mechanism, so as to overcome the problems of insufficient accuracy, poor dynamic adaptability, and lag in real-time feedback in the prior art, ensure the smooth progress of high-precision ophthalmic surgery, and especially reduce the occurrence of errors during complex surgical procedures. Summary of the Invention

[0004] The purpose of the present invention is to: in view of the current lack of an auxiliary tool for real-time standardized navigation of a doctor's operation based on the dynamic tracking of a patient's eyeball during surgery, provide an ophthalmic surgery real-time navigation system and method based on dynamic visual field tracking. Through a pre-established standard surgical operation path, during the surgery, the images collected by the image acquisition module are continuously compared with the standard surgical operation path, and the image navigation of the next surgical operation can be displayed on the comparison display module in real time.

[0005] To achieve the above-mentioned invention purpose, the present invention provides the following technical solutions:

[0006] In the first aspect of the present application, a real-time navigation system for ophthalmic surgery based on dynamic vision tracking is provided, including:

[0007] A navigation generation module, which matches the operation data closest to the current surgery from the database according to the surgery type and location of the patient, disassembles the steps of the operation data, and generates a standard operation navigation for the current surgery;

[0008] A comparison and display module, which compares the real-time image of the surgery process with the image of the standard operation navigation in real time to determine the current surgery step, and simultaneously displays the image navigation for the subsequent surgery of the patient;

[0009] A patient tracking module, which collects the corneal limbus feature points on the surface of the patient's eyeball in real time through a near-infrared light source and an image sensor, calculates the three-dimensional motion trajectory, and generates a spatial mapping matrix between the eyeball motion trajectory and the target area in combination with the target area boundary data segmented from the preoperative OCT image and the intraoperative navigation coordinate system;

[0010] An offset compensation module, which dynamically corrects the virtual navigation path of the surgical instrument according to the real-time eyeball motion trajectory, adjusts the relative position between the corrected instrument path and the target area boundary, and aligns the surgical instrument with the target area.

[0011] In an embodiment of the present application, it further includes a physician tracking module, which collects the physician's eye image in real time through an image sensor to obtain the gaze point coordinates of the physician, calculates the ratio of the coverage area of the gaze point coordinates to the current navigation target area. If the average ratio within a unit time is lower than a set threshold, a visual alarm signal is triggered.

[0012] In an embodiment of the present application, the output data of the patient tracking module and the physician tracking module are input into an eye movement fusion unit, and a comprehensive compensation amount is generated through the eye movement fusion unit, specifically including:

[0013] Establish a first transformation matrix T1 according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path;

[0014] Establish a second transformation matrix T2 according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view;

[0015] Obtain the three-dimensional coordinate system correction parameters through weighted fusion of T1 and T2, and generate a comprehensive compensation amount.

[0016] In an embodiment of the present application, the physician tracking module further includes a status monitoring sub-module, which increases the navigation prompt intensity when an abnormal status of the physician is detected according to the physician's eye image; the abnormal status includes tension and fatigue;

[0017] The state of tension is detected by the change rate of the pupil diameter of the doctor in the eye image. When the pupil diameter expands beyond a threshold within a unit time, it is determined as a state of tension;

[0018] The state of exhaustion is detected by the contraction decline rate k1 of the levator palpebrae superioris muscle, the pupil diameter increase rate k2, the pupil light reflex speed delay rate k3 in the eye image, and the doctor's pulse is detected to obtain the doctor's heart rate variability decline rate k4. When k1, k2, k3, and k4 reach a threshold through weighted fusion within a unit time, it is determined as a state of fatigue.

[0019] In an embodiment of the present application, it further includes a second display module, and the second display module performs the following operations:

[0020] Generate a step flowchart according to the pre-operative planned surgical path, highlight the current surgical stage in real time, and mark the estimated time of the remaining steps;

[0021] Predict the type of instrument required for the subsequent steps based on a convolutional neural network, and dynamically display a tool preparation list, which includes instrument specification parameters and disinfection status identifiers;

[0022] When it is detected that the completion degree of the current step reaches a threshold, a prompt signal is triggered to enable the assistant to complete the loading and positioning of the instruments for the next stage in advance;

[0023] In an emergency operation scenario, including sudden bleeding, automatically switch to a warning interface, and synchronously display the location of the emergency tool kit and the usage priority guide.

[0024] In an embodiment of the present application, it further includes an assistant tracking module, which collects the eye images of the assistant in real time through an image sensor, and collects the pupil focus coordinates and the scanning frequency of the assistant in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the entropy value of the scanning path is lower than the threshold, it is determined as a state of distraction; an LSTM network is used to analyze the matching degree between the eye movement pattern and the current surgical stage, generate an attention score, and trigger a reminder when the score is lower than the threshold.

[0025] In an embodiment of the present application, the assistant tracking module further includes an instrument guiding sub-module, which tracks the spatial coordinates of the instrument to be prepared in real time through a positioning chip and compares them with the preset position of the second display module; when the deviation distance of the instrument is greater than the preset value, a highlighted arrow guide is superimposed in the assistant's AR glasses, and the specification parameters of the target instrument are synchronously prompted by voice; at the same time, a feature vector library is established for the frequently mis-taken instruments, and the convolutional network is used to identify abnormal grasping actions and issue an early warning in advance.

[0026] In an embodiment of the present application, it further includes a physician database module that stores the historical surgical records of each physician, including the time-consuming distribution of surgical stages, the physiological parameter fluctuation curve, and the instrument usage correction amount; extracts feature patterns through an LSTM network, reversely optimizes the prediction model parameters based on the postoperative quality assessment report, establishes an individualized tension-fatigue time window prediction model, and adjusts the navigation parameters in advance for the high-frequency warning stage.

[0027] In an embodiment of the present application, the second display module further includes an emergency exemption sub-module that automatically shields the conventional warning logic and switches to the augmented reality mode in the event of sudden intraocular pressure elevation or bleeding: superimposes a 3D vascular pressure distribution heat map on the surgical field; and rearranges the instrument priorities: hides non-critical tooltips and highlights the positions of hemostatic instruments.

[0028] In a second aspect of the present application, there is provided a real-time navigation method for ophthalmic surgery based on dynamic visual field tracking, characterized by including:

[0029] Match the operation data closest to the current surgery from the database according to the patient's surgery type and location, disassemble the steps of the operation data, and generate the standard operation navigation for the current surgery.

[0030] Compare the real-time image of the surgical process with the image of the standard operation navigation in real time to determine the current surgical step, and at the same time display the image navigation for the subsequent patient surgery.

[0031] Collect the limbus feature points on the patient's eye surface in real time through a near-infrared light source and an image sensor, calculate the three-dimensional motion trajectory, and generate a spatial mapping matrix between the eye movement trajectory and the target area in combination with the target area boundary data segmented from the preoperative OCT image and the intraoperative navigation coordinate system.

[0032] Dynamically correct the virtual navigation path of the surgical instrument according to the real-time eye movement trajectory, adjust the relative position between the corrected instrument path and the target area boundary, and align the surgical instrument with the target area.

[0033] In an embodiment of the present application, the eye image of the physician is collected in real time through an image sensor to obtain the gaze point coordinates of the physician, calculate the ratio of the coverage area between the gaze point coordinates and the current navigation target area, and if the average ratio within a unit time is lower than the set threshold, trigger a visual alarm signal.

[0034] In an embodiment of the present application, a comprehensive compensation amount is generated through the patient's eye movement data and the physician's eye movement data, specifically including:

[0035] Establish a first transformation matrix T1 according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path.

[0036] A second transformation matrix T2 is established based on the eye movement data of the physician to describe the dynamic adjustment amount of the microscope field of view;

[0037] The three-dimensional coordinate system correction parameters are obtained by weighted fusion of T1 and T2 to generate a comprehensive compensation amount.

[0038] In an embodiment of the present application, when an abnormal state of the physician is detected based on the eye image of the physician, the navigation prompt intensity is increased; the abnormal states include tension and fatigue;

[0039] For the abnormal state of tension, the change rate of the pupil diameter of the physician is detected through the eye image. When the pupil diameter expands beyond the threshold within a unit time, it is determined as a tense state;

[0040] For the abnormal state of fatigue, the contraction decline rate k1 of the levator palpebrae superioris muscle, the pupil diameter increase rate k2, the pupil light reflex speed delay rate k3 are detected through the eye image, and the physician's pulse is detected to obtain the heart rate variability decline rate k4 of the physician. When k1, k2, k3, and k4 reach the threshold within a unit time through weighted fusion, it is determined as a fatigue state.

[0041] In an embodiment of the present application, a step flow chart is generated according to the pre-operative planned surgical path, the current surgical stage is highlighted in real time, and the estimated time of the remaining steps is marked;

[0042] Based on the convolutional neural network, the type of instrument required for the subsequent steps is predicted, and a tool preparation list is dynamically displayed. The list includes instrument specification parameters and disinfection status identification;

[0043] When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading and positioning of the instruments in the next stage in advance;

[0044] In an emergency operation scenario, including sudden bleeding, it automatically switches to a warning interface, and synchronously displays the location of the emergency tool kit and the usage priority guide.

[0045] In an embodiment of the present application, the eye image of the assistant is collected in real time through an image sensor, the pupil focus coordinates and the scanning frequency of the assistant are collected in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than the preset time or the entropy value of the scanning path is lower than the threshold, it is determined as a distracted state; the LSTM network is used to analyze the matching degree between the eye movement pattern and the current surgical stage to generate an attention score, and when the score is lower than the threshold, a reminder is triggered.

[0046] In an embodiment of the present application, the spatial coordinates of the instrument to be prepared are tracked in real time by a positioning chip and compared with the preset position of the second display module; when the deviation distance of the instrument is greater than the preset value, a highlighted arrow guide is superimposed in the assistant AR glasses, and the specification parameters of the target instrument are synchronously prompted by voice; at the same time, a feature vector library is established for the instruments frequently mis-taken, and the convolutional network is used to identify abnormal grasping actions and give early warnings in advance.

[0047] In an embodiment of the present application, the historical surgical records of each physician are stored, including the time-consuming distribution of surgical stages, the physiological parameter fluctuation curve, and the instrument use correction amount; the feature patterns are extracted through the LSTM network, the prediction model parameters are reversely optimized based on the postoperative quality assessment report, and an individualized stress-fatigue time window prediction model is established to adjust the navigation parameters in advance for the high-frequency warning stage.

[0048] In an embodiment of the present application, in the event of sudden intraocular pressure elevation or bleeding, the conventional warning logic is automatically blocked and switched to the augmented reality mode: a 3D vascular pressure distribution heat map is superimposed on the surgical field; and the instrument priority is rearranged: the non-critical tool tips are hidden, and the position of the hemostatic instrument is highlighted.

[0049] The present application has the following beneficial effects:

[0050] 1. Compare the real-time images of the surgical process with the images of the standard surgical operation path. After determining the path position of the current surgery, display the image navigation of the subsequent standard surgical operation path for the patient. Therefore, during the process of accumulating experience, doctors can perform surgeries according to the image navigation of the standard surgical operation path. By forming the standard operation path specifications based on the successful surgical processes of other doctors and learning the surgical operations at this surgical position and of this surgical type, the dependence on surgical clinical experience is reduced. The surgeries performed by young doctors are standardized by the standard paths of this surgical type and surgical site. For example, the cutting positions and angles of the surgeries are standardized, reducing problems such as unnecessary skin and tissue injuries to patients. The postoperative recovery and postoperative quality of life of patients are improved. At the same time, since the longer the operation time, the greater the harm to the patient and the longer the recovery time, the operation time will be minimized while ensuring the surgical effect. Therefore, doctors can standardize surgical operations according to the standard surgical path, reduce the surgical thinking time, shorten the operation time, and reduce the pain of patients. More deeply, the system dynamically matches surgical data with real-time images, transforming the traditional eye operations that rely on doctors' experience into a standardized navigation process, significantly reducing the risk of instrument deviation caused by the slight movement of the eyeball. Through the superimposed display of the three-dimensional virtual path and real-time images, the surgeon can intuitively master the spatial relationship between the instrument and the target area, avoiding misoperations caused by visual errors, especially applicable to high-precision surgical scenarios such as corneal transplantation and glaucoma drainage valve implantation. The system's real-time tracking and compensation mechanism for eye movement can actively adapt to the unconscious displacement or physiological tremor of the patient during the operation, ensuring the continuous accuracy of the navigation path and reducing the time-consuming operation of repeatedly adjusting the instrument position during the operation. In addition, the multi-level warning mechanism (visual prompt → interface simplification → forced pause) can effectively intercept potential operation errors and enhance the surgical safety. By integrating preoperative imaging data and intraoperative dynamic information, the system further optimizes the coherence of surgical planning and execution, shortens the learning curve of the surgeon, and provides technical support for the standardized promotion of complex ophthalmic surgeries.

[0051] 2. The physician tracking module solves the risk of attention deviation in traditional surgeries caused by the surgeon's distraction, visual fatigue, or limited operating vision by quantifying the spatial correlation between the surgeon's fixation point and the target area in real time. This module can actively identify the abnormal state where the surgeon's line of sight deviates from the key operation area and intervene in a timely manner through a hierarchical warning mechanism, reducing the probability of instrument misoperation caused by visual misjudgment. Combined with the dynamic path correction function of the navigation system, it further optimizes the man-machine cooperation efficiency and ensures the precise synchronization of the surgeon's attention and the instrument movement trajectory. Experiments show that this module can significantly improve the operation stability of high-precision surgeries. Especially in long and complex surgeries, by continuously monitoring the surgeon's visual focus, it helps to maintain the best operation state, reduce the surgical failures caused by incorrect operations, and reduce the risk of postoperative complications caused by human factors.

[0052] 3. By fusing the dual visual dynamic data of the patient and the physician, it solves the problem of mismatch between the instrument path and the operation field of view caused by the traditional navigation system relying only on single-dimensional compensation. In the traditional method, the slight movement compensation of the patient's eyeball may cause instrument path deviation due to the lag in the physician's field of view adjustment. However, this solution realizes the collaborative correction of the instrument path and the microscope field of view by real-time fusing the spatial transformation parameters of both. For example, when the patient's eyeball undergoes a horizontal displacement, T1 drives the instrument path compensation, and at the same time, T2 synchronously adjusts the center position of the microscope field of view to avoid the surgeon misjudging the relative position between the instrument and the target area due to the field of view deviation. It significantly reduces the risk of misoperation caused by visual dislocation. In addition, the dynamic optimization of the three-dimensional coordinate system correction parameters further enhances the system's adaptability to complex surgical scenarios, ensuring the coherence and stability of high-precision operations.

[0053] 4. Monitor the physician's state. When the physician's state is abnormal, through dynamically enhanced visual guidance, it helps the physician be aware of and quickly recover the state; furthermore, through the fusion analysis of multi-modal physiological parameters, it solves the problem of misjudgment caused by the traditional intraoperative monitoring relying only on a single indicator. For example, during a long operation, the levator palpebrae superioris of the physician may show slight relaxation, but if the pupillary light reflex is normal, the system will not misjudge it as a fatigue state, avoiding ineffective alarms from interfering with the surgical process. When tension or fatigue is detected, the enhanced navigation prompt can actively correct the risk of instrument deviation caused by the physician's distracted attention. It can effectively reduce the instrument path deviation caused by the abnormal state of the operator. Especially in the high-risk surgical stage, through dynamically enhanced visual guidance, it helps the physician quickly recover the operation accuracy. In addition, combined with heart rate variability analysis, the system can identify the early cumulative fatigue state and trigger preventive prompts before the critical nodes of the operation, optimizing the initiative and safety of human-machine collaboration.

[0054] 5. The high-light marking and time estimation function of the flow chart decomposes complex surgeries into controllable units, especially suitable for multi-stage surgeries such as glaucoma drainage valve implantation, reducing the cognitive load of novice surgeons; resource optimization: the dynamic tool list reduces the probability of incorrect instrument selection, and the disinfection status identification blocks the risk of cross-infection; safety improvement: the emergency interface improves the bleeding control efficiency through the coordination of three-dimensional space marking and voice guidance; human-machine collaboration, the multi-modal feedback of the prompt signal and the microscope AR overlay display realize the closed-loop optimization of the "hand-eye-brain" operation link, reducing the risk of instrument slippage caused by the surgeon's distracted attention.

[0055] 6. Quantify the visual attention distribution state through the eye movement entropy value, and combine the time series modeling ability of the LSTM network to accurately identify the attention shift caused by fatigue, distraction or information overload; hierarchical intervention mechanism, when the score is lower than the threshold, the system triggers in turn:

[0056] First-level reminder: The edge of the AR glasses gradually changes to red light (frequency 2Hz);

[0057] Secondary reminder: vibration tactile feedback (intensity 0.3G);

[0058] Level 3 intervention: automatically pause the instrument conveyor and send an alarm to the surgeon’s interface;

[0059] Surgical stage adaptation: The LSTM network dynamically adjusts the scoring weights according to different surgical stages, so that the intervention strategy is accurately matched with the key nodes of the surgery.

[0060] 7. AR arrows dynamically mark the offset direction; voice synchronization emphasizes key parameters; prediction of mispick behavior through grasping action analysis: detection of the contact area between the hand and the instrument through a convolutional network; trajectory abnormality warning, triggering an anti-fall prompt when the acceleration of the instrument movement changes suddenly; self-learning optimization mechanism, automatic clustering analysis of the feature library every week to identify new mispick patterns. Personalized risk prediction, by analyzing individual characteristics in historical surgical data, the system can trigger preparatory prompts 2-3 minutes in advance to avoid uncontrolled operations in sudden situations; in predicted high-risk periods, the system automatically enhances navigation prompts to match the physician's real-time cognitive load level. The heat map intuitively displays the distribution of vascular pressure to help the surgeon quickly locate the source of bleeding and reduce secondary injuries caused by blind exploration; hiding non-critical tool prompts can reduce the complexity of interface information and allow the surgeon to focus on hemostasis operations; in emergency mode, the system automatically removes the speed limit of the instrument movement to support rapid intervention operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0062] Figure 1 A schematic diagram of the electronic device structure of the hardware operating environment involved in an embodiment of the present application.

[0063] Figure 2 It is a schematic diagram of the functional modules of the real-time navigation system for ophthalmic surgery based on dynamic visual field tracking provided in an embodiment of the present application.

[0064] Figure 3 This is a module construction diagram of a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking provided in an embodiment of the present application.

[0065] Figure 4 It is a flowchart of the steps of the real-time navigation method for ophthalmic surgery based on dynamic visual field tracking provided in an embodiment of the present application.

[0066] Symbols in the figure: 1001 - processor, 1002 - communication bus, 1003 - user interface, 1004 - network interface, 1005 - memory. DETAILED DESCRIPTION

[0067] To make the above objects, features, and advantages of the present application more apparent and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0068] The solution of the present application will be further described below with reference to the accompanying drawings.

[0069] As Figure 1 shown, the electronic device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0070] Those skilled in the art can understand that Figure 1 the structure shown in

[0071] As Figure 1 shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and a data storage module.

[0072] In Figure 1In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be arranged in the electronic device, and the electronic device calls, through the processor 1001, a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking in the memory 1005, and executes a real-time navigation method for ophthalmic surgery based on dynamic visual field tracking provided by an embodiment of the present application.

[0073] Based on the foregoing hardware operating environment and system architecture, in the first aspect of the present application, with reference to Figure 2 and Figure 3 shown, a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking is provided, including:

[0074] A navigation generation module, according to the surgical type and location of a patient, matches the operation data closest to the current surgery from a database, disassembles the steps of the operation data, and generates a standard operation navigation for the current surgery;

[0075] It should be noted that the database contains three-dimensional image data of historical surgical cases, instrument movement trajectories, and key operation node parameters, and matches biometric features such as the patient's corneal curvature and anterior chamber depth through the dynamic time warping algorithm (DTW) (error threshold ±0.3 mm). When disassembling the steps, the STFT network is used to extract the surgical stage features, and the complex operations are decomposed into quantifiable subtasks (for example, cataract surgery is divided into 4 stages: incision making, capsulotomy, lens implantation, and suture). As the data in the database becomes richer, each subtask can be further subdivided, and a three-dimensional virtual path (accuracy ±0.05 mm) is generated;

[0076] A comparison and display module, which compares the real-time image of the surgical process with the image of the standard operation navigation in real time, determines the current step of the surgery, and simultaneously displays the image navigation for the subsequent surgery of the patient;

[0077] It should be noted that the real-time image is collected by a surgical microscope at a frame rate of 60 fps, 256 feature points are extracted by the SIFT algorithm for registration, and the optical flow method is used to compensate for the image offset caused by eye movement (error ≤0.1 pixel). The subsequent navigation image is displayed on the microscope eyepiece in a semi-transparent overlay form, including the expected path of the instrument (red dotted line), the boundary of the target area (green contour), and the safety distance prompt (yellow warning area);

[0078] A patient tracking module, which collects the corneal limbus feature points on the surface of the patient's eyeball in real time through a near-infrared light source and an image sensor, calculates the three-dimensional movement trajectory, and combines the boundary data of the target area segmented from the preoperative OCT image and the intraoperative navigation coordinate system to generate a spatial mapping matrix between the eyeball movement trajectory and the target area;

[0079] It should be noted that the near-infrared light source uses a wavelength of 850 nm (pulse frequency of 200 Hz), and the CMOS sensor (IMX585) captures the displacements of 8 characteristic points on the limbus corneae with an exposure time of 0.1 ms, and calculates the three-dimensional motion trajectory of the eyeball through extended Kalman filtering. The spatial mapping matrix fuses the preoperative OCT tomography data (accuracy of 5 μm) with the intraoperative navigation coordinate system, and the dynamic update formula is:

[0080]

[0081] Among them, the rotation matrix R is solved by the quaternion method, and the translation vector T calibrates the target area offset in real time;

[0082] The offset compensation module dynamically corrects the virtual navigation path of the surgical instrument according to the real-time eyeball motion trajectory, adjusts the relative position between the corrected instrument path and the target area boundary, and aligns the surgical instrument with the target area;

[0083] It should be noted that the correction algorithm uses a PID controller. When the detected eyeball displacement > 0.03 mm, the robotic arm is driven to adjust the instrument motion trajectory. The coincidence degree between the corrected path and the target area boundary is displayed on the operation interface in real time (a green indicator is triggered when ≥ 98%). If the coincidence degree < 90% lasts for 2 seconds, an audible and visual alarm is activated;

[0084] In this embodiment, the real-time image of the surgical process is compared with the image of the standard surgical operation path. After determining the path position of the current surgery, the image navigation of the subsequent standard surgical operation path of the patient is displayed. Therefore, during the process of accumulating experience, doctors can perform surgeries according to the image navigation of the standard surgical operation path. By forming the standard operation path specification of the successful surgical process of other doctors and learning the surgical operations of this surgical position and surgical type, the dependence on surgical clinical experience is reduced. The surgeries performed by young doctors are standardized by the standard path of this surgical type and surgical site. For example, the cutting position and angle of the surgery are standardized, reducing problems such as unnecessary skin and tissue injuries to patients, and improving aspects such as the postoperative recovery and postoperative quality of life of patients. At the same time, since the longer the operation time, the greater the harm to the patient and the longer the recovery time, the operation time will be minimized as much as possible while ensuring the surgical effect. Therefore, doctors can standardize surgical operations according to the standard surgical path, reduce the surgical thinking time, shorten the operation time, and reduce the pain of patients. More deeply, the system dynamically matches surgical data with real-time images, transforming the traditional intraocular operation that relies on the experience of doctors into a standardized navigation process, significantly reducing the risk of instrument deviation caused by the slight movement of the eyeball. Through the superimposed display of the three-dimensional virtual path and the real-time image, the surgeon can intuitively master the spatial relationship between the instrument and the target area, avoiding misoperations caused by visual errors, especially applicable to high-precision surgical scenarios such as corneal transplantation and glaucoma drainage valve implantation. The real-time tracking and compensation mechanism of the system for eye movement can actively adapt to the unconscious displacement or physiological tremor of the patient during the operation, ensuring the continuous accuracy of the navigation path and reducing the time-consuming operation of repeatedly adjusting the instrument position during the operation. In addition, the multi-level warning mechanism (visual prompt → interface simplification → forced pause) can effectively intercept potential operation mistakes and improve the surgical safety. By integrating preoperative image data and intraoperative dynamic information, the system further optimizes the coherence of surgical planning and execution, shortens the learning curve of the surgeon, and provides technical support for the standardized promotion of complex ophthalmic surgeries.

[0085] In an embodiment of the present application, it further includes a physician tracking module that continuously acquires the eye image of the physician through an image sensor to obtain the fixation point coordinates of the physician, calculates the ratio of the coverage area between the fixation point coordinates and the current navigation target area. If the average ratio within a unit time is lower than the set threshold, a visual warning signal is triggered.

[0086] It should be noted that the image sensor uses a global shutter CMOS (such as IMX585) to capture the eye movements of the physician at a sampling rate of 200 Hz, extracts the pupil center coordinates through a convolutional neural network (CNN), and calculates the three-dimensional spatial position of the fixation point by combining the corneal reflection spots. The calculation of the coverage area ratio uses the convex hull algorithm, dynamically analyzes the proportion of the spatial overlap area between the fixation point and the target area boundary, and sets the threshold to be dynamically adjusted according to the surgical type (such as 85% for cataract surgery and 92% for glaucoma surgery). The visual alarm signal includes the interface color gradient (yellow → red), the target area contour flashing, and the simplification of the operation interface (only the core navigation information is retained) to avoid interfering with the physician's attention;

[0087] In this embodiment, the physician tracking module solves the risk of attention deviation caused by the surgeon's distraction, visual fatigue, or limited operating vision in traditional surgeries by real-time quantifying the spatial correlation between the surgeon's fixation point and the target area. This module can actively identify the abnormal state of the surgeon's line of sight deviating from the key operation area, and intervene in a timely manner through a hierarchical alarm mechanism to reduce the probability of instrument misoperation caused by visual misjudgment. Combining with the dynamic path correction function of the navigation system, it further optimizes the human-machine cooperation efficiency and ensures the precise synchronization of the surgeon's attention and the instrument movement trajectory. Experiments show that this module can significantly improve the operation stability of high-precision surgeries (such as macular hole repair and retinal vascular anastomosis). Especially in long and complex surgeries, by continuously monitoring the surgeon's visual focus, it helps to maintain the best operation state, reduce the surgical failure caused by incorrect operations, and reduce the risk of postoperative complications caused by human factors.

[0088] In an embodiment of the present application, the output data of the patient tracking module and the physician tracking module are input into the eye movement fusion unit, and a comprehensive compensation amount is generated through the eye movement fusion unit, specifically including:

[0089] Establish a first transformation matrix T1 according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path;

[0090] Establish a second transformation matrix T2 according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view;

[0091] Obtain the three-dimensional coordinate system correction parameters by weighted fusion of T1 and T2 to generate a comprehensive compensation amount.

[0092] It should be noted that the first transformation matrix T1 is calculated based on the boundary data of the target area segmented from the preoperative OCT image and the displacement of the limbus feature points collected in real time during the operation. The rotation matrix is solved by the quaternion method and the target area offset is calibrated in real time. The second transformation matrix T2 is based on the spatial mapping relationship between the physician's fixation point coordinates and the optical axis of the microscope, and compensates for the visual field offset caused by the slight movement of the physician's head through the optical flow method (error ≤ 0.1 pixel). During weighted fusion, the weight coefficient of T1 is dynamically adjusted according to the movement amplitude of the patient's eyeball (for example, when the displacement > 0.1 mm, the weight is increased to 0.7), and the weight coefficient of T2 is set in combination with the magnification of the microscope (for example, the weight is 0.3 under a 40x microscope).

[0093] In this embodiment, by fusing the dual visual dynamic data of the patient and the physician, the problem of mismatch between the instrument path and the operation visual field caused by the traditional navigation system relying only on single-dimensional compensation is solved. In the traditional method, the compensation for the slight movement of the patient's eyeball may cause an instrument path deviation due to the lag in the physician's visual field adjustment. However, in this solution, by fusing the spatial transformation parameters of the two in real time, the collaborative correction of the instrument path and the microscope field of view is realized. For example, when the patient's eyeball undergoes a horizontal displacement, T1 drives the instrument path compensation, and at the same time, T2 synchronously adjusts the center position of the microscope field of view to avoid the operator misjudging the relative position between the instrument and the target area due to the visual field offset. The risk of misoperation caused by visual dislocation is significantly reduced. In addition, the dynamic optimization of the three-dimensional coordinate correction parameters further enhances the adaptability of the system to complex surgical scenarios (such as retinal vascular anastomosis), ensuring the coherence and stability of high-precision operations.

[0094] In an embodiment of the present application, the physician tracking module further includes a status monitoring sub-module, which increases the navigation prompt intensity when an abnormal status of the physician is detected based on the physician's eye image; the abnormal status includes tension and fatigue;

[0095] For the abnormal status of tension, the change rate of the physician's pupil diameter is detected through the eye image. When the pupil diameter expands beyond the threshold within a unit time, it is determined as the tension status;

[0096] For the abnormal status of fatigue, the contraction decline rate k1 of the levator palpebrae superioris muscle, the pupil diameter increase rate k2, the pupil light reflex speed delay rate k3 are detected through the eye image, and the physician's pulse is detected to obtain the heart rate variability decline rate k4 of the physician. When k1, k2, k3, and k4 reach the threshold within a unit time through weighted fusion, it is determined as the fatigue status.

[0097] It should be noted that the pupil diameter change rate is captured by a global shutter CMOS sampled at 200 Hz. Gaussian filtering is used to eliminate blinking interference, and the dilation threshold is set at 15% of the baseline value. In fatigue determination, the weight coefficients of k1 - k4 are 0.3, 0.25, 0.25, and 0.2 respectively, and the fusion threshold is dynamically optimized through a support vector machine (SVM) classification model. Navigation prompt enhancement includes: increasing the blinking frequency of the target area contour to 5 Hz, expanding the warning area of the instrument path by 20%, and switching the operation interface to a high - contrast mode (red - green color scheme).

[0098] In this embodiment, for the monitoring of the physician's state, when the physician's state is abnormal, through dynamically enhanced visual guidance, it helps the physician become aware and quickly recover the state; furthermore, through the fusion analysis of multi - modal physiological parameters, it solves the problem of misjudgment caused by traditional intraoperative monitoring relying only on a single indicator (such as operation duration). For example, during a long - term operation, the physician may have a slight relaxation of the levator palpebrae superioris muscle (k1 increases), but if the pupil light reflex is normal (k3 does not exceed the standard), the system will not misjudge it as a fatigue state, avoiding interference from invalid alarms to the surgical process. When tension or fatigue is detected, the enhanced navigation prompt can actively correct the risk of instrument deviation caused by the physician's distracted attention. It can effectively reduce the instrument path deviation caused by the abnormal state of the operator, especially in high - risk surgical stages (such as internal limiting membrane peeling for macular hole). Through dynamically enhanced visual guidance, it helps the physician quickly restore the operation accuracy. In addition, combined with heart rate variability analysis (k4), the system can identify the early stage of cumulative fatigue, trigger preventive prompts before key surgical nodes, and optimize the initiative and safety of human - machine cooperation.

[0099] In an embodiment of the present application, it further includes a second display module, and the second display module performs the following operations:

[0100] Generate a step - by - step flowchart according to the pre - operative planned surgical path, highlight the current surgical stage in real - time, and mark the estimated time for the remaining steps;

[0101] Based on a convolutional neural network, predict the type of instrument required for the subsequent steps, and dynamically display a tool preparation list, which includes instrument specification parameters and disinfection status identification;

[0102] When it is detected that the completion degree of the current step reaches the threshold, trigger a prompt signal to enable the assistant to complete the loading and positioning of the next - stage instrument in advance;

[0103] In an emergency operation scenario, including sudden bleeding, automatically switch to a warning interface, and synchronously display the location of the emergency tool kit and the usage priority guidance.

[0104] It should be noted that for the generation of the step flow chart, a temporal segmentation network (TSN) is used to parse the preoperative three-dimensional imaging data, and the surgical path is decomposed into quantifiable stage nodes (such as incision positioning, tissue separation, hemostasis operation, etc.). The flow chart supports touch zooming and perspective switching; the remaining time estimation is based on the instrument operation duration database of historical surgical cases and is dynamically corrected in combination with the current operator's operation speed (collected through the instrument movement trajectory sensor), with the error controlled within ±15 seconds; for instrument prediction and list management, the surgical video frames and corresponding instrument usage records in the convolutional neural network (CNN) training dataset are used. The input layer fuses the current operative field image features (SIFT key points) and the status of the stage flow chart, and the output layer predicts the confidence level of the instrument type (threshold ≥ 0.85); the disinfection status indicator is linked with the central disinfection system in the operating room through an RFID chip to update the sterilization expiration date and usage times of the instrument in real time; the collaborative prompt mechanism is set differently according to the surgical type based on the completion threshold (such as 95% for the phacoemulsification stage of cataract and 90% for the vitrectomy stage), and the coincidence degree between the instrument movement trajectory in the operative field and the standard path is calculated through the optical flow method. The prompt signals include the color change of the LED ring light strip (blue → yellow) in the instrument preparation area and the vibration of the tactile feedback device to avoid acoustic interference; for emergency interface switching, sudden bleeding is jointly determined by the sudden change of the blood flow signal (flow velocity > 5 mm / s) and the hemoglobin concentration threshold in the intraoperative OCT image, and the trigger response delay < 0.3 seconds; the position of the emergency tool kit is marked in the operator's microscope field of view through AR projection (guided by a red arrow), and the usage priority is automatically sorted according to the anatomical risk level of the bleeding site.

[0105] In this implementation, the technical problems solved include that the traditional surgical navigation system relies on the surgeon's experience to judge the timing of instrument switching, which is prone to interruption of operation due to preparation delays. This solution achieves seamless connection of instrument flow through intelligent prediction and collaborative prompts; in emergency scenarios, the traditional system lacks active emergency guidance, and the surgeon needs to distract himself to recall the treatment process. The automatic switching mechanism of this module projects key information (such as hemostatic forceps specifications and gelatin sponge storage location) directly into the operating field of view, shortening the emergency response cycle. The technical effects achieved include operation standardization, highlighting and time estimation functions of flowcharts, breaking down complex surgeries into controllable units, which is especially suitable for multi-stage surgeries such as glaucoma drainage valve implantation, reducing the cognitive load of new surgeons; resource optimization: dynamic tool lists reduce the probability of instrument misselection (such as avoiding the use of 5mm ultrasonic scalpels in 3mm blood vessel closure scenarios), and disinfection status identification blocks the risk of cross-infection; safety improvement: the emergency interface improves the efficiency of bleeding control through the collaboration of three-dimensional spatial annotation (such as projecting high-frequency electrocoagulation to the 10 o'clock position of the surgical field) and voice guidance ("preferentially use 4-0 absorbable sutures"); human-machine collaboration, multimodal feedback of prompt signals (visual + tactile) and microscope AR overlay display, realize closed-loop optimization of the "hand-eye-brain" operation link, and reduce the risk of instrument slippage caused by the surgeon's distraction.

[0106] In one embodiment of the present application, it also includes an assistant tracking module, which uses an image sensor to collect the assistant's eye images in real time, and collects the assistant's pupil focus coordinates and scanning frequency in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the scanning path entropy value is lower than a threshold, it is determined to be a distracted state; an LSTM network is used to analyze the match between the eye movement pattern and the current surgical stage, and an attention score is generated, and a reminder is triggered when the score is lower than the threshold.

[0107] It should be noted that the image sensor uses a global shutter CMOS (such as IMX585) to capture eye movement at a sampling rate of 240Hz, segment the iris area through the U-Net network, and calculate the three-dimensional coordinates of the pupil focus (accuracy ±0.1mm) in combination with the corneal reflection spot;

[0108] The scan path entropy value is calculated using the Shannon entropy formula:

[0109]

[0110] in pi The entropy value is lower than the threshold, which indicates that the gaze trajectory is too concentrated or disordered.

[0111] The input layer of the LSTM network contains eye movement features (gaze duration, scanning speed), surgical stage codes (such as the incision making stage is coded as 01) and instrument usage records, and the output layer generates an attention score of 0-100.

[0112] In this embodiment, the technical problems to be solved are as follows: In traditional surgeries, the assistant's attention is distracted (such as visual fatigue caused by long-term operation), which is likely to lead to delays in instrument preparation. The risk is exacerbated especially in scenarios of multi-instrument switching (such as more than 20 kinds of instruments are continuously used in coronary artery bypass grafting). Manual supervision has a lag and cannot quantitatively evaluate the cognitive load and task matching degree of the assistant in real time. The achieved technical effects include dynamic cognitive assessment: quantifying the visual attention distribution state through the eye movement entropy value, and combining the time series modeling ability of the LSTM network to accurately identify attention shifts caused by fatigue, distraction or information overload (such as the error operation rate will increase when the entropy value is less than 0.8); a hierarchical intervention mechanism. When the score is lower than the threshold, the system triggers the following in sequence:

[0113] First-level reminder: The edge of the AR glasses gradually turns red (frequency 2Hz);

[0114] Second-level reminder: Vibration tactile feedback (intensity 0.3G);

[0115] Third-level intervention: Automatically pause the instrument conveyor belt and send an alarm to the interface of the surgeon in charge;

[0116] Surgical stage adaptation: The LSTM network dynamically adjusts the scoring weights according to different surgical stages (such as paying more attention to the focus stability during the suture stage), so that the intervention strategy is accurately matched with the key nodes of the surgery.

[0117] In an embodiment of the present application, the assistant tracking module further includes an instrument guidance sub-module, which real-time tracks the spatial coordinates of the instrument to be prepared through a positioning chip and compares them with the preset positions of the second display module; when the deviation distance of the instrument is greater than the preset value, a highlighted arrow is superimposed in the assistant's AR glasses for guidance, and the specification parameters of the target instrument are synchronously prompted by voice; at the same time, a feature vector library is established for high-frequency mis-taken instruments, and the convolutional network is used to identify abnormal grasping actions and give early warnings.

[0118] It should be noted that the positioning system adopts UWB+IMU integrated positioning (accuracy ±1cm), and each instrument is embedded with a passive RFID tag (such as EPC Gen2) and is calibrated in real time with the surgical table coordinate system; the AR guidance interface uses the SLAM algorithm to construct spatial anchor points, the dynamic rendering delay of the arrow is <10ms, and the voice prompt is generated by the TTS engine (speech rate 3 characters / second); the mis-taken feature library includes three-dimensional features of the instrument shape (extracted by ResNet50), grasping posture (hand key points of OpenPose) and movement trajectory (encoded by LSTM), and the misjudgment rate is <2%.

[0119] In this embodiment, the technical problems to be solved include the large variety of instruments in complex surgeries (such as more than 150 types in neurosurgery), where traditional label recognition is easily affected by bloodstain occlusion, leading to the risk of incorrect instrument selection (such as misidentifying bipolar forceps as monopolar); and incorrect instrument placement by the assistant due to lack of experience, etc., which prolongs the surgical preparation time.

[0120] The achieved technical effects include spatial-semantic collaborative guidance, with AR arrows dynamically marking the deviation direction (such as a red arrow deflected 30° horizontally); voice synchronization emphasizing key parameters (such as "Attention: This instrument is made of titanium alloy and ultrasonic cleaning is prohibited"); pre-judgment of incorrect instrument selection behavior, analyzed through grasping actions: detecting the contact area between the hand and the instrument through a convolutional network (such as incorrectly holding the head of the aspirator instead of the handle); trajectory anomaly warning, triggering a fall prevention prompt when the acceleration mutation of the instrument movement (such as >2m / s²); self-learning optimization mechanism, with the feature library automatically clustering and analyzing every week to identify new incorrect instrument selection patterns (such as confusion caused by similar packaging of a certain batch of instruments).

[0121] In an embodiment of the present application, it further includes a physician database module that stores the historical surgical records of each physician, including the time-consuming distribution of surgical stages, the fluctuation curve of physiological parameters, and the instrument usage correction amount; extracting feature patterns through an LSTM network, and reversely optimizing the parameters of the prediction model based on the postoperative quality assessment report to establish an individualized stress-fatigue time window prediction model, and adjusting the navigation parameters in advance for the high-frequency warning stage.

[0122] It should be noted that for data storage and modeling, the standard deviation of the duration of each operation node (such as incision making, hemostasis, suture) is recorded through the time-consuming distribution of surgical stages, and the time axes of different surgical cases are aligned through the dynamic time warping algorithm; the physiological parameter fluctuation curve integrates the change rate of pupil diameter, the decline gradient of heart rate variability (HRV), and the hand tremor frequency, with a sampling rate of 100Hz, and the data is processed by differential privacy (ε = 0.6); the input layer of the LSTM network includes sequential physiological data and instrument correction amount (such as the number of path offsets, average compensation distance), and the output layer predicts the fatigue accumulation risk index (0 - 100 points) within the next 10 minutes; the stress-fatigue time window prediction model dynamically adjusts the warning threshold according to individual operation habits. For example, for physicians who are accustomed to rapid operations, the pupil dilation rate threshold is increased by 20%.

[0123] In this embodiment, the technical problems to be solved include that traditional navigation systems use fixed threshold warnings and cannot adapt to the operation habits and physiological characteristics of different physicians, resulting in a high false alarm rate or missed reports of key risks.

[0124] There is a lack of forward-looking prediction for sudden fatigue or stress during surgery, and it can only respond passively after an abnormality occurs.

[0125] The technical effect achieved is personalized risk prediction. By analyzing individual characteristics in historical surgical data (e.g., a physician's HRV decline rate significantly accelerates at 45-60 minutes into an operation), the system can trigger preparatory prompts 2-3 minutes in advance (e.g., reducing interface information density, starting instrument speed limit), to avoid loss of control in emergencies.

[0126] Dynamic parameter adjustment: During predicted high-risk periods (such as fatigue accumulation peaks), the system automatically enhances navigation prompts (such as increasing the virtual path width by 50% and extending the voice command interval) to match the physician's real-time cognitive load level.

[0127] In one embodiment of the present application, the second display module also includes an emergency exemption submodule, which automatically shields the conventional warning logic and switches to augmented reality mode in the event of sudden increase in intraocular pressure or bleeding: superimposes a 3D vascular pressure distribution heat map on the surgical field; and rearranges the instrument priority: hides non-critical tool prompts, and highlights the location of the hemostatic device.

[0128] It should be noted that event judgment and response are based on the sudden increase in intraocular pressure, which is determined by the angle closure degree (>75%) of the intraoperative OCT image combined with the intraocular pressure sensor data (>30 mmHg), with a response delay of <0.5 seconds. The 3D vascular pressure heat map is generated based on the fusion of preoperative CTA images and intraoperative blood flow Doppler data, and the pressure is displayed in grades (blue: <15 mmHg; red: >30 mmHg).

[0129] The device priority management includes AR highlight box annotation (gold frame + pulse animation) of hemostatic devices (such as bipolar electrocoagulation forceps and gelatin sponge), and the transparency of non-critical tools (such as microscissors and forceps) is increased to 80%. The device position guidance uses SLAM algorithm for real-time positioning, combined with voice prompts (such as "hemostatic forceps is located at position 2 in area C of the instrument table").

[0130] In this embodiment, the technical problems solved include that the traditional system continues to push routine warnings (such as instrument path deviation prompts) in emergency scenarios, interfering the operator's focus on core issues; in the event of sudden bleeding, the operator needs to manually retrieve the location of the hemostatic tool, delaying the golden treatment time.

[0131] The heat map intuitively displays the distribution of vascular pressure, helping the surgeon to quickly locate the source of bleeding (such as the rupture point of the posterior ciliary artery) and reduce secondary damage caused by blind exploration; hiding non-critical tool prompts can reduce the complexity of interface information and allow the surgeon to focus on hemostasis operations (such as electrocoagulation forceps power adjustment and gelatin sponge filling); in emergency mode, the system automatically removes the device movement speed limit to support rapid intervention operations.

[0132] In a second aspect of the present application, a real-time navigation method for ophthalmic surgery based on dynamic visual field tracking is provided.Figure 4 As shown, it includes:

[0133] According to the surgical type and location of the patient, match the operation data closest to the current surgery from the database, disassemble the steps of the operation data, and generate the standard operation navigation for the current surgery;

[0134] Compare the real-time image of the surgical process with the image of the standard operation navigation in real time to determine the current surgical step, and at the same time display the image navigation for the subsequent patient surgery;

[0135] Collect the limbal feature points on the surface of the patient's eyeball in real time through a near-infrared light source and an image sensor, calculate the three-dimensional motion trajectory, and combine the target area boundary data segmented from the preoperative OCT image and the intraoperative navigation coordinate system to generate a spatial mapping matrix between the eyeball motion trajectory and the target area;

[0136] Dynamically correct the virtual navigation path of the surgical instrument according to the real-time eyeball motion trajectory, adjust the relative position between the corrected instrument path and the target area boundary, so that the surgical instrument is aligned with the target area.

[0137] In an embodiment of the present application, the eye image of the physician is collected in real time through an image sensor to obtain the gaze point coordinates of the physician, and the ratio of the coverage area of the gaze point coordinates to the current navigation target area is calculated. If the average ratio within a unit time is lower than the set threshold, a visual alarm signal is triggered.

[0138] In an embodiment of the present application, a comprehensive compensation amount is generated through the patient's eye movement data and the physician's eye movement data, which specifically includes:

[0139] Establish a first transformation matrix T1 according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path;

[0140] Establish a second transformation matrix T2 according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view;

[0141] Obtain the three-dimensional coordinate system correction parameters through weighted fusion of T1 and T2 to generate a comprehensive compensation amount.

[0142] In an embodiment of the present application, when an abnormal state of the physician is detected according to the physician's eye image, the navigation prompt intensity is increased; the abnormal states include tension and fatigue;

[0143] For the abnormal state of tension, the change rate of the pupil diameter of the physician is detected through the eye image. When the pupil diameter expands beyond the threshold within a unit time, it is determined as a tense state;

[0144] The abnormal state of fatigue is detected by the contraction decline rate k1 of the levator palpebrae superioris muscle, the pupil diameter increase rate k2, the pupil light reflex speed delay rate k3 in the eye image, and the physician's pulse is detected to obtain the physician's heart rate variability decline rate k4. When the weighted fusion of k1, k2, k3, and k4 reaches the threshold within the unit time, it is determined as the fatigue state.

[0145] In an embodiment of the present application, a step flowchart is generated according to the pre-operative planned surgical path, the current surgical stage is highlighted in real time, and the estimated time of the remaining steps is marked.

[0146] Based on the convolutional neural network, the type of instrument required for the subsequent steps is predicted, and the tool preparation list is dynamically displayed. The list includes the instrument specification parameters and the disinfection status identification.

[0147] When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading and positioning of the instruments in the next stage in advance.

[0148] In an emergency operation scenario, including sudden bleeding, it automatically switches to the warning interface, and synchronously displays the location of the emergency tool kit and the usage priority guide.

[0149] In an embodiment of the present application, the eye images of the assistant are collected in real time through an image sensor, and the pupil focus coordinates and the scanning frequency of the assistant are collected in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than the preset time or the entropy value of the scanning path is lower than the threshold, it is determined as the distracted state; the LSTM network is used to analyze the matching degree between the eye movement pattern and the current surgical stage, and an attention score is generated. When the score is lower than the threshold, a reminder is triggered.

[0150] In an embodiment of the present application, the spatial coordinates of the instrument to be prepared are tracked in real time through a positioning chip and compared with the preset position of the second display module; when the deviation distance of the instrument is greater than the preset value, a highlighted arrow guide is superimposed in the assistant's AR glasses, and the target instrument specification parameters are synchronously prompted by voice; at the same time, a feature vector library is established for the high-frequency mis-taken instruments, and the abnormal grasping actions are identified through a convolutional network to issue an early warning.

[0151] In an embodiment of the present application, the historical surgical records of each physician are stored, including the time-consuming distribution of the surgical stage, the physiological parameter fluctuation curve, and the instrument usage correction amount; the LSTM network is used to extract the feature patterns, and the prediction model parameters are reversely optimized based on the postoperative quality assessment report to establish an individualized stress-fatigue time window prediction model, and the navigation parameters are adjusted in advance for the high-frequency warning stage.

[0152] In one embodiment of the present application, in the event of sudden intraocular pressure elevation or bleeding, the conventional warning logic is automatically blocked and switched to the augmented reality mode: a 3D vascular pressure distribution heat map is superimposed on the surgical field; and the instrument priority is rearranged: non-critical tooltips are hidden, and the positions of hemostatic instruments are highlighted.

[0153] It should be noted that the specific implementation of the method for a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking in the embodiments of the present application refers to the specific implementation of the real-time navigation system for ophthalmic surgery based on dynamic visual field tracking proposed in the first aspect of the embodiments of the present application as described above, and will not be elaborated here.

[0154] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that an article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such article or device. Without more limitations, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the article or device including the elements.

[0155] The above provides a detailed introduction to a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking. Specific examples are used in this text to elaborate on the principle and implementation of the present application. The description of the above embodiments is only used to help understand the real-time navigation system for ophthalmic surgery based on dynamic visual field tracking of the present application and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An ophthalmic surgery real-time navigation system based on dynamic vision tracking, characterized in that, Comprising: A navigation generation module, which matches the operation data closest to the current surgery from the database according to the patient's surgery type and location, disassembles the steps of the operation data, and generates a standard operation navigation for the current surgery. The database contains three-dimensional image data of historical surgery cases, instrument movement trajectories, and key operation node parameters, and matches the patient's corneal curvature and anterior chamber depth biometrics through the dynamic time warping algorithm; A comparison and display module, which compares the real-time image of the surgery process with the image of the standard operation navigation in real time to determine the current surgery step, and at the same time displays the image navigation for the subsequent surgery of the patient. The subsequent navigation image is displayed on the microscope eyepiece in a semi-transparent overlay form, including the expected path of the instrument, the target area boundary, and the safety distance prompt; A patient tracking module, which uses a near-infrared light source and an image sensor to collect the corneal limbus feature points on the patient's eye surface in real time, calculates the three-dimensional movement trajectory, and combines the target area boundary data segmented by the preoperative OCT image and the intraoperative navigation coordinate system to generate a spatial mapping matrix between the eye movement trajectory and the target area; An offset compensation module, which dynamically corrects the virtual navigation path of the surgical instrument according to the real-time eye movement trajectory, adjusts the relative position between the corrected instrument path and the target area boundary, so that the surgical instrument is aligned with the target area. When the detected eye displacement > 0.03mm, the robotic arm is driven to adjust the instrument movement trajectory, and the coincidence degree between the corrected path and the target area boundary is displayed on the operation interface in real time. If the coincidence degree < 90% for 2 consecutive seconds, an audible and visual alarm is activated; A physician tracking module, which uses an image sensor to collect the physician's eye image in real time to obtain the fixation point coordinates of the physician, and calculates the ratio of the coverage area of the fixation point coordinates to the current navigation target area. If the average ratio within a unit time is lower than the set threshold, a visual alarm signal is triggered; The output data of the patient tracking module and the physician tracking module are input into the eye movement fusion unit, and a comprehensive compensation amount is generated through the eye movement fusion unit, specifically including: Establishing a first transformation matrix T1 according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path; Establishing a second transformation matrix T2 according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view; obtaining the three-dimensional coordinate system correction parameters through weighted fusion of T1 and T2 to generate a comprehensive compensation amount; when performing weighted fusion, the weight coefficient of T1 is dynamically adjusted according to the amplitude of the patient's eye movement, and the weight coefficient of T2 is set in combination with the microscope magnification.

2. The real-time navigation system for ophthalmic surgery based on dynamic vision tracking according to claim 1, characterized in that, The physician tracking module further includes a status monitoring sub-module, which increases the navigation prompt intensity when an abnormal status of the physician is detected according to the physician's eye image; the abnormal status includes tension and fatigue; For the abnormal status of tension, the change rate of the physician's pupil diameter is detected through the eye image. When the pupil diameter expands beyond the threshold within a unit time, it is determined as a tense state; For the abnormal status of fatigue, the contraction and decline rate k1 of the levator palpebrae superioris muscle, the pupil diameter increase rate k2, the pupil light reflex speed delay rate k3 are detected through the eye image, and the physician's pulse is detected to obtain the physician's heart rate variability decline rate k4. When the weighted fusion of k1, k2, k3 and k4 reaches the threshold within a unit time, it is determined as a fatigue state.

3. An ophthalmic surgery real-time navigation system based on dynamic visual field tracking according to any one of claims 1-2, characterized in that, It further includes a second display module, which performs the following operations: Generate a step flow chart according to the pre-operative planned surgical path, highlight the current surgical stage in real time, and mark the estimated time of the remaining steps; Based on a convolutional neural network, predict the types of instruments required for the subsequent steps, and dynamically display a tool preparation list, which includes instrument specification parameters and disinfection status identifiers; When it is detected that the completion degree of the current step reaches the threshold, trigger a prompt signal to enable the assistant to complete the loading and positioning of the instruments for the next stage in advance; In an emergency operation scenario, including sudden bleeding, automatically switch to a warning interface, and synchronously display the location of the emergency tool kit and the usage priority guide.

4. The real-time navigation system for ophthalmic surgery based on dynamic vision tracking according to claim 3, wherein, It further includes an assistant tracking module, which uses an image sensor to collect the assistant's eye images in real time, and collect the assistant's pupil focus coordinates and scanning frequency in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the entropy value of the scanning path is lower than the threshold, it is determined as a distracted state; use an LSTM network to analyze the matching degree between the eye movement pattern and the current surgical stage, generate an attention score, and trigger a reminder when the score is lower than the threshold.

5. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking according to claim 4, wherein, The assistant tracking module further includes an instrument guidance sub-module, which uses a positioning chip to track the spatial coordinates of the instrument to be prepared in real time and compare them with the preset position of the second display module; when the deviation distance of the instrument is greater than the preset value, superimpose a highlighted arrow in the assistant's AR glasses to guide, and synchronously voice the target instrument specification parameters; at the same time, establish a feature vector library for frequently mis-taken instruments, identify abnormal grasping actions through a convolutional network, and issue an early warning in advance.

6. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking according to claim 5, wherein, It further includes a physician database module, which stores the historical surgical records of each physician, including the time-consuming distribution of surgical stages, the fluctuation curve of physiological parameters, and the instrument usage correction amount; extract the feature patterns through an LSTM network, reverse-optimize the prediction model parameters based on the postoperative quality assessment report, establish an individualized stress-fatigue time window prediction model, and adjust the navigation parameters in advance for the high-frequency warning stage.

7. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking according to claim 3, wherein, The second display module further includes an emergency exemption sub-module, which automatically shields the conventional warning logic and switches to the augmented reality mode in the event of sudden intraocular pressure elevation or bleeding: superimpose a 3D vascular pressure distribution heat map on the surgical field; and re-arrange the instrument priorities: hide the non-critical tool tips and highlight the location of the hemostatic instruments.

8. A real-time navigation method for ophthalmic surgery based on dynamic visual field tracking, characterized in that, It includes: According to the patient's surgical type and location, match the operation data closest to the current operation from the database, disassemble the steps of the operation data, and generate a standard operation navigation for the current operation. The database includes the three-dimensional image data of historical surgical cases, the instrument movement trajectories, and the key operation node parameters, and matches the patient's corneal curvature and anterior chamber depth biometric features through the dynamic time warping algorithm; Compare the real-time image of the surgical process with the image of the standard operation navigation in real time to determine the current step of the operation, and at the same time display the image navigation for the subsequent operation of the patient. The subsequent navigation images are displayed in a semi-transparent overlay form on the microscope eyepiece, including the expected path of the instrument, the target area boundary, and the safety distance prompt; The corneal limbus feature points on the surface of the patient's eyeball are collected in real time by a near-infrared light source and an image sensor, and the three-dimensional motion trajectory is calculated. Combining the target area boundary data segmented from the preoperative OCT image and the intraoperative navigation coordinate system, a spatial mapping matrix between the eyeball motion trajectory and the target area is generated; According to the real-time eyeball motion trajectory, the virtual navigation path of the surgical instrument is dynamically corrected, and the relative position between the adjusted instrument path and the target area boundary is adjusted to align the surgical instrument with the target area. When the detected eyeball displacement > 0.03 mm, the robotic arm is driven to adjust the instrument motion trajectory, and the coincidence degree between the corrected path and the target area boundary is displayed on the operation interface in real time. If the coincidence degree < 90% lasts for 2 seconds, an audible and visual alarm is activated.

Images