Ophthalmologic operation real-time navigation system and method based on dynamic visual field tracking

Abstract

The invention discloses an ophthalmologic operation real-time navigation system and method based on dynamic visual field tracking, and relates to the technical field of medical systems. Comprising the following steps: matching operation data closest to a current operation from a database, and generating standard operation navigation of the current operation; the real-time image of the operation process is compared with the image of the standard operation navigation in real time, the step of the current operation is determined, and meanwhile subsequent image navigation of the operation of the patient is displayed; generating a spatial mapping matrix between an eyeball movement track and a target region by combining target region boundary data of preoperative OCT image segmentation and an intra-operative navigation coordinate system; the virtual navigation path of the surgical instrument is dynamically corrected according to the real-time eyeball movement track, so that the surgical instrument is aligned with the target area, the spatial relation between the instrument and the target area can be visually mastered through the overlapping display of the three-dimensional virtual path and the real-time image, unconscious displacement or physiological tremor of a patient in an operation is actively adapted, and the operation accuracy is improved. And the continuous accuracy of the navigation path is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of medical systems, and in particular to a real-time navigation system and method for ophthalmic surgery based on dynamic visual field tracking. Background Art

[0002] With the continuous development of modern medicine, especially the progress in the field of ophthalmic surgery, how to improve the accuracy and safety of surgery in ophthalmic surgery has become an important research direction. In ophthalmic surgery, especially precision surgery such as retinal surgery and cataract surgery, the patient's eyes may still move slightly after anesthesia, so the doctor's operation accuracy and experience are extremely high. With the development of technology, surgical navigation technology and surgical assistance systems have improved the success rate of surgery to a certain extent, but the existing ophthalmic surgical navigation systems mostly rely on preoperative images and video data for auxiliary operations. Although these technologies can achieve the positioning of the surgical area to a certain extent, their accuracy and dynamic adaptability are still insufficient. For example, in the prior art, patent CN202010516777.9 (ophthalmic surgery navigation system) aims to assist in adjusting the positioning of surgical instruments through real-time eye dynamic monitoring. However, the patent only proposes dynamic tracking based on feature points on the surface of the eyeball. Although it can provide certain auxiliary effects for surgery, its tracking accuracy and dynamic adaptability are relatively limited. Specific problems include: relying on feature points on the surface of the eyeball for tracking, but lacking comprehensive calculation and processing of the eyeball's motion trajectory in three-dimensional space, it is difficult to provide sufficiently fine eyeball movement data.

[0003] Therefore, the technical problem to be solved by the present application is: how to provide a real-time navigation system for ophthalmic surgery based on the three-dimensional dynamic motion trajectory of the patient's eyeball, combined with preoperative image data and surgical navigation coordinate system, which can track the dynamic field of view of the patient's eyeball and dynamically adjust the path of surgical instruments in real time, and improve surgical accuracy through a real-time feedback mechanism, thereby overcoming the problems of insufficient accuracy, poor dynamic adaptability, and real-time feedback lag in the prior art, ensuring the smooth progress of high-precision ophthalmic surgery, especially reducing the occurrence of errors during complex operations. Summary of the invention

[0004] The purpose of the present invention is to provide an ophthalmic surgery real-time navigation system and method based on dynamic visual field tracking, in view of the current lack of an auxiliary tool for real-time normative navigation of the doctor's operation according to the dynamic tracking of the patient's eyeball during the operation. By using a standard surgical operation path established in advance, during the operation, the images collected by the image acquisition module are continuously compared with the standard surgical operation path, and the image navigation of the next surgical operation can be displayed in real time on the comparison display module.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions: In a first aspect of the present application, a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking is provided, comprising: The navigation generation module matches the operation data closest to the current operation from the database according to the patient's operation type and location, breaks down the steps of the operation data, and generates the standard operation navigation for the current operation; The comparison display module compares the real-time image of the surgical process with the image of the standard operation navigation in real time to determine the current surgical step and display the image navigation for the patient's subsequent surgery; The patient tracking module collects the corneal limbus feature points on the surface of the patient's eyeball in real time through a near-infrared light source and an image sensor, calculates the three-dimensional motion trajectory, and combines the target area boundary data of the preoperative OCT image segmentation and the intraoperative navigation coordinate system to generate a spatial mapping matrix between the eyeball motion trajectory and the target area; The offset compensation module dynamically corrects the virtual navigation path of the surgical instrument according to the real-time eye movement trajectory, adjusts the relative position of the corrected instrument path and the target area boundary, and aligns the surgical instrument with the target area.

[0006] In one embodiment of the present application, it also includes a physician tracking module, which collects the physician's eye image in real time through an image sensor to obtain the physician's gaze point coordinates, and calculates the ratio of the gaze point coordinates to the coverage area of ​​the current navigation target area. If the average ratio per unit time is lower than the set threshold, a visual alarm signal is triggered.

[0007] In one embodiment of the present application, the output data of the patient tracking module and the physician tracking module are input into the eye movement fusion unit, and the eye movement fusion unit generates a comprehensive compensation amount, which specifically includes: A first transformation matrix T1 is established according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path; A second transformation matrix T2 is established according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view; The three-dimensional coordinate system correction parameters are obtained by weighted fusion of T1 and T2 to generate a comprehensive compensation value.

[0008] In one embodiment of the present application, the physician tracking module further includes a state monitoring submodule, which increases the intensity of the navigation prompt when an abnormal state of the physician is detected according to the physician's eye image; the abnormal state includes tension and fatigue; The abnormal state of tension is to detect the change rate of the doctor's pupil diameter through the eye image, and when the pupil diameter expands beyond a threshold value within a unit time, it is determined to be a state of tension; The abnormal state of fatigue is detected by the eye image to obtain the contraction decrease rate k1 of the levator palpebrae superioris muscle, the increase rate k2 of the pupil diameter, the delay rate k3 of the pupil light reflex speed, and the physician's pulse, and the physician's heart rate variability decrease rate k4 is obtained. When the threshold is reached within a unit time through weighted fusion of k1, k2, k3 and k4, it is determined to be a fatigue state.

[0009] In an embodiment of the present application, the device further includes a second display module, and the second display module performs the following operations: Generate a step flow chart based on the preoperatively planned surgical path, highlight the current surgical stage in real time, and mark the estimated time for the remaining steps; Predict the type of instrument required for subsequent steps based on a convolutional neural network and dynamically display a tool preparation list that includes instrument specifications and disinfection status indicators; When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading of the next stage of the instrument in advance; In emergency operation scenarios, including sudden bleeding, the system automatically switches to the warning interface, simultaneously displaying the location of the emergency tool kit and usage priority instructions.

[0010] In one embodiment of the present application, it also includes an assistant tracking module, which uses an image sensor to collect the assistant's eye images in real time, and collects the assistant's pupil focus coordinates and scanning frequency in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the scanning path entropy value is lower than a threshold, it is determined to be a distracted state; an LSTM network is used to analyze the match between the eye movement pattern and the current surgical stage, and an attention score is generated, and a reminder is triggered when the score is lower than the threshold.

[0011] In one embodiment of the present application, the assistant tracking module also includes an instrument guidance submodule, which tracks the spatial coordinates of the instrument to be prepared in real time through a positioning chip, and compares it with the preset position of the second display module; when the instrument offset distance is greater than a preset value, a highlighted arrow guidance is superimposed in the assistant AR glasses, and a synchronous voice prompt of the target instrument specification parameters is given; at the same time, a feature vector library is established for high-frequency mis-picked instruments, and abnormal grasping actions are identified through a convolutional network, and an early warning is issued.

[0012] In one embodiment of the present application, it also includes a physician database module, which stores the historical surgical records of each physician, including the time distribution of the surgical stages, the fluctuation curve of physiological parameters, and the correction amount of instrument use; extracts feature patterns through the LSTM network, reversely optimizes the prediction model parameters based on the postoperative quality assessment report, establishes an individualized tension-fatigue time window prediction model, and adjusts the navigation parameters in advance for the high-frequency warning stage.

[0013] In one embodiment of the present application, the second display module also includes an emergency exemption submodule, which automatically shields the conventional warning logic and switches to augmented reality mode in the event of sudden increase in intraocular pressure or bleeding: superimposes a 3D vascular pressure distribution heat map on the surgical field; and rearranges the instrument priority: hides non-critical tool prompts, and highlights the location of the hemostatic device.

[0014] In a second aspect of the present application, a real-time navigation method for ophthalmic surgery based on dynamic visual field tracking is provided, characterized in that it includes: According to the patient's surgery type and location, the operation data closest to the current surgery is matched from the database, the steps of the operation data are disassembled, and the standard operation navigation of the current surgery is generated; Compare the real-time images of the surgical process with the images of the standard operation navigation in real time to determine the current surgical step and display the image navigation for the patient's subsequent surgery; The near-infrared light source and image sensor are used to collect the feature points of the corneal limbus on the surface of the patient's eyeball in real time, and the three-dimensional motion trajectory is calculated. The target area boundary data of the preoperative OCT image segmentation and the intraoperative navigation coordinate system are combined to generate a spatial mapping matrix between the eyeball movement trajectory and the target area. The virtual navigation path of the surgical instrument is dynamically corrected according to the real-time eye movement trajectory, and the relative position of the corrected instrument path and the target area boundary is adjusted to align the surgical instrument with the target area.

[0015] In one embodiment of the present application, the doctor's eye image is collected in real time by an image sensor to obtain the coordinates of the doctor's gaze point, and the ratio of the gaze point coordinates to the coverage area of ​​the current navigation target area is calculated. If the average ratio per unit time is lower than the set threshold, a visual alarm signal is triggered.

[0016] In one embodiment of the present application, a comprehensive compensation amount is generated by using the patient's eye movement data and the physician's eye movement data, specifically including: A first transformation matrix T1 is established according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path; A second transformation matrix T2 is established according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view; The three-dimensional coordinate system correction parameters are obtained by weighted fusion of T1 and T2 to generate a comprehensive compensation value.

[0017] In one embodiment of the present application, when an abnormal state of the physician is detected based on the physician's eye image, the intensity of the navigation prompt is increased; the abnormal state includes tension and fatigue; The abnormal state of tension is to detect the change rate of the doctor's pupil diameter through the eye image, and when the pupil diameter expands beyond a threshold value within a unit time, it is determined to be a state of tension; The abnormal state of fatigue is detected by the eye image to obtain the contraction decrease rate k1 of the levator palpebrae superioris muscle, the increase rate k2 of the pupil diameter, the delay rate k3 of the pupil light reflex speed, and the physician's pulse, and the physician's heart rate variability decrease rate k4 is obtained. When the threshold is reached within a unit time through weighted fusion of k1, k2, k3 and k4, it is determined to be a fatigue state.

[0018] In one embodiment of the present application, a step flow chart is generated according to the surgical path planned before surgery, the current surgical stage is highlighted in real time, and the estimated time of the remaining steps is marked; Predict the type of instrument required for subsequent steps based on a convolutional neural network and dynamically display a tool preparation list that includes instrument specifications and disinfection status indicators; When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading of the next stage of the instrument in advance; In emergency operation scenarios, including sudden bleeding, the system automatically switches to the warning interface, simultaneously displaying the location of the emergency tool kit and usage priority instructions.

[0019] In one embodiment of the present application, an image sensor is used to collect the assistant's eye images in real time, and the assistant's pupil focus coordinates and scanning frequency are collected in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the scanning path entropy value is lower than a threshold, it is determined to be a distracted state; an LSTM network is used to analyze the match between the eye movement pattern and the current surgical stage, and an attention score is generated. When the score is lower than the threshold, a reminder is triggered.

[0020] In one embodiment of the present application, the spatial coordinates of the instrument to be prepared are tracked in real time by a positioning chip and compared with the preset position of the second display module; when the instrument offset distance is greater than the preset value, a highlighted arrow guide is superimposed in the assistant AR glasses, and a synchronous voice prompt is given for the target instrument specification parameters; at the same time, a feature vector library is established for high-frequency mis-picked instruments, and abnormal grasping actions are identified through a convolutional network, and an early warning is issued.

[0021] In one embodiment of the present application, the historical surgical records of each physician are stored, including the time distribution of the surgical stages, the fluctuation curve of physiological parameters, and the correction amount of instrument use; the feature patterns are extracted through the LSTM network, and the prediction model parameters are reversely optimized based on the postoperative quality assessment report, and an individualized tension-fatigue time window prediction model is established, and the navigation parameters are adjusted in advance for the high-frequency warning stage.

[0022] In one embodiment of the present application, in the event of sudden increase in intraocular pressure or bleeding, the conventional warning logic is automatically shielded and switched to augmented reality mode: a 3D vascular pressure distribution heat map is superimposed on the surgical field; and the instrument priority is rearranged: non-critical tool prompts are hidden, and the location of the hemostatic device is highlighted.

[0023] This application has the following beneficial effects: 1. Compare the real-time image of the surgical process with the image of the standard surgical operation path, determine the path position of the current surgery, and display the image navigation of the standard surgical operation path for subsequent patients. Therefore, in the process of accumulating experience, doctors can perform surgery according to the image navigation of the standard surgical operation path. Through the successful surgical processes of other doctors, the standard surgical operation path specifications are formed and the surgical operations of the surgical location and type are learned, reducing the dependence on clinical surgical experience. The operations performed by young doctors are regulated by the standard path of the surgical type and surgical site, such as standardizing the position and angle of the surgical incision, reducing unnecessary skin and tissue injuries to patients, and improving the postoperative recovery and quality of life of patients. At the same time, since the longer the operation time, the greater the harm to the patient and the longer the recovery time, the operation time will be minimized while ensuring the surgical effect. Therefore, doctors can standardize the surgical operation according to the standard surgical path, reduce the surgical thinking time, shorten the operation time, and reduce the patient's pain. At a deeper level, the system dynamically matches the surgical data with the real-time image, transforming the traditional intraocular operation that relies on the experience of the physician into a standardized navigation process, significantly reducing the risk of instrument deviation caused by micro-movement of the eyeball. Through the superposition display of three-dimensional virtual paths and real-time images, the surgeon can intuitively grasp the spatial relationship between the instrument and the target area, avoiding misoperation caused by visual errors. It is especially suitable for high-precision surgical scenarios such as corneal transplantation and glaucoma drainage valve implantation. The system's real-time tracking and compensation mechanism for eye movement can actively adapt to the patient's unconscious displacement or physiological tremor during surgery, ensure the continuous accuracy of the navigation path, and reduce the time-consuming operation of repeatedly adjusting the position of the instrument during surgery. In addition, the multi-level alarm mechanism (visual prompts → interface simplification → forced pause) can effectively intercept potential operational errors and improve surgical safety. By integrating preoperative imaging data and intraoperative dynamic information, the system further optimizes the consistency of surgical planning and execution, shortens the surgeon's learning curve, and provides technical support for the standardization and promotion of complex ophthalmic surgeries.

[0024] 2. The physician tracking module solves the risk of attention deviation caused by operator distraction, visual fatigue or limited operating field in traditional surgery by quantifying the spatial correlation between the operator's gaze point and the target area in real time. This module can actively identify the abnormal state of the operator's line of sight away from the key operating area, and intervene in time through a graded alarm mechanism to reduce the probability of instrument misoperation caused by visual misjudgment. Combined with the dynamic path correction function of the navigation system, the efficiency of human-machine collaboration is further optimized to ensure the precise synchronization of the operator's attention and the movement trajectory of the instrument. Experiments have shown that this module can significantly improve the operational stability of high-precision surgery, especially in long-term and complex operations. By continuously monitoring the operator's visual focus, it helps maintain the optimal operating state, reduces surgical failures caused by incorrect operations, and reduces the risk of postoperative complications caused by human factors.

[0025] 3. By integrating the dual visual dynamic data of patients and physicians, the problem of mismatch between instrument path and operating field of view caused by traditional navigation systems relying only on single-dimensional compensation is solved. In traditional methods, compensation for patient eye micro-movements may cause instrument path deviations due to the lag in the physician's field of view adjustment. This solution achieves coordinated correction of instrument path and microscope field of view by integrating the spatial transformation parameters of the two in real time. For example, when the patient's eyeball shifts horizontally, T1 drives the instrument path compensation, while T2 synchronously adjusts the center position of the microscope field of view to prevent the operator from misjudging the relative position of the instrument and the target area due to field of view offset. Significantly reduce the risk of misoperation caused by visual dislocation. In addition, the dynamic optimization of the three-dimensional coordinate system correction parameters further enhances the system's adaptability to complex surgical scenarios and ensures the consistency and stability of high-precision operations.

[0026] 4. Monitoring the status of the physician. When the physician's status is abnormal, dynamically enhanced visual guidance is used to help the physician become aware and quickly recover. Further, through the fusion analysis of multimodal physiological parameters, the misjudgment problem caused by traditional intraoperative monitoring relying only on a single indicator is solved. For example, the physician may experience slight relaxation of the levator palpebrae superioris muscle during a long operation, but if the pupil reflexes to light normally, the system will not misjudge it as fatigue, avoiding invalid alarms that interfere with the surgical process. When tension or fatigue is detected, enhanced navigation prompts can actively correct the risk of instrument deviation caused by the physician's distraction. It can effectively reduce the instrument path deviation caused by abnormal operator status, especially in high-risk surgical stages, and help physicians quickly restore operating accuracy through dynamically enhanced visual guidance. In addition, combined with heart rate variability analysis, the system can identify early fatigue accumulation states, trigger preventive prompts before key surgical nodes, and optimize the initiative and safety of human-machine collaboration.

[0027] 5. The highlighting and time estimation functions of the flowchart break down complex surgeries into controllable units, which is especially suitable for multi-stage surgeries such as glaucoma drainage valve implantation, reducing the cognitive load of new surgeons; resource optimization: the dynamic tool list reduces the probability of misselection of instruments, and the disinfection status mark blocks the risk of cross-infection; safety improvement: the emergency interface improves the efficiency of bleeding control through the collaboration of three-dimensional space annotation and voice guidance; human-machine collaboration, multimodal feedback of prompt signals and microscope AR overlay display, realize closed-loop optimization of the "hand-eye-brain" operation link, and reduce the risk of instrument slippage caused by the surgeon's distraction.

[0028] 6. Quantify the distribution of visual attention through eye movement entropy, and combine the time series modeling capability of LSTM network to accurately identify attention deviation caused by fatigue, distraction or information overload; graded intervention mechanism, when the score is lower than the threshold, the system triggers in sequence: Level 1 reminder: The edge of the AR glasses gradually turns red (frequency 2Hz); Secondary reminder: vibration tactile feedback (intensity 0.3G); Level 3 intervention: automatically pause the instrument conveyor and send an alarm to the surgeon’s interface; Surgical stage adaptation: The LSTM network dynamically adjusts the scoring weights according to different surgical stages, so that the intervention strategy is accurately matched with the key nodes of the surgery.

[0029] 7. AR arrows dynamically mark the offset direction; voice synchronization emphasizes key parameters; prediction of mispick behavior through grasping action analysis: detection of the contact area between the hand and the instrument through a convolutional network; trajectory abnormality warning, triggering an anti-fall prompt when the acceleration of the instrument movement changes suddenly; self-learning optimization mechanism, automatic clustering analysis of the feature library every week to identify new mispick patterns. Personalized risk prediction, by analyzing individual characteristics in historical surgical data, the system can trigger preparatory prompts 2-3 minutes in advance to avoid uncontrolled operations in sudden situations; in predicted high-risk periods, the system automatically enhances navigation prompts to match the physician's real-time cognitive load level. The heat map intuitively displays the distribution of vascular pressure to help the surgeon quickly locate the source of bleeding and reduce secondary injuries caused by blind exploration; hiding non-critical tool prompts can reduce the complexity of interface information and allow the surgeon to focus on hemostasis operations; in emergency mode, the system automatically removes the speed limit of the instrument movement to support rapid intervention operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0031] Figure 1 A schematic diagram of the electronic device structure of the hardware operating environment involved in an embodiment of the present application.

[0032] Figure 2 It is a schematic diagram of the functional modules of the real-time navigation system for ophthalmic surgery based on dynamic visual field tracking provided in an embodiment of the present application.

[0033] Figure 3 This is a module construction diagram of a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking provided in an embodiment of the present application.

[0034] Figure 4 It is a flowchart of the steps of the real-time navigation method for ophthalmic surgery based on dynamic visual field tracking provided in an embodiment of the present application.

[0035] Symbols in the figure: 1001 - processor, 1002 - communication bus, 1003 - user interface, 1004 - network interface, 1005 - memory. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0037] The solution of the present application is further described below in conjunction with the accompanying drawings.

[0038] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0039] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0040] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a data storage module.

[0041] exist Figure 1In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls a real-time navigation system for ophthalmic surgery based on dynamic field of view tracking in the memory 1005 through the processor 1001, and executes a real-time navigation method for ophthalmic surgery based on dynamic field of view tracking provided in an embodiment of the present application.

[0042] Based on the aforementioned hardware operating environment and system architecture, in the first aspect of the present application, refer to Figure 2 and Figure 3 As shown, a real-time navigation system for ophthalmic surgery based on dynamic visual field tracking is provided, comprising: The navigation generation module matches the operation data closest to the current operation from the database according to the patient's operation type and location, breaks down the steps of the operation data, and generates the standard operation navigation for the current operation; It should be noted that the database contains 3D image data, instrument motion trajectories and key operation node parameters of historical surgical cases, and matches biological features such as corneal curvature and anterior chamber depth of patients through the dynamic time warping algorithm (DTW) (error threshold ±0.3mm). During the disassembly steps, the STFT network is used to extract surgical stage features and decompose complex operations into quantifiable subtasks (such as cataract surgery is divided into 4 stages: incision making, capsulorhexis, lens implantation, and suturing). As the database data becomes richer, each subtask can be further subdivided and a 3D virtual path can be generated (accuracy ±0.05mm); The comparison display module compares the real-time image of the surgical process with the image of the standard operation navigation in real time to determine the current surgical step and display the image navigation for the patient's subsequent surgery; It should be noted that the real-time image is collected at a frame rate of 60fps through a surgical microscope, 256 feature points are extracted for registration using the SIFT algorithm, and the image offset caused by eye movement is compensated by the optical flow method (error ≤ 0.1 pixel). The subsequent navigation image is displayed in the form of a semi-transparent overlay on the microscope eyepiece, including the expected path of the instrument (red dotted line), the target area boundary (green outline) and the safety distance prompt (yellow warning area); The patient tracking module collects the corneal limbus feature points on the surface of the patient's eyeball in real time through a near-infrared light source and an image sensor, calculates the three-dimensional motion trajectory, and combines the target area boundary data of the preoperative OCT image segmentation and the intraoperative navigation coordinate system to generate a spatial mapping matrix between the eyeball motion trajectory and the target area; It should be noted that the near-infrared light source uses a wavelength of 850nm (200Hz pulse frequency), and the CMOS sensor (IMX585) captures the displacement of 8 characteristic points of the corneal limbus with an exposure time of 0.1ms, and the three-dimensional movement trajectory of the eyeball is calculated by extended Kalman filtering. The spatial mapping matrix fuses the preoperative OCT tomographic data (accuracy 5μm) with the intraoperative navigation coordinate system, and the dynamic update formula is:

[0043] The rotation matrix R is solved by the quaternion method, and the translation vector T is used to calibrate the target area offset in real time; The offset compensation module dynamically corrects the virtual navigation path of the surgical instrument according to the real-time eye movement trajectory, and adjusts the relative position between the corrected instrument path and the target area boundary to align the surgical instrument with the target area; It should be noted that the correction algorithm uses a PID controller. When the eyeball displacement is detected to be greater than 0.03mm, the robot arm is driven to adjust the movement trajectory of the instrument. The overlap between the corrected path and the target area boundary is displayed in real time on the operation interface (a green mark is triggered when it is ≥98%). If the overlap is less than 90% for 2 seconds, an audible and visual alarm is activated. In this embodiment, the real-time image of the surgical process is compared with the image of the standard surgical operation path, and after the path position of the current surgery is determined, the image navigation of the standard surgical operation path for subsequent patients is displayed. Therefore, in the process of accumulating experience, the doctor can perform the surgery according to the image navigation of the standard surgical operation path during the surgery, form the standard surgical operation path specification and learn the surgical operation of the surgical position and surgical type through the successful surgical process of other doctors, reduce the dependence on clinical surgical experience, and the surgery performed by young doctors is subject to the standard of the surgical type and the standard path of the surgical site, for example, the surgical incision position and angle are standardized, and unnecessary skin and tissue injuries of patients are reduced, and the patient's postoperative recovery and postoperative quality of life are improved; at the same time, because the longer the operation time, the greater the harm to the patient and the longer the recovery time, the operation time will be minimized while ensuring the surgical effect, so the doctor can standardize the surgical operation according to the standard surgical path, reduce the surgical thinking time, shorten the operation time, and reduce the patient's pain; at a deeper level, the system dynamically matches the surgical data with the real-time image, transforms the traditional intraocular operation that relies on the experience of the physician into a standardized navigation process, and significantly reduces the risk of instrument deviation caused by micro-movement of the eyeball. Through the superposition display of three-dimensional virtual paths and real-time images, the surgeon can intuitively grasp the spatial relationship between the instrument and the target area, avoiding misoperation caused by visual errors. It is especially suitable for high-precision surgical scenarios such as corneal transplantation and glaucoma drainage valve implantation. The system's real-time tracking and compensation mechanism for eye movement can actively adapt to the patient's unconscious displacement or physiological tremor during surgery, ensure the continuous accuracy of the navigation path, and reduce the time-consuming operation of repeatedly adjusting the position of the instrument during surgery. In addition, the multi-level alarm mechanism (visual prompts → interface simplification → forced pause) can effectively intercept potential operational errors and improve surgical safety. By integrating preoperative imaging data and intraoperative dynamic information, the system further optimizes the consistency of surgical planning and execution, shortens the surgeon's learning curve, and provides technical support for the standardization and promotion of complex ophthalmic surgeries.

[0044] In one embodiment of the present application, it also includes a physician tracking module, which collects the physician's eye image in real time through an image sensor to obtain the physician's gaze point coordinates, and calculates the ratio of the gaze point coordinates to the coverage area of ​​the current navigation target area. If the average ratio per unit time is lower than the set threshold, a visual alarm signal is triggered.

[0045] It should be noted that the image sensor uses a global shutter CMOS (such as IMX585) to capture the physician's eye movement at a sampling rate of 200Hz, extracts the pupil center coordinates through a convolutional neural network (CNN), and calculates the three-dimensional spatial position of the gaze point in combination with the corneal reflection spot. The coverage area ratio is calculated using a convex hull algorithm to dynamically analyze the proportion of the spatial overlap area between the gaze point and the target area boundary, and the threshold is set and dynamically adjusted according to the type of surgery (such as 85% for cataract surgery and 92% for glaucoma surgery). Visual warning signals include interface color gradients (yellow→red), target area outline flashing, and simplified operation interfaces (retaining only core navigation information) to avoid distracting the physician's attention; In this embodiment, the physician tracking module solves the risk of attention deviation caused by operator distraction, visual fatigue or limited operating field in traditional surgery by quantifying the spatial correlation between the operator's gaze point and the target area in real time. This module can actively identify the abnormal state of the operator's line of sight away from the key operating area, and intervene in time through a graded alarm mechanism to reduce the probability of instrument misoperation caused by visual misjudgment. Combined with the dynamic path correction function of the navigation system, the efficiency of human-machine collaboration is further optimized to ensure the precise synchronization of the operator's attention and the movement trajectory of the instrument. Experiments have shown that this module can significantly improve the operational stability of high-precision surgeries (such as macular hole repair and retinal vascular anastomosis), especially in long and complex operations, by continuously monitoring the operator's visual focus, assisting in maintaining the optimal operating state, reducing surgical failures caused by incorrect operations, and reducing the risk of postoperative complications caused by human factors.

[0046] In one embodiment of the present application, the output data of the patient tracking module and the physician tracking module are input into the eye movement fusion unit, and the eye movement fusion unit generates a comprehensive compensation amount, which specifically includes: A first transformation matrix T1 is established according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path; A second transformation matrix T2 is established according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view; The three-dimensional coordinate system correction parameters are obtained by weighted fusion of T1 and T2 to generate a comprehensive compensation value.

[0047] It should be noted that the first transformation matrix T1 is calculated by the target area boundary data of the preoperative OCT image segmentation and the displacement of the corneal limbus feature points collected in real time during the operation. The quaternion method is used to solve the rotation matrix and calibrate the target area offset in real time. The second transformation matrix T2 is based on the spatial mapping relationship between the physician's gaze point coordinates and the optical axis of the microscope. The optical flow method is used to compensate for the field of view offset caused by the physician's head micro-movement (error ≤ 0.1 pixel). During weighted fusion, the T1 weight coefficient is dynamically adjusted according to the patient's eye movement amplitude (such as the weight is increased to 0.7 when the displacement is greater than 0.1mm), and the T2 weight coefficient is set in combination with the microscope magnification (such as the weight is 0.3 under a 40x microscope).

[0048] In this embodiment, by fusing the dual visual dynamic data of the patient and the physician, the problem of mismatch between the instrument path and the operating field of view caused by the traditional navigation system relying only on single-dimensional compensation is solved. In the traditional method, the compensation for the patient's eye movement may cause instrument path deviation due to the lag in the physician's field of view adjustment. This solution achieves the coordinated correction of the instrument path and the microscope field of view by fusing the spatial transformation parameters of the two in real time. For example, when the patient's eyeball moves horizontally, T1 drives the instrument path compensation, and T2 synchronously adjusts the center position of the microscope field of view to prevent the operator from misjudging the relative position of the instrument and the target area due to field of view offset. Significantly reduce the risk of misoperation caused by visual dislocation. In addition, the dynamic optimization of the three-dimensional coordinate system correction parameters further enhances the system's adaptability to complex surgical scenarios (such as retinal vascular anastomosis), ensuring the consistency and stability of high-precision operations.

[0049] In one embodiment of the present application, the physician tracking module further includes a state monitoring submodule, which increases the intensity of the navigation prompt when an abnormal state of the physician is detected according to the physician's eye image; the abnormal state includes tension and fatigue; The abnormal state of tension is to detect the change rate of the doctor's pupil diameter through the eye image, and when the pupil diameter expands beyond a threshold value within a unit time, it is determined to be a state of tension; The abnormal state of fatigue is detected by the eye image to obtain the contraction decrease rate k1 of the levator palpebrae superioris muscle, the increase rate k2 of the pupil diameter, the delay rate k3 of the pupil light reflex speed, and the physician's pulse, and the physician's heart rate variability decrease rate k4 is obtained. When the threshold is reached within a unit time through weighted fusion of k1, k2, k3 and k4, it is determined to be a fatigue state.

[0050] It should be noted that the pupil diameter change rate is captured by a global shutter CMOS with 200Hz sampling, and Gaussian filtering is used to eliminate blink interference, and the dilation threshold is set to 15% of the baseline value. In fatigue judgment, the weight coefficients of k1-k4 are 0.3, 0.25, 0.25, and 0.2, respectively, and the fusion threshold is dynamically optimized through the support vector machine (SVM) classification model. Navigation prompt enhancements include: the target area outline flashing frequency is increased to 5Hz, the instrument path warning area is expanded by 20%, and the operation interface is switched to high contrast mode (red and green color scheme).

[0051] In this embodiment, the physician's state is monitored. When the physician's state is abnormal, dynamically enhanced visual guidance is used to help the physician to be aware and quickly recover. Furthermore, multimodal physiological parameter fusion analysis is used to solve the misjudgment problem caused by traditional intraoperative monitoring relying only on a single indicator (such as operation time). For example, the physician may experience slight relaxation of the levator palpebrae superioris muscle (k1 increase) during a long operation, but if the pupil light reflex is normal (k3 is not exceeded), the system will not misjudge it as fatigue state, avoiding invalid alarms interfering with the surgical process. When tension or fatigue is detected, enhanced navigation prompts can actively correct the risk of instrument deviation caused by the physician's distraction. The instrument path deviation caused by abnormal operator state can be effectively reduced, especially in high-risk surgical stages (such as macular hole internal limiting membrane peeling), and dynamically enhanced visual guidance can help physicians quickly restore operation accuracy. In addition, combined with heart rate variability analysis (k4), the system can identify early fatigue accumulation state, trigger preventive prompts before key surgical nodes, and optimize the initiative and safety of human-machine collaboration.

[0052] In an embodiment of the present application, the device further includes a second display module, and the second display module performs the following operations: Generate a step flow chart based on the preoperatively planned surgical path, highlight the current surgical stage in real time, and mark the estimated time for the remaining steps; Predict the type of instrument required for subsequent steps based on a convolutional neural network and dynamically display a tool preparation list that includes instrument specifications and disinfection status indicators; When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading of the next stage of the instrument in advance; In emergency operation scenarios, including sudden bleeding, the system automatically switches to the warning interface, simultaneously displaying the location of the emergency tool kit and usage priority instructions.

[0053] It should be noted that the step flowchart is generated by using the time segmentation network (TSN) to analyze the preoperative three-dimensional imaging data, decomposing the surgical path into quantifiable stage nodes (such as incision positioning, tissue separation, hemostasis operation, etc.), and the flowchart supports touch zooming and perspective switching; the remaining time estimation is based on the instrument operation time database of historical surgical cases, combined with the current operator's operation speed (collected by the instrument motion trajectory sensor) for dynamic correction, and the error is controlled within ±15 seconds; the instrument prediction and list management uses the surgical video frames and corresponding instrument usage records in the convolutional neural network (CNN) training data set. The input layer fuses the current surgical field image features (SIFT key points) and the stage flowchart status, and the output layer predicts the instrument type confidence (threshold ≥ 0.85); the disinfection status identification is linked to the central disinfection system of the operating room through the RFID chip, and the sterilization validity period and number of uses of the instrument are updated in real time; the collaborative prompt mechanism is set differently according to the type of surgery through the completion threshold (such as 95% for cataract phacoemulsification stage and 90% for vitrectomy stage), and the overlap calculation of the surgical field instrument motion trajectory and the standard path is analyzed by the optical flow method. Prompt signals include color change of the LED ring light strip in the instrument preparation area (blue→yellow) and vibration of the tactile feedback device to avoid acoustic interference; the emergency interface switching is determined by sudden bleeding through the blood flow signal mutation of the intraoperative OCT image (flow rate > 5mm / s) and the hemoglobin concentration threshold, and the trigger response delay is <0.3 seconds; the location of the emergency tool kit is marked in the surgeon's microscope field of view through AR projection (guided by a red arrow), and the priority of use is automatically sorted according to the anatomical risk level of the bleeding site.

[0054] In this implementation, the technical problems solved include that the traditional surgical navigation system relies on the surgeon's experience to judge the timing of instrument switching, which is prone to interruption of operation due to preparation delays. This solution achieves seamless connection of instrument flow through intelligent prediction and collaborative prompts; in emergency scenarios, the traditional system lacks active emergency guidance, and the surgeon needs to distract himself to recall the treatment process. The automatic switching mechanism of this module projects key information (such as hemostatic forceps specifications and gelatin sponge storage location) directly into the operating field of view, shortening the emergency response cycle. The technical effects achieved include operation standardization, highlighting and time estimation functions of flowcharts, breaking down complex surgeries into controllable units, which is especially suitable for multi-stage surgeries such as glaucoma drainage valve implantation, reducing the cognitive load of new surgeons; resource optimization: dynamic tool lists reduce the probability of instrument misselection (such as avoiding the use of 5mm ultrasonic scalpels in 3mm blood vessel closure scenarios), and disinfection status identification blocks the risk of cross-infection; safety improvement: the emergency interface improves the efficiency of bleeding control through the collaboration of three-dimensional spatial annotation (such as projecting high-frequency electrocoagulation to the 10 o'clock position of the surgical field) and voice guidance ("preferentially use 4-0 absorbable sutures"); human-machine collaboration, multimodal feedback of prompt signals (visual + tactile) and microscope AR overlay display, realize closed-loop optimization of the "hand-eye-brain" operation link, and reduce the risk of instrument slippage caused by the surgeon's distraction.

[0055] In one embodiment of the present application, it also includes an assistant tracking module, which uses an image sensor to collect the assistant's eye images in real time, and collects the assistant's pupil focus coordinates and scanning frequency in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the scanning path entropy value is lower than a threshold, it is determined to be a distracted state; an LSTM network is used to analyze the match between the eye movement pattern and the current surgical stage, and an attention score is generated, and a reminder is triggered when the score is lower than the threshold.

[0056] It should be noted that the image sensor uses a global shutter CMOS (such as IMX585) to capture eye movement at a sampling rate of 240Hz, segment the iris area through the U-Net network, and calculate the three-dimensional coordinates of the pupil focus (accuracy ±0.1mm) in combination with the corneal reflection spot; The scan path entropy value is calculated using the Shannon entropy formula:

[0057] in pi The entropy value is lower than the threshold, which indicates that the gaze trajectory is too concentrated or disordered.

[0058] The input layer of the LSTM network contains eye movement features (gaze duration, scanning speed), surgical stage codes (such as the incision making stage is coded as 01) and instrument usage records, and the output layer generates an attention score of 0-100.

[0059] In this embodiment, the technical problems solved are that the distraction of assistants in traditional surgery (such as visual fatigue caused by long-term operation) can easily lead to delays in instrument preparation, especially in scenarios where multiple instruments are switched (such as heart bypass surgery, which requires the continuous use of 20+ instruments); manual supervision has a lag and cannot quantify the assistant's cognitive load and task matching in real time. The technical effects achieved include dynamic cognitive evaluation: quantifying the distribution state of visual attention through eye movement entropy, combined with the time series modeling ability of the LSTM network, accurately identifying attention deviations caused by fatigue, distraction or information overload (such as the error rate will increase when the entropy value is <0.8); graded intervention mechanism, when the score is lower than the threshold, the system triggers in sequence: Level 1 reminder: The edge of the AR glasses gradually turns red (frequency 2Hz); Secondary reminder: vibration tactile feedback (intensity 0.3G); Level 3 intervention: automatically pause the instrument conveyor and send an alarm to the surgeon’s interface; Surgical stage adaptation: The LSTM network dynamically adjusts the scoring weights according to different surgical stages (e.g., the suturing stage pays more attention to focal stability), so that the intervention strategy is accurately matched with the key nodes of the surgery.

[0060] In one embodiment of the present application, the assistant tracking module also includes an instrument guidance submodule, which tracks the spatial coordinates of the instrument to be prepared in real time through a positioning chip, and compares it with the preset position of the second display module; when the instrument offset distance is greater than a preset value, a highlighted arrow guidance is superimposed in the assistant AR glasses, and a synchronous voice prompt of the target instrument specification parameters is given; at the same time, a feature vector library is established for high-frequency mis-picked instruments, and abnormal grasping actions are identified through a convolutional network, and an early warning is issued.

[0061] It should be noted that the positioning system uses UWB+IMU fusion positioning (accuracy ±1cm), and each instrument is embedded with a passive RFID tag (such as EPC Gen2), which is calibrated with the operating table coordinate system in real time; the AR guidance interface uses the SLAM algorithm to build spatial anchor points, the dynamic rendering delay of the arrow is <10ms, and the voice prompts are generated by the TTS engine (speech rate 3 words / second); the misjudgment feature library contains three-dimensional features of instrument shape (ResNet50 extraction), grasping posture (OpenPose hand key points) and motion trajectory (LSTM encoding), and the misjudgment rate is <2%.

[0062] In this embodiment, the technical problems solved include the large variety of instruments in complex surgeries (e.g., more than 150 types of neurosurgery), the susceptibility of traditional label recognition to blood obstruction, resulting in the risk of mistaking instruments (e.g., mistaking bipolar electrocoagulation forceps for monopolar ones), and the incorrect placement of instruments due to assistants' lack of experience, which prolongs the surgical preparation time.

[0063] The technical effects achieved include spatial-semantic collaborative guidance, AR arrows dynamically marking the offset direction (such as a red arrow with a horizontal deflection of 30°); voice synchronization emphasizing key parameters (such as "Note: This instrument is made of titanium alloy and ultrasonic cleaning is prohibited"); prediction of mispick behavior through grasping action analysis: detection of the contact area between the hand and the instrument through a convolutional network (such as mistakenly holding the attractor head instead of the handle); trajectory abnormality warning, triggering an anti-fall prompt when the instrument movement acceleration changes suddenly (such as >2m / s²); self-learning optimization mechanism, automatic clustering analysis of the feature library every week, and identification of new mispick patterns (such as confusion caused by similar packaging of a batch of instruments).

[0064] In one embodiment of the present application, it also includes a physician database module, which stores the historical surgical records of each physician, including the time distribution of the surgical stages, the fluctuation curve of physiological parameters, and the correction amount of instrument use; extracts feature patterns through the LSTM network, reversely optimizes the prediction model parameters based on the postoperative quality assessment report, establishes an individualized tension-fatigue time window prediction model, and adjusts the navigation parameters in advance for the high-frequency warning stage.

[0065] It should be noted that data storage and modeling record the standard deviation of the duration of each operation node (such as incision making, hemostasis, and suturing) through the time distribution of the surgical stage, and align the time axis of different surgical cases through the dynamic time warping algorithm; the physiological parameter fluctuation curve integrates the pupil diameter change rate, heart rate variability (HRV) decline gradient and hand tremor frequency, the sampling rate is 100Hz, and the data is processed by differential privacy (ε=0.6); the LSTM network input layer contains time-series physiological data and instrument corrections (such as the number of path deviations and the mean compensation distance), and the output layer predicts the fatigue cumulative risk index (0-100 points) within the next 10 minutes; the tension-fatigue time window prediction model dynamically adjusts the warning threshold according to individual operating habits. For example, for physicians who are accustomed to fast operations, the pupil dilation rate threshold is increased by 20%.

[0066] In this embodiment, the technical problems solved include that the traditional navigation system adopts a fixed threshold warning, which cannot adapt to the operating habits and physiological characteristics of different doctors, resulting in a high false alarm rate or missed reporting of key risks.

[0067] There is a lack of forward-looking prediction of sudden fatigue or tension during surgery, and the patient can only respond passively after the abnormality occurs.

[0068] The technical effect achieved is personalized risk prediction. By analyzing individual characteristics in historical surgical data (e.g., a physician's HRV decline rate significantly accelerates at 45-60 minutes into an operation), the system can trigger preparatory prompts 2-3 minutes in advance (e.g., reducing interface information density, starting instrument speed limit), to avoid loss of control in emergencies. Dynamic parameter adjustment: During predicted high-risk periods (such as fatigue accumulation peaks), the system automatically enhances navigation prompts (such as increasing the virtual path width by 50% and extending the voice command interval) to match the physician's real-time cognitive load level.

[0069] In one embodiment of the present application, the second display module also includes an emergency exemption submodule, which automatically shields the conventional warning logic and switches to augmented reality mode in the event of sudden increase in intraocular pressure or bleeding: superimposes a 3D vascular pressure distribution heat map on the surgical field; and rearranges the instrument priority: hides non-critical tool prompts, and highlights the location of the hemostatic device.

[0070] It should be noted that event judgment and response are based on the sudden increase in intraocular pressure, which is determined by the angle closure degree (>75%) of the intraoperative OCT image combined with the intraocular pressure sensor data (>30 mmHg), with a response delay of <0.5 seconds. The 3D vascular pressure heat map is generated based on the fusion of preoperative CTA images and intraoperative blood flow Doppler data, and the pressure is displayed in grades (blue: <15 mmHg; red: >30 mmHg).

[0071] The device priority management includes AR highlight box annotation (gold frame + pulse animation) of hemostatic devices (such as bipolar electrocoagulation forceps and gelatin sponge), and the transparency of non-critical tools (such as microscissors and forceps) is increased to 80%. The device position guidance uses SLAM algorithm for real-time positioning, combined with voice prompts (such as "hemostatic forceps is located at position 2 in area C of the instrument table").

[0072] In this embodiment, the technical problems solved include that the traditional system continues to push routine warnings (such as instrument path deviation prompts) in emergency scenarios, interfering the operator's focus on core issues; in the event of sudden bleeding, the operator needs to manually retrieve the location of the hemostatic tool, delaying the golden treatment time.

[0073] The heat map intuitively displays the distribution of vascular pressure, helping the surgeon to quickly locate the source of bleeding (such as the rupture point of the posterior ciliary artery) and reduce secondary damage caused by blind exploration; hiding non-critical tool prompts can reduce the complexity of interface information and allow the surgeon to focus on hemostasis operations (such as electrocoagulation forceps power adjustment and gelatin sponge filling); in emergency mode, the system automatically removes the device movement speed limit to support rapid intervention operations.

[0074] In a second aspect of the present application, a real-time navigation method for ophthalmic surgery based on dynamic visual field tracking is provided. Figure 4 As shown, including: According to the patient's surgery type and location, the operation data closest to the current surgery is matched from the database, the steps of the operation data are disassembled, and the standard operation navigation of the current surgery is generated; Compare the real-time images of the surgical process with the images of the standard operation navigation in real time to determine the current surgical step and display the image navigation for the patient's subsequent surgery; The near-infrared light source and image sensor are used to collect the feature points of the corneal limbus on the surface of the patient's eyeball in real time, and the three-dimensional motion trajectory is calculated. The target area boundary data of the preoperative OCT image segmentation and the intraoperative navigation coordinate system are combined to generate a spatial mapping matrix between the eyeball movement trajectory and the target area. The virtual navigation path of the surgical instrument is dynamically corrected according to the real-time eye movement trajectory, and the relative position of the corrected instrument path and the target area boundary is adjusted to align the surgical instrument with the target area.

[0075] In one embodiment of the present application, the doctor's eye image is collected in real time by an image sensor to obtain the coordinates of the doctor's gaze point, and the ratio of the gaze point coordinates to the coverage area of ​​the current navigation target area is calculated. If the average ratio per unit time is lower than the set threshold, a visual alarm signal is triggered.

[0076] In one embodiment of the present application, a comprehensive compensation amount is generated by using the patient's eye movement data and the physician's eye movement data, specifically including: A first transformation matrix T1 is established according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path; A second transformation matrix T2 is established according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view; The three-dimensional coordinate system correction parameters are obtained by weighted fusion of T1 and T2 to generate a comprehensive compensation value.

[0077] In one embodiment of the present application, when an abnormal state of the physician is detected based on the physician's eye image, the intensity of the navigation prompt is increased; the abnormal state includes tension and fatigue; The abnormal state of tension is to detect the change rate of the doctor's pupil diameter through the eye image, and when the pupil diameter expands beyond a threshold value within a unit time, it is determined to be a state of tension; The abnormal state of fatigue is detected by the eye image to obtain the contraction decrease rate k1 of the levator palpebrae superioris muscle, the increase rate k2 of the pupil diameter, the delay rate k3 of the pupil light reflex speed, and the physician's pulse, and the physician's heart rate variability decrease rate k4 is obtained. When the threshold is reached within a unit time through weighted fusion of k1, k2, k3 and k4, it is determined to be a fatigue state.

[0078] In one embodiment of the present application, a step flow chart is generated according to the surgical path planned before surgery, the current surgical stage is highlighted in real time, and the estimated time of the remaining steps is marked; Predict the type of instrument required for subsequent steps based on a convolutional neural network and dynamically display a tool preparation list that includes instrument specifications and disinfection status indicators; When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading of the next stage of the instrument in advance; In emergency operation scenarios, including sudden bleeding, the system automatically switches to the warning interface, simultaneously displaying the location of the emergency tool kit and usage priority instructions.

[0079] In one embodiment of the present application, an image sensor is used to collect the assistant's eye images in real time, and the assistant's pupil focus coordinates and scanning frequency are collected in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the scanning path entropy value is lower than a threshold, it is determined to be a distracted state; an LSTM network is used to analyze the match between the eye movement pattern and the current surgical stage, and an attention score is generated. When the score is lower than the threshold, a reminder is triggered.

[0080] In one embodiment of the present application, the spatial coordinates of the instrument to be prepared are tracked in real time by a positioning chip and compared with the preset position of the second display module; when the instrument offset distance is greater than the preset value, a highlighted arrow guide is superimposed in the assistant AR glasses, and a synchronous voice prompt is given for the target instrument specification parameters; at the same time, a feature vector library is established for high-frequency mis-picked instruments, and abnormal grasping actions are identified through a convolutional network, and an early warning is issued.

[0081] In one embodiment of the present application, the historical surgical records of each physician are stored, including the time distribution of the surgical stages, the fluctuation curve of physiological parameters, and the correction amount of instrument use; the feature patterns are extracted through the LSTM network, and the prediction model parameters are reversely optimized based on the postoperative quality assessment report, and an individualized tension-fatigue time window prediction model is established, and the navigation parameters are adjusted in advance for the high-frequency warning stage.

[0082] In one embodiment of the present application, in the event of sudden increase in intraocular pressure or bleeding, the conventional warning logic is automatically shielded and switched to augmented reality mode: a 3D vascular pressure distribution heat map is superimposed on the surgical field; and the instrument priority is rearranged: non-critical tool prompts are hidden, and the location of the hemostatic device is highlighted.

[0083] It should be noted that the specific implementation of the real-time navigation method for ophthalmic surgery based on dynamic field of view tracking in the embodiment of the present application refers to the specific implementation of the real-time navigation system for ophthalmic surgery based on dynamic field of view tracking proposed in the first aspect of the aforementioned embodiment of the present application, and will not be repeated here.

[0084] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the existence of other identical elements in the article or device including the elements.

[0085] The above is a detailed introduction to a real-time navigation system for ophthalmic surgery based on dynamic field of view tracking. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the real-time navigation system for ophthalmic surgery based on dynamic field of view tracking and its core idea of ​​the present application; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A real-time navigation system for ophthalmic surgery based on dynamic visual field tracking, characterized in that: include: The navigation generation module matches the operation data closest to the current operation from the database according to the patient's operation type and location, breaks down the steps of the operation data, and generates the standard operation navigation for the current operation; The comparison display module compares the real-time image of the surgical process with the image of the standard operation navigation in real time to determine the current surgical step and display the image navigation for the patient's subsequent surgery; The patient tracking module collects the corneal limbus feature points on the surface of the patient's eyeball in real time through a near-infrared light source and an image sensor, calculates the three-dimensional motion trajectory, and combines the target area boundary data of the preoperative OCT image segmentation and the intraoperative navigation coordinate system to generate a spatial mapping matrix between the eyeball motion trajectory and the target area; The offset compensation module dynamically corrects the virtual navigation path of the surgical instrument according to the real-time eye movement trajectory, adjusts the relative position of the corrected instrument path and the target area boundary, and aligns the surgical instrument with the target area.

2. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 1, characterized in that: It also includes a physician tracking module, which collects the physician's eye images in real time through an image sensor, obtains the physician's gaze point coordinates, calculates the ratio of the gaze point coordinates to the coverage area of ​​the current navigation target area, and triggers a visual alarm signal if the average ratio per unit time is lower than a set threshold.

3. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 2, characterized in that: The output data of the patient tracking module and the physician tracking module are input into the eye movement fusion unit, and the eye movement fusion unit generates a comprehensive compensation amount, specifically including: A first transformation matrix T1 is established according to the patient's eye movement trajectory to describe the real-time compensation amount of the instrument path; A second transformation matrix T2 is established according to the physician's eye movement data to describe the dynamic adjustment amount of the microscope field of view; The three-dimensional coordinate system correction parameters are obtained by weighted fusion of T1 and T2 to generate a comprehensive compensation value.

4. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 2, characterized in that: The doctor tracking module also includes a state monitoring submodule, which increases the intensity of navigation prompts when an abnormal state of the doctor is detected based on the doctor's eye image; the abnormal state includes tension and fatigue; The abnormal state of tension is to detect the change rate of the doctor's pupil diameter through the eye image, and when the pupil diameter expands beyond a threshold value within a unit time, it is determined to be a state of tension; The abnormal state of fatigue is detected by the eye image to obtain the contraction decrease rate k1 of the levator palpebrae superioris muscle, the increase rate k2 of the pupil diameter, the delay rate k3 of the pupil light reflex speed, and the physician's pulse, and the physician's heart rate variability decrease rate k4 is obtained. When the threshold is reached within a unit time through weighted fusion of k1, k2, k3 and k4, it is determined to be a fatigue state.

5. A real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as described in any one of claims 1 to 4, characterized in that: The device further includes a second display module, and the second display module performs the following operations: Generate a step flow chart based on the preoperatively planned surgical path, highlight the current surgical stage in real time, and mark the estimated time for the remaining steps; Predict the type of instrument required for subsequent steps based on a convolutional neural network and dynamically display a tool preparation list that includes instrument specifications and disinfection status indicators; When it is detected that the completion degree of the current step reaches the threshold, a prompt signal is triggered to enable the assistant to complete the loading of the next stage of the instrument in advance; In emergency operation scenarios, including sudden bleeding, the system automatically switches to the warning interface, simultaneously displaying the location of the emergency tool kit and usage priority instructions.

6. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 5, characterized in that: It also includes an assistant tracking module, which uses image sensors to collect the assistant's eye images, pupil focus coordinates and scanning frequency in real time. When it is detected that the line of sight deviates from the instrument preparation area for more than a preset time or the scanning path entropy value is lower than a threshold, it is determined to be a distracted state; an LSTM network is used to analyze the match between the eye movement pattern and the current surgical stage, and an attention score is generated. When the score is lower than the threshold, a reminder is triggered.

7. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 6, characterized in that: The assistant tracking module also includes an instrument guidance submodule, which tracks the spatial coordinates of the instrument to be prepared in real time through the positioning chip, and compares it with the preset position of the second display module; when the instrument offset distance is greater than the preset value, a highlighted arrow guidance is superimposed in the assistant AR glasses, and the target instrument specification parameters are synchronously prompted by voice; at the same time, a feature vector library is established for high-frequency mis-picked instruments, and abnormal grasping actions are identified through a convolutional network, and an early warning is issued.

8. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 6, characterized in that: It also includes a physician database module, which stores the historical surgical records of each physician, including the time distribution of the surgical stages, the fluctuation curve of physiological parameters, and the correction amount of instrument use; it extracts feature patterns through the LSTM network, reversely optimizes the prediction model parameters based on the postoperative quality assessment report, establishes an individualized tension-fatigue time window prediction model, and adjusts the navigation parameters in advance in the high-frequency warning stage.

9. The real-time navigation system for ophthalmic surgery based on dynamic visual field tracking as claimed in claim 5, characterized in that: The second display module also includes an emergency exemption sub-module, which automatically blocks the conventional warning logic and switches to augmented reality mode in the event of sudden increase in intraocular pressure or bleeding: superimposes a 3D vascular pressure distribution heat map on the surgical field; and rearranges the device priority: hides non-critical tool prompts, and highlights the location of the hemostatic device.

10. A real-time navigation method for ophthalmic surgery based on dynamic visual field tracking, characterized in that: include: According to the patient's surgery type and location, the operation data closest to the current surgery is matched from the database, the steps of the operation data are disassembled, and the standard operation navigation of the current surgery is generated; Compare the real-time images of the surgical process with the images of the standard operation navigation in real time to determine the current surgical step and display the image navigation for the patient's subsequent surgery; The near-infrared light source and image sensor are used to collect the feature points of the corneal limbus on the surface of the patient's eyeball in real time, and the three-dimensional motion trajectory is calculated. The target area boundary data of the preoperative OCT image segmentation and the intraoperative navigation coordinate system are combined to generate a spatial mapping matrix between the eyeball movement trajectory and the target area. The virtual navigation path of the surgical instrument is dynamically corrected according to the real-time eye movement trajectory, and the relative position of the corrected instrument path and the target area boundary is adjusted to align the surgical instrument with the target area.

Images