Retinal traction tracked by digital image correlation

Abstract

A tracking system for quantifying retinal traction on the retina of a patient's eye includes an indicator device, a stereo camera, and an electronic control unit (ECU). The stereo camera collects and outputs stereo image data. The ECU, communicating with the indicator device, performs the method by receiving the stereo image data from the stereo camera during ophthalmic surgery and then assigning tracking points as overlapping pixels within a stereo image pair. The ECU also uses these stereo image pairs to automatically perform digital image correlation (DIC) processing to determine the relative motion of these tracking points and correlates this relative motion with the retinal traction using a traction map. A digital traction coefficient indicating the magnitude of the retinal traction is generated. The ECU uses the indicator device to perform control actions based on this digital traction coefficient.

Description

Technical Field

[0001] This disclosure relates to an imaging-based strategy for quantifying, tracking, and mitigating traction forces on the retina during retinal surgery. Background Technology

[0002] Vitrectomy and other invasive surgeries of the eye require surgeons to insert specialized surgical instruments into the vitreous cavity of the patient's eyeball and then manipulate these instruments while performing specific surgical techniques. The surgeon's actions are guided in real-time by highly magnified images of the retina and surrounding intraocular tissues. For this purpose, magnified images of the retina are typically displayed within the field of vision of the surgeon and other attending clinicians. Simultaneously, the magnified retina can be viewed in other ways, such as through the eyepiece of a high-powered ophthalmic microscope.

[0003] As understood in the art, the vitreous cavity extends between the lens and retina of the human eye, with the lens and retina located in the anterior and posterior regions of the eyeball, respectively. The vitreous cavity is occupied by a transparent gel-like substance called vitreous fluid, which is itself encased in a thin membrane called the vitreous cortex. The retina is separated from the vitreous body by another thin intermediate tissue, namely the internal limiting membrane (ILM).

[0004] Because the interretinal membrane (ILM) adheres to the surface of the retina, common ophthalmic surgeries such as repair of macular holes or retinal tears, removal of scar tissue, and other delicate eye surgeries may require the surgeon to firmly grasp the ILM and carefully peel it away from the supporting retina. ILM peeling has the effect of relaxing the disturbed retina while also providing the surgeon with unobstructed access to the retinal surface. The execution of an ILM peel involves the intentional application of traction force by the surgeon to the ILM. As the surgeon manipulates the ILM away from the retina, this force is transferred to the attached retina. The resulting traction across the retinal surface is referred to herein and in general techniques as retinal traction. Summary of the Invention

[0005] This article discloses an automated imaging-based system and method for quantifying retinal traction during ophthalmic surgeries (primarily, but not limited to, vitrectomy and other invasive ophthalmic surgeries). As understood in the art, alternative methods for manipulating the internal limiting membrane (ILM) include forceps-assisted “pinching and peeling” techniques and friction-based techniques, the latter utilizing specialized scraping rings, such as FINESSE, commercially available from Alcon, Inc. TM Flexible, toroidal, bent nickel-titanium rings. Using either of these exemplary tools can cause retinal traction, which can sometimes lead to iatrogenic ILM tears and other potential ocular trauma.

[0006] The structural integrity of the retinal membrane (ILM) in a given patient's eye often varies due to factors such as genetics, age, injury, and disease. Therefore, it is difficult to predict the effects of applying retinal traction of a specific amplitude and duration on a given patient's eye. Similarly, variations in the surgeon's skill level, the inherent capabilities and limitations of the surgical instruments used by a given surgeon, and other factors can produce drastically different end results. Therefore, this solution aims to reduce uncertainty during retinal surgery while improving surgical outcomes and increasing the surgeon's overall confidence.

[0007] To achieve these and other possible objectives, this method relies on collecting and processing stereoscopic images of the retina, and automatically and / or, under the guidance of a surgeon, assigning target pixels (“tracking points”) as overlapping pixels within stereoscopic image pairs of stereoscopic image data. In various embodiments, the relative motion of the assigned tracking points is closely monitored during ophthalmic surgery using an electronic control unit (ECU) (e.g., a standalone or distributed computer device, or associated hardware partially or fully integrated with a stereoscopic camera).

[0008] The ECU described herein uses a traction model to automatically quantify retinal traction, whereby the ECU ultimately outputs a digital traction coefficient indicating the magnitude of this retinal traction. In a simplified approach, the digital traction coefficient can be a dimensionless value indicating the magnitude, such as a normalized value with a maximum value of 1 and a minimum value of 0. The ECU automatically alerts the surgeon in real time based on the digital traction coefficient, such as when the digital traction coefficient, which may be averaged across the entire surface area of ​​the retina or within a designated area or region, exceeds a corresponding traction threshold. In some embodiments, real-time audio, visual, and / or tactile alerts may be generated to enable the surgeon to make more informed adjustments to the pulling or traction force applied to the ILM by the surgeon.

[0009] In a non-limiting exemplary embodiment, the tracking system for quantifying such retinal traction during ophthalmic surgery includes an indicator device, a stereo camera, and an ECU. In this particular embodiment, the ECU, which communicates with the indicator device via wired or wireless communication, is configured to receive stereo image data from the stereo camera. The ECU may be integrated with the stereo camera or exist as separate hardware communicating with it. The controller then autonomously or using surgeon-guided input signals assigns target pixels within the stereo image pair; for simplicity, these assigned target pixels are hereinafter referred to as "tracking points."

[0010] The ECU in this representative configuration is programmed with the traction model described above. While various methods can be used to implement the envisioned traction model, one possible solution involves a software-based logic block configured to automatically perform digital image correlation (DIC) processing on the collected stereo images, whereby the DIC processing is used to determine the relative motion of the tracking points. The ECU then, for example, uses a lookup table to correlate the relative motion of the tracking points with the specific retinal traction that caused this motion, and then outputs the digital traction coefficients described above. Thus, the provided digital traction coefficients indicate the magnitude of the retinal traction as described above. The ECU then performs appropriate control actions on the indicator device based on the digital traction coefficients, for example, when the magnitude of the digital traction coefficients exceeds a pre-calibrated threshold or a user-calibrable threshold.

[0011] To ensure that the relative motion tracked by the ECU is primarily caused by retinal traction applied by the surgeon rather than by baseline motion caused by other forces such as eye movements induced by the patient and / or external factors, the ECU can be configured to apply a free-body / solid-body motion filter to the relative motion to account for and ultimately filter out this baseline motion.

[0012] The indicator device may include one or more high-resolution displays, such as 4K or higher resolution LED-backlit medical-grade monitors. In such embodiments, the ECU may be configured to automatically present an intuitive "thermal map" of the retinal surface via the displays(s), either individually or in combination with text messages or prompts. In some configurations, for example, on a pixel-by-pixel or region-by-region basis, the displayed thermal map (which graphically represents the current level of retinal traction) can precisely pinpoint locations of relatively high traction. Such a thermal map may be displayed, for example, as an overlay on top of a displayed stereoscopic image of the retina, to indicate corresponding areas of high traction.

[0013] An auxiliary method for quantifying retinal traction during ophthalmic surgery is also disclosed. According to an exemplary embodiment, the method includes receiving stereoscopic image data from a stereo camera via an ECU during ophthalmic surgery, wherein the stereoscopic image data includes stereoscopic image pairs. The method further includes assigning tracking points as overlapping pixels within the stereoscopic image pairs and automatically performing DIC processing via the ECU using the stereoscopic image pairs. In this manner, the ECU determines the relative motion of the tracking points.

[0014] Furthermore, the method in this particular embodiment includes using a traction map of the ECU to correlate the relative motion of the tracking points with retinal traction, thereby determining a digital traction coefficient that indicates the magnitude of retinal traction. The ECU then uses an indicator device to perform a control action, wherein the control action is based on the digital traction coefficient and indicates the magnitude of retinal traction.

[0015] This document also discloses a system for quantifying retinal traction on the retina of a patient's eye, wherein the system includes a central processing unit (CPU) and a computer-readable medium on which a set of instructions is recorded. The instructions, executed by the CPU, cause the CPU to receive stereoscopic image data, comprising one or more stereoscopic image pairs, from a stereoscopic camera during ophthalmic surgery, and to assign tracking points as overlapping pixels within the stereoscopic image pairs(s). The CPU also automatically performs DIC processing using the stereoscopic images to determine the relative motion of the tracking points. In this particular embodiment, the CPU uses a traction map to correlate the relative motion of the tracking points with retinal traction as a digital traction coefficient indicating the magnitude of retinal traction. Then, when the digital traction coefficient exceeds one or more calibrated traction thresholds, the CPU transmits a control signal to an external indicator device.

[0016] The above-described features and advantages of this disclosure, as well as other possible features and advantages, will become apparent from the following detailed description of the best mode for carrying out this disclosure, taken in conjunction with the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a schematic illustration of an operating room setup using a tracking system that is used to automatically quantify and track retinal traction during representative ophthalmic surgeries using digital image correlation (DIC) processing.

[0018] Figure 2 It is a schematic depiction of a representative ophthalmic surgery in which the internal limiting membrane (ILM) is grasped and detached from the retina, thereby applying traction to the retina.

[0019] Figure 3 yes Figure 1 An exemplary embodiment of the tracking system shown is illustrated.

[0020] Figure 4 It is a schematic illustration of a thermal map depicting a region or area of ​​retinal traction elevation according to aspects of this disclosure.

[0021] Figure 5 It is a schematic illustration of stereoscopic image data and associated tracking points that can be used as part of the disclosed solution.

[0022] Figure 6 It describes the use of Figure 1 The flowchart illustrates an exemplary method for quantifying and tracking retinal traction using a traction system.

[0023] The foregoing and other features of this disclosure will become more fully apparent from the following description taken in conjunction with the accompanying drawings and the appended claims. Detailed Implementation

[0024] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms. These drawings are not necessarily drawn to scale. Some features may be enlarged or minimized to show details of specific components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as a representative basis for teaching those skilled in the art to employ the present disclosure in different ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any of the drawings may be combined with features illustrated in one or more other drawings to produce embodiments not explicitly illustrated or described. The illustrated combinations of features provide representative embodiments for typical applications. However, for a particular application or implementation, various combinations and modifications of features consistent with the teachings of this disclosure may be required.

[0025] In the following description, certain terms may be used for illustrative purposes only and are therefore not intended to be limiting. For example, terms such as “above” and “below” refer to directions referenced in the accompanying drawings. Terms such as “front,” “rear,” “before,” “after,” “left,” “right,” “rear,” and “side” describe the orientation and / or position of portions of a component or element within a consistent but arbitrary frame of reference, as will become clear from the text describing the component or element under discussion and the associated accompanying drawings. Furthermore, terms such as “first,” “second,” and “third” may be used to describe individual components. Such terms may include the words specifically mentioned above, their derivatives, and words with similar meanings.

[0026] Referring to the accompanying drawings, similar reference numerals refer to similar parts. Figure 1 A representative surgical operating room 10 is schematically depicted. This operating room 10 may be equipped with a multi-axis surgical robot 12 and an operating platform 14, such as a table or an adjustable / tilting surgical chair as shown. When the operating room 10 is occupied by a surgical team (not shown), the multi-axis surgical robot 12 can be connected to a stereo camera 16, through which the patient's intraocular anatomy can be visualized in three dimensions at high magnification, as understood in the art. Using associated hardware and software, a surgeon using three-dimensional (3D) viewing glasses (not shown) in a heads-up manner can accurately visualize target tissue using highly magnified 3D images 18 and 118 of the retina 25 and surrounding anatomy, which can be displayed or projected using corresponding high-resolution medical displays 20 and / or 200. Such displays 20 and 200 are as follows... Figure 3The illustrated example is a possible embodiment of an indicator device (IND) 20A that allows the surgeon to view at eye level. Viewing at eye level in this manner has the benefit of reducing stress and strain on the surgeon's neck and back, compared to the conventional method of viewing target tissue from top to bottom through the eyepiece of an ophthalmic microscope.

[0027] Stereo camera 16 (which can be like) Figure 1 The exemplary embodiment shown is configured, or may be configured in various other sizes and / or shapes suitable for the application, including or communicatively connected to a local control processor (LCP) 36. The LCP 36 may be embodied as a microprocessor, an application-specific integrated circuit (ASIC), a central processing unit, etc., and is configured to collect and output stereoscopic image data 38. That is, for each moment according to a predetermined sampling interval, two digital images are simultaneously collected as a stereoscopic image pair 38P (image 1, image 2). As understood in the art, when a surgeon and other attending clinicians view the stereoscopic image pair 38P through 3D glasses, the stereoscopic image pair 38P converges into a 3D image.

[0028] The ever-evolving capabilities of modern ophthalmic / surgical quality optics enable useful levels of optical and / or digital magnification, as well as digital imaging, for surgeries involving the retina 25 and other intraocular anatomy. Stereo camera 16 is one such device. Therefore, the real-time availability of stereoscopic image data 38 from stereo camera 16 during vitrectomy or other ophthalmic surgeries is for reference below. Figures 2 to 6 The enabling technology of this solution is described. Compared to the method of surgeons viewing the retina 25 through an optical eyepiece using a conventional ophthalmic microscope throughout the surgical procedure, this solution allows surgeons to maintain a more ergonomic "eye-level" posture, thereby reducing stress and strain on the surgeon's neck and back.

[0029] Figure 1 The exemplary surgical operating room 10 also includes a cabinet 22, which contains components related to... Figure 3 The indicator device 20A communicates with the electronic control unit (ECU) 50. In different possible implementations, it is embodied as follows: Figure 6 The computer-readable instructions of the illustrated method 70 can reside on ECU 50. ECU 50 can also be as follows: Figure 1The system can be a standalone computer, a distributed / networked system, or partially or fully integrated with the stereo camera 16. A cabinet 22, optionally depicted juxtaposed with the display screen 20, can be positioned elsewhere within the surgical room 10. Such a cabinet 22 can be constructed of lightweight and easily sterilizable materials (such as painted aluminum or stainless steel) and serves to protect the component hardware from potential dust, debris, and moisture intrusion. To improve visibility, light can be emitted by a lamp 17 mounted to the optical head of the stereo camera 16, and possibly by light inserted into... Figure 3 The patient's eye 30 is illuminated by an internal illuminator 32.

[0030] Figure 1 The ECU 50 is configured to receive stereoscopic image data 38, namely, a sequential stereoscopic image pair 38P schematically represented by arrow images 1 and 2, wherein the ECU 50 receives such stereoscopic image data 38 from the stereoscopic camera 16 in real time. As part of method 70, the ECU 50 autonomously or with the assistance of a surgeon assigns tracking points to overlapping pixels within the stereoscopic image pair 38P, and then automatically performs digital image correlation (DIC) processing on the stereoscopic image pair 38P to determine the relative motion of the tracking points. Additionally, the ECU 50 may use methods such as... Figure 3 The depicted traction diagram 55 correlates the relative motion of the tracking point with retinal traction, wherein the ECU 50 does so as a digital traction coefficient indicating the magnitude of retinal traction. The ECU 50 then targets the retinal traction coefficient... Figure 3 The (multiple) indicator devices 20A and / or 20B and / or other audio, visual or tactile devices perform control actions.

[0031] Brief Reference Figure 2 The image shows a representative patient's eye 30 undergoing an internal limiting membrane (ILM) peeling procedure, in which the ILM 31 is carefully separated and peeled from the backing retina 25 using surgical instruments 34. To complete the ILM peeling, the surgeon carefully manipulates the surgical instruments 34 to apply traction to the ILM 31 (arrow F). T In this way, the surgeon is able to expose the retina 25 in preparation for work thereon. In practice, the ILM 31 is thin, flexible, and transparent, making it difficult to see. Therefore, the surgeon typically applies a small amount of contrast-enhancing staining dye (such as indocyanine green (ICG) or membrane blue bicolor (DORC International)) to the ILM 31 to lightly stain it, thereby increasing contrast. With the visualization of the ILM 31 enhanced in this way, the surgeon can begin to peel away the ILM 31 to expose the retina 25, as understood in the art.

[0032] As stated above, the structural integrity of the ILM 31 in a given patient can vary considerably due to factors such as genetics, age, injury, and disease. Therefore, without this guidance, it is difficult to predict the effects of retinal traction during a specific surgical instance. Similarly, changes in surgical skill and intrinsic ability over time, along with the specific choice of surgical tools, can collectively produce different surgical outcomes. Therefore, this solution aims to facilitate real-time monitoring of retinal traction during ILM peeling and other procedures, while also providing surgeons with real-time, intuitive feedback. Monitoring and intuitive feedback together allow surgeons to work both alone and in conjunction with... Figure 3 With the help of an exemplary surgical robot 60, any necessary adjustments can be made to achieve the best surgical results.

[0033] Now for reference Figure 3 This illustration shows a patient's eye 30 undergoing a representative ophthalmic surgery 13 during operation of a tracking system 100 constructed as detailed herein, in this example, an invasive vitreoretinal surgery. The tracking system 100 is operable to quantify retinal traction within the patient's eye 30. For this purpose, the tracking system 100 includes indicator devices 20A and 20B, a stereo camera 16, and an ECU 50, the latter two of which may be integrated into a single, monolithic device in some embodiments. During the ophthalmic surgery 13, an internal illuminator 32 may be inserted into the vitreous cavity 15 of the patient's eye 30. Light LL emitted from the distal end E1 of the internal illuminator 32 and light from... Figure 1 Some of the light from lamp 17 is used to illuminate the vitreous cavity 15. A variety of different lighting techniques can be used to emit light LL, such as, but not limited to, red / green / blue (RGB) lasers, light-emitting diodes (LEDs), halogen bulbs, etc.

[0034] exist Figure 3 During a typical ophthalmic surgery 13, the surgeon may need to insert surgical instruments 34 into the vitreous cavity 15 to perform surgical tasks on or near the retina 25. As those skilled in the art will understand, Figure 2 The conventional methods for manipulating the ILM 31 shown include tweezers-assisted "pinching and peeling" techniques, or alternative uses of specialized scraping tools, such as the Finesse from Alcon. TM Flexible annular bending nickel-titanium ring. Therefore, the surgical tool 34 within the scope of this disclosure may encompass one or two devices.

[0035] The aforementioned techniques typically rely on the surgeon's dexterity to manually manipulate the surgical instrument 34. However, evolving machine vision-assisted robotic surgery techniques enable the construction of semi-automatic or automated dissection procedures in some embodiments, for example, using a multi-axis surgical robot 60. For instance, the surgical instrument 34 can be attached to an end effector 60E of the surgical robot 60, which in turn can be positioned to communicate with an ECU 50. This surgical robot 60 can be remotely operated by the surgeon either directly through interaction with the ECU 50 or via a human-machine interface (not shown) such as a surgical workstation. Alternatively, the surgical robot 60 can have limited support functions, such as relieving stress on the surgeon by supporting the weight of the surgical instrument 34, reducing tremors, etc., while leaving only the manipulation actions to the surgeon. As understood in the art, such automated dissection techniques performed or supported to some extent by the surgical robot 60 will require a deep understanding of retinal traction when measuring retinal indentation forces.

[0036] Regarding the internal illuminator 32, directional light LL is incident on the exposed surface of the retina 25 to produce an irradiated retinal surface 25I, including... Figure 2 The ILM 31 is attached to the retina 25. The internal illuminator 32 is connected to an auxiliary power supply (PS) 37, which can be powered via an illumination control signal (arrow CC) from the ECU 50 or another control processor. L Control measures include, for example, a filtered wall socket or battery pack and power inverter suitable for ensuring reliable generation and transmission of directional light (arrow LL). During ophthalmic surgery 13, stereo camera 16 collects stereoscopic image data 38 of the irradiated retinal surface 25I and ILM 31. Figure 2 The collected stereoscopic image data 38 is then transmitted to the ECU 50 for processing according to the Retinal Traction (R-TRAC) method 70.

[0037] Indicator device (IND) 20A (e.g., Figure 1 The displays 20 and / or 200 also communicate with the ECU 50 and are configured to respond to indicator control signals (arrow CC) from the ECU 50. 20A Activated / turned on in response to indicator control signals (arrow CC). 20A And depending on the specific configuration of the indicator device 20A, the indicator device 20A can provide appropriate visual alarms, such as... Figure 4 The thermal image 45 is shown and described below. Therefore, the ECU 50 uses the indicator device 20A to present a visual graphical description of the level of retinal traction relative to the surface area of ​​the retina 25.

[0038] Another indicator device 20B with a similar configuration can be used in conjunction with indicator device 20A to provide multiple alarms, which may escalate in response to the result of method 70. For example, indicator device 20B can provide audio, visual, and / or tactile alarms or warnings. A possible implementation of indicator device 20B is an audio speaker, in which case the indicator control signal (arrow CC) 20B This can cause the indicator device 20B to emit an audible beep or warning sound. Alternatively, the indicator device 20B may include a color-coded light that receives an indicator control signal (arrow CC). 20B This causes the indicator device 20B to light up in an easily identifiable manner (e.g., using red light). Tactile feedback, such as low-level vibration, can be presented to... Figure 1 The surgeon or another clinician in the surgical operating room 10, which may include wearable devices, floor mats, etc.

[0039] Although for the sake of clarity and simplicity, Figure 3 The ECU 50 is schematically depicted as a monolithic housing, but the ECU 50 may include one or more networking devices, each having a central processing unit (CPU) 52 and sufficient memory 54, i.e., computer-readable medium, including non-transitory (e.g., tangible) media involved in providing data / instructions that can be read by the CPU 52. Instructions embodying method 70 and the accompanying traction diagram 55 may be stored in memory 54 and executed by the CPU 52 to cause the CPU 52 to perform the various functions described herein, thereby potentially enabling method 70 in conjunction with calibrable input 58 (arrow CAL).

[0040] The memory 54 can take many forms, including but not limited to non-volatile and volatile media. As understood in the art, non-volatile media may include optical discs and / or magnetic disks or other persistent storage, while volatile media may include dynamic random access memory (DRAM), static RAM (SRAM), etc., any or all of which may constitute the main memory of the ECU 50. The input / output (I / O) circuitry 56 may be used to facilitate connection and communication with various peripheral devices used during ophthalmic surgery 13, including the stereo camera 16, the internal illuminator 32, and the indicator devices 20A and / or 20B. Other hardware not depicted but commonly used in the art may be included as part of the ECU 50, including but not limited to local oscillators or high-speed clocks, signal buffers, filters, etc.

[0041] Brief Reference Figure 4The retina 25 is shown as a representative fundus image 42. As understood in the art, fundus image 42 typically represents a color, black-and-white, or grayscale image of various key structures of the retina 25, primarily the optic disc 44, the retinal arteries 46 and the peripheral veins originating therefrom, and the macula 48. Fundus image 42, commonly used in ophthalmic practice and therefore familiar to attending clinicians, can serve as a visual background for the displayed heatmap 45. In such a configuration, ECU 50 can be configured to digitally subdivide or otherwise separate the retina 25 into multiple virtual regions, wherein, Figure 4 Two such regions, Z1 and Z2, are depicted, and retinal traction is mapped onto the retinal surface 25. In this way, multiple regions can have corresponding levels of retinal traction, which can be diagnosed and responded to individually, as an alternative to, for example, averaging retinal traction over the entire surface area of ​​the retina 25 and applying a single traction threshold.

[0042] exist Figure 4 In the optional display configuration, the ECU 50 can overlay the thermal image 45 onto the fundus image 42 (i.e., by...). Figure 1 The information is presented in real time via a stereoscopic image formed on 38P, wherein the information is presented via (multiple) indicator devices 20A and / or 20B. Thus, the thermal image 45 provides intuitive information that clearly indicates the distribution or concentration of the amplitude of retinal traction on the retina 25.

[0043] Now for reference Figure 6 Execute storage or recording in such Figure 3 The instructions in memory 54 of the ECU 50 shown cause the CPU 52 and other hardware of the ECU 50 to execute method 70. A representative embodiment of method 70 begins with logic block B72, which includes, for example, instructions from... Figure 3 The directional light LL from the internal illuminator 32 and possible light from Figure 1 Some additional light illumination from lamp 17 Figure 2 The retina 25. Surgical steps prior to implementing logic box B72 will include Figure 3 An incision is created in the patient's eye 30, and an internal illuminator 32 and surgical instruments 34 are carefully inserted into the vitreous cavity 15. Method 70 then proceeds to logic block B74.

[0044] Figure 6 The logic block B74 needs to receive data from the stereo camera 16 via the ECU 50. Figure 3The stereoscopic image data 38 is generated. This occurs in real time during ophthalmic surgery 13. In some embodiments, the ECU 50 may be separate from the stereoscopic camera 16, as depicted for clarity. Alternatively, the ECU 50 may include the LCP 36 of the stereoscopic camera 16, such that the LCP 36 is integrated with the ECU 50, i.e., the ECU 50 and the stereoscopic camera 16 are essentially a single functional unit, thereby reducing the number of parts and potentially providing additional processing efficiencies and reduced communication latency.

[0045] Brief Reference Figure 5 The collected stereoscopic image data 38 is formed from image pixels in the digital embodiment, and therefore Figure 6 The logic block B74 also includes identifying the corresponding pixel field of each component image in the stereo image pair 38P. For example, images 1 and 2 are shown as 8-pixel by 8-pixel (8×8) digital images arranged in a nominal (X, Y) pixel coordinate system. Since images 1 and 2 were collected at the same time, a specified pixel (4, 5) in image 1 coincides with the same pixel (4, 5) in image 2, and so on. That is, within a given stereo image pair 38P, pixels (4, 5) in image 1 coincide with pixels (4, 5) in image 2. Figure 6 As part of logic block B74, ECU 50 automatically, or with the assistance or guidance of a surgeon, assigns tracking points to at least one overlapping pixel within the stereo image pair 38P, for example, via an input signal (not shown). Within the scope of method 70, this action may require identifying specific overlapping pixels in each image of the stereo image pair 38P. Alternatively, if a single overlapping pixel provides insufficient resolution for tracking, a defined cluster of such overlapping pixels may be assigned. Method 70 then proceeds to logic block B76.

[0046] exist Figure 6 At logic block B76, Figure 3 The ECU 50 shown next uses the collected stereo image data 38 to quantify retinal traction. As part of logic block B76, ECU 50 can automatically perform the aforementioned DIC processing on the stereo image(s)(s)38P in the stereo image data 38 to determine the relative motion of the tracking points assigned to be part of logic block B74. As understood in the art, DIC is an optical technique in digital image processing used to quantify static or dynamic deformation, contours, strain, vibration, and other displacements in an imaging object. In this example, the imaging object is the retina 25. Figure 2 The ILM 31 and surrounding intraocular tissues, where the tracking points identified from logic box B74 are applied in relation to the ILM 31 and surrounding intraocular tissues.

[0047] Also as part of logic block B76, ECU 50 correlates the relative motion of such tracking points with retinal traction, as quantified in logic block B74. For example, ECU 50 may refer to... Figure 3 The traction diagram 55, schematically shown, serves as, for example, a lookup table indexed by the relative motion of the tracking points. For example, a given relative motion value may correspond to a value referred to herein as a digital traction coefficient, where such a value indicates the magnitude of retinal traction. In a possible embodiment, this digital traction coefficient may be normalized, where 0 corresponds to no traction on the retina 25 and 1 corresponds to the maximum amount of traction. Alternatively, a non-normalized embodiment may be used, or the actual corresponding traction value may be used.

[0048] Eye movements can sometimes occur due to patient movement or external forces. For example, a patient may move during surgery, whether by the patient's own will or in response to a surgeon bumping into the patient and / or... Figure 1 Platform 14, or other reasons. The resulting motion is referred to in the art as free body or solid body motion. Regarding this motion, the relative distance between two overlapping image pixels or tracking points in a stereo image pair of 38P remains the same under motion. Therefore, the relative motion considered in this paper for quantifying retinal traction is excluded, for example, by applying a solid body motion filter. This allows ECU 50 to consider the solid body motion of retina 25 as part of the DIC processing occurring within logic block B76. Once ECU 50 has determined the digital traction coefficients, method 70 proceeds to logic block B78.

[0049] exist Figure 6 At logic block B78, ECU 50 next compares the digital traction coefficient from logic block B76 with a calibrated traction threshold (“Is the traction ≤ CAL?”) or with a plurality of such traction thresholds corresponding to different regions or areas of the retina 25 as described above. When no calibrated traction threshold is exceeded, i.e., when the retinal traction is less than or equal to the aforementioned calibrated traction threshold(s), method 70 repeats logic block B72. When ECU 50 definitively determines that one or more of the calibrated traction thresholds have been exceeded, method 70 proceeds to logic block B80 in an alternative manner.

[0050] Logic block B80 relates to performing control actions based on a digital traction coefficient, wherein the control action indicates retinal traction. For example, ECU 50 can activate indicator devices 20A and / or 20B appropriately in response to the digital traction coefficient exceeding a calibrated traction threshold. In use Figure 3In embodiments of the multi-axis surgical robot 60, control actions may include transmitting motion control signals to the surgical robot 60, and in particular to one or more of its rotary joints as understood in the art, to change the location of force unloading, or, in possible autonomous embodiments, to change scraping forces, pull / drag forces, and / or other values ​​required in response to a digital traction coefficient.

[0051] As part of logic block B80, when determining which of a number of possible control actions ECU 50 should execute, ECU 50 can take into account the magnitude of exceeding a given traction threshold in logic block B78. That is, the control action can be proportionate to the magnitude of the difference between the current level of retinal traction and the exceeded traction threshold, where ECU 50 may escalate the corresponding alarm as the magnitude increases. Executing the control action may optionally include adjusting the traction coefficient when it exceeds a calibrated traction threshold. Figure 1 The settings of one or more displays 20 and / or 200.

[0052] Illustrative examples include those for Figure 4 The representative regions Z1 and Z2 are used to establish corresponding traction thresholds. ECU 50 can be accessed via... Figure 1 The display screen has 20 and / or 200 displays. Figure 4 A color-coded version of the heatmap 45 will allow surgeons to readily identify whether excessive traction has been applied in a particular area relative to another. In an embodiment, the ECU 50 can automatically adjust the color and / or brightness of the "hotter" area as retinal traction in a given area gradually increases, such as by gradually changing the color of that area from yellow to red as retinal traction increases. When the traction threshold of a given area is exceeded, the ECU 50 can be activated in a specific complementary manner (such as by emitting a warning sound, vibration, displaying a warning message, etc.). Figure 3 The indicator device 20B. In addition to or besides this warning message, the ECU 50 can prompt the surgeon to use different surgical tools 34, such as suggesting the use of a circular scraper instead of forceps.

[0053] By using Figure 1 and Figure 3 ECU 50 Figure 4 Intuitive heatmap 45 and Figure 6 Method 70 shown is executed. Figure 3 The surgeon performing the ophthalmic procedure 13 shown is aware of the actual level of traction applied to the retina 25 in a visual and potentially local manner (if necessary). Because an alarm is only triggered if and only when a given threshold is exceeded, this method is minimally invasive and easily accessible via calibrable input 58. Figure 3The arrow (CAL) can be customized to meet the preferences of a given surgeon.

[0054] Regarding this optional customization, this document recognizes that surgical outcomes are highly dependent on the individual skill and technique employed by a given surgeon. To this end, the ECU 50 of this disclosure can be configured to present a range of threshold sensitivity options, possibly including a default setting that uses a calibrated set of traction thresholds for all patients. However, in some implementations, the surgeon may wish to deviate from this default setting to appropriately account for surgical preferences or to consider different patient-specific / variable parameters such as age, sex, health condition (e.g., diabetes, glaucoma, hypertension, etc.), prior injury, disease and / or surgery, past outcomes, etc. For example, in some methods, the surgeon may be prompted via the ECU 50 to answer a set of questions about the patient, including any or all of the example parameters above, and the ECU 50 may then suggest or automatically select the corresponding threshold to be applied in conjunction with the traction chart 55.

[0055] Regarding threshold adjustment, in addition to the use of a method 70 tailored to a given patient and / or surgeon, the traction threshold can be adjusted over time using historical outcomes. As an example, using historical outcomes from multiple surgeries over many days, months, or years, it can be determined over time that specific control actions associated with exceeding one or more traction thresholds may be unnecessary or premature. In this case, the ECU 50 can selectively increase the traction threshold during subsequent surgeries to allow the surgeon to utilize their professional judgment to apply a higher level of retinal traction when needed, without concern for damaging the ILM 31 or retina 25. Alternatively, the ECU 50 can decrease the traction threshold when historical outcomes indicate that the applied traction threshold is too low, potentially leading to damage or unsatisfactory results.

[0056] Those skilled in the art will recognize that method 70 can be implemented as recorded in Figure 3 Computer-readable instructions in memory 54 or a separate memory location, wherein the instructions executed by CPU 52 allow CPU 52 to quantize Figure 3 The diagram schematically illustrates retinal traction on the retina 25 of the patient's eye 30. That is, the execution of the instructions embodying method 70 causes the system comprised of CPU 52 and memory 54 to operate during ophthalmic surgery 13. Figure 3 The stereo camera 16 receives stereo image data 38 and assigns tracking points within the stereo image pair 38P.

[0057] The execution of the instructions also causes the CPU 52 to automatically perform the aforementioned DIC processing on the 38P using the stereo image, thereby determining the relative motion of the tracking points, for example, as... Figure 5 As shown, and using Figure 3 The traction diagram 55 correlates the relevant motion of the tracking point with retinal traction as a digital traction coefficient indicating the magnitude of retinal traction. When the digital traction coefficient exceeds one or more calibrated traction thresholds, the CPU 52, alone or using other associated hardware, sends a control signal (arrow CC). 20A The signal is transmitted to the indicator device 20A. For example, a control signal (arrow CC) 20A It can be configured to display Figure 4 The color-coded heatmap 45 shown above, thereby via Figure 1 The display screens 20 and / or 200 visually display retinal traction.

[0058] The detailed descriptions and accompanying drawings are supportive and descriptive of this disclosure, but the scope of this disclosure is defined only by the claims. While some best modes and other embodiments for implementing the claimed disclosure have been described in detail, various alternative designs and embodiments exist to practice the disclosure as defined in the appended claims.

[0059] Furthermore, the features of the embodiments shown in the accompanying drawings or the various embodiments mentioned in this specification are not necessarily to be construed as independent embodiments. Rather, each feature described in one of these examples of embodiments may be combined with one or more other desired features from other embodiments to produce other embodiments not described in words or with reference to the accompanying drawings. Therefore, such other embodiments fall within the scope of the appended claims.

Claims

1. A tracking system for quantifying retinal traction in a patient's eye during ophthalmic surgery, the tracking system comprising: Indicator device; A stereo camera, configured to collect and output stereo image data of the retina of the patient's eye; as well as An electronic control unit (ECU) communicates with the indicator device and the stereo camera, wherein the ECU is configured to: During the ophthalmic surgery, the stereoscopic image data, including stereoscopic image pairs, is received from the stereoscopic camera. The tracking points are assigned to the overlapping pixels of the stereo image pair; The stereo image pair is automatically subjected to digital image correlation (DIC) processing to determine the relative motion of the tracking points; The relative motion of the tracking point is correlated with the retinal traction using the traction map of the ECU, serving as a digital traction coefficient indicating the amplitude of the retinal traction; and The control action is performed based on the digital traction coefficient, wherein the control action instructs the retinal traction.

2. The tracking system of claim 1, wherein, The stereo camera includes a local control processor integrated with the ECU.

3. The tracking system as described in claim 1, wherein, As part of the DIC processing, the ECU is configured to apply a solid-body motion filter to the relative motion to account for the solid-body motion of the retina.

4. The tracking system as described in claim 1, wherein, The indicator device includes one or more high-resolution displays.

5. The tracking system as described in claim 4, wherein, The ECU is configured to perform the control action based on the digital traction coefficient by adjusting the settings of the one or more high-resolution displays when the digital traction coefficient exceeds a calibrated traction threshold.

6. The tracking system as described in claim 5, wherein, Adjusting the settings of the one or more high-resolution displays includes displaying a color-coded heatmap of the retinal traction via the one or more high-resolution displays.

7. The tracking system as claimed in claim 6, wherein, The ECU is configured to display a color-coded heatmap of the retinal traction on top of a stereoscopic image formed by the stereoscopic image pair.

8. The tracking system as claimed in claim 1, wherein, The ECU is configured to perform the control action in response to the digital traction coefficient exceeding a calibrated traction threshold.

9. The tracking system as claimed in claim 8, wherein, The ECU is configured to receive calibrable input from the user of the tracking system and adjust the calibrated traction threshold based on the calibrable input.

10. The tracking system of claim 8, wherein, The calibrated traction thresholds include multiple traction thresholds, each corresponding to a different area of ​​the retina.

11. The tracking system according to any one of claims 5 and 8-10, wherein, The ECU is configured to adjust the calibrated traction threshold over time in response to historical data indicating past surgical outcomes.

Images