Automatic image guidance for ophthalmic surgery

Through automated detection and image registration, the ophthalmic system reduces manual interaction in surgical procedures, solves the shortcomings of manual initialization in traditional systems, and improves the efficiency and safety of surgical procedures.

CN118946304APending Publication Date: 2024-11-12ALCON INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380028451.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-09-12
Filing Date
2023-09-07
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional image guidance systems require manual initialization by the surgeon during surgery, resulting in additional steps and distractions, and may occupy operating room space and use non-sterile equipment.

Method used

An automatically initialized ophthalmic system is designed to monitor in-operative scenes through an imaging device, detect the patient's eyes and automatically initiate image guidance, and generate coverage content using computer-assisted image registration to reduce or eliminate manual interactions.

Benefits of technology

It significantly improves the surgical experience of surgeons, reduces obstacles to image guidance technology, reduces relevant time and space requirements, and improves the safety and effectiveness of the surgery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118946304A_ABST
    Figure CN118946304A_ABST
Patent Text Reader

Abstract

In some embodiments, an ophthalmic system and computer-implemented method for automatically initializing image-guided procedures is described. The initialization includes monitoring the scene using a plurality of images captured by the first imaging device. An eye of a user is detected within a first image of the plurality of images. In response to detecting the user's eye, a registration procedure is initiated using the first image and a reference image of the user's eye to generate a set of transformation information. A set of overlay content is generated based on the set of transformation information. The set of overlay content includes a transformed first image or a transformed reference image. The overlay content is presented onto the scene via the second imaging device.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Image guidance generally refers to any form of medical imaging used to plan, perform, and evaluate surgical procedures, including but not limited to ophthalmic microsurgery (e.g., vitreoretinal surgery (such as retinotomy, retinal resection, autologous retinal transplantation, etc.), and anterior segment surgery (such as cataract surgery, minimally invasive glaucoma surgery (MIGS), etc.)). For example, image guidance can involve real-time tracking and guiding instruments (e.g., lasers, probes, etc.) during surgery, providing visual feedback to the surgeon (or physician) performing the surgery, supporting manual execution of treatment via visual aids (e.g., augmented reality using a surgical microscope), etc. Summary of the invention

[0002] In certain embodiments, an ophthalmic system is provided. The ophthalmic system includes a first imaging device, a second imaging device, a memory including executable instructions, and a processor in data communication with the memory. The first imaging device is suitable for capturing multiple images of a scene within an intraoperative environment. The second imaging device is suitable for visualizing the scene. The processor is configured to execute the executable instructions to monitor the scene using the multiple images captured by the first imaging device. The processor is also configured to execute the executable instructions to automatically initialize image-guided surgery when the user's eye is detected in the first image of the multiple images, which includes starting a registration procedure based at least in part on the first image and a reference image of the user's eye to generate a set of transformation information. The processor is also configured to execute the executable instructions to generate a set of overlay content based on the set of transformation information, wherein the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image. The processor is further configured to execute the executable instructions to present the overlay content onto the scene via the second imaging device.

[0003] In certain embodiments, a computer-implemented method is provided. The computer-implemented method includes monitoring a scene in an intraoperative environment using a plurality of images captured by a first imaging device. The computer-implemented method also includes automatically initializing image-guided surgery when an eye of a user is detected within a first image of the plurality of images, wherein automatically initializing the image-guided surgery includes initiating a registration procedure based at least in part on the first image and a reference image of the user's eye to generate a set of transformation information. The computer-implemented method further includes generating a set of overlays based on the set of transformation information, wherein the set of overlays includes (i) the transformed first image or (ii) the transformed reference image. The computer-implemented method further includes presenting the overlays onto the scene via a second imaging device.

[0004] In certain embodiments, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium has computer-executable instructions stored thereon. The computer-executable instructions may be executed by one or more processors to perform operations. The operations include monitoring a scene in an intraoperative environment using a plurality of images captured by a first imaging device. The operations also include automatically initializing image-guided surgery when an eye of a user is detected in a first image of the plurality of images, wherein automatically initializing the image-guided surgery includes starting a registration procedure based at least in part on the first image and a reference image of the user's eye to generate a set of transformation information. The operations further include generating a set of overlay content based on the set of transformation information, wherein the set of overlay content includes (i) a transformed first image or (ii) a transformed reference image. The operations further include presenting the overlay content onto the scene via a second imaging device. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order to be able to understand the above-mentioned features of the present disclosure in detail, the present disclosure briefly summarized above can be described in more detail with reference to the embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings only show exemplary embodiments and are therefore not to be considered as limiting the scope thereof, and other equally effective embodiments may be allowed.

[0006] Figure 1 An example ophthalmic system for automatically initiating image-guided surgery in accordance with certain embodiments is presented.

[0007] Figure 2 An example workflow for automatically initiating image-guided surgery in accordance with certain embodiments is presented.

[0008] Figure 3 is a flow chart of a method for automatically initiating image-guided surgery, according to certain embodiments.

[0009] Figure 4 is a flow chart of another method for automatically initiating image-guided surgery in accordance with certain embodiments.

[0010] Figure 5 An example computing system for automatically initiating image-guided surgery in accordance with certain embodiments is presented.

[0011] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation. DETAILED DESCRIPTION

[0012] A critical component of an image-guided system, particularly in an ophthalmic surgical setting, is the ability to register one or more preoperative images of a patient with one or more intraoperative images of the patient. For example, during image registration, different image data sets are typically transformed into a coordinate system with matching imaging content (or features). Image registration can be a computer-assisted process in which features in (multiple) intraoperative images are detected based on predefined descriptions, such as pattern matching of manual markers, registration against preoperative images (also referred to as reference images), etc. Establishing image correspondences through image registration is critical for many clinical tasks, including but not limited to ophthalmic microsurgery, etc. For example, images transformed through image registration can be used during a surgical procedure (e.g., in real time) to help the surgeon orient to the location of various features / structures associated with the patient's eye.

[0013] One problem with conventional image-guided systems is that they typically rely on the surgeon (or another medical operator) to manually initialize image-guided surgery, for example, by manually triggering an image registration procedure. For example, an exemplary conventional image-guided system may include an initial alignment step in which the surgeon must position the eye under a microscope. Once the surgeon is satisfied with the position of the patient's eye, the surgeon typically must manually press a button to trigger the image-guided system to attempt image registration. This manual intervention by the surgeon creates an additional step that requires time and attention and may distract the surgeon from the actual surgical procedure at hand. In addition, relying on the surgeon's manual intervention to initialize image guidance may involve additional input devices (e.g., foot pedals, touch screens, etc.), which may take up space in the operating room and / or even involve additional non-sterile equipment (e.g., circulators) to operate the input devices. Therefore, it is desirable to provide techniques, systems, and devices for automatically initializing image-guided operations for surgical procedures (e.g., image-guided surgery).

[0014] Embodiments described herein provide systems, techniques, and apparatus for automatic initialization of performing image-guided surgery. Automatic initialization can include automatically initiating image registration of an intraoperative image of a patient's eye with a reference image of the patient's eye (such as a preoperative image). In certain embodiments, automatic initiation of image registration can reduce (or completely eliminate) the manual interaction typically required to begin image-guided surgery.

[0015] In an exemplary embodiment, an ophthalmic system includes an imaging device (e.g., a camera) and a visualization tool (e.g., a surgical microscope). The ophthalmic system can monitor (via the imaging device) an intraoperative scene under the visualization tool, and automatically trigger the initialization of image guidance of a surgical procedure once an eye of a patient is detected in the intraoperative scene. For example, the ophthalmic system can trigger the initialization by automatically initiating a computer-assisted image registration of an intraoperative image (captured via the imaging device) with a reference image (e.g., a preoperative image of the patient's eye) without any user input.

[0016] The ophthalmic system can generate overlay content based on image registration and present the overlay content to the surgeon via a visualization tool. For example, the overlay content can include a transformed reference image and / or a transformed intraoperative image, which is displayed as an overlay on the field of view (FOV) of the visualization tool. When the surgeon views the eye via the visualization tool, the overlay display can provide the surgeon with preoperative information about the eye, thereby improving the safety and / or effectiveness of the surgical procedure. For example, the overlay content can be used for patient verification, left / right eye examination, image guidance, etc.

[0017] Additionally or alternatively, in some embodiments, the ophthalmic system can provide feedback to the surgeon via the visualization tool regarding the proper centering, magnification, and focus of the patient's eye. Such feedback can be provided during monitoring of the scene and can be provided via augmented reality in the surgeon's FOV through the visualization tool.

[0018] Thus, certain embodiments of the ophthalmic systems described herein can reduce or completely eliminate the manual interactions that conventional systems typically require when starting an image-guided procedure. By reducing or eliminating these manual interactions, the embodiments described herein can significantly improve the surgeon's experience during surgery, lower the barriers to adopting image-guided technology, reduce the time and / or space associated with performing image-guided surgery, and the like.

[0019] As used herein, hyphenated reference numerals refer to specific instances of an element, while unhyphenated reference numerals refer to the element collectively. Thus, for example, device '12-1' refers to an instance of a class of devices, which may be collectively referred to as device '12', and any one of the class of devices may be generally referred to as device '12'.

[0020] Figure 1An example ophthalmic system 100 for performing automated image-guided surgery according to certain embodiments is shown. The example ophthalmic system 100 includes an imaging device 150, a computing system 110, and a visualization tool 140. The imaging device 150 represents various imaging devices that can capture images of a surgical scene. The example imaging device 150 is a digital camera and other imaging devices now known or later developed. The computing system 110 represents various computing systems (or devices), including, for example, a laptop computer, a mobile computer (e.g., a tablet computer or a smart phone), a server computer, a desktop computer, an imaging system, a system embedded in a medical device, and the like.

[0021] The visualization tool 140 represents an intraoperative imaging device used in an operating room during a surgical procedure to generate an image of a patient's eye 160. Example visualization tools 140 are digital microscopes (e.g., three-dimensional (3D) stereo digital microscopes), digital cameras, digital fundus cameras, analog microscopes with separate or integrated displays, monitor(s), heads-up display(s), augmented reality devices added using overlays, and other imaging devices now known or later developed. The computing system 110 includes an automatic image guidance tool 115, which is generally configured to automatically initialize image guidance for a surgical procedure. The automatic image guidance tool 115 includes an eye detection component 120, a registration component 125, and an overlay component 130, each of which may include hardware components, software components, or a combination thereof.

[0022] In some embodiments, the automatic image guidance tool 115 monitors the scene of the patient's eye 160 via the imaging device 150, and determines whether to automatically trigger the initialization of image guidance for the surgical procedure based on the monitoring. For example, the imaging device 150 can continuously capture one or more intraoperative images 180 of the scene and send the (multiple) intraoperative image 180 to the automatic image guidance tool 115. In some embodiments, the automatic image guidance tool 115 evaluates the (multiple) intraoperative image 180 using one or more predefined criteria and selects at least one intraoperative image in the (multiple) intraoperative image 180 that meets the predetermined conditions. For example, the automatic image guidance tool 115 can select the intraoperative image 180 in which the patient's eye 160 is detected.

[0023] In certain embodiments, after selecting the intraoperative image 180, the automatic image guidance tool 115 automatically initializes image guidance for the surgical procedure based at least in part on the intraoperative image 180. In an exemplary embodiment, the automatic image guidance tool 115 initializes image guidance by automatically attempting image registration of the intraoperative image 180 with the reference image 190 using the registration component 125. The intraoperative image 180 and the reference image 190 may include different views of the eye. The different views within the intraoperative image 180 and the reference image 190 may be due in part to the intraoperative image 180 and the reference image 190 being captured at different times, at different viewpoints, using different modalities, etc. In the depicted embodiment, the reference image 190 is an image of the patient's eye 160 captured during a preoperative phase, and the intraoperative image 180 is an image of the patient's eye 160 captured during an intraoperative phase. In another example, the reference image 190 and the intraoperative image 180 may be captured during gyration of the eye when the patient changes from a sitting position to a supine position during surgery.

[0024] In some embodiments, reference image 190 includes one or more annotations (or marks) made by a user (e.g., a surgeon or surgical staff member). For example, during the preoperative stage, the user can assess the clinical characteristics of the eye based on reference image 192 and annotate reference image 190 with information associated with the clinical characteristics.

[0025] In some embodiments, the automatic image guidance tool 115 generates a set of transformation information based on the image registration, and the set of transformation information is used to transform the reference image 190 into the coordinate system of the intraoperative image 180, or to transform the intraoperative image 180 into the coordinate system of the reference image 190. For example, the set of transformation information may include translation information, scaling information, and / or rotation information of at least one of the intraoperative image 180 or the reference image 190.

[0026] In some embodiments, the automatic image guidance tool 115 generates overlay content 175 (using overlay component 130) based on the transformed output from the image registration. The overlay content 175 may include a transformed intraoperative image, a transformed reference image 190, and / or additional information related to the reference image 190 and / or the intraoperative image (e.g., patient data, treatment data, etc.). The automatic image guidance tool 115 sends the overlay content 175 to the visualization tool 140, so that the overlay content 175 can be displayed (or presented) in the user's FOV by the visualization tool 140. In some embodiments, the overlay content 175 may include features selected from the intraoperative image 180 and / or the reference image 190, which can be used to verify the intended patient and / or perform a left / right eye examination. It should be noted that the following description of Figure 2 The eye detection component 120, the registration component 125, and the covering component 130 are described in more detail.

[0027] It should be noted that Figure 1 Reference examples of ophthalmic systems for automatically initializing image-guided surgery are presented, and in other embodiments, the ophthalmic systems may have different configurations. For example, although Figure 1 Computing system 110 is depicted as being separate from imaging device 150, but in some embodiments, computing system 110 may be part of imaging device 150. Similarly, although Figure 1 Computing system 110 is depicted as being separate from visualization tool 140, but in some embodiments, computing system 110 may be part of visualization tool 140. Figure 1 The imaging device 150 and the visualization tool 140 are depicted as separate devices, but in some embodiments, the operations of the imaging device 150 and the visualization tool 140 may be performed by a single device (e.g., a digital microscope including optical components (such as an objective lens) and a digital camera for outputting images). Furthermore, in some embodiments, the computing system 110, the imaging device 150, and the visualization tool 140 may be part of a single computing device (or system) that performs automatic initialization of image-guided surgery.

[0028] Figure 2 An example workflow 200 for automatically initiating image-guided surgery according to some embodiments is shown. The workflow 200 can be performed by the automatic image guidance tool 115. In the depicted embodiment, the eye detection component 120 receives one or more intraoperative images 180 of the scene. As described above, the intraoperative image(s) 180 can be captured by the imaging device 150.

[0029] The eye detection component 120 is generally configured to detect when the patient's eye 160 is in the scene and has a particular arrangement suitable for performing image registration. For example, when one or more visualization parameters (e.g., focus, magnification, lighting, position of the eye relative to the visualization tool 140, etc.) are being adjusted, the eye detection component 120 can use the eye detection algorithm 220 to evaluate the intraoperative image 180 to detect when the patient's eye has a particular arrangement. The visualization parameter(s) can be adjusted manually by a user (e.g., a surgeon or surgical staff), via a hands-free user device (e.g., a foot pedal), or automatically by the visualization tool 140.

[0030] In some embodiments, the eye detection algorithm 220 may determine whether the patient's eye is present within the intraoperative image 180 based on whether the intraoperative image 180 includes one or more features (or landmarks) associated with eye tissue / structure. These features may include, but are not limited to, for example, a speculum, a limbus, a center point, etc. In some embodiments, the eye detection algorithm 220 may determine the placement of the patient's eye within the intraoperative image 180 based on one or more of these features. For example, the eye detection algorithm 220 may determine the position of the patient's eye relative to the visualization tool 140 based on the speculum. In another example, the eye detection algorithm 220 may determine the dimensions of the patient's eye (relative to the dimensions of the reference image 190) based on the limbus. In yet another example, the eye detection algorithm 220 may determine the center point based on the center of the limbus.

[0031] In some embodiments, the eye detection algorithm 220 generates positioning feedback 230 based on the evaluation of the intraoperative image(s) 180. The positioning feedback 230 may include information about the position of the patient's eye 160 (relative to the visualization tool 140), the orientation of the patient's eye 160, the center point, etc. In some embodiments, the positioning feedback 230 may be displayed using augmented reality in the FOV of the visualization tool 140. The positioning feedback 230 may provide guidance to the user on how to adjust one or more parameters of the visualization tool 140 (e.g., the position of the visualization tool 140 relative to the patient's eye 160, focus, magnification, lighting, etc.), how to center the patient's eye, etc. For example, the positioning feedback 230 may be displayed to the user (via the visualization tool 140) while the user is adjusting one or more parameters of the visualization tool 140 and / or is positioning the patient's eye 160 within the scene.

[0032] like Figure 2 As shown, in some embodiments, the eye detection algorithm 220 selects one of the intraoperative images 180 that satisfies a predetermined condition as a candidate intraoperative image 235. In an exemplary embodiment, the eye detection algorithm 220 may select an intraoperative image 180 having a threshold number of detectable features as a candidate intraoperative image 235. In another exemplary embodiment, the eye detection algorithm may select the first intraoperative image 180 in which the limbus and / or the speculum are detected.

[0033] In some embodiments, the eye detection algorithm 220 sends the candidate intraoperative image 235 to the registration component 125 to trigger the initiation of image registration of the candidate intraoperative image 235 with the reference image 190. As described above, image registration can generally be performed as part of the initialization of image-guided surgery. The registration component 125 includes one or more registration algorithms 240. Examples of the registration algorithm 240 can include an image processing-based registration algorithm, a machine learning-based registration algorithm, an image processing and machine learning-based registration algorithm, and the like.

[0034] In some embodiments, as part of the image registration, the registration component 125 generates transformation information 255, which may include at least one of scaling information, translation information (including center information), and / or rotation information for transforming at least one of the candidate intraoperative image 235 or the reference image 190. In an exemplary embodiment, the transformation information 255 includes at least one of scaling information, translation information (including center information), and / or rotation information for transforming the reference image 190.

[0035] The registration component 125 sends the transformation information 255 to the overlay component 130, which includes an overlay generator 260 and a verification tool 265. The overlay generator 260 is generally configured to generate overlay content 275 based on the transformation information 255. In some embodiments, the overlay content 275 includes an overlay of the candidate intraoperative image 235 and the reference image 190 aligned using the transformation information 255. For example, the overlay content 275 may include a (transformed) candidate intraoperative image and / or a (transformed) reference image transformed according to the transformation information 255.

[0036] The verification tool 265 is generally configured to verify that the transformation information 255 meets a predetermined quality level. For example, in some embodiments, the verification tool 265 can use multiple different registration algorithms 240 to perform image registration and verify that the corresponding transformation information 255 output from each registration algorithm is the same or similar (e.g., the difference between the corresponding transformation information 255 is within a threshold). In some embodiments, the verification tool 265 can verify the transformation information 255 before generating the overlay content 275.

[0037] It should be noted that Figure 2 Reference example configurations of a workflow 200 that can be used to automatically initiate image-guided surgery are shown, and the workflow 200 can have other configurations consistent with the functionality described herein. For example, although the workflow 200 is described as being implemented with an eye detection component 120, a registration component 125, and an overlay component 130, the workflow 200 can also be implemented using any number of components (e.g., a single component, multiple components, etc.).

[0038] Figure 3 is a flow chart of an example method 300 for automatically initiating image-guided surgery according to some embodiments. The method 300 may be performed by an automated image guidance tool (eg, automated image guidance tool 115).

[0039] The method 300 enters at block 305, where the automated image-guided tool monitors a surgical scene in an intraoperative environment via a camera device (e.g., imaging device 150). As described above, the camera device can be configured to continuously capture multiple intraoperative images (e.g., intraoperative images 180) of the scene as a user (e.g., a surgeon or surgical staff member) adjusts one or more parameters (e.g., focus, magnification, lighting, position relative to the patient's eye, etc.) of a visualization tool (e.g., visualization tool 140) and / or the center position of the patient's eye.

[0040] At block 310, the automated image guidance tool detects an eye of a patient (e.g., the patient's eye 160) in an intraoperative environment based on monitoring. For example, the automated image guidance tool may employ an eye detection algorithm to assess whether one or more intraoperative images include one or more features indicative of eye tissue / structure.

[0041] At box 315, the automatic image guidance tool automatically initializes image-guided surgery using one of the images of the eye captured within the intraoperative environment and a reference image of the eye (e.g., reference image 190). In some embodiments, automatically initializing the image-guided surgery includes starting an image registration procedure using one of the images captured within the intraoperative environment and the reference image. In some embodiments, the automatic image guidance tool can select one of the intraoperative images that meets a predetermined condition as the image for the image registration procedure. The predetermined condition can include detecting a threshold number of features within the intraoperative image, being the first image in which (multiple) predefined features are detected, etc.

[0042] At block 320, the automatic image guidance tool presents an overlay generated from the registration procedure within the intraoperative environment. In some embodiments, the overlay may include an overlay of the selected intraoperative image and the reference image aligned using a set of transformation information (e.g., transformation information 255) generated from the registration procedure. The overlay may be presented (or displayed) onto the scene via the visualization tool (e.g., in the FOV of the visualization tool).

[0043] Figure 4 is a flow chart of an example method 400 for automatically initiating image-guided surgery according to some embodiments. The method 400 may be performed by an automatic image guidance tool (eg, the automatic image guidance tool 115).

[0044] The method 400 is entered at block 405, where an automated image-guided tool monitors a scene in an intraoperative environment via a camera device (eg, imaging device 150). The operations in block 405 may be similar to Figure 3 The operations in box 305 of method 300 in .

[0045] The operations in one or more of blocks 410, 415, 420, 425, 430, 435, 440, and 445 may be performed for each (intraoperative) image captured during monitoring. At block 410, the automated image guidance tool determines whether an eye is detected within the captured image. For example, the automated image guidance tool may determine that an eye is present (as opposed to another structure or material) when a speculum and / or limbus is detected within the captured image.

[0046] If an eye is not detected within the captured image at block 410, the method 400 proceeds to process another captured image. If an eye is detected within the captured image at block 410, the automatic image guidance tool determines a set of features associated with the eye from the image at block 415. The set of features may include one or more landmarks (or points of the eye), examples of which may include a speculum, limbus, center point, limbus scale, blood vessel(s), etc.

[0047] At box 420, the automatic image guidance tool determines whether the features satisfy one or more predetermined conditions. The predetermined condition(s) at box 420 may be based on a threshold number of features, a predefined scale of the limbus (e.g., based on the scale of the limbus within a reference image), a predefined center point. If the automatic image guidance tool determines at box 420 that the features do not satisfy the predetermined conditions (e.g., a threshold number of features are not detected, the eye is in an incorrect arrangement due to at least one of an incorrect position, scale, and / or center point), then at box 425, the automatic image guidance tool provides feedback to the user (e.g., a surgeon or surgical staff) regarding the positioning of the eye and proceeds to process another captured image.

[0048] If the automatic image-guided tool determines at block 420 that the feature satisfies the predetermined condition, then at block 430, the automatic image-guided tool performs a registration procedure on the image and the reference image to generate a set of transformation information (e.g., transformation information 255). At block 435, the automatic image-guided tool determines whether the transformation information satisfies the predetermined condition(s). For example, the predetermined condition(s) at block 435 may be associated with a threshold quality level. In some embodiments, the automatic image-guided tool may determine that the transformation information satisfies the predetermined condition when multiple registration algorithms output the same or similar set of transformation information.

[0049] If the automatic image guidance tool determines at block 435 that the transformation information satisfies the predetermined condition(s), then at block 440, the automatic image guidance tool generates and presents overlay content based on the transformation information. On the other hand, if the automatic image guidance tool determines at block 435 that the transformation information does not satisfy the predetermined condition(s), then at block 445, the automatic image guidance tool determines whether a manual trigger is satisfied. For example, in some embodiments, the automatic image guidance tool may determine that a manual trigger is satisfied when multiple image registration attempts have been initiated.

[0050] If the automatic image guidance tool determines at box 445 that the manual trigger is not satisfied, the method 400 proceeds to the above-mentioned box 425. If the automatic image guidance tool determines at box 445 that the manual trigger is satisfied, the method 400 can exit. In some embodiments, in response to the manual trigger, the automatic image guidance tool can prompt the user to provide input about at least a portion of the transformation information. For example, the automatic image guidance tool can prompt the user to adjust the rotation of at least one of the intraoperative image or the reference image of the image-guided surgery. In some embodiments, the user can provide input via a hands-free user device (e.g., a foot pedal).

[0051] Figure 5 An example computing system 500 configured to automatically initiate image-guided surgery according to certain embodiments is shown. As shown, the computing system 500 includes, but is not limited to, a processing unit 505, a network interface 515, a memory 520, and a storage device 560, each of which is connected to a bus 517. The computing system 500 may also include an I / O device interface 510 that connects an I / O device 512 (e.g., a keyboard, a display, and a mouse device) to the computing system 500. The computing system 500 is typically under the control of an operating system (not shown). Examples of operating systems include the UNIX operating system, multiple versions of the Microsoft Windows operating system, and distributions of the Linux operating system. (UNIX is a registered trademark of the International Open Standards Organization in the United States and other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.) More generally, any operating system that supports the functionality disclosed herein may be used.

[0052] The processing unit 505 may include one or more central processing units (CPUs) and / or one or more graphics processing units (GPUs). The processing unit 505 obtains and executes programming instructions stored in the memory 520 and stored in the storage device 560. The bus 517 is used to transmit programming instructions and application data between the processing unit 505, the I / O device interface 510, the storage device 560, the network interface 515 and the memory 520. Note that the processing unit 505 is included to represent a single CPU, multiple CPUs, a single CPU with multiple processing cores, a single GPU, multiple GPUs, a single GPU with multiple processing cores, or any combination thereof. The memory 520 is generally included to represent a random access memory. The storage device 560 may be a disk drive or a flash storage device. Although shown as a single unit, the storage device 560 may be a combination of fixed and / or removable storage devices, such as a fixed disk drive, a removable memory card, an optical storage device, a network attached storage device (NAS) or a storage area network (SAN). Illustratively, the memory 520 includes the automated image guidance tool 115 , which is discussed in greater detail above. Further, the storage device 560 includes the reference image 190 , the intraoperative image(s) 180 , the positioning feedback 230 , the transformation information 255 , and the overlay content 175 described above.

[0053] In summary, certain embodiments of the present disclosure allow for reduction or complete elimination of the manual interactions that are typically required by conventional systems when starting an image-guided surgery. By reducing or eliminating these manual interactions, embodiments can significantly improve the surgeon's experience during surgery, lower barriers to adopting image-guided technology, reduce the time and / or space associated with performing image-guided surgery, and the like.

[0054] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. For example, "at least one of a, b, or c" is intended to cover any combination of a, b, c, ab, ac, bc, and abc, as well as multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc or any other order of a, b, and c).

[0055] The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the claims are not intended to be limited to the embodiments shown herein, but are to be given the full scope consistent with the language of the claims.

[0056] In the claims, unless otherwise specified, references to singular elements are not intended to mean "one and only one", but "one or more". Unless otherwise specifically stated, the term "some" means one or more. All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or later known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, regardless of whether such disclosure is explicitly recited in the claims, the content disclosed herein is not intended to be dedicated to the public. Pursuant to 35 U.S.C. §112 (f), the elements of any claim will not be interpreted unless they are explicitly recited using the phrase "means for..." or, in the case of a method claim, the phrase "step for..." is used to recite these elements. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

Claims

1. An ophthalmic system comprising: a first imaging device adapted to capture a plurality of images of a scene within an intraoperative environment; a second imaging device adapted to visualize the scene; a memory including executable instructions; as well as a processor in data communication with the memory and configured to execute the executable instructions to perform the following operations: monitoring the scene using the plurality of images captured by the first imaging device; automatically initiating image-guided surgery when an eye of a user is detected within a first image of the plurality of images, comprising initiating a registration procedure based at least in part on the first image and a reference image of the eye of the user to generate a set of transformation information; generating a set of overlay content based on the set of transformation information, wherein the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image; as well as The overlay content is rendered onto the scene via the second imaging device.

2. The ophthalmic system of claim 1, wherein: The processor is further configured to execute the executable instructions to provide feedback on the position of the eye based on the monitoring.

3. The ophthalmic system of claim 2, wherein: The feedback is provided when the plurality of images are captured by the first imaging device.

4. The ophthalmic system of claim 1, wherein: The transformation information includes at least one of: (i) scaling information, (ii) translation information, or (iii) rotation information.

5. The ophthalmic system of claim 1, wherein: Detecting the user's eyes includes detecting, via an eye detection algorithm, a set of features associated with eye structures within the first image.

6. The ophthalmic system of claim 5, wherein: The set of features includes at least one of a speculum or a limbus.

7. The ophthalmic system of claim 1, wherein: The processor is further configured to execute the executable instructions to validate the set of transformation information prior to generating the overlay content.

8. The ophthalmic system of claim 1, wherein: The first image is an intraoperative image; and The reference image is a pre-operative image.

9. A computer-implemented method comprising: monitoring a scene in an intraoperative environment using a plurality of images captured by a first imaging device; automatically initiating an image-guided surgery when an eye of a user is detected within a first image of the plurality of images, wherein automatically initiating the image-guided surgery comprises initiating a registration procedure based at least in part on the first image and a reference image of the eye of the user to generate a set of transformation information; generating a set of overlay content based on the set of transformation information, wherein the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image; and The overlay content is rendered onto the scene via a second imaging device.

10. The computer-implemented method of claim 9, further comprising providing feedback of the position of the eye based on the monitoring.

11. The computer-implemented method of claim 10, wherein: The feedback is provided when the plurality of images are captured by the first imaging device.

12. The computer-implemented method of claim 9, wherein: The transformation information includes at least one of: (i) scaling information, (ii) translation information, or (iii) rotation information.

13. The computer-implemented method of claim 9, wherein: Detecting the user's eyes includes detecting, via an eye detection algorithm, a set of features associated with eye structures within the first image.

14. The computer-implemented method of claim 13, wherein: The set of features includes at least one of a speculum or a limbus.

15. The computer-implemented method of claim 9, further comprising validating the set of transformation information prior to generating the overlay content.

16. The computer-implemented method of claim 9, wherein: The first image is an intraoperative image; and The reference image is a pre-operative image.

17. A non-transitory computer readable medium having computer executable instructions stored thereon, the computer executable instructions executable by one or more processors to perform operations comprising: monitoring a scene in an intraoperative environment using a plurality of images captured by a first imaging device; automatically initiating an image-guided surgery when an eye of a user is detected within a first image of the plurality of images, wherein automatically initiating the image-guided surgery comprises initiating a registration procedure based at least in part on the first image and a reference image of the eye of the user to generate a set of transformation information; generating a set of overlay content based on the set of transformation information, wherein the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image; and The overlay content is rendered onto the scene via a second imaging device.

18. The non-transitory computer readable medium of claim 17, the operations further comprising providing feedback of the position of the eye based on the monitoring.

19. The non-transitory computer readable medium of claim 18, wherein: The feedback is provided when the plurality of images are captured by the first imaging device.

20. The non-transitory computer readable medium of claim 17, wherein: Detecting the user's eyes comprises detecting, via an eye detection algorithm, a set of features associated with eye structures within the first image; and The set of features includes at least one of a speculum or a limbus.