Automated image guidance for ophthalmic surgery

Abstract

In certain embodiments, an ophthalmic system and computer-implemented method for automatically initializing image-guided surgery are described. The initialization includes monitoring a scene using a plurality of images captured by a first imaging device. A user's eye is detected in a first image of the plurality of images. In response to detecting the user's eye, a registration procedure is initiated using the first image and a reference image of the user's eye to generate a set of transformation information. A set of overlay content is generated based on the set of transformation information. The set of overlay content includes the transformed first image or the transformed reference image. The overlay content is presented over the scene via a second imaging device.

Description

[Technical Field]

[0001] Image guidance generally refers to any form of medical imaging for planning, performing, and evaluating surgical procedures, including, but not limited to, microscopic ophthalmic procedures (e.g., vitreoretinal procedures such as retinotomy, retinal resection, retinal autotransplantation, etc., and anterior segment surgical procedures such as cataract surgery, minimally invasive glaucoma surgery (MIGS)). For example, image guidance may include tracking and guiding instruments (e.g., lasers, probes, etc.) in real time during a surgical procedure, providing visual feedback to a surgeon (or physician) performing a surgical procedure, supporting manual execution of a treatment via visual aids (e.g., augmented reality using a surgical microscope), etc. Summary of the Invention [Means for solving the problem]

[0002] In certain embodiments, an ophthalmic system is provided. The ophthalmic system includes a first imaging device, a second imaging device, a memory including executable instructions, and a processor in data communication with the memory. The first imaging device is adapted to capture multiple images of a scene within an intraoperative environment. The second imaging device is adapted to visualize the scene. The processor is configured to execute the executable instructions to monitor the scene using the multiple images captured by the first imaging device. The processor is also configured to execute the executable instructions to automatically initialize image-guided surgery upon detecting a user's eye in a first image of the multiple images, where the automatically initializing includes initiating a registration procedure to generate a set of transformation information based at least in part on the first image and a reference image of the user's eye. The processor is also configured to execute the executable instructions to generate a set of overlay content based on the set of transformation information, where the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image. The processor is further configured to execute the executable instructions to present the overlay content over the scene via the second imaging device.

[0003] In certain embodiments, a computer-implemented method is provided. The computer-implemented method includes monitoring a scene in an intraoperative environment using a plurality of images captured by a first imaging device. The computer-implemented method also includes automatically initializing image-guided surgery upon detecting a user's eye in a first image of the plurality of images, where automatically initializing image-guided surgery includes initiating a registration procedure to generate a set of transformation information based at least in part on the first image and a reference image of the user's eye. The computer-implemented method further includes generating a set of overlay content based on the set of transformation information, where the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image. The computer-implemented method further includes presenting the overlay content over the scene via a second imaging device.

[0004] In certain embodiments, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium has computer-executable instructions stored thereon. The computer-executable instructions are executable by one or more processors to perform operations. The operations include monitoring a scene in an intraoperative environment using a plurality of images captured by a first imaging device. The operations also include automatically initializing image-guided surgery upon detecting a user's eye in a first image of the plurality of images, where automatically initializing image-guided surgery includes initiating a registration procedure to generate a set of transformation information based at least in part on the first image and a reference image of the user's eye. The operations further include generating a set of overlay content based on the set of transformation information, where the set of overlay content includes (i) the transformed first image or (ii) the transformed reference image. The operations further include presenting the overlay content over the scene via a second imaging device.

[0005] So that the above-mentioned features of the present disclosure can be understood in detail, a more particular description of the present disclosure briefly summarized above can be had by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only exemplary embodiments and therefore should not be considered as limiting the scope of the invention, as other equally effective embodiments are possible. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an example ophthalmic system for automatically initiating image-guided surgery, according to certain embodiments. [Figure 2] FIG. 2 illustrates an example workflow for automatically initiating image-guided surgery, according to certain embodiments. [Figure 3] FIG. 3 is a flowchart of a method for automatically initiating image-guided surgery, according to certain embodiments. [Figure 4] FIG. 4 is a flowchart of another method for automatically initiating image-guided surgery, according to certain embodiments. [Figure 5] FIG. 5 illustrates an example computing system for automatically initiating image-guided surgery, according to certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] For ease of understanding, where possible, identical reference numerals will be used to designate identical elements common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further reference.

[0008] A critical element of image-guided systems, particularly in ophthalmic surgery settings, is the ability to register one or more preoperative images of a patient with one or more intraoperative images of the patient. For example, during image registration, different image datasets are typically transformed into a single coordinate system consistent with the imaged content (e.g., features). Image registration can be a computer-assisted process in which features within intraoperative images are detected based on previously known descriptions, such as pattern matching of manual marks, registration with preoperative images (also called reference images), etc. Establishing image correspondence through image registration is crucial for many clinical tasks, including, but not limited to, ophthalmic microscopy procedures. For example, the transformed images resulting from image registration can be used during a surgical procedure (in real time) to facilitate the surgeon's orientation regarding the location of various features / structures associated with the patient's eye.

[0009] One problem with conventional image-guided systems is that they typically rely on the surgeon (or another medical operator) to manually initialize image-guided surgery, for example, by manually triggering an image registration procedure. For example, an exemplary conventional image-guided system may include an initial alignment step that requires the surgeon to position the eye under a microscope. Once the surgeon is satisfied with the position of the patient's eye, the surgeon typically must manually press a button to trigger the image-guided system to attempt image registration. This manual intervention by the surgeon creates an additional step that requires time and attention and may distract the surgeon from the actual surgical procedure at hand. Furthermore, relying on manual intervention by the surgeon to initialize image guidance may involve additional input devices (e.g., foot pedals, touch screens, etc.), which may occupy space in the operating room and / or may even involve additional non-sterile equipment (e.g., circulators) to operate the input devices. Therefore, it would be desirable to provide techniques, systems, and devices that automatically initialize image-guided operations in a surgical procedure (e.g., image-guided surgery).

[0010] The embodiments described herein provide systems, techniques, and devices for automatic initialization of image-guided surgery. Automatic initialization can include automatically initiating image registration of intraoperative images of a patient's eye with a reference image, such as a preoperative image, of the patient's eye. In certain embodiments, automatic initiation of image registration can reduce (or completely eliminate) the manual interaction typically required to begin image-guided surgery.

[0011] In an exemplary embodiment, the ophthalmic system includes an imaging device (e.g., a camera) and a visualization tool (e.g., a surgical microscope). The ophthalmic system monitors an intraoperative scene under the visualization tool (via the imaging device) and may automatically trigger initialization of image guidance for the surgical procedure when the patient's eye is detected in the intraoperative scene. For example, the ophthalmic system may trigger initialization by automatically initiating computer-assisted image registration of the intraoperative image (captured via the imaging device) with a reference image (e.g., a preoperative image of the patient's eye) without any user input.

[0012] The ophthalmic system may generate overlay content based on the image registration and present the overlay content to the surgeon via the visualization tool. For example, the overlay content may include a transformed reference image and / or a transformed intraoperative image displayed as an overlay on the field of view (FOV) of the visualization tool. Displaying the overlay may provide the surgeon with preoperative information about the eye as the surgeon views the eye through the visualization tool, thereby improving the safety and / or effectiveness of the surgical procedure. For example, the overlay content may be used for patient identification, left eye / right eye checks, image guidance, etc.

[0013] Additionally or alternatively, in certain embodiments, the ophthalmic system may provide feedback to the surgeon via the visualization tool regarding proper centering, magnification, and focus of the patient's eye. Such feedback may be provided during scene monitoring and / or via augmented reality in the surgeon's FOV through the visualization tool.

[0014] Accordingly, certain embodiments of the ophthalmic systems described herein can reduce or completely eliminate the manual interactions typically required by conventional systems to initiate image-guided surgery. By reducing or eliminating these manual interactions, the embodiments described herein can significantly improve the surgeon's experience during a surgical procedure, lower the barrier to adoption of image-guided surgery, reduce the time and / or space associated with performing image-guided surgery, etc.

[0015] As used herein, hyphenated forms of reference numerals refer to specific instances of elements, while unhyphenated forms of reference numerals refer to collective elements. Thus, for example, device "12-1" refers to an instance of a class of devices that may be collectively referred to as device "12," and any one of which may be generically referred to as device "12."

[0016] 1 illustrates an example ophthalmic system 100 for performing automated image-guided surgery, according to certain embodiments. The ophthalmic system 100 includes an imaging device 150, a computing system 110, and a visualization tool 140. The imaging device 150 represents various imaging devices capable of capturing images of a surgical scene. An example imaging device 150 is a digital camera and other imaging devices now known or later developed. The computing system 110 represents various computing systems (or devices) including, for example, a laptop computer, a mobile computer (e.g., a tablet or smartphone), a server computer, a desktop computer, an imaging system, a system integrated into a medical device, etc.

[0017] The visualization tool 140 represents an intraoperative (intra-operative) imaging device used in an operating room during a surgical procedure to generate images of a patient's eye 160. Example visualization tools 140 are digital microscopes (e.g., three-dimensional (3D) stereo digital microscopes), digital cameras, digital fundus cameras, analog microscopes with separate or integrated displays, heads-up displays, augmented reality devices using overlay injection, and other imaging devices now known or later developed. The computing system 110 includes an automatic image guidance tool 115, which is generally configured to automatically initiate image guidance of a surgical procedure. The automatic image guidance tool 115 includes an eye detection component 120, a registration component 125, and an overlay component 130, each of which may include hardware components, software components, or a combination thereof.

[0018] In certain embodiments, the automatic image guidance tool 115 monitors the scene for the patient's eye 160 via the imaging device 150 and, based on the monitoring, determines whether to automatically trigger initialization of image guidance of the surgical procedure. For example, the imaging device 150 may continuously capture one or more intraoperative images 180 of the scene and transmit the intraoperative images 180 to the automatic image guidance tool 115. In certain embodiments, the automatic image guidance tool 115 evaluates the intraoperative images 180 using one or more predefined criteria and selects at least one of the intraoperative images 180 that meets the predetermined conditions. For example, the automatic image guidance tool 115 may select an intraoperative image 180 in which the patient's eye 160 is detected.

[0019] In certain embodiments, after selecting the intraoperative image 180, the automatic image guidance tool 115 automatically initiates image guidance of the surgical procedure based at least in part on the intraoperative image 180. In an exemplary embodiment, the automatic image guidance tool 115 initializes image guidance by automatically attempting image registration between the intraoperative image 180 and the reference image 190 using the registration component 125. The intraoperative image 180 and the reference image 190 may include different views of the eye. The different views in the intraoperative image 180 and the reference image 190 may be due, in part, to the intraoperative image 180 and the reference image 190 being captured at different times, from different perspectives, using different modalities, etc. In the illustrated embodiment, the reference image 190 is an image of the patient's eye 160 captured during a preoperative phase, and the intraoperative image 180 is an image of the patient's eye 160 captured during an intraoperative phase. In another example, the reference image 190 and the intraoperative image 180 may be captured during cyclorotation of the eye as the patient changes from a sitting position to a supine position during surgery.

[0020] In certain embodiments, the reference image 190 includes one or more annotations (or markings) made by a user, such as a surgeon or a surgical staff member. For example, during the preoperative phase, the user may assess the clinical characteristics of the eye based on the reference image 192 and annotate the reference image 190 with information related to the clinical characteristics.

[0021] In certain embodiments, the automatic image guidance tool 115 generates, based on the image registration, a set of transformation information for transforming the reference image 190 into the coordinate system of the intraoperative image 180 or for transforming the intraoperative image 180 into the coordinate system of the reference image 190. For example, the set of transformation information may include translation information, scaling information, and / or rotation information for at least one of the intraoperative image 180 or the reference image 190.

[0022] In certain embodiments, the automatic image guidance tool 115 generates overlay content 175 (using the overlay component 130) based on the transformation output from the image registration. The overlay content 175 may include the transformed intraoperative image, the transformed reference image 190, and / or additional information related to the reference image 190 and / or the intraoperative image (e.g., patient data, treatment data, etc.). The automatic image guidance tool 115 sends the overlay content 175 to the visualization tool 140, which may display (or present) the overlay content 175 within the user's FOV through the visualization tool 140. In certain embodiments, the overlay content 175 may include selected features from the intraoperative image 180 and / or the reference image 190, which may be used to verify the expected patient and / or perform a left eye / right eye check. Note that the eye detection component 120, the registration component 125, and the overlay component 130 are described in further detail below with respect to FIG. 2 .

[0023] It should be noted that while FIG. 1 illustrates a reference example of an ophthalmic system for performing automatic initialization of image-guided surgery, in other embodiments, the ophthalmic system may have a different configuration. For example, while FIG. 1 illustrates computing system 110 as separate from imaging device 150, in certain embodiments, computing system 110 may be part of imaging device 150. Similarly, while FIG. 1 illustrates computing system 110 as separate from visualization tool 140, in certain embodiments, computing system 110 may be part of visualization tool 140. Similarly, while FIG. 1 illustrates imaging device 150 and visualization tool 140 as separate devices, in certain embodiments, the operations of imaging device 150 and visualization tool 140 may be performed by a single device (e.g., a digital microscope including optical components (such as an objective lens) and a digital camera for outputting images). Furthermore, in certain embodiments, computing system 110, imaging device 150, and visualization tool 140 may be part of a single computing device (or system) for performing automatic initialization of image-guided surgery.

[0024] 2 illustrates an example workflow 200 for automatically initiating image-guided surgery, according to certain embodiments. Workflow 200 may be performed by automatic image guidance tool 115. In the illustrated embodiment, eye detection component 120 receives one or more intraoperative images 180 of a scene. As previously mentioned, intraoperative images 180 may be captured by imaging device 150.

[0025] Eye detection component 120 is generally configured to detect when a patient's eye 160 is in a scene and has a particular geometry suitable for performing image registration. For example, while one or more visualization parameters (e.g., focus, magnification, lighting, eye position relative to visualization tool 140, etc.) are being adjusted, eye detection component 120 can evaluate intraoperative images 180 using eye detection algorithm 220 to detect when a patient's eye has a particular geometry. The visualization parameters may be adjusted manually by a user (e.g., a surgeon or surgical staff member), may be adjusted via a hands-free user device (e.g., a foot pedal), or may be adjusted automatically by visualization tool 140.

[0026] In certain embodiments, the eye detection algorithm 220 may determine that the patient's eye is present in the intra-operative image 180 based on whether the intra-operative image 180 includes one or more features (or landmarks) associated with ocular tissue / structures. Such features may include, by way of example and not limitation, a speculum, a limbus, a centration point, etc. In certain embodiments, the eye detection algorithm 220 may identify the placement of the patient's eye in the intra-operative image 180 based on one or more of the features. For example, the eye detection algorithm 220 may identify the position of the patient's eye relative to the visualization tool 140 based on the speculum. In another example, the eye detection algorithm 220 may identify the scale of the patient's eye (relative to the scale of the reference image 190) based on the limbus. In yet another example, the eye detection algorithm 220 may identify the centration point based on the center of the limbus.

[0027] In certain embodiments, eye detection algorithm 220 generates positioning feedback 230 based on evaluation of intra-operative images 180. Positioning feedback 230 may include information regarding the position of patient's eye 160 (relative to visualization tool 140), the orientation of patient's eye 160, the centration point, etc. In certain embodiments, positioning feedback 230 may be displayed in the FOV of visualization tool 140 using augmented reality. Positioning feedback 230 may provide guidance to the user regarding how to adjust one or more parameters of visualization tool 140 (e.g., position, focus, magnification, lighting, etc. of visualization tool 140 relative to patient's eye 160), how to center the patient's eye, etc. For example, positioning feedback 230 may be displayed to the user (via visualization tool 140) while the user is adjusting one or more parameters of visualization tool 140 and / or while the user is positioning patient's eye 160 within the scene.

[0028] 2, in certain embodiments, the eye detection algorithm 220 selects one of the intraoperative images 180 that meets predetermined conditions as the candidate intraoperative image 235. In one exemplary embodiment, the eye detection algorithm 220 may select the intraoperative image 180 that has a threshold number of detectable features as the candidate intraoperative image 235. In another exemplary embodiment, the eye detection algorithm may select the first intraoperative image 180 in which the limbus and / or lid speculum are detected.

[0029] In certain embodiments, the eye detection algorithm 220 sends the candidate intraoperative image 235 to the registration component 125 to trigger initialization of image registration of the candidate intraoperative image 235 and the reference image 190. As noted, image registration may generally occur as part of the initialization of image-guided surgery. The registration component 125 includes one or more registration algorithms 240. Examples of the registration algorithms 240 may include an image processing-based registration algorithm, a machine learning-based registration algorithm, an image processing and machine learning-based registration algorithm, etc.

[0030] In certain embodiments, as part of image registration, registration component 125 generates transformation information 255, which may include at least one of scaling information, translation information (including centration information), and / or rotation information for transforming at least one of the candidate intraoperative image 235 or the reference image 190. In an exemplary embodiment, transformation information 255 includes at least one of scaling information, translation information (including centration information), and / or rotation information for transforming the reference image 190.

[0031] The registration component 125 sends the transformation information 255 to the overlay component 130, which includes an overlay generator 260 and a verification tool 265. The overlay generator 260 is generally configured to generate overlay content 275 based on the transformation information 255. In certain embodiments, the overlay content 275 includes an overlay of the candidate intraoperative image 235 and the reference image 190 aligned using the transformation information 255. For example, the overlay content 275 may include the candidate intraoperative image 235 and / or the reference image transformed according to the transformation information 255.

[0032] The validation tool 265 is generally configured to ensure that the transformation information 255 meets a predetermined quality level. For example, in certain embodiments, the validation tool 265 may perform image registration using multiple different registration algorithms 240 and verify that the transformation information 255 output from each registration algorithm is the same or similar (e.g., the difference between the transformation information 255 is within a threshold). In certain embodiments, the validation tool 265 may verify the transformation information 255 before generating the overlay content 275.

[0033] 2 shows a reference example configuration of a workflow 200 that may be used to automatically initiate image-guided surgery, it being noted that workflow 200 may have other configurations consistent with the functionality described herein. For example, although workflow 200 is described as being implemented using eye detection component 120, registration component 125, and overlay component 130, workflow 200 may be implemented using any number of components (e.g., a single component, multiple components, etc.).

[0034] 3 is a flowchart of an example method 300 for automatically initiating image-guided surgery, according to certain embodiments. Method 300 may be performed by an automatic image-guided tool (e.g., automatic image-guided tool 115).

[0035] Method 300 enters block 305, where an automated image guidance tool monitors a surgical scene in an intraoperative environment via a camera device (e.g., imaging device 150). As mentioned, the camera device may be configured to sequentially capture several intraoperative images of the scene (e.g., intraoperative images 180) while a user (e.g., a surgeon or surgical staff member) adjusts one or more parameters (e.g., focus, magnification, lighting, position relative to the patient's eye, etc.) of a visualization tool (e.g., visualization tool 140) and / or the centering of the patient's eye.

[0036] In block 310, the automated image guidance tool detects the patient's eye (e.g., patient's eye 160) in the intraoperative environment based on the monitoring. For example, the automated image guidance tool may employ an eye detection algorithm to evaluate whether one or more intraoperative images contain one or more features indicative of ocular tissue / structure.

[0037] In block 315, the automated image guidance tool automatically initializes image-guided surgery using one of the images of the eye captured within the intraoperative environment and a reference image of the eye (e.g., reference image 190). In certain embodiments, automatically initializing image-guided surgery includes starting an image registration procedure using one of the images captured within the intraoperative environment and the reference image. In certain embodiments, the automated image guidance tool may select one of the intraoperative images that meets predetermined conditions as the image to use in the image registration procedure. The predetermined conditions may include a threshold number of features detected in the intraoperative image, the first image in which a predefined feature is detected, etc.

[0038] In block 320, the automated image guidance tool presents overlay content generated from the registration procedure within the intraoperative environment. In certain embodiments, the overlay content may include an overlay of the selected intraoperative image and the reference image aligned using a set of transformation information (e.g., transformation information 255) generated from the registration procedure. The overlay content may be presented (or displayed) over the scene (e.g., in the FOV of the visualization tool) via the visualization tool.

[0039] 4 is a flowchart of another method 400 for automatically initiating image-guided surgery, according to certain embodiments. Method 400 may be performed by an automatic image-guided tool (e.g., automatic image-guided tool 115).

[0040] Method 400 enters block 405, where the automated image guidance tool monitors a scene within the intraoperative environment via a camera device (e.g., imaging device 150). The operations at block 405 may be similar to the operations at block 305 of method 300 in FIG.

[0041] One or more of the operations of blocks 410, 415, 420, 425, 430, 435, 440, and 445 may be performed for each (intra-operative) image captured during monitoring. In block 410, the automated image guidance tool determines whether an eye is detected in the captured image. For example, the automated image guidance tool may determine that an eye is present if the lid speculum and / or limbus are detected in the captured image (as opposed to another structure or material).

[0042] At block 410, if an eye is not detected in the captured image, method 400 proceeds to process another captured image. If an eye is detected in the captured image at block 410, the automated image guidance tool identifies a set of features associated with the eye from the image at block 415. The set of features may include one or more landmarks (or ocular points), examples of which may include the lid retractor, limbus, centration point, limbal scale, blood vessels, etc.

[0043] In block 420, the automatic image guidance tool determines whether the features satisfy one or more predetermined conditions. The predetermined conditions in block 420 may be based on a threshold number of features, a predetermined scale of the contour limbus (e.g., based on the scale of the limbus in the reference image), or a predetermined centration point. If the automatic image guidance tool determines in block 420 that the features do not satisfy the predetermined conditions (e.g., the threshold number of features were not detected, the eye is incorrectly positioned due to at least one of an incorrect position, scale, and / or centration point), the automatic image guidance tool in block 425 provides positioning feedback about the eye to a user (e.g., a surgeon or surgical staff member) and proceeds to process another captured image.

[0044] If the automatic image guidance tool determines in block 420 that the features meet the predetermined criteria, then in block 430, the automatic image guidance tool performs a registration procedure using the image and a reference image to generate a set of transformation information (e.g., transformation information 255). In block 435, the automatic image guidance tool determines whether the transformation information meets a predetermined condition. For example, the predetermined condition in block 435 may be related to a threshold quality level. In certain embodiments, the automatic image guidance tool may determine that the transformation information meets the predetermined condition if multiple registration algorithms output the same or similar sets of transformation information.

[0045] If the automatic image guidance tool determines that the transformation information satisfies the predetermined condition in block 435, then the automatic image guidance tool generates and presents overlay content based on the transformation information in block 440. On the other hand, if the automatic image guidance tool determines that the transformation information does not satisfy the predetermined condition in block 435, then the automatic image guidance tool determines whether a manual trigger is satisfied in block 445. For example, in certain embodiments, the automatic image guidance tool may determine that a manual trigger is satisfied if several image registration attempts have been initiated.

[0046] If the automatic image guidance tool determines at block 445 that the manual trigger is not satisfied, method 400 proceeds to block 425, described above. If the automatic image guidance tool determines at block 445 that the manual trigger is satisfied, method 400 may end. In certain embodiments, in response to the manual trigger, the automatic image guidance tool may prompt the user to provide input regarding at least a portion of the transformation information. For example, the automatic image guidance tool may prompt the user to adjust the rotation of at least one of the intraoperative image or the reference image for image-guided surgery. In certain embodiments, the user may provide input via a hands-free user device (e.g., a foot pedal).

[0047] FIG. 5 illustrates an example computing system 500 configured to automatically initiate image-guided surgery, according to certain embodiments. As shown, computing system 500 includes, without limitation, a processing unit 505, a network interface 515, memory 520, and storage 560, each connected to a bus 517. Computing system 500 may also include an I / O device interface 510 that connects I / O devices 512 (e.g., a keyboard, a display, and a mouse device) to computing system 500. Computing system 500 is typically under the control of an operating system (not shown). Examples of operating systems include the UNIX operating system, versions of the Microsoft Windows operating system, and distributions of the Linux operating system (UNIX is a registered trademark of The Open Group in the United States and / or other countries. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both). More generally, any operating system that supports the functionality disclosed herein may be used.

[0048] Processing unit 505 may include one or more central processing units (CPUs) and / or one or more graphics processing units (GPUs). Processing unit 505 retrieves and executes programming instructions stored in memory 520 and storage 560. Bus 517 is used to transfer programming instructions and application data between processing unit 505, I / O device interface 510, storage 560, network interface 515, and memory 520. Note that processing unit 505 is included to represent a single CPU, multiple CPUs, a single CPU with multiple processing cores, a single GPU, multiple GPUs, a single GPU with multiple processing cores, or any combination thereof. Memory 520 is included to generally represent random access memory. Storage 560 may be a disk drive or flash storage device. Although shown as a single unit, storage 560 may be a combination of fixed or removable storage devices, such as a fixed disk drive, a removable memory card, optical storage, network-attached storage (NAS), or a storage area network (SAN), etc. Illustratively, memory 520 includes automatic image guidance tool 115, which is discussed in more detail above. Additionally, storage 560 includes reference images 190, intraoperative images 180, positioning feedback 230, transformation information 255, and overlay content 175, as described above.

[0049] In summary, certain embodiments of the present disclosure may reduce or completely eliminate the manual interactions typically required by conventional systems to initiate image-guided surgery. By reducing or eliminating these manual interactions, embodiments may significantly improve the surgeon's experience during a surgical procedure, lower the barrier to adopting image-guided technology, reduce the time and / or space associated with performing image-guided surgery, etc.

[0050] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single elements. By way of example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or a, b, and c in any other order).

[0051] The foregoing description is provided to enable those skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Accordingly, the claims are not intended to be limited to the embodiments shown herein, but are to be accorded the full scope consistent with the language of the claims.

[0052] Reference to an element in the claims in the singular is intended to mean one or more, not one and only one, unless specifically stated otherwise. The term "some" refers to one or more, unless specifically stated otherwise. All structural and functional equivalents to the elements of the various embodiments described throughout this disclosure that are known or later become known to those skilled in the art are expressly incorporated by reference herein and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended as a dedication to the public, regardless of whether such disclosure is expressly recited in the claims. No element of a claim is to be construed under 35 U.S.C. § 112(f) unless the element is expressly recited using the phrase "means for" or, in the case of a method claim, the element is recited using the phrase "step for." The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

Claims

1. 1. An ophthalmic system comprising: a first imaging device adapted to capture a plurality of images of a scene within an intraoperative environment; a second imaging device adapted to visualize the scene; and a memory containing executable instructions; a processor in data communication with the memory, executing the executable instructions to: monitoring the scene using the plurality of images captured by the first imaging device; automatically initiating image-guided surgery upon detecting an eye of a user in a first image of the plurality of images, the automatic initialization including initiating a registration procedure to generate a set of transformation information based at least in part on the first image and a reference image of the eye of the user; generating a set of overlay content based on the set of transformation information, the set of overlay content including (i) a transformed first image or (ii) a transformed reference image; presenting the overlay content onto the scene via the second imaging device; and a processor configured to: Ophthalmic system including.

2. The ophthalmic system of claim 1 , wherein the processor is further configured to execute the executable instructions to provide feedback about the position of the eye based on the monitoring.

3. The ophthalmic system of claim 2 , wherein the feedback is provided while the plurality of images are being captured by the first imaging device.

4. The ophthalmic system of claim 1 , wherein the transformation information includes at least one of: (i) scaling information, (ii) translation information, or (iii) rotation information.

5. The ophthalmology system of claim 1 , wherein detecting the user's eyes comprises detecting a set of features in the first image associated with ocular structures via an eye detection algorithm.

6. The ophthalmic system of claim 5 , wherein the set of features includes at least one of a speculum or a limbus.

7. The ophthalmic system of claim 1 , wherein the processor is further configured to execute the executable instructions to verify the set of transformation information before generating the overlay content.

8. the first image is an intraoperative image; and The ophthalmic system of claim 1 , wherein the reference image is a pre-operative image.

9. 1. A computer-implemented method comprising: monitoring a scene within an intraoperative environment using a plurality of images captured by a first imaging device; automatically initiating image-guided surgery upon detecting an eye of a user in a first image of the plurality of images, the automatic initialization including initiating a registration procedure to generate a set of transformation information based at least in part on the first image and a reference image of the eye of the user; generating a set of overlay content based on the set of transformation information, the set of overlay content including (i) a transformed first image or (ii) a transformed reference image; presenting the overlay content onto the scene via a second imaging device; and 20. A computer-implemented method comprising:

10. The computer-implemented method of claim 9 , further comprising providing feedback about the eye position based on the monitoring.

11. The computer-implemented method of claim 10 , wherein the feedback is provided while the plurality of images are being captured by the first imaging device.

12. The computer-implemented method of claim 9 , wherein the transformation information includes at least one of: (i) scaling information, (ii) translation information, or (iii) rotation information.

13. 10. The computer-implemented method of claim 9, wherein detecting the eyes of the user comprises detecting a set of features in the first image associated with ocular structures via an eye detection algorithm.

14. The computer-implemented method of claim 13 , wherein the set of features includes at least one of a speculum or a limbus.

15. The computer-implemented method of claim 9 , further comprising validating the set of transformation information before generating the overlay content.

16. the first image is an intraoperative image; and The computer-implemented method of claim 9 , wherein the reference image is a pre-operative image.

17. A non-transitory computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions comprising: monitoring a scene within an intraoperative environment using a plurality of images captured by a first imaging device; automatically initiating image-guided surgery upon detecting an eye of a user in a first image of the plurality of images, the automatic initialization including initiating a registration procedure to generate a set of transformation information based at least in part on the first image and a reference image of the eye of the user; generating a set of overlay content based on the set of transformation information, the set of overlay content including (i) a transformed first image or (ii) a transformed reference image; presenting the overlay content onto the scene via a second imaging device; and 1. A non-transitory computer-readable medium executable by one or more processors to perform operations including:

18. 20. The non-transitory computer-readable medium of claim 17, wherein the actions further include providing feedback about the position of the eye based on the monitoring.

19. The non-transitory computer-readable medium of claim 18 , wherein the feedback is provided while the plurality of images are being captured by the first imaging device.

20. Detecting the eyes of the user includes detecting a set of features in the first image associated with ocular structures via an eye detection algorithm; and 20. The non-transitory computer-readable medium of claim 17, wherein the set of features includes at least one of a speculum or a limbus.

Images