System, method, and computer program product for optimizing the optical system of a surgical microscope having an integrated imaging system - Patents.com

Abstract

A system for optimizing an optical system is provided. The system is configured to calibrate a position of a reference arm of an interferometric imaging system to provide an initial calibrated position so that an image of the sample is visible when the sample is positioned at a working distance of an objective. An image is acquired using the initial calibrated position. The image quality of the acquired image is evaluated to determine whether the acquired image is a valid image. The path length of the reference arm is adjusted if it is determined that the acquired image is not a valid image. A difference between the calibrated position of the reference arm and an adjusted position of the reference arm is calculated. Elements of the system are adjusted based on the calculated difference so that the sample is visible when the sample is positioned at the working distance in the adjusted position.

Description

[Technical field]

[0001] Priority claim This application claims priority to U.S. Provisional Patent Application No. 62 / 987,481, filed March 10, 2020, entitled “Methods, Systems and Devices for Guided Microscope Optimization,” the contents of which are incorporated by reference herein as if set forth in their entirety.

[0002] Field The inventive concepts relate generally to imaging, and more particularly to microscopes having integrated OCT systems and related systems and methods. [Background technology]

[0003] The use of optical coherence tomography (OCT), for example during ophthalmic surgery, typically requires the integration of the OCT system with a surgical microscope. The surgical microscope provides the surgeon with an expanded view of the surgical field. Ophthalmic surgical microscopes are typically stereo zoom microscopes with a binocular viewing port for the surgeon and often have one or two observer viewing ports at 90 degrees (left and right) relative to the surgeon. A "working distance" is provided between the microscope objective and the surface of the patient's eye (sample) to ensure the surgeon has an adequate working area.

[0004] The integration of OCT with a surgical microscope allows the introduction of at least two different optical systems within the microscope, one for the OCT system and one for a standard white light imaging path through the microscope eyepiece. To obtain ideal image quality across each modality, the microscope is generally parfocal, i.e., each optical subsystem has a coincident focus that should correspond to the focal plane of the objective lens, as shown, for example, in FIG. 1. Thus, FIG. 1 represents the focal plane of a microscope objective lens with the ideal working distance (WD) as measured from the last glass surface of the lens.

[0005] Due to the presence of multiple optical components in the white light path, it is possible to set the microscope objective at a position other than the working distance and still obtain an image that appears to be in focus by compensating with other system components such as the ocular eyepiece. For example, FIG. 2 shows a working distance (WDshort) that is set too short by compensation using the diopter adjustment of the viewing eyepiece. FIG. 3 shows a working distance (WDlong) that is set too long by compensation using the diopter adjustment of the viewing eyepiece. Using the system shown in FIGS. 2 and 3 can lead to poor quality OCT images because the sample may not be in focus relative to the microscope objective, and this also applies to additional imaging modalities that may be added to the microscope.

[0006] As shown in FIG. 4, OCT systems are designed to have optimal image quality at the working distance of the microscope objective. FIG. 4 shows an OCT system with an imaging plane set to coincide with the focal plane of the objective. OCT uses the principle of low-coherence interferometry to obtain a three-dimensional (3D) image of a sample. The optical path length through the sample arm of an OCT system generally matches the optical path through the reference arm of the OCT system, which typically consists of a mirror mounted on a translatable stage. The specific position of this reference mirror in the reference arm generally corresponds to the working distance of the microscope objective. If the sample is placed at a position outside the working distance of the microscope objective, the reference mirror is moved a corresponding distance to ensure that the optical path lengths match. An initial calibration of the reference arm position to the working distance allows deviations from this position to be detected. Conventional methods for adjusting elements of the system are generally trial and error, i.e., no specific coordinates are provided for the adjustments made. Summary of the Invention [Means for solving the problem]

[0007] Some embodiments of the inventive concept provide a system for optimizing the optical system of a surgical microscope having an integrated imaging system. The system includes one or more processors and one or more storage devices, a first optical system associated with the surgical microscope, and a second optical system, different from the first optical system, associated with an interferometric imaging system, such as OCT. The system is configured to calibrate a position of a reference arm of the imaging system such that an image of the sample is visible when the sample is placed at a working distance of an objective lens of the surgical microscope to provide an initial calibrated position. The system is further configured to acquire an image using the initial calibrated position of the reference arm of the imaging system. An image quality of the acquired image is evaluated to determine whether the acquired image is a valid image of an intended sample target. If it is determined that the acquired image is not a valid sample target image, a path length of the reference arm of the imaging system is adjusted until it is determined that the acquired image is a valid image to provide an adjusted position of the reference arm. A difference between the original calibrated position of the reference arm and the adjusted position of the reference arm is calculated. The elements of the system are adjusted based on the calculated difference so that the sample is visible when placed at the working distance of the objective lens of the surgical microscope in the adjusted position of the reference arm.

[0008] In some embodiments, the system may be further configured to determine whether the imaging system is in focus at the adjusted position of the reference arm before adjusting an element of the system. When it is determined that the system is out of focus, the element of the system may be adjusted until it is determined that the imaging system is in focus.

[0009] In a further embodiment, the system may be further configured to iteratively adjust the path length of a reference arm of the imaging system, calculate the difference between the current reference arm position and the original calibrated position, and adjust elements of the system until the system is determined to be in focus.

[0010] Additionally, in a further embodiment, the system may be further configured to determine whether the acquired image contains a valid image of the sample target based on one of signal quality and the presence or absence of features from the acquired image.

[0011] In some embodiments, the imaging plane of the imaging system can coincide with the focal plane of the surgical microscope when the system is in focus.

[0012] In a further embodiment, the system may be further configured to adjust elements of the system by adjusting at least one of an eyepiece associated with the surgical microscope and other optical components of the system.

[0013] Additionally, in a further embodiment, the system may be further configured to adjust the position of the reference arm by adjusting a mirror in the reference arm to provide a match between the optical path length and working distance in the reference arm and sample arm of the imaging system such that the sample is visible when placed at the working distance of the objective lens of the surgical microscope.

[0014] In some embodiments, the working distance of the surgical microscope objective may be the distance between the surgical microscope objective and the surface of the sample.

[0015] In a further embodiment, the imaging system may be one of an optical coherence tomography (OCT) imaging system.

[0016] Additionally, in a further embodiment, the system may be configured to manually adjust the path length of the reference arm of the imaging system and / or elements of the system.

[0017] In some embodiments, the system may be further configured to automatically adjust the path length of a reference arm and / or elements of the imaging system.

[0018] In a further embodiment, the system may further include a demagnifying lens between the objective lens and the sample to reduce the focal length of the objective lens. A relay lens may be provided adjacent the sample to relay the objective lens image plane to the retina of the sample to adapt the system for retinal imaging of the sample.

[0019] Additionally, in a further embodiment, the objective lens may be a multifocal objective lens, and the system may further include a relay lens adjacent to the sample to relay the objective lens image plane to the retina of the sample, to adapt the system for retinal imaging of the sample.

[0020] Some embodiments of the inventive concept provide a method for optimizing the optical system of a surgical microscope having an integrated imaging system, the system including one or more processors and one or more storage devices, a first optical system associated with the surgical microscope, and a second optical system associated with the imaging system, different from the first optical system, the method including calibrating a position of a reference arm of the imaging system such that an image of the sample is visible when the sample is placed at a working distance of an objective of the surgical microscope to provide an initial calibrated position, acquiring an image using the initial calibrated position of the reference arm of the imaging system, evaluating an image quality of the acquired image to determine whether the acquired image is a valid image, if it is determined that the acquired image is not a valid image, adjusting a path length of the reference arm of the imaging system until it is determined that the acquired image is a valid image to provide an adjusted position of the reference arm, calculating a difference between the original calibrated position of the reference arm and the adjusted position of the reference arm, and adjusting elements of the system based on the calculated difference such that the sample is visible when the sample is placed at a working distance of the objective of the surgical microscope in the adjusted position of the reference arm.

[0021] A related computer program product is also provided. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a diagram of the focal plane of a microscope objective showing the correct "working distance." [Diagram 2] FIG. 1 is a diagram of the focal plane of a microscope objective showing a working distance set too short. [Diagram 3] FIG. 1 is a diagram of the focal plane of a microscope objective showing a working distance set too long. [Figure 4] FIG. 1 illustrates a system with an OCT imaging plane set to coincide with the focal plane of the objective lens. [Figure 5A] FIG. 1 is a block diagram illustrating an exemplary OCT retinal (posterior) imaging system. [Figure 5B] FIG. 1 is a block diagram illustrating an exemplary OCT corneal (anterior) imaging system. [Figure 6] 1 is a flowchart illustrating operations for optimization using manual tuning, in accordance with some embodiments of the inventive concepts. [Figure 7] 1 is a flowchart illustrating operations for optimization using auto-tuning, in accordance with some embodiments of the inventive concepts. [Figure 8] 1 is a flowchart illustrating operations for automated microscope movement and OCT focus adjustment in accordance with some embodiments of the inventive concepts. [Figure 9] 1 is a flowchart illustrating operations for manual microscope movement and OCT focus adjustment, according to some embodiments of the inventive concepts. [Figure 10] FIG. 2 is a diagram for explaining a microscope objective lens set for imaging the cornea. [Figure 11] FIG. 1 illustrates a retina viewing lens system including a reduction lens in combination with a relay lens. [Figure 12] FIG. 1 illustrates that wide field of view (FOV) imaging requires close proximity of the retinal relay lens to the cornea. [Figure 13]FIG. 1 is a diagram of a microscope objective lens design consisting of split elements. [Figure 14] FIG. 1 illustrates a system having an extension of the split lens design, including the elimination of multifocal power. [Figure 15] The multifocal design of the system allows for full retinal FOV imaging. [Figure 16] FIG. 2 is a block diagram of a data processing system that may be used to implement processes in accordance with various embodiments of the inventive concept. [Figure 17] FIG. 1 is a block diagram of a system according to some embodiments of the inventive concepts including a microscope. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The inventive concepts will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the inventive concepts are shown. The inventive concepts may be embodied in many alternative forms and should not be construed as limited to the embodiments set forth herein.

[0024] Thus, while the inventive concepts are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that there is no intention to limit the inventive concepts to the specific forms disclosed, but on the contrary, the inventive concepts are intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the inventive concepts as defined by the appended claims. Like numbers refer to like elements throughout the drawings.

[0025] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the inventive concept. As used herein, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise. It will be further understood that the terms "comprise," "comprising," "including," and / or "comprising" as used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when an element is referred to as "responsive to" or "connected" to another element, it can be directly responsive to or connected to the other element, or intervening elements may be present. In contrast, when an element is referred to as "directly responsive to" or "directly connected" to another element, there are no intervening elements. As used in this specification, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0026] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the inventive concept belongs. Furthermore, it will be understood that the terms used herein should be interpreted to have a meaning consistent with their meaning in the context of the present specification and the related technical field, and should not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0027] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step, and similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0028] In this specification, terms such as first, second, etc. may be used to describe various elements, but it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the teachings of the present disclosure. Some of the drawings include arrows on communication paths to indicate a primary direction of communication, but it should be understood that communication may also occur in the opposite direction to the depicted arrows.

[0029] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device to create a machine, whereby the instructions executed by the processor of the computer or other programmable instruction execution device create a mechanism for performing the functions / operations specified in one or more blocks of the flowchart and / or block diagram. As used herein, a "processor" may refer to one or more processors.

[0030] These computer program instructions may be stored on a computer-readable medium that, when executed, can direct a computer, other programmable data processing apparatus, or other device to function in a particular manner, such that, when these instructions are stored on a computer-readable medium, an article of manufacture is produced that includes instructions that, when executed, cause a computer to perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer program instructions may also be loaded onto a computer, other programmable instruction execution apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to produce a computer-implemented process such that the instructions executing on the computer or other programmable apparatus provide a process for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0031] As used herein, "subject" refers to a person or thing, or a part of a person or thing, being imaged. Although embodiments of the inventive concepts are discussed herein in the context of an eye being a subject, it will be understood that embodiments of the inventive concepts are not limited to this configuration. The subject may be any subject, including, for example, a veterinary, cadaveric, or human subject, without departing from the scope of the inventive concepts.

[0032] Although many of the examples discussed herein refer to the sample / subject being the eye, specifically the retina, cornea, anterior segment, and lens of the eye, embodiments of the inventive concepts are not limited to this type of sample. Any type of sample that may be used in combination with the embodiments discussed herein may be used without departing from the scope of the inventive concepts.

[0033] Although embodiments of the inventive concepts focus on the use of OCT to scan a sample, embodiments of the inventive concepts are not limited to the use of OCT, and it will be appreciated that any method and system used to scan a sample may be used without departing from the scope of the inventive concepts.

[0034] Furthermore, imaging as discussed herein can be performed in any manner known to those of skill in the art. For example, in some embodiments, the imaging system may be incorporated into a microscope or surgical microscope. Various of these embodiments are discussed, for example, in U.S. Pat. No. 8,774,12 and U.S. Patent Application Publication Nos. 2015 / 0168250 and 2015 / 0359426, the disclosures of which are incorporated herein by reference in their entireties.

[0035] As used herein, "working distance" refers to the distance from the last glass surface of the objective lens to the surface of the sample. Working distance is sometimes referred to as WD throughout this specification.

[0036] A system according to some embodiments of the inventive concept includes a microscope that includes both a surgical optical system (first optical system) and a second interferometric-based optical system (second optical system). In some embodiments, the interferometer-based optical system is an OCT imaging system. However, embodiments of the inventive concept are not so limited. For example, as used herein, "interferometric imaging system" refers to an imaging system that utilizes optical signals generated from the interference of light between a reference optical path and a sample optical path. Some embodiments may further include one or more additional imaging systems that have separate optical paths within the microscope but share the same objective lens. For example, these additional imaging systems may be fundus fluorescence, scanning laser ophthalmoscopy, wide-field imaging systems, etc., without departing from the scope of the inventive concept.

[0037] With reference to Figures 5A and 5B, exemplary interferometric imaging systems for use in accordance with some embodiments of the inventive concepts are discussed. It will be understood that these systems are provided for illustrative purposes only, and thus, the embodiments of the inventive concepts should not be limited thereto. Conventional Fourier Domain OCT (FDOCT) systems are now discussed to provide some background related to these systems. First, with reference to Figure 5A, a block diagram of an FDOCT retinal imaging system is discussed. As shown in Figure 5A, the system includes a broadband source 100, a reference arm 110, and a sample arm 140 coupled to each other by a beam splitter 120. The beam splitter 120 may be, for example, a fiber optic coupler or a bulk or micro-optic coupler. The beam splitter 120 may provide a splitting ratio of about 50 / 50 to about 90 / 10. As further shown in Figure 5A, the beam splitter 120 is also coupled to a wavelength or frequency sampling detection module 130 via a detection path 106, which may be provided by an optical fiber.

[0038] As further shown in FIG. 5A, source 100 is coupled to beam splitter 120 by source path 105. Source 100 may be, for example, a continuous wave broadband superluminescent diode, a pulsed broadband source, or a tunable source. Reference arm 110 is coupled to beam splitter 120 via reference arm path 107. Similarly, sample arm 140 is coupled to beam splitter 120 via sample arm path 108. Source path 105, reference arm path 107, and sample arm path 108 may all be provided by optical fiber or a combination of optical fiber, free space, and bulk or micro-optical elements.

[0039] 5A, the reference arm of the FDOCT retinal imaging system can include a collimator assembly 180, a variable attenuator 181, which can include a neutral density filter or a variable aperture, a mirror assembly 182, a reference arm variable path length adjustment 183, and a path length matching position 150, i.e., an optical path length matching between the reference arm path length and the sample arm path length relative to the subject region of interest. As further shown, the sample arm 140 can include a two-axis scanner assembly 190 and an objective lens with a variable focus 191.

[0040] 5A is an eyeball including a cornea 195, an iris / pupil 194, an ocular lens 193, and a retina 196. A representation of an FDOCT imaging window 170 is shown proximate the retina 196. This retinal imaging system relies on the optics of the subject's eye, specifically the cornea 195 and the ocular lens 193, as well as an objective lens, to image the posterior structures of the eyeball. As further shown, a region of interest 170 within the subject is selected through adjustment of the focal position 196 and the reference arm pathlength adjuster 183 such that a pathlength matching position 197 within the subject is at a desired location.

[0041] With reference to Figure 5B, a block diagram showing an FDOCT corneal (anterior) imaging system will be discussed. As shown therein, the system of Figure 5B is very similar to the system of Figure 5A. However, objective lens varifocality need not be included and is not included in Figure 5B. The anterior imaging system of Figure 5B directly images the anterior structures without relying on the subject's optics to focus on the anterior structures.

[0042] As discussed in the background above, OCT systems are designed to have optimal image quality at the working distance WD of the microscope objective, as shown in FIG. 4. The optical path length through the sample arm of an OCT system generally matches the optical path through the reference arm of the OCT system, which consists of a mirror mounted on a translatable stage. As used herein, "optical path length (OPL)" refers to the geometric length of the path followed by light through a given system multiplied by the optical index of refraction for the simultaneous medium through which the light passes. The specific position of this reference mirror in the reference arm generally corresponds to the working distance of the microscope objective. If the sample is placed at a position outside the working distance of the microscope objective, the reference mirror is moved a corresponding distance to ensure that the optical path lengths match. An initial calibration of the reference arm position to the working distance allows deviations from this position to be detected. Conventional methods for adjusting elements of the system are generally trial and error, without providing specific coordinates for the adjustments made.

[0043] Thus, some embodiments of the inventive concepts provide methods and systems that detect the presence of a valid OCT image of a sample target, set the OCT reference arm in a position to detect the image, and optimize the optics of multiple optical subsystems to obtain the best image. In other words, instead of traditional trial-and-error methods, embodiments of the inventive concepts use OCT to measure where the target is and then instruct the system operator where to go based on the OCT measurements. In some embodiments, these adjustments may be performed automatically, as discussed further below.

[0044] With reference to FIG. 6, a flow chart illustrating the operations of detecting the presence of a valid OCT image of a sample target, setting the OCT reference arm in a position to detect the image, and optimizing the optical system of the multiple optical subsystems to obtain the best image quality is discussed. As shown in FIG. 6, the operations for optimization using manual adjustment of the microscope begin in block 600 by calibrating the reference arm of the system. A factory calibration or a service calibration of the instrument may be performed. The OCT reference arm is set at a distance such that an OCT image of the sample target is visible when the target (sample) is placed at the working distance of the microscope objective, as shown, for example, in FIG. 4. Conventionally, the target is first viewed through the microscope visible light path set to infinity focus and maximum magnification to ensure that the target is accurately placed in the focal plane of the microscope objective. This is a subjective technique and therefore can lead to inaccuracies. Thus, according to some embodiments of the inventive concept, the methods discussed herein are used to increase the likelihood that the target is placed in the correct (virtual) focal plane of the objective. It will be understood that an "exact" position refers to a position as close as possible to the exact position, since it may be impossible to achieve the exact position. As used herein, "focal plane" refers to the distance from the objective lens where parallel light collected on the lens aperture converge together.

[0045] The signal used to generate the OCT image can be generated from the interference between light in the sample and reference arms (cross-correlation signal) or between different light reflectors in a single arm (auto-correlation signal). To generate a cross-correlation signal, the optical path lengths in the reference and sample arms must be closely matched to fall within the detection bandwidth of the system's photodetector. For the auto-correlation signal to be visible, the light must be focused sufficiently on the sample to generate sufficient optical signal strength from a single arm. The Fourier transform of the auto-correlation signal results in a signal near zero frequency, which is subject to the same signal reduction effects as the cross-correlation signal. The auto-correlation signal provides a means to determine whether the sample is in the focal plane of the microscope objective. The light from the reference arm may be blocked or the reference arm may be moved to move the cross-correlation signal outside the detection bandwidth of the system. A collimated (parallel) OCT beam may be incident through the objective and a maximum signal strength evaluation of the OCT auto-correlation signal generated by the reflected light from the target may be performed to determine whether the sample is in the correct position. The focus of the microscope objective is adjusted to increase, or in some cases maximize, this signal. This may involve moving the position of the objective or adjusting optical elements to shift the focal plane of the objective and can be done manually or in an automated manner. Evaluation of the autocorrelation signal may include analysis of the signal-to-noise ratio or the intensity of the Fourier transform magnitude near zero frequency. Once the autocorrelation is maximized, the reference path is adjusted until the OCT cross-correlation signal is visible and located near the zero frequency position in the OCT B-scan.

[0046] Once the calibration is performed (Block 600), an image is detected and the quality of the image is evaluated (Block 610). For example, the presence of a valid OCT image is detected via software algorithms that evaluate signal quality or detect specific features within a given OCT image. In some embodiments, these methods may include, but are not limited to, maximum intensity, intensity distribution, or variance of intensity across the image, signal-to-noise ratio of the image, or detection of specific sample target features through machine learning or classical object detection methods. It is determined whether a valid OCT image is present (Block 630). If a valid image is not present (Block 610), the reference arm is moved through a series of positions (Block 620) until a position is found that results in a valid OCT image (Block 630). System optics that affect beam focus, polarization, or other factors that may affect image quality are adjusted to optimize the OCT signal intensity, which may be done sequentially or simultaneously with the reference arm movement. If a valid image is determined to be present (Block 630), the difference from the calibrated reference arm position relative to the microscope working distance is calculated (Block 640).

[0047] It is determined whether the microscope is in focus (block 645). If it is determined that the microscope is not in focus (block 645), manual adjustments may be made (block 650) until it is determined that the microscope is in focus (block 645). Once in focus (block 645), the operation proceeds to block 660 where other subsystem optics may be adjusted. For example, the user may command other optical components that may affect image quality, i.e., microscope eyepieces, to be adjusted to ensure parfocality across each imaging modality integrated with the microscope. If other imaging modalities, e.g., fluorescence microscope, scanning laser ophthalmoscope, etc., are included within the microscope, this information may also be used to optimize the focus of the associated optics to improve image quality without departing from the scope of the inventive concept. In other words, any other additional imaging system that has a separate optical path within the microscope but shares the same objective lens may be used without departing from the scope of the inventive concept. For example, fundus fluorescence, scanning laser ophthalmoscope, wide field imaging system, etc.

[0048] Although the embodiment of the inventive concept shown in FIG. 6 illustrates manual adjustment of the microscope (block 650), the embodiment of the inventive concept is not limited to this configuration. FIG. 7 illustrates operations similar to those discussed above in connection with FIG. 6, and therefore the details of the blocks will not be repeated herein in the interest of brevity. However, instead of manual microscope movement as shown in FIG. 7, block 751 of FIG. 7 illustrates that microscope movement may be automated. Thus, systems according to the embodiments discussed herein may be programmed to automatically adjust without departing from the scope of the inventive concept. For example, the microscope working distance may be automatically adjusted as shown in FIG. 7, or information regarding the direction and distance to move the microscope may be communicated to the user as shown in FIG. 6.

[0049] Referring to the flowcharts of FIG. 8 and FIG. 9, these flowcharts are very similar to the flowcharts of FIG. 6 and FIG. 7. Therefore, details regarding similar elements will not be repeated for brevity. In addition to the details discussed in FIG. 7, FIG. 8 includes an OCT focus adjustment (block 815) in response to a determination that the image is not valid (block 630). Thus, when it is determined that the image is not valid (block 630), the reference arm path length is adjusted (block 620) and the OCT focus is adjusted (block 815). The remaining operations proceed as discussed above in connection with FIG. 7 with the automated microscope adjustment (block 751). In the flowchart of FIG. 9, the same OCT adjustment (block 815) is added to the manual microscope adjustment (block 650) discussed above in connection with FIG. 6.

[0050] The addition of an auxiliary lens system to an objective lens set will be discussed with reference to Figures 10 and 11. Figure 10 shows a standard microscope objective lens set for imaging of the cornea. Figure 11 shows a system including a standard retina viewing lens system including additional reduction and relay lenses. Additional complications may arise when these auxiliary retina viewing lens systems are attached to a microscope. These lens systems are generally required to relay the focal plane of the microscope objective from a position on the cornea to the retina of the surgical eye. Standard workflow requires these imaging systems to operate without the requirement to move the microscope position between corneal and retinal imaging, so the relay system consists of two optical elements working together. The first element 1170 (reduction lens) of Figure 11 effectively reduces the focal length of the microscope objective so that the relay lens 1175 of Figure 11 is mechanically positioned within close proximity to the cornea and can relay the microscope objective image plane to the retina of the surgical eye.

[0051] As shown in Figure 12, the close proximity of the relay lens 1175 to the cornea is generally necessary for a large field of view (FOV). Thus, if the microscope is improperly set up as discussed above in connection with Figures 2 and 3, the addition of the retinal viewing system (1170 / 1175) will cause further perturbation of the microscope imaging optics from their optimal working distance, forcing other imaging modalities, such as OCT, to effectively operate outside the normal imaging window.

[0052] An additional complication arises from the adjustability of the retina viewing lens system relayed to the lens for accommodation of refractive errors of the surgical eye. These errors can compound to produce non-functional results for additional imaging modalities such as OCT. Some embodiments of the inventive concept address this situation by providing a multi-focal length objective lens as opposed to a fixed focus objective lens or relay lens 1435 as shown in Figures 13-15.

[0053] Current microscope lens designs use split elements as a way to reduce optical aberrations in an image, as shown in FIG. 13. This concept can be extended to allow the air gap between elements to be increased or decreased, making the focal length of the entire lens adjustable. FIG. 14 shows that increasing the air gap between elements shortens the focal length, allowing the retina viewing relay lens 1435 to be used for retinal imaging without the need for a secondary reduction lens as shown in the standard configuration of FIG. 11. FIG. 15 shows that a multifocal design can enable full retinal field imaging.

[0054] Additionally, the gap between elements can be adjusted to accommodate refractive errors in the operated eye. One advantage of using a multifocal objective design is that the optical paths of all imaging modalities are aligned together through the objective, effectively eliminating the requirement for individual focusing forces between the optical paths.

[0055] Independent feedback from an external distance finder can also be incorporated into the positioning control loop to further reduce the risk of impact of the patient's eye from the retinal relay lens.

[0056] As is evident from the description of the embodiments of the inventive concept above, many of the methods discussed herein require processing provided by a computing device. Referring now to FIG. 16, an exemplary embodiment of a data processing system 1630 configured in accordance with an embodiment of the inventive concept will be discussed in connection with FIG. 16. As will be appreciated, the data processing system may be included in, for example, the system of FIG. 5A and FIG. 5B, the microscope, or may be a separate device in communication with the system of FIG. 5A and FIG. 5B or the microscope without departing from the scope of the inventive concept. The data processing system 1630 may include a user interface 1644 including, for example, an input device such as a keyboard or keypad, a display, a speaker and / or a microphone, and a memory 1636 in communication with a processor 1638. The data processing system 1630 may further include an I / O data port 1646 that also communicates with the processor 1638. The I / O data port 1646 may be used to transfer information between the data processing system 1630 and another computer system or network, for example, using an Internet Protocol (IP) connection. These components may be conventional components such as those found in many conventional data processing systems, and may be configured to operate as described herein.

[0057] As further shown in FIG. 16, the data processing system 1630 communicates with a microscope 1610 including a first optical system 1637, a second interferometric optical system 1647, and optionally one or more additional systems 1657 (the dotted lines in FIG. 16 indicate optional additional elements). The first optical system 1637 may be associated with the microscope, and the second interferometric based optical system 1647 may be provided by an OCT imaging system as discussed above. However, embodiments of the inventive concept are not so limited. As further shown, additional imaging modalities may be provided by optional systems associated with the first and second systems. For example, some embodiments may further include one or more additional imaging systems 1657 having separate optical paths within the microscope but sharing the same objective lens. For example, these additional imaging systems may be fundus fluorescence, scanning laser ophthalmoscopy, wide field imaging systems, etc., without departing from the scope of the inventive concept.

[0058] Some embodiments of the inventive concept relate to a microscope including a system as described in connection with one or more of Figures 1 to 16. Alternatively, the microscope may be part of a system as described in connection with one or more of Figures 1 to 16 or may be connected to a system as described in connection with one or more of Figures 1 to 16. Figure 17 shows a schematic diagram of a system 1700 configured to perform the methods described herein. The system 1700 includes a microscope 1710 and a computer system 1720. The microscope 1710 is configured to capture images and is connected to the computer system 1720. The computer system 1720 is configured to perform at least a portion of the methods described herein. The computer system 1720 may be configured to execute a machine learning algorithm. The computer system 1720 and the microscope 1710 may be separate entities, but may be integrated in one common housing. The computer system 1720 may be part of a central processing system of the microscope 1710 and / or the computer system 1720 may be part of a subordinate component of the microscope 1710, such as a sensor, actor, camera or lighting unit of the microscope 1710.

[0059] The computer system 1720 may be a local computing device (e.g., a personal computer, laptop, tablet computer, or mobile phone) with one or more processors and one or more storage devices, or may be a distributed computing system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, such as local clients and / or one or more remote server farms and / or data centers). The computer system 1720 may include any circuit or combination of circuits. In one embodiment, the computer system 1720 may include one or more processors, which may be of any type. As used herein, a processor may contemplate any type of computing circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit, for example, of a microscope or a microscope component (e.g., a camera). Other types of circuits that may be included in computer system 1720 may be custom circuits, application specific integrated circuits (ASICs), such as one or more circuits (such as communications circuits) used in wireless devices such as cell phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. Computer system 1020 may also include one or more storage devices, which may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives handling removable media, such as compact discs (CDs), flash memory cards, digital video discs (DVDs), and the like.Computer system 1720 may also include a display device, one or more speakers and a keyboard and / or controller which may include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that enables a user of the system to input information to and receive information from computer system 1720.

[0060] Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, any one or more of the crucial steps may be performed by such an apparatus.

[0061] Depending on certain implementation requirements, the embodiments of the inventive concept may be implemented in hardware or software. The implementation may be performed by a non-transitory recording medium, such as a digital recording medium, for example a floppy disk, DVD, Blu-ray, CD, ROM, PROM and EPROM, EEPROM or FLASH memory, on which electronically readable control signals are stored that cooperate (or can cooperate) with a programmable computer system to perform the respective methods. Thus, the digital recording medium may be computer readable.

[0062] Some embodiments of the inventive concept include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system such that any of the methods described herein are performed.

[0063] In general, embodiments of the inventive concept may be implemented as a computer program product comprising program code which is operable to perform any of the methods when the computer program product is run on a computer, the program code may for example be stored on a machine readable carrier.

[0064] Another embodiment comprises the computer program for performing any of the methods described herein, stored on a machine readable carrier.

[0065] In other words, therefore, an embodiment of the inventive concept is, therefore, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.

[0066] Therefore, another embodiment of the inventive concept is a recording medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the inventive concept is an apparatus as described herein, including a processor and a recording medium.

[0067] A further embodiment of the inventive concept is therefore a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or the sequence of signals being for example adapted to be transferred via a data communication connection, for example the Internet.

[0068] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.

[0069] Another embodiment comprises a computer having the computer program installed thereon for performing any of the methods described herein.

[0070] Another embodiment of the inventive concept includes a device or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The device or system may include, for example, a file server to transfer the computer program to the receiver.

[0071] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus.

[0072] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects of the disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code that includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, as well as combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special-purpose hardware-based system that performs the specified functions or acts, or a combination of special-purpose hardware and computer instructions.

[0073] The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the disclosed form. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the present disclosure. The aspects of the disclosure herein have been selected and described in order to best explain the principles and practical applications of the present disclosure, and have been selected and described so that others skilled in the art can understand the disclosure with various modifications suitable for the particular use contemplated.

Claims

1. A system for optimizing an optical system of a surgical microscope having an interferometric imaging system, the system comprising: one or more processors and one or more storage devices; a first optical system associated with the surgical microscope; and a second optical system associated with the interferometric imaging system and distinct from the first optical system; and Including, The system comprises: calibrating a position of a reference arm of the interferometric imaging system such that the sample is visible by the interferometric imaging system when the sample is positioned at a working distance of an objective lens of the surgical microscope to provide an initial calibrated position with the surgical microscope focused on the sample; acquiring an optical coherence tomography (OCT) image using the initial calibrated position of the reference arm of an intended sample target; evaluating an image quality of the acquired OCT image to determine whether the acquired OCT image is a valid image; if it is determined that the acquired OCT image is not a valid image of the intended sample target, adjusting a path length of the reference arm to provide an adjusted position of the reference arm until it is determined that the acquired OCT image is a valid image; calculating a difference between the initial calibrated position of the reference arm and the adjusted position of the reference arm; and moving a position of the surgical microscope based on the calculated difference such that the sample is visible when the sample is placed at the working distance of the objective lens of the surgical microscope in the adjusted position of the reference arm. system.

2. The system comprises: determining whether the interferometric imaging system is in focus at the adjusted position of the reference arm before moving the position of the surgical microscope; and further configured, when it is determined that the interferometric imaging system is out of focus, to move the position of the surgical microscope to adjust the focus of the interferometric imaging system. The system of claim 1 .

3. the system is further configured to iteratively adjust the path length of the reference arm, calculate a difference between a current reference arm position and the initial calibrated position, move the position of the surgical microscope, and adjust the focus of the interferometric imaging system.

3. The system according to claim 1 or 2.

4. The system is further configured to determine whether the acquired OCT image includes a valid image of the sample target based on one of signal quality and a presence or absence of a feature from the acquired OCT image. A system according to any one of claims 1 to 3.

5. an imaging plane of the interferometric imaging system coincides with a focal plane of the surgical microscope when the interferometric imaging system is in focus; A system according to any one of claims 1 to 4.

6. The system is further configured to move the position of the surgical microscope by adjusting at least one of a plurality of eyepieces associated with the surgical microscope. A system according to any one of claims 1 to 5.

7. the system is further configured to adjust a position of the reference arm by adjusting a mirror in the reference arm to provide a match between an optical path length and the working distance in the reference arm and a sample arm of the interferometric imaging system such that the sample is visible when the sample is positioned at the working distance of the objective of the surgical microscope. A system according to any one of claims 1 to 6.

8. the working distance of the objective of the surgical microscope is the distance between the objective of the surgical microscope and the surface of the sample; A system according to any one of claims 1 to 7.

9. the interferometric imaging system includes one of an OCT imaging system, a first imaging system, and at least one additional optical imaging system different from a second interferometric based optical imaging system; A system according to any one of claims 1 to 8.

10. the system is configured to manually adjust the path length of the reference arm; A system according to any one of claims 1 to 9.

11. the system is configured to automatically adjust the path length of the reference arm; A system according to any one of claims 1 to 9.

12. The system comprises: a reduction lens between the objective lens and the sample to reduce the focal length of the objective lens; a relay lens adjacent to the sample for relaying an objective lens image plane to a retina of the sample to adapt the interferometric imaging system for retinal imaging of the sample; Further comprising: A system according to any one of claims 1 to 11.

13. the objective lens is a multifocal objective lens, the system further comprising a relay lens adjacent the sample for relaying an objective lens image plane to a retina of the sample to adapt the interferometric imaging system for retinal imaging of the sample. A system according to any one of claims 1 to 11.

14. A method for optimizing an optical system of a surgical microscope having an interferometric imaging system, comprising: the interferometric imaging system includes one or more processors and one or more storage devices; a first optical system associated with the surgical microscope; and a second optical system associated with the interferometric imaging system, the second optical system being interferometric based and distinct from the first optical system; The method comprises: With the surgical microscope focused on a sample, calibrating the position of a reference arm of the interferometric imaging system such that the sample is visible by the interferometric imaging system when the sample is positioned at a working distance of an objective lens of the surgical microscope to provide an initial calibrated position; acquiring an OCT image using the initial calibrated position of the reference arm; evaluating an image quality of the acquired OCT image to determine whether the acquired OCT image is a valid image; if it is determined that the acquired OCT image is not a valid image, adjusting a path length of the reference arm to provide an adjusted position of the reference arm until it is determined that the acquired OCT image is a valid image; calculating a difference between the initial calibrated position of the reference arm and the adjusted position of the reference arm; moving a position of the surgical microscope based on the calculated difference such that the sample is visible when the sample is placed at the working distance of the objective lens of the surgical microscope in the adjusted position of the reference arm; The method includes:

15. The method comprises: determining whether the interferometric imaging system is in focus at the adjusted position of the reference arm before moving the position of the surgical microscope; when it is determined that the interferometric imaging system is out of focus, moving the position of the surgical microscope to adjust the focus of the interferometric imaging system; Further comprising:

15. The method of claim 14.

16. the method further comprising iteratively adjusting the path length of the reference arm; 16. The method according to claim 14 or 15.

17. The method further includes determining whether the acquired OCT image is a valid image based on one of signal quality and the presence or absence of features from the acquired OCT image.

17. The method according to any one of claims 14 to 16.

18. an imaging plane of the interferometric imaging system coincides with a focal plane of the surgical microscope when the interferometric imaging system is in focus; 18. The method according to any one of claims 14 to 17.

19. and wherein the step of moving the position of the surgical microscope includes adjusting at least one of a plurality of eyepieces associated with the surgical microscope.

19. The method according to any one of claims 14 to 18.

20. adjusting the position of the reference arm includes adjusting a mirror in the reference arm to provide a match between an optical path length and a working distance in the reference arm and a sample arm of the interferometric imaging system such that the sample is visible when the sample is positioned at the working distance of the objective lens of the surgical microscope; 20. The method according to any one of claims 14 to 19.

21. the working distance of the objective of the surgical microscope is the distance between the objective of the surgical microscope and the surface of the sample; 21. The method according to any one of claims 14 to 20.

22. the interferometric imaging system includes one of an optical coherence tomography (OCT) imaging system, an additional imaging system different from the first and second imaging systems; 22. The method according to any one of claims 14 to 21.

23. the adjusting step further comprises manually adjusting a path length of the reference arm.

23. The method according to any one of claims 14 to 22.

24. The adjusting step further comprises automatically adjusting the path length of the reference arm.

23. The method according to any one of claims 14 to 22.

25. The method comprises: reducing the focal length of the objective lens; relaying an objective image plane to a retina of the sample to adapt the interferometric imaging system for retinal imaging of the sample; Further comprising:

25. The method according to any one of claims 14 to 24.

26. 26. A computer program having a program code for performing a method according to any one of claims 14 to 25, when the computer program is run on a processor.

Images