Systems, methods, and computer program products for optimizing optics of a surgical microscope with an integrated imaging system

Abstract

A system for optimizing an optical device is provided. The system is configured to calibrate a position of a reference arm of an interferometric imaging system such that an image of a sample is visible when the sample is at a working distance of an objective lens to provide an initial calibration position. An image is acquired using the initial calibration position. An image quality of the acquired image is evaluated to determine whether the acquired image is a valid image. If it is determined that the acquired image is not a valid image, a path length of the reference arm is adjusted. A difference between the calibration position of the reference arm and the adjusted position of the reference arm is calculated. System elements are adjusted based on the calculated difference such that the sample is visible when the sample is at the working distance at the adjusted position.

Description

TECHNICAL FIELD

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 62 / 987,481, filed March 10, 2020, entitled “Methods, Systems, and Apparatus for Guided Microscope Optimization,” the contents of which are incorporated by reference as if fully set forth herein.

[0002] The inventive concept relates generally to imaging, and more particularly, to microscopes with integrated OCT systems and related systems and methods. BACKGROUND

[0003] For example, the use of Optical Coherence Tomography (OCT) during ophthalmic surgery often requires the integration of an OCT system with a surgical microscope. The surgical microscope provides the surgeon with a magnified view of the surgical area. Ophthalmic surgical microscopes are typically stereoscopic zoom microscopes with binocular viewing ports for the surgeon to use, and typically have one or two observer viewing ports at ninety degrees (left and right) from the surgeon. A “working distance” is provided between the objective lens of the microscope and the surface of the patient’s eye (sample) to give the surgeon enough working area.

[0004] The integration of a surgical microscope with OCT can result in at least two different optical systems within the microscope, one for the OCT system and one for the standard white light imaging path through the microscope eyepiece. To achieve ideal image quality in each mode, the microscope is typically parfocal, i.e., each optical subsystem has a matching focal point that should correspond to the focal plane of the objective lens, e.g., as shown in Figure 1 Thus, Figure 1 represents the focal plane of a microscope objective lens with an ideal working distance (WD) when measured from the last glass surface of the lens.

[0005] Because of the multiple optical components in the white light path, the microscope objective lens can be set at a position outside the working distance, and still achieve an image that appears to be in focus through compensation by other system components, such as the eyepiece. For example, Figure 2 shows a working distance that is set too short (WDshort) compensated for by using viewing eyepiece diopter adjustment. Figure 3 shows a working distance that is set too long (WDlong) compensated for by using viewing eyepiece diopter adjustment. Using Figure 2 and Figure 3 The systems shown in and can result in lower quality OCT images because the sample can be out of focus with respect to the microscope objective lens, as can be the case for other imaging modes that can be added to the microscope.

[0006] The OCT system is designed to have optimal image quality at the working distance of the microscope objective lens, as shown inFigure 4 are shown. Figure 4 An OCT system is shown, with its imaging plane set to match the focal plane of the objective lens. OCT uses the principle of low coherence interferometry to obtain three-dimensional (3D) images of a sample. The optical path length through the sample arm of the OCT system is usually matched to the optical path through the reference arm of the OCT system, which usually consists of a mirror mounted on a translatable stage. The specific position of this reference mirror in the reference arm is usually mapped to the working distance of the microscope objective lens. If the sample is placed at a position outside the working distance of the microscope objective lens, the reference mirror is moved by a corresponding distance to ensure that the optical path lengths are matched. The initial calibration of the position of the reference arm to the working distance allows for the detection of deviations from this position. Conventional methods for adjusting the elements of the system are usually trial and error, i.e. no specific coordinates are provided for the adjustments made. SUMMARY

[0007] Some embodiments of the inventive concept provide a system for optimizing optics of a surgical microscope with an integrated imaging system. The system includes one or more processors and one or more storage devices; a first optical system associated with the surgical microscope; and a second optical system different from the first optical system associated with an interferometric imaging system such as OCT. The system is configured to calibrate a position of a reference arm of the imaging system such that when a sample is at a working distance of an objective lens of the surgical microscope, an image of the sample is visible to provide an initial calibration position. The system is further configured to obtain an image using the initial calibration position of the reference arm of the imaging system. An image quality of the obtained image is evaluated to determine whether the obtained image is a valid image of an intended sample target. If it is determined that the obtained image is not a valid image of the sample target, a path length of the reference arm of the imaging system is adjusted until it is determined that the obtained image is a valid image to provide an adjusted position of the reference arm. A difference between the initial calibration position of the reference arm and the adjusted position of the reference arm is calculated. Elements of the system are adjusted based on the calculated difference such that when a sample is at the working distance of the objective lens of the surgical microscope at the adjusted position of the reference arm, the sample is visible.

[0008] In some embodiments, the system can be further configured to determine whether the imaging system is in focus at the adjusted position of the reference arm before adjusting the elements of the system. When it is determined that the system is not in focus, the elements of the system are adjusted until it is determined that the imaging system is in focus.

[0009] In further embodiments, the system can be further configured to repeatedly adjust the path length of the reference arm of the imaging system, calculate a difference between a current position of the reference arm and the initial calibration position, and adjust the elements of the system until it is determined that the system is in focus.

[0010] In still further embodiments, the system can be further configured to determine whether the acquired image contains a valid image of the sample target based on one of a quality of the signal, and a presence or absence of a feature in the acquired image.

[0011] In some embodiments, an imaging plane of the imaging system can match a focal plane of the surgical microscope when the system is in focus.

[0012] In further embodiments, the system can be further configured to adjust elements of the system by adjusting at least one of a plurality of eyepieces associated with the surgical microscope and other optical components of the system.

[0013] In still further embodiments, the system can be further configured to adjust a position of the reference arm by adjusting mirrors in the reference arm to provide matching optical path lengths in the reference arm and the sample arm of the imaging system, and a working distance, such that the sample is visible when the sample is located at the working distance of the objective lens of the surgical microscope.

[0014] In some embodiments, a working distance of the objective lens of the surgical microscope is a distance between the objective lens of the surgical microscope and a surface of the sample.

[0015] In further embodiments, the imaging system can be one of an optical coherence tomography (OCT) imaging system.

[0016] In still further embodiments, the system can be configured to manually adjust a path length of the reference arm of the imaging system and / or elements of the system.

[0017] In some embodiments, the system can be further configured to automatically adjust a path length of the reference arm of the imaging system and / or elements of the system.

[0018] In further embodiments, the system can further include a demagnification lens between the objective lens and the sample to demagnify a focal length of the objective lens. A relay lens is provided adjacent the sample to relay the objective lens image plane to the retina of the sample to adapt the system for imaging the retina of the sample.

[0019] In still further embodiments, the objective lens can be a multi-focal objective lens, and wherein the system further includes a relay lens adjacent the sample to relay the objective lens image plane to the retina of the sample to adapt the system for imaging the retina of the sample.

[0020] Some embodiments of the inventive concept provide a method for optimizing optics of a surgical microscope having an integrated imaging system, the system comprising one or more processors and one or more storage devices; a first optical system associated with the surgical microscope; a second interferometric measurement optical system associated with the imaging system different from the first optical system. The method comprises calibrating a position of a reference arm of the imaging system such that an image of a sample is visible when the sample is at a working distance of an objective lens of the surgical microscope to provide an initial calibration position; acquiring an image using the initial calibration position of the reference arm of the imaging system; evaluating an image quality of the acquired image to determine whether the acquired image is a valid image; adjusting a path length of the reference arm of the imaging system until the acquired image is determined to be a valid image to provide an adjusted position of the reference arm if the acquired image is determined not to be a valid image; calculating a difference between the initial calibration position of the reference arm and the adjusted position of the reference arm; and adjusting elements of the system based on the calculated difference such that the sample is visible when the sample is at the working distance of the objective lens of the surgical microscope at the adjusted position of the reference arm.

[0021] Related computer program products are also provided. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a focal plane diagram of a microscope objective showing an appropriate "working distance".

[0023] Figure 2 is a focal plane diagram of a microscope objective showing a working distance set too short.

[0024] Figure 3 is a focal plane diagram of a microscope objective showing a working distance set too long.

[0025] Figure 4 is a block diagram showing a system with an OCT imaging plane set to match the focal plane of the objective lens.

[0026] Figure 5A is a block diagram showing an example OCT retinal (back) imaging system.

[0027] Figure 5B is a block diagram showing an example OCT corneal (front) imaging system.

[0028] Figure 6 is a flowchart showing operations for optimization using manual adjustment according to some embodiments of the inventive concept.

[0029] Figure 7 is a flowchart showing operations for optimization using automatic adjustment according to some embodiments of the inventive concept.

[0030] Figure 8 is a flowchart illustrating operations for automated microscope movement and OCT focus adjustment according to some embodiments of the inventive concepts.

[0031] Figure 9 is a flowchart illustrating operations for manual microscope movement and OCT focus adjustment according to some embodiments of the inventive concepts.

[0032] Figure 10 is a diagram of a microscope objective group for imaging a cornea.

[0033] Figure 11 is a diagram of a retinal field lens system including a combination of a reducing lens and a relay lens group.

[0034] Figure 12 is a diagram showing that wide field of view (FOV) imaging requires the retinal relay lens to be very close to the cornea.

[0035] Figure 13 is a diagram of a microscope objective design consisting of separate elements.

[0036] Figure 14 is a diagram showing an extension of the system with separate lens design to include multifocal capability elimination.

[0037] Figure 15 is a multifocal design of the system that allows full retinal FOV imaging.

[0038] Figure 16 is a block diagram of a data processing system that can be used to implement processes in accordance with various embodiments of the inventive concepts.

[0039] Figure 17 is a block diagram of a system including a microscope according to some embodiments of the inventive concepts. DETAILED DESCRIPTION

[0040] The inventive concepts will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the inventive concepts are shown. The inventive concepts may, however, be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein.

[0041] Therefore, although the inventive concepts are susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the inventive concepts are not to be construed as being limited to the particular forms disclosed, but to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the inventive concepts as defined by the claims. Like numbers refer to like elements throughout the description of the figures.

[0042] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the inventive concepts. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Additionally, when an element is referred to as being "responsive" or "connected" to another element, it can be directly responsive or connected to the other element, or indirectly responsive or connected to the other element through one or more other elements. In contrast, when an element is referred to as being "directly responsive" or "directly connected" to another element, there are no intervening elements. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items and can be abbreviated as " / ".

[0043] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this inventive concept belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0044] Although some aspects are described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where the blocks or features of the apparatus correspond to method steps or method features. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0045] It should be understood that, although the terms first, second, etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the teachings of the present disclosure. Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication can occur in the opposite direction to the depicted arrows.

[0046] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0047] These computer program instructions can also be stored in a computer readable medium that when executed can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions

[0048] As used herein, "subject" refers to a person or thing or a part of a person or thing that is imaged. It should be understood that while embodiments of the present inventive concepts are directed to eyes as the subject as discussed herein, embodiments of the present inventive concepts are not limited to this configuration. The subject can be any subject including, for example, a veterinary, cadaver study, or human subject without departing from the scope of the present inventive concepts.

[0049] While many of the examples discussed herein refer to the sample / subject being an eye, specifically the retina, cornea, anterior segment, and lens of the eye, embodiments of the present inventive concepts are not limited to this type of sample. Any type of sample that can be used in conjunction with the embodiments discussed herein can be used without departing from the scope of the present inventive concepts.

[0050] While embodiments of the present inventive concepts focus on using OCT to scan the sample, embodiments of the present inventive concepts are not limited to using OCT. It should be understood that any method and system for scanning a sample can be used without departing from the scope of the present inventive concepts.

[0051] Further, imaging as discussed herein can be performed in any manner known to those of skill in the art. For example, in some embodiments, an imaging system can be incorporated into a microscope or surgical microscope. Various embodiments of these are discussed in, for example, U.S. Patent No. 8.77,412 and U.S. Patent Publication Nos. 2015 / 0168250 and 2015 / 0359426, the disclosures of which are incorporated by reference herein as if fully set forth in their entirety.

[0052] As used herein, "working distance" refers to the distance from the last glass surface of the objective to the surface of the sample. Throughout this specification, the working distance can be referred to as WD.

[0053] Systems according to some embodiments of the inventive concept include a microscope that includes a surgical optical system (first optical system) and a second, interferometry-based optical system (second optical system). In some embodiments, the interferometry-based optical system is an OCT imaging system. However, embodiments of the inventive concept are not limited to this. For example, as used herein, an "interferometric imaging system" refers to an imaging system that utilizes light signals generated by interference of light between a reference optical path and a sample optical path. Some embodiments can further include one or more additional imaging systems that have separate optical paths in the microscope, but share the same objective. For example, these additional imaging systems can be fundus fluorescein, scanning laser ophthalmoscopy, widefield imaging systems, etc., without departing from the scope of the inventive concept.

[0054] Example interferometric imaging systems used according to some embodiments of the inventive concept will now be discussed with reference to Figure 5A and 5B It should be understood that these systems are provided for example purposes only, and thus, embodiments of the inventive concept should not be limited thereto. A conventional Fourier-domain OCT (FDOCT) system will now be discussed to provide some background knowledge related to these systems. Reference is first made to Figure 5A A block diagram of a FDOCT retinal imaging system will be discussed. As shown in Figure 5A the system includes a broadband source 100, a reference arm 110, and a sample arm 140 coupled to each other by a beamsplitter 120. The beamsplitter 120 can be, for example, a fiber coupler or bulk or micro-optic coupler. The beamsplitter 120 can provide a splitting ratio from about 50 / 50 to about 90 / 10. As further shown in Figure 5A the beamsplitter 120 is also coupled to a wavelength or frequency sampling detection module 130 by a detection path 106, which can be provided by an optical fiber.

[0055] As shown in Figure 5AFurther shown, the source 100 is coupled to the beamsplitter 120 by a source path 105. The source 100 can be, for example, a continuous wave broadband superluminescent diode, a pulsed broadband source, or a tunable source. The reference arm 110 is coupled to the beamsplitter 120 by a reference arm path 107. Similarly, the sample arm 140 is coupled to the beamsplitter 120 by a sample arm path 108. The source path 105, the reference arm path 107, and the sample arm path 108 can all be provided by optical fiber or a combination of optical fiber, free space, and bulk or micro-optics.

[0056] As Figure 5A shown, the reference arm of the FDOCT retinal imaging system can include a collimator assembly 180, a variable attenuator 181 which can include a neutral density filter or a variable aperture, a mirror assembly 182, a reference arm variable path length adjustment 183, and a path length matching location 150, i.e., a matching of the optical path length between the reference arm path length and the sample arm path length to the region of interest of the subject. As further shown, the sample arm 140 can include a biaxial scanner assembly 190 and an objective lens 191 with variable focal length.

[0057] Figure 5A The sample shown in is an eye, including a cornea 195, an iris / pupil 194, an ocular lens 193, and a retina 196. An indication of the FDOCT imaging window 170 is shown near the retina 196. The retinal imaging system relies on the optics of the subject eye, particularly the cornea 195 and the ocular lens 193, to be added on top of the objective lens to image the posterior structures of the eye. As further shown, the region of interest 170 within the subject is selected by coordination of the focal position 196 and the reference arm path length adjustment 183 coordination so that the path length matching location 197 within the subject is at the desired location.

[0058] Figure 5B Reference will now be made to Figure 5B a block diagram showing an FDOCT corneal (anterior) imaging system. As shown therein, Figure 5A the system is very similar to the system of Figure 5B . However, the objective lens variable focus is not necessarily included, nor is it included in Figure 5B the anterior imaging system directly images the anterior structures without relying on the optics of the subject to focus the anterior structures.

[0059] As discussed in the background above, OCT systems are designed to have the best image quality at the working distance WD of the microscope objective, as Figure 4As shown. The optical path length of the sample arm through the OCT system is typically matched to the optical path length of the reference arm through the OCT system, which usually consists of a mirror mounted on a translational stage. As used herein, "optical path length (OPL)" refers to the geometric length of the path light follows through a given system multiplied by the optical refractive index of the concurrent medium through which the light travels. The specific position of this reference mirror in the reference arm is typically mapped to the working distance of the microscope objective. If the sample is placed outside the working distance of the microscope objective, the reference mirror is moved accordingly to ensure optical path length matching. Initial calibration of the reference arm's position relative to the working distance allows for the detection of deviations from this position. The conventional method for adjusting system components is often trial and error, i.e., without providing specific coordinates for the adjustments made.

[0060] Therefore, some embodiments of the present invention provide methods and systems for detecting the presence of a valid OCT image of a sample target, positioning an OCT reference arm at the location of the detected image, and optimizing the optics of multiple optical subsystems to obtain the best possible image. In other words, instead of using conventional methods of trial and error, embodiments of the present invention use OCT to measure where the target is and then instruct the system operator to proceed based on the OCT measurement. In some embodiments, these adjustments can be performed automatically, as will be discussed further below.

[0061] Now for reference Figure 6 The discussion will present a flowchart illustrating the existence of a valid OCT image of the detected sample target, the placement of the OCT reference arm in the detected image, and the optimization of the optics of multiple optical subsystems to obtain optimal image quality. (See flowchart for details.) Figure 6 As shown, the operation optimized by manually adjusting the microscope using the reference arm of the calibration system begins at block 600. Factory or servicing calibration of the instrument is possible. The distance of the OCT reference arm is set such that the OCT image of the sample target is visible when the target (sample) is at the working distance of the microscope objective, for example, as... Figure 4 As shown. Traditionally, the target is first observed through the visible light path of a microscope set at infinity focus and maximum magnification to ensure that the target is accurately located on the focal plane of the microscope objective. This is a subjective technique and can therefore lead to inaccuracies. Therefore, according to some embodiments of the concept of the invention, the methods discussed herein are used to increase the likelihood that the target is placed at the accurate (substantially) focal plane of the objective. It should be understood that "accurate" position means a position as close as possible to an accurate position, as an accurate position may not be achievable. As used herein, "focal plane" refers to the distance from the objective where the collimated light collected at the lens aperture converges.

[0062] The signal used to generate the OCT image can be produced from the interference between light in the sample and reference arms (cross-correlation signal) or from the interference of light between different optical reflectors in a single arm (auto-correlation signal). The optical path lengths in the reference and sample arms must be closely matched to generate a cross-correlation signal within the detection bandwidth of the system's optical detector. To view the auto-correlation signal, the light must be sufficiently focused on the sample to generate sufficient optical signal strength from a single arm. The Fourier transform of the auto-correlation signal results in the signal near zero frequency and is subject to the same signal decay effects as the cross-correlation signal. The auto-correlation signal provides a means to determine if the sample is located in the focal plane of the microscope objective. Light from the reference arm can be blocked or the reference arm moved to a position to move the cross-correlation signal outside the detection bandwidth of the system. A collimated (parallel) OCT beam can be injected through the objective and the maximum signal strength of the OCT auto-correlation signal generated by the reflected light from the target evaluated to determine if the sample is in the correct position. The focus of the microscope objective is adjusted to increase or possibly maximize this signal. This can require moving the position of the objective or adjusting optical elements to move the focal plane of the objective and can be done manually or in an automated fashion. The evaluation of the auto-correlation signal can include an analysis of the signal-to-noise ratio or intensity of the Fourier transform amplitude near zero frequency. Once the auto-correlation is maximized, the reference path is adjusted until the OCT cross-correlation signal is visible and located near the zero frequency position within the OCT B-Scan.

[0063] Once the calibration has been performed (block 600), the image is detected and the quality of the image is evaluated (block 610). For example, the presence of a valid OCT image is detected via a software algorithm that evaluates the signal quality or detects specific features in a given OCT image. In some embodiments, these methods can include but are not limited to the maximum intensity, intensity distribution or intensity variation of the entire image, the signal-to-noise ratio of the image or by machine learning to detect specific sample target features or classical object detection methods. It is determined if a valid OCT image is present (block 630). If a valid image is not present (block 610), the reference arm is moved through a series of positions (block 620) until a position is found that produces a valid OCT image (block 630). Other system optics that affect the focusing, polarization or other factors that can affect the image quality of the optical beam are adjusted to optimize the OCT signal strength and can be done in sequence or simultaneously with the reference arm movement. If it is determined that a valid image is present (block 630), the difference in position of the calibrated reference arm relative to the microscope working distance is calculated (block 640).

[0064] Determine if the microscope is in focus (block 645). If it is determined that the microscope is not in focus (block 645), manual adjustment can be performed (block 650) until it is determined that the microscope is in focus (block 645). Once focused (block 645), the operation can proceed to block 660, where other subsystem optics can be adjusted. For example, the user can be instructed to adjust other optical components that may affect image quality, i.e., microscope eyepieces, to ensure parfocality (focus on the same plane) for each imaging mode integrated with the microscope. If the microscope includes other imaging modes, such as fluorescence microscopy, scanning laser ophthalmoscope, etc., this information can also be used to optimize the focus of the relevant optics to improve image quality without departing from the scope of the invention. In other words, any other additional imaging system with a separate optical path but sharing the same objectives in the microscope can be used without departing from the scope of the invention. For example, fundus fluorescence, scanning laser ophthalmoscope, wide-field imaging systems, etc.

[0065] although Figure 6 The embodiment of the inventive concept shown illustrates manual adjustment of the microscope (block 650), but the embodiment of the inventive concept is not limited to this configuration. Figure 7 It shows the above about Figure 6 The operations discussed are similar, therefore, for the sake of brevity, the details of the blocks will not be repeated here. However, as... Figure 7 As shown, instead of manually moving the microscope, Figure 7 Block 751 indicates that the movement of the microscope can be automatic. Therefore, the system according to the embodiments discussed herein can be programmed to adjust automatically without departing from the scope of the inventive concept. For example, the working distance of the microscope can be adjusted automatically, such as... Figure 7 As shown, or information about the direction and distance of moving the microscope can be conveyed to the user, such as... Figure 6 As shown.

[0066] Now for reference Figure 8 and 9 Flowchart, flowchart and Figure 6 and Figure 7 The flowcharts are very similar. Therefore, for the sake of brevity, details about similar components will not be repeated. Except in Figure 7 Beyond the details discussed in the article, Figure 8 This includes OCT focus adjustment (block 815) in response to determining that an image is invalid (block 630). Therefore, when an image is determined to be invalid (block 630), the reference arm path length is adjusted (block 620) and the OCT focus is adjusted (block 815). The remaining operations are as described above regarding microscopes with automatic adjustment. Figure 7 Proceed as discussed (block 751). Figure 9 The flowchart adds the same OCT adjustment (block 815) to the above regarding...Figure 6 The manual microscope adjustments discussed (block 650).

[0067] Reference is now made to Figure 10 and 11 The addition of an auxiliary lens system to the objective lens group will be discussed. Figure 10 A standard microscope objective lens group for corneal imaging is shown. Figure 11 A system comprising a standard retinal viewing lens system is shown, which includes an additional reduction lens and a relay lens. When these axillary retinal viewing lens systems are attached to a microscope, other complications can arise. These lens systems typically require the relay of the microscope objective focal plane from the corneal location to the retina of the eye under surgery. Since the standard workflow requires these imaging systems to operate without the need to move the microscope position between corneal and retinal imaging, the relay system is composed of two cooperating optical elements. Figure 11 The first element 1170 (reduction lens) of the system of Figure 11 The relay lens 1175 of the system of

[0068] As shown in Figure 12 Large field of view (FOV) typically requires the relay lens 1175 to be in close proximity to the cornea. Therefore, if the microscope setup is not correct, as discussed above with respect to Figure 2 and 3 then the addition of the retinal viewing system (1170 / 1175) will cause further disturbance to the microscope imaging optics from the optimal working distance, thereby forcing other imaging modalities, such as OCT to effectively operate outside the normal imaging window.

[0069] The adjustability of the retinal viewing lens system relay lens can cause further complications to accommodate for refractive errors of the eye under surgery. These errors can compound in such a way that creates non-functional results for the additional imaging modalities, such as OCT. Some embodiments of the inventive concept address this situation by providing a multi- focal objective lens instead of a fixed focal objective lens and relay 1435, as shown in Figures 13 to 15

[0070] Current microscope lens designs use split elements as a method to reduce optical aberrations in the image, as depicted in Figure 13 Extending this concept to allow for the increase or decrease of the air space between the elements, thereby enabling the adjustability of the overall lens focal length. Figure 14 Increasing the air space between the elements is shown to shorten the focal length, thereby allowing the retinal viewing relay lens 1435 to be used for imaging the retina without the need to move the microscope position as Figure 11 ​a secondary downscaling lens as shown in the standard configuration. Figure 15 It is shown that a multi-focal design can allow for full retinal field of view imaging.

[0071] Further, the air spaces between the adjustment elements can accommodate refractive errors of the eye during surgery. One advantage of using a multi-focal objective design is that the requirement for individual focal adjustability between the light paths is effectively eliminated since the light paths for all imaging modes are co-aligned through the objective.

[0072] Independent feedback from an external rangefinder can also be integrated into the positioning control loop to further reduce the risk of the retinal relay lens affecting the patient’s eye.

[0073] As is clear from the above discussion of embodiments of the inventive concept, many of the methods discussed herein require processing provided by a computing device. Reference is now made to Figure 16 An example embodiment of a data processing system 1630 configured in accordance with embodiments of the inventive concept will be discussed with reference to Figure 16 As will be appreciated, the data processing system can be included in a system, for example, as Figure 5A and 5B In a microscope, as shown, or as a separate device in communication with the system in Figure 5A and 5B The data processing system 1630 can include a user interface 1644, including, for example, an input device, such as a keyboard or keypad, a display, a speaker and / or a microphone, in communication with the processor 1638, and a memory 1636. The data processing system 1630 can further include an I / O data port 1646 also in communication with the processor 1638. The I / O data port 1646 can be used to transfer information between the data processing system 1630 and another computer system or network, for example, connected using an Internet Protocol (IP) connection. These components can be conventional components, such as those used in many conventional data processing systems, which can be configured to operate as described herein.

[0074] As Figure 16 Further shown, the data processing system 1630 is in communication with a microscope 1610, which includes a first optical system 1637, a second interferometric measurement optical system 1647, and optionally one or more additional systems 1657 Figure 16The dashed lines in the figure indicate optional elements). The first optical system 1637 can be associated with a microscope, and the second, interference-based optical system 1647 can be provided by an OCT imaging system as described above. However, embodiments of the inventive concept are not limited thereto. As further shown, additional imaging modalities can also be provided by optional systems associated with the first and second systems. For example, some embodiments can further include one or more additional imaging systems 1657 that have separate optical paths in the microscope, but share the same objective lens. For example, these additional imaging systems can be fundus fluorescence, scanning laser ophthalmoscopy, wide-field imaging systems, etc., without departing from the scope of the inventive concept.

[0075] Some embodiments of the inventive concept relate to a microscope comprising a combination of the systems described in one or more of Figures 1 to 16 Alternatively, the microscope can be part of or connected to a system comprising one or more of the systems described in one or more of Figures 1 to 16 Figure 17 A schematic of a system 1700 configured to perform the methods described herein is shown. The system 1700 comprises a microscope 1710 and a computer system 1720. The microscope 1710 is configured to take images and is connected to the computer system 1720. The computer system 1720 is configured to perform at least part of the methods described herein. The computer system 1720 can be configured to perform a machine learning algorithm. The computer system 1720 and the microscope 1710 can be separate entities, but can also be integrated together in a common housing. The computer system 1020 can be part of a central processing system of the microscope 1710 and / or the computer system 1720 can be part of a subcomponent of the microscope 1710, such as a sensor, actuator, camera or illumination unit of the microscope 1710, etc.

[0076] ​The computer system 1720 can be a local computer device (e.g., a personal computer, a laptop, a tablet computer, or a mobile phone) having one or more processors and one or more storage devices, or can be a distributed computer system (e.g., a cloud computing system having one or more processors distributed in various locations, such as at a local client and / or one or more remote server farms and / or data centers). The computer system 1020 can include any circuit or combination of circuits. In one embodiment, the computer system 1-20 can include one or more processors which can be any type of processors. As used herein, a processor can refer to any type of computing circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), for example, a microscope or a microscope assembly (e.g., a camera), or any other type of processor or processing circuit. Other types of circuit that can be included in the computer system 1720 can be custom circuit, an application specific integrated circuit (ASIC), etc., such as, for example, one or more circuits (such as a communication circuit) for a wireless device, such as a mobile phone, a tablet, a laptop, a two-way radio, and similar electronic systems. The computer system 1020 can include one or more storage devices, which can include one or more memory elements suitable for the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CDs), flash memory cards, digital video disks (DVDs), and the like. The computer system 1720 can also include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that allows a system user to input information to and receive information from the computer system 1720.

[0077] Some or all of the method steps can be performed by (or using) a hardware apparatus, such as, for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps can be performed by such an apparatus.

[0078] Depending on certain implementation requirements, embodiments of the inventive concept can be implemented in hardware or in software. The implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

[0079] Some embodiments according to the inventive concept comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0080] Generally, embodiments of the present inventive concept can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code can for example be stored on a machine readable carrier.

[0081] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0082] In other words, an embodiment of the inventive concept is, therefore, a computer program for performing one of the methods described herein, when the computer program runs on a computer.

[0083] A further embodiment of the inventive concept is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when the program is run by a processor. The data carrier, the digital storage medium or the record medium is typically tangible and / or non-transitionary. A further embodiment of the inventive concept is an apparatus comprising a processor and the storage medium.

[0084] A further embodiment of the inventive concept is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be configured to be transferred via a data communication connection, for example via the Internet.

[0085] A further embodiment comprises processing means, for example a computer or a programmable logic device, configured to or adapted for performing one of the methods described herein.

[0086] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0087] A further embodiment according to the concept of the present application comprises an apparatus or system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0088] In some embodiments, portions or all of the functionality described herein can be performed by a programmable logic device (for example, a field programmable gate array). In some embodiments, a field programmable gate array can cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0089] The flow and block diagrams in the drawings represent possible architectures, functions, and operations for systems, methods, and computer program products according to various aspects of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may, sometimes, be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0090] The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The aspects disclosed herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others skilled in the art to understand the disclosure with various modifications being suited to the particular use contemplated.

Claims

1. A system for optimizing the optics of a surgical microscope with an integrated imaging system, the system comprising: One or more processors and one or more storage devices; The first optical system associated with a surgical microscope; as well as A second interference-based optical system, distinct from the first optical system, associated with the imaging system, wherein the system for optimizing the optics of a surgical microscope with an integrated imaging system is configured as follows: The position of the reference arm of the second interference-based optical system is calibrated such that an image of the sample is visible when the sample is at the working distance of the objective lens of the surgical microscope, in order to provide an initial calibration position; Image acquisition is performed using the initial calibration position of the reference arm of the imaging system for the intended sample target; The image quality of the acquired images is evaluated to determine whether the acquired images are valid images; If it is determined that the acquired image is not a valid image of the expected sample target, the path length of the reference arm of the imaging system is adjusted until it is determined that the acquired image is a valid image, so as to provide the adjusted position of the reference arm; Calculate the difference between the initial calibration position of the reference arm and the adjusted position of the reference arm; as well as The components of the system for optimizing the optics of a surgical microscope with an integrated imaging system are adjusted based on the calculated difference, such that the sample is visible when it is located at the working distance of the surgical microscope objective at the adjusted position of the reference arm.

2. The system according to claim 1, wherein, The system is further configured as follows: Before adjusting the components of the system, determine whether the imaging system is in focus at the adjusted position of the reference arm; and When it is determined that the system is out of focus, adjust the system components until it is determined that the imaging system is in focus.

3. The system according to claim 1, wherein, The system is further configured to repeatedly adjust the path length of the reference arm of the imaging system, calculate the difference between the current position of the reference arm and the initial calibration position, and adjust the components of the system until the system is determined to be in focus.

4. The system according to claim 1, wherein, The system is further configured to determine whether the acquired image contains a valid image of the sample target based on one of the quality of the signal and the presence or absence of features in the acquired image.

5. The system according to claim 1, wherein, When the system is focused, the imaging plane of the imaging system matches the focal plane of the surgical microscope.

6. The system according to claim 1, wherein, The system is further configured to adjust the elements of the system by adjusting at least one of a plurality of eyepieces associated with the surgical microscope and other optical components of the system.

7. The system according to claim 1, wherein, The system is further configured to adjust the position of the reference arm by adjusting a mirror in the reference arm to provide matching optical path lengths in the reference arm and sample arm of the imaging system, as well as the working distance, such that the sample is visible when it is located at the working distance of the objective lens of the surgical microscope.

8. The system according to claim 1, wherein, The working distance of the objective lens of the surgical microscope is the distance between the objective lens of the surgical microscope and the sample surface.

9. The system according to claim 1, wherein, The imaging system includes one of the following: an optical coherence tomography (OCT) imaging system, and at least one additional imaging system that is optionally different from the first optical system and the second interference-based optical system.

10. The system according to claim 1, wherein, The system is configured to manually adjust the path length of the reference arm of the imaging system and / or the components of the system.

11. The system according to claim 1, wherein, The system is configured to automatically adjust the path length of the reference arm of the imaging system and / or the components of the system.

12. The system according to any one of claims 1-11, wherein, The system further includes: A reducing lens located between the objective lens and the sample, used to reduce the focal length of the objective lens; and A relay lens adjacent to the sample is used to relay the image plane of the object to the retina of the sample, so as to adapt the system to retinal imaging of the sample.

13. The system according to any one of claims 1-11, wherein, The objective lens is a multifocal objective lens, and the system further includes a relay lens adjacent to the sample for relaying the image plane of the objective lens to the retina of the sample, so as to adapt the system to imaging the retina of the sample.

14. A method for optimizing the optics of a surgical microscope with an integrated imaging system, the system for optimizing the optics of a surgical microscope with an integrated imaging system comprising one or more processors and one or more storage devices; The first optical system associated with a surgical microscope; A second interference-based optical system, different from the first optical system, associated with the imaging system, wherein the method includes: The position of the reference arm of the imaging system is calibrated such that an image of the sample is visible when the sample is at the working distance of the objective lens of the surgical microscope, in order to provide an initial calibration position; Images are acquired using the initial calibration position of the reference arm of the imaging system; The image quality of the acquired images is evaluated to determine whether the acquired images are valid images; If it is determined that the acquired image is not a valid image, the path length of the reference arm of the imaging system is adjusted until it is determined that the acquired image is a valid image, so as to provide the adjusted position of the reference arm; Calculate the difference between the initial calibration position and the adjusted position of the reference arm; and The components of the system for optimizing the optics of a surgical microscope with an integrated imaging system are adjusted based on the calculated difference, such that the sample is visible when it is located at the working distance of the surgical microscope objective at the adjusted position of the reference arm.

15. The method of claim 14, further comprising: Before adjusting the components of the system, determine whether the imaging system is in focus at the adjusted position of the reference arm; When it is determined that the system is out of focus, adjust the system components until it is determined that the imaging system is in focus.

16. The method of claim 14, wherein, The method further includes repeatedly adjusting the path length of the reference arm of the imaging system, calculating the difference, and adjusting the components of the system until the system is determined to be in focus.

17. The method of claim 14, wherein, The method further includes determining whether the acquired image is a valid image based on one of the quality of the signal and the presence or absence of features in the acquired image.

18. The method according to claim 14, wherein, When the system is focused, the imaging plane of the imaging system matches the focal plane of the surgical microscope.

19. The method of claim 14, wherein, Adjusting the components of the system includes adjusting at least one eyepiece among the eyepieces associated with the surgical microscope, as well as other optical components of the system.

20. The method of claim 14, wherein adjusting the position of the reference arm comprises adjusting a mirror in the reference arm to provide matching optical path lengths in the reference arm and sample arm of the imaging system, and the working distance, such that the sample is visible when it is located at the working distance of the objective lens of the surgical microscope.

21. The method according to claim 14, wherein, The working distance of the objective lens of the surgical microscope is the distance between the objective lens of the surgical microscope and the sample surface.

22. The method according to claim 14, wherein, The imaging system includes one of the following: an optical coherence tomography (OCT) imaging system, and an additional imaging system different from the first optical system and the second interference-based optical system.

23. The method according to any one of claims 14-22, wherein, The adjustment further includes manually adjusting the path length of the reference arm of the imaging system and / or the components of the system.

24. The method according to any one of claims 14-22, wherein, The adjustment further includes automatically adjusting the path length of the reference arm of the imaging system and / or the components of the system.

25. The method of claim 14, further comprising: Reduce the focal length of the objective lens; as well as The object's mirror plane is relayed to the sample's retina to adapt the system for retinal imaging of the sample.

26. A computer program having program code for performing the method according to any one of claims 14-25 when the computer program is run on a processor.

Images