Self-calibration and directed focusing system and method for infinity-corrected microscopes

Abstract

A system and method for automatic focusing of an objective lens in a microscope system. A self-calibrating autofocus system includes a light source, an eccentric hole, and an image capture device. The system can be connected to a microscope system so that the light source generates light in an optical path including an objective lens and a plate having a reference surface at the proximal end of a sample holding component. When the objective lens is moved to a series of z positions, the image capture device images the light reflection from the reference surface. A reference calibration slope for the objective lens is generated by determining the positions of multiple images taken at the z positions of the objective lens. At least one image with a specific attribute corresponds to a best focus position. The objective lens is moved to predict the best focus or a preferred offset relative to the calibrated best focus using the reference calibration slope.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 16 / 397,665, filed on April 29, 2019, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present subject matter relates to systems and methods for focusing a microscope, and more particularly to systems and methods for calibrating an optimal focus position for automatic focusing of an objective of a microscope. Background Art

[0004] Microscopes have evolved from stand-alone arrangements of optics that allow a user to observe images from a single slide to high-throughput imaging systems that generate images of samples deposited in multiple sample wells (also called "microwells," "microtiters," and "microplates") formed in a tray or plate. The optics in a microscope system may be controlled by electromechanical devices under computer program control. Images may be collected using image capture devices such as cameras, charge-coupled devices, or other image capture devices.

[0005] Modern microscope systems can be used to detect and measure optical signal characteristics. In some systems, the sample hole can contain materials such as biological materials, which have been treated with markers or reactive components that cause light emission. In contrast to simply capturing images, microscopes can be used to detect the intensity level of light and the pattern of light. Therefore, the microscope can use a signal detector device and an image capture device to obtain the measured value of the light level. Therefore, in addition to the image capture device, the detection component of the microscope can also include, for example, a photomultiplier tube, a photodetector, and a photodetector array. The microscope system can also provide an alternative lighting system, such as a coherent (laser) or incoherent light source, or a light source filtered at a selected wavelength. Filters, beam splitters and other optical devices can be inserted at various positions in the light path between the light source and the sample or between the sample and the detection component to excite the sample in a desired manner, or to measure selected optical properties including intensity and other characteristics at certain wavelengths.

[0006] When capturing or discerning patterns in an image, or when measuring an optical signal, optical devices such as lenses, mirrors, filters, etc. are arranged along the optical path from the light source to the image capture device. When the objective lens is focused on a reference surface, the imaging lens is adjusted to focus the light source on the image plane of the image capture device. Loss of focus appears as blur or lack of sharpness in the image. Similarly, optical measurements taken from an unfocused objective lens may produce inaccurate intensity levels and pixels that may be brighter or darker than they are supposed to be due to the expansion of the image when out of focus. Prior to imaging or optical measurements through a microscope system, the objective lens should generally be positioned to focus on the sample.

[0007] In high-throughput microscopy systems, where the microscope can be used to capture images or optical signal measurements of a large number of samples, the focusing step may need to be performed multiple times as quickly as possible. The focusing process should also be sensitive and accurate while adding as little complexity as possible and not disturb the sample. The focusing process should also be an automated focusing process or a process that is performed with as little user involvement as possible. Fluorescence microscopy is used to analyze biological samples labeled with one or more fluorophores. It is important to avoid photodamage to the fluorophores and the biological sample.

[0008] For each sample being processed, the microscope system typically performs hardware-based autofocus. In a high-throughput microscope system, a sample well supported on a plate has a sample well bottom surface and a plate bottom surface. The sample well bottom surface and the plate bottom surface are at least partially reflective. In one method of a hardware-based autofocus system, the sample well is illuminated by an objective lens, which will be used to perform imaging or optical measurements on the sample in the sample well. The objective lens is positioned at a series of positions along the optical axis (along which the objective lens travels), while checking the image and pixel intensity at each position. The optical axis along which the objective lens travels can be referred to as the z-axis, and the movement of the objective lens can be referred to as scanning along the z-axis.

[0009] FIG. 1A is a schematic diagram of a microscope 100, which illustrates the operation of a process for performing hardware-based autofocusing of the microscope 100 so as to image a sample 114 in a sample well 112 disposed on a plate 110. The microscope 100 in FIG. 1A includes the following main components for autofocusing: an objective lens 102, a detector 104, a beam splitter 106, and a light source 108. The microscope 100 can be implemented as a system including components configured to control the state and movement of the objective lens 102 and the plate 110. These components can also be configured to control the state of the light source 108 and the operation of the detector 104. For example, a linear actuator with a motor under the control of a computer program can be used to control the movement of the objective lens 102 along the z-axis. The objective lens 102 is moved to a position on the z-axis (i.e., the z position). The detector 104 can be implemented as a charge coupled device or other image capture device, which is also controlled to capture an image and transmit the digital content of the image to a computer through a computer program.

[0010] In an exemplary hardware-based autofocus process, a user interface may be provided for inputting configuration data to configure the microscope according to the type of objective lens 102, the type of plate 110, and the light source 108. The system may then position the objective lens 102 at a starting position that is appropriately close to the plate 110 on the z-axis. The starting position may depend on the type of plate 110, as different plates may have plate bottom surfaces at different positions along the z-axis. The objective lens 102 may also be controlled to move along a working distance or scan range through which the best focus position may be found. The process typically involves positioning the objective lens 102 at each z position in the scan range and capturing an image at each z position.

[0011] Light from light source 108 follows a first optical path at 101 from light source 108 to beam splitter 106. Beam splitter 106 reflects a portion of the light toward objective 102, which directs the light along optical path 105 to sample well 112. Light is reflected back from the top and / or bottom surface of plate 110 toward objective 102 and passes through beam splitter 106 along optical path 107 to detector 104. If the position of objective 102 is close enough to the position where objective 102 is focused on the reflective surface of plate 110, the light appears as a beam spot having a diameter and intensity at detector 104. As objective 102 moves along the z-axis, the system controls detector 104 to capture an image at each z-axis position. Each image is analyzed to determine a focus score based on a focus metric, which may include, for example, the average intensity, maximum signal, standard deviation, variance or covariance, size, morphology, position, etc. of the pixels in each beam spot. The smallest and brightest beam spot is generally considered to have the "best focus." The focus score may be plotted relative to the z position of each image captured in graph 120. In graph 120 (see FIG. 1B ), the smallest and brightest beam spot may have its peak value for intensity and z position at 124. Graph 120 may also identify the brightest pixels as peaks 122 and 126. The z position at the highest focus score (at 124 in FIG. 1B ) is identified as the best focus position.

[0012] It should be noted that two peaks may be formed in the graph, representing reflections from two surfaces on the plate 110. Once one or more peaks are identified and a corresponding z position identified for each peak, the process identifies the best focus position as the z position corresponding to a selected one of the peaks. That is, the best focus position is the position where the beam spot formed by light reflected from the top or bottom surface of the plate 110 has a best focus score.

[0013] The best focus position in the context of the autofocus procedure described with reference to FIG1A is the z position of the objective lens 102 that produces the sharpest and clearest projection of the light beam reflected from the reference surface. In the system shown in FIG1A, the reference surface is the top surface or the bottom surface of the plate 110. In order to image with good focus on a sample such as a cell collection (114 in FIG1A), the objective lens 102 should be moved to focus on the sample 114. In some embodiments, the offset along the z-axis can be defined based on the sample type, the objective lens, the plate type, and / or other suitable parameters. When the best focus position is determined, the objective lens 102 can be moved according to the offset to start imaging. In some embodiments, the system can perform image-based autofocus after hardware-based autofocus. Image-based focusing involves moving the objective lens 102 along the z-axis while checking the actual image of the sample until the sample is in focus.

[0014] The autofocus process described with reference to FIG. 1A may include hardware-based autofocusing to find a reference surface on the plate 110, and moving with a certain offset to focus on the sample, or image-based focusing to find the sample 114. Even if the objective lens is not replaced, each time a new sample hole is provided for imaging, it may be necessary to perform hardware-based autofocusing or image-based autofocusing or both. The microplate on which the sample hole is disposed may not be completely flat, and the thickness of the bottom of the sample hole may not be consistent. Different samples or different light sources may also require refocusing.

[0015] Current hardware-based autofocus processes are slowed by the need to collect data points from every z position in the working distance of the objective 102. In systems that use a microscope to image a large number of samples, even a short delay in focusing the microscope objective may be too long. Other processes used are accompanied by tradeoffs and other limitations, such as complexity, reduced accuracy, reduced sensitivity to surfaces that reflect the beam imaged during focusing, and high cost. Summary of the invention

[0016] According to one aspect, a method for performing a self-calibrating autofocus procedure on an objective in a microscope system is provided. In an exemplary method, a reference calibration slope is generated by determining corresponding positions of a plurality of images taken at a series of z positions of the objective. The images are analyzed to determine specific properties corresponding to a best focus position. Then, an autofocus procedure can be performed on the objective based at least in part on the reference calibration slope and the best focus position.

[0017] In another aspect, the reference calibration slope indicates a linear relationship between a range of objective lens positions and the lateral position on the image plane as the reference image is translated in the image plane.

[0018] In another aspect, the reference image may be a beam spot reflected from a reference surface and projected onto the image capture device. Alternatively, the reference image may be a patterned image.

[0019] In another aspect, a system for autofocusing is provided. An example of an autofocusing system includes a light source, an eccentric aperture, and optical components forming an optical path toward an image capture device. The autofocusing system can be connected to a microscope system having an optical path including an objective lens of the microscope system. The autofocusing system can also operate as a subsystem of the microscope system. The autofocusing system also includes a controller or operates using a controller operating in the microscope system. The controller can include a storage device for machine instructions that, when executed by a processor, perform a self-calibrating autofocusing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings illustrate the design and utility of the embodiments of the disclosed invention, wherein similar elements are represented by common reference numerals. The drawings are not necessarily drawn to scale. The drawings depict only typical embodiments of the disclosed invention and therefore should not be considered as limiting the scope of the disclosed invention.

[0021] FIG. 1A is an example of a known microscope system configured to perform a hardware-based or image-based autofocus process.

[0022] 1B is an example of a graph of focus scores measured for the intensity of light reflected from two surfaces at the sample aperture scanned against the z position of the objective.

[0023] Figure 2A An exemplary embodiment of a system for calibrating an optimal focus position and automatically focusing an objective in a microscope system.

[0024] Figure 2B is a flow chart illustrating the operation of an exemplary system for calibrating an optimal focus position and autofocusing an objective in a microscope system.

[0025] Figure 3A is a schematic illustration of an example of multiple images showing the lateral translation of a beam spot superimposed on a representation of a linear array detector during calibration of the best focus position of an objective in a microscope system.

[0026] Figure 3B is an example centroid diagram of the lateral translation beam spot position on the detector during the z position scan of the objective lens.

[0027] FIG. 4A to FIG. 4C is a schematic illustration of examples of images captured at different z positions of the objective, which shows the autofocus process when a reference calibration slope is available for the objective.

[0028] Figure 4D is an example of a patterned image that can be projected on an image capture device as a reference image.

[0029] Figure 5 is another example of a microscope system configured to perform self-calibrating autofocus using an off-center hole 502 in the collimated illumination path.

[0030] Figure 6 is another example of a microscope system configured to perform self-calibrating autofocus utilizing an off-center hole 602 in the collimated return emission path on the detector side.

[0031] Figure 7 is another example of a microscope system configured to perform self-calibrating autofocus. DETAILED DESCRIPTION

[0032] Whether or not explicitly stated, all numerical values ​​herein are assumed to be modified by the term "about" or "approximately", wherein the term "about" or "approximately" generally refers to a numerical range that is considered by those skilled in the art to be equivalent to the cited value (i.e., having the same function or result). In some cases, the term "about" or "approximately" may include numbers rounded to the nearest significant figure. Numerical ranges expressed by endpoints include all numerical values ​​within the range (e.g., 1 to 5 include 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

[0033] Unless the content clearly indicates otherwise, as used in this specification and the appended claims, the singular forms "a", "an", and "the" include plural referents. As used in this specification and the appended claims, the term "or" is generally used in its meaning that includes "and / or", unless the content clearly indicates otherwise. When describing the depicted embodiments of the disclosed invention shown in the accompanying drawings, specific terms are employed for clarity and ease of description. However, the disclosure of this patent specification is not intended to be limited to the specific terms so selected, and it should be understood that each specific element includes all technical equivalents that operate in a similar manner. It should also be understood that within the scope of the present disclosure and the appended claims, various elements and / or features of different illustrative embodiments may be combined with each other and / or replaced with each other.

[0034] Various embodiments of the disclosed invention are described below with reference to the accompanying drawings. It should be noted that the drawings are not drawn to scale, and in all drawings, elements of similar structure or function are represented by similar reference numerals. It should also be noted that the drawings are intended only to facilitate the description of the embodiments. They are not intended to serve as an exhaustive description of the invention or as a limitation on the scope of the disclosed invention, which is limited only by the appended claims and their equivalents. In addition, the illustrated embodiments of the disclosed invention do not need to have all the aspects or advantages shown. For example, an aspect or advantage described in conjunction with a specific embodiment of the disclosed invention is not necessarily limited to that embodiment, and may be practiced in any other embodiment even if not shown as such.

[0035] Described below are systems and methods for determining the optimal focus position of an objective lens in a microscope system. The methods and systems described below disclose a calibration process that produces reference slopes corresponding to various known parameters. Specific to the concepts of the present invention, an eccentric aperture is used to provide wavefront sampling at multiple z positions, which is strategically mapped to form a calibration reference slope. This reference slope obtained by the calibration process can then be advantageously used to determine an offset relative to the optimal focus position, as will be discussed in further detail below. This approach allows the system to quickly calibrate and return to optimal focus during image acquisition, thereby minimizing errors and improving throughput.

[0036] As used in the following description, the term "best focus position" shall mean the z position of the objective lens at which the objective lens provides the best focus on the reference image projected onto the reference surface. Therefore, the best focus position may not be the position at which the objective lens is focused on the sample. When the sample is presented for imaging, the reference surface used to determine the best focus position may be below or above the z position level of the sample. In some exemplary embodiments, for certain imaging procedures or optical measurement procedures, an offset (amount) relative to the best focus position may be defined based on the type of sample being imaged and the objective lens being used for imaging. When higher precision is required, hardware focusing may be performed similar to that described above with reference to FIGS. 1A and 1B . The advantage of such hardware-based focusing starting from the closest best focus position is that fewer z positions are used compared to the above-described procedures that do not start from the best focus position.

[0037] As used in the following description, the terms "imaging", "image capture", "capturing an image", or "detecting an image" shall refer to any process for collecting optical data from an image capture device. The image may be a reference image for obtaining a reference calibration slope, an image of a sample, including capturing a digital image for storage, or measuring optical properties such as intensity, color, or other types of data.

[0038] As used in the following description, the term "sample well" shall refer to any suitable structure for accommodating a sample to be presented for imaging or for obtaining optical measurements. A sample well may include, for example, a recessed structure formed in a microplate as defined below in which a sample may be deposited. A sample well, as the term is defined herein, may also include a slide with or without a cover or any other suitable structure for accommodating a sample.

[0039] As used in the following description, the term "plate" shall refer to any sample holding structure configured to support a container in which a sample can be deposited. In particular, the term "plate" may include a tray or similar structure, including such sample holding structures known in the art by terms including "microwell," "consumable," "microtiter," and "microplate." "Plate" should be understood to refer to a structure capable of holding a single sample well or a plurality of sample wells. In the examples described below, the plate is used to provide a reference surface for determining an optimal focus position. The term "plate" in the following description should be understood to mean any structure that can provide a reference surface for the process described below, unless otherwise specified.

[0040] As briefly discussed above, the present invention discloses systems and methods for calibrating the best focus position of an objective lens in a microscope system, and then using the best focus position calibration to focus the objective lens faster and more efficiently than conventional autofocus techniques. When positioning a sample for measurement using the calibrated objective lens, and after the objective lens may have been repositioned, the best focus position calibration is used to find the position of the objective lens relative to its best focus position. To focus on the sample, the objective lens may be moved to the best focus position or to a predetermined offset relative to the best focus.

[0041] Calibration of the best focus position for a given objective lens includes determining the relationship between the position on the axis of travel of the objective lens and the translation of the image (or reference image) on the image plane of the image capture device. An autofocus subsystem or module in a microscope system may include a suitable light source, an eccentric hole, a reference surface, imaging optics, and an image capture device. The reference surface may be a plate (i.e., a microplate, a microwell plate, a microtiter, etc.) that may be used to hold sample wells for samples on which optical measurements are to be performed. These components may be arranged to illuminate the reference surface through the objective lens of the microscope system using a light source. For example, Figure 2A and Figures 5 to 7 As shown, the optical components project a reference image onto the image capture device. The location and size of the eccentric hole are set to sample a portion of the total light passing through the entrance pupil of the objective lens to be projected onto the image capture device as, for example, a beam spot. As the objective lens moves along its axis of travel, the beam spot translates linearly and changes focus (focus). This pattern is strategically used to create a reference slope that maps the position of the beam spot at multiple z positions, as will be discussed in further detail below.

[0042] Figure 2A 2 is an exemplary embodiment of an autofocus system 200 for calibrating an optimal focus position for a microscope system and autofocusing an objective lens 202. The autofocus system 200 includes an image capture device 204, a light source 206, a first imaging lens 208, a beam splitter 210, a second imaging lens 240, and an autofocus aperture 230. The objective lens 202 is a component of the microscope system that is configured to perform imaging and / or optical measurements on a sample that may be deposited into a sample aperture 214. Other components of the microscope system include any sample holding structure that supports a plate 212 having reference surfaces 212a, 212b.

[0043] Figure 2A The autofocus system 200 in the embodiment is implemented as a module or subsystem of a microscope system. Other components of the microscope system for imaging a sample (e.g., an excitation light source, a filter, a beam splitter, and a sample image capture device) are provided in the embodiment of the present invention. Figure 2A200 is represented as sample imaging component 221. Sample imaging component 221 may include lenses, filters, or other optical devices, such as optical devices that form an optical path including objective lens 202 and plate 212 when the microscope system is used to image a sample. Microscope system 200 may also use different light sources or different sample image capture devices based on the type of imaging or measurement performed. The optical device may be in Figure 2A It is inserted below the objective lens in the embodiment, at 221a above the beam splitter 210, or at 221b above the eccentric hole 230. Figure 2A The sample well 214 in the example shown is formed on a plate 212 which is provided with a bottom reference surface 212a and a top reference surface 212b for use in an autofocus procedure.

[0044] Once a self-calibration of the best focus position has been performed for a given objective 202, the process of focusing that objective 202 in subsequent imaging or optical measurements can be performed with minimal further imaging. The calibration of the best focus position can be stored as a reference calibration slope, which can be stored or included in the data characterizing the objective 202 in the system data storage system 223.

[0045] The first imaging lens 208 collimates light from the light source 206 along the optical path 201 and passes the collimated light toward the beam splitter 210. The beam splitter 210 reflects a portion of the light along the optical path 203 toward the objective 202 and on the optical path 205 toward the plate 212. The plate 212 reflects the light back to the objective 202 and on the optical path 207 toward the beam splitter 210. The beam splitter 210 passes a portion of the light along the optical path 209 toward the eccentric aperture 230. The light passes through an eccentric opening on the aperture 230 that is smaller than the total beam incident on the aperture 230. The remaining portion of the aperture 230 blocks the portion of the beam that does not pass through the aperture. The light that passes through the aperture 230 is directed to the second imaging lens 240 and the image capture device 204. The eccentric aperture 230 operates by sampling a portion of the plane wavefront from the objective 202. The sampled portion of the light is focused by the second imaging lens 240 and directed toward the detector, but is constrained to an asymmetric marginal ray by the eccentric aperture. This allows the position and best focus of the light to be observed without changing any component setup. It will be appreciated that different eccentric apertures of different sizes and / or positions may be used to allow for adjustment of sensitivity or different sizes of pupil diameter of objective 202.

[0046] It should be noted that Figure 2A The optical paths 201, 203, 205, 207, and 209 shown in FIG. 20 show only the light along the optical path forming the light beam incident on the image capture device 204. The portion of light not shown is the portion of light blocked by the blocking portion of the autofocus aperture 230.

[0047] The objective lens 202 is configured to have a Figure 2A 202 is a plane that moves along optical paths 203 and 205, the z-axis being perpendicular to the xy plane, and the plate 212 extending along the xy plane. For the sake of clarity, the following description refers to the position of the objective lens as being on the z-axis, and the position or lateral position of the reference image as being on the xy plane. The reference image translates along a straight path on the image plane. The straight path is referred to as the y-axis. It should be understood that the use of the z-axis or y-axis to provide a spatial reference is not intended to be limiting. Any suitable coordinate system may be used. It should also be noted that exemplary embodiments may involve an objective lens 202 that travels in a non-vertical direction.

[0048] The plate 212 may include Figure 2A The sample well 214 shown may be positioned for imaging a sample that may be deposited in the sample well 214 in accordance with normal function and operation of the microscope system. Figure 2A In the exemplary system shown, the plate 212 has a first surface 212a and a second surface 212b, the second surface 212b being the bottom surface of the sample well. The first surface 212a and / or the second surface 212b may be at least partially reflective, and thereby provide a reflective reference surface for use during a self-calibration autofocus procedure. The reference surface may also be provided on a cover glass, or on the surface of a slide, or on other planar material provided in the optical path near the bottom surface of the sample well 214.

[0049] The objective lens 202 may be moved along the z-axis using a linear actuator motor controlled by the controller 220. Figure 2A 2 is schematically represented as including a linear actuator motor that moves the objective lens 202. The objective lens 202 includes selected optics that are configured to focus light from a light source on a sample 216 when the microscope system 200 is controlled to image on the sample. During an autofocus procedure, the objective lens 202 is controlled to focus on a reference surface (212a or 212b on plate 212). In some embodiments, the motor that moves the objective lens 202 can be a stepper motor or a servo motor with a linear actuator.

[0050] Light along the optical path through the eccentric aperture 230 travels through the second imaging lens 240 to the image capture device 204 where the projection source is imaged on the detector plane. When defocused, the light beam on the image capture device 204 is expanded in size, has lower intensity and / or low contrast. When the objective lens 202 is in focus, a reference image is captured at maximum intensity, minimum size and its highest contrast. The process of focusing the objective lens 202 includes moving the objective lens 202 to find the best focus position on the z-axis. Each beam spot in each reference image captured at the z position of each objective lens 202 appears at a position in the image plane that is offset relative to the beam spot position on the previous image.

[0051] In an exemplary embodiment, the image capture device 204 in the autofocus system 200 can be a linear array detector, a charge coupled device, a position sensitive diode, a 2-D sensor array as an image capture device, or any suitable device that can be controlled by the controller 220 to capture a reference image when the objective lens 202 is controlled to move to a series of z positions. The light source 206 in the autofocus system 200 can be any suitable light emitting device, such as a laser, a light emitting diode (LED) or an LED array, an incandescent lamp, a fluorescent light source, an infrared light, etc.

[0052] The controller 220 can be implemented using any computer programmable system having a hardware interface connected to at least the image capture device 204 and a motor configured to move the objective lens 202. In some embodiments, the controller 220 can also be the component of the microscope system 221 that causes the objective lens to be automatically focused. The self-calibration autofocus routine can be a function stored as software in a data storage medium 223 accessible to the controller 220.

[0053] The self-calibration and autofocusing procedure for a given objective lens includes performing a first process to obtain a reference calibration slope. The reference calibration slope can then be used to subsequently autofocus the objective lens. Figure 2B The flowchart in 250 shows the operation of an exemplary method 250 for automatically focusing an objective lens by first obtaining a reference calibration slope at 260 and determining a best focus position of the objective lens 202 by using the reference calibration slope at 270. Then, an offset relative to the best focus position or other methods can be used to move the objective lens 202 to focus on the sample, such as performing hardware-based focusing on the sample similar to the hardware-based method described above with reference to FIG. 1A .

[0054] refer to Figure 2A (for the components referenced below) and Figure 2B , at step 252, when the objective 202 is inserted for use in the microscope system 200, a self-calibration and autofocus routine can be initiated. At step 254, the system can perform startup or setup functions for using the particular objective 202 that has been selected. One startup step can include determining at decision block 256 whether a reference calibration slope has been generated for the objective 202. If a reference calibration slope has not been generated (indicated by the "no" path to method 260), then at step 262, a starting location for the autofocus routine is determined.

[0055] When the objective lens 202 has been inserted or installed in the microscope system, the position of the objective lens 202 on the z-axis is generally unknown to the system. The system 200 can perform a home function of positioning the objective lens 202 at a known position based on a home structure that can be sensed by the objective lens 202 or a sensor attached to the objective lens structure when the objective lens 202 is moved. In other embodiments, the objective lens 202 can be positioned at its farthest position from the plate 212.

[0056] Once the objective lens 202 is in a known or home position, the controller 220 may position the objective lens 202 at a predetermined distance from the home position to begin an autofocus procedure at step 263. The predetermined distance may be a starting position stored in a data description of the objective lens 202. The starting position may be a parameter for all or most objective lenses.

[0057] At step 264, a reference image is captured at the current position of the objective lens 202. The objective lens 202 is then positioned at a series of z positions to scan the reference surface while moving the objective lens. The scan includes capturing a reference image at each z position. As shown in decision block 265, the z position is checked to determine whether the scan is complete. Figure 2B An example of a check used in decision block 265 is used to determine whether the scan is complete by checking whether a reference image has been captured for each z position in the scan. The scan can be performed using a set number of z positions, or until the travel limit on the objective lens is reached. If the scan is not complete, the objective lens 202 is moved to the next z position at step 266 (along the "no" decision path). At step 264, a reference image is captured at the next z position. If the scan is complete, the reference images captured at each z position are analyzed at step 268 (the "yes" path of decision block 265).

[0058] Figure 3A is a schematic illustration of an example of a plurality of images 300 showing lateral translation (along the y-axis) of beam spots 302a to 302o superimposed on a representation of a linear array detector 304 during calibration of the best focus position of an objective in a microscope system.

[0059] For the objective lens 202 ( Figure 2A ), Figure 3A The beam spots 302a to 302o in FIG. 3 are shown at 16 corresponding positions on the image plane of the linear array detector 304. It should be noted that Figure 3A 16 positions are shown to illustrate the process, but a single image plane is acquired at each z position at a time.While the illustrated embodiment shows 16 beam spots, it will be appreciated that any number of z positions and corresponding beam spots may similarly be used.

[0060] The beam spots 302 in each image are analyzed to determine the position of each beam spot. The determination of the position of the center of mass of the imaged object is well known and does not need to be described in further detail. The y-axis position of the center of mass of each beam spot 302 in each reference image is determined and the y-axis position is associated with the z position of the objective lens when each image was captured. Figure 3A The lateral translation of the beam spot 302 is shown when the reference images are superimposed on each other. The beam spot 302 is positioned along the y-axis, where the beam spot appears in each reference image in which the beam spot is captured.

[0061] Each beam spot 302a to 302o is Figure 3A 302a to 302e are shown as being unfocused at initial y-axis positions (e.g., y=1 to y=6). The focus level is determined by the measured properties of the beam spot 302. The beam spots 302a to 302e have, for example, a lower intensity at Figure 3A In the example of y=7, beam spot 302f is more focused, as determined by the smaller size and greater intensity of beam spot image 302f. The greater intensity is indicated by the darker shading of beam spot 302f. Beam spot 302h is Figure 3A The darkest (highest light intensity) and smallest beam spot.

[0062] As the objective lens 202 moves further along the series of z positions in the scan past y=9, the beam spots 302i to 302o become larger and have lighter shades, indicating that the reference image is out of focus. Figure 2B As shown at step 268, the best focus position can be determined by identifying the smallest and brightest beam spot and setting the z position of the captured image as the best focus position. Figure 3A In the example shown, the best focus position is the z position corresponding to the y position of the beam spot 302h. As described above, other optical properties besides intensity may also be used to determine the best focus position, and the above example should not be construed as limiting. In addition, each beam spot 302( Figure 3A The determination of the intensity of a beam spot can be based on the average intensity of all pixels forming the beam spot, or other measurements. In an exemplary embodiment, intensity or contrast can be used to determine the focus score of each beam spot.

[0063] At step 268, further analysis will be beam spot ( Figure 3A The y position of the reference calibration slope 302a to 302o) is compared with the z position to determine the linear relationship between the y position and the z position. Then at step 269, the linear relationship can be stored as a reference calibration slope.

[0064] Although plotting the data is not necessary to store the reference calibration slope, Figure 3BAn example of a plot of reference image position y versus objective lens position z of distance plate 212a is shown. For convenience, the y-axis and x-axis have been shifted relative to the best focus position at the origin. Figure 3B As shown, according to the reference Figure 3A The yz points identified by the described scan may lie on a curve C that may be approximated by a line L. The reference calibration slope of the objective 202 may be indicated as a linear expression of the line L (i.e., y=mz+B), which in the illustrated example is y=1.37z+8.814. For the objective 202, a reference image such as the beam spot 302 may be captured regardless of the position of the objective 202. The position of the beam spot 302 on the y-axis may be determined. The current distance to the best focus position may then be determined using y=1.37z+8.814 and solving for z from one or more reference images within the linear range of the system.

[0065] Preferably, the reference calibration slope is a non-zero slope m. If no eccentric aperture or imaging lens is used in the autofocus system, a reference calibration slope of m=0 will result. A flat horizontal line will not allow the best focus position to be determined, since the beam spot will be at the same position in each reference image regardless of the position of the objective lens. The reference calibration slope can be adjusted by selecting an eccentric aperture according to the aperture size and the position of the light-passing portion of the aperture. An imaging lens can also be added to the autofocus system before or after the eccentric aperture to adjust the magnitude of the reference calibration slope. It should also be noted that the alignment of the sensor with the trajectory of the reference image positions on the sensor can be easily set by the orientation of the eccentric aperture relative to the sensor.

[0066] Return to reference Figure 2B , at decision box 256, the objective lens inserted at step 252 may have been calibrated to the best focus position. If so, a reference calibration slope is found for the objective lens 202 (following the "yes" decision path), and method 270 is executed to obtain focus. The reference calibration slope of the objective lens 202 is retrieved at step 272, and the objective lens 202 is moved to the starting position at step 276. At step 278, an image is captured at the starting position. At step 280, the image is analyzed to determine the position of the beam spot centroid in the image on the y-axis. Then, at step 282, the y position of the image is used in the reference calibration slope to determine the current distance between the current position of the objective lens 202 and the best focus position.

[0067] The sample imaging component 221 of the microscope system ( Figure 2A202 to focus on the sample. The objective lens 202 may be moved to focus on the sample by moving the objective lens to the best focus position plus a predetermined offset along the z-axis. The predetermined offset may be determined based on the type of sample being imaged or the type of plate being used to hold the sample. In some embodiments, the offset may depend on the objective lens, the sample, or a longitudinal color shift. The objective lens may also be controlled to move to a predetermined offset by moving the target reference image away from the best focus and scaling the displacement by a calibration slope.

[0068] Some optical measurements may require a higher sensitivity to defocus. As mentioned above, hardware-based focusing as described above with reference to FIGS. 1A and 1B may be used, but from using Figure 2B Starting from the best focus position determined by method 250 in , the objective is focused on the sample. Combining the profile method described with reference to FIGS. 1A and 1B with the predictive properties of the eccentric hole and calibration slope will increase the speed of the method by defining a smaller search range. The smaller search range can be based on one or more measurements that predict the appropriate objective position while maximizing sensitivity to defocus.

[0069] Return to reference Figure 2B If a more precise distance to the best focus position is needed, the objective lens can be moved to several different z positions, and steps 278 to 282 can be repeated for each z position to which the objective lens 202 is moved. In this way, the linear relationship defining the reference calibration slope can be confirmed, and the best focus can be found based on the focus metrics around the best focus by fitting, interpolation, or other methods to improve the resolution of defocus. FIG. 4A to FIG. 4C Describe the process. FIG. 4A to FIG. 4C Shows the use of Figure 2B An example of the method 270 in FIG. 2 is a graphical representation of images 400 , 402 , 406 of a beam spot 420 captured using an objective lens at different z positions. FIG. 4A to FIG. 4C Each point 420 in is shown as having a centroid 421, which is a smaller spot generally at the center of the spot. Figure 4B The image 402 in FIG. 4 shows a beam spot 420 at or near the best focus position. Figure 4A The image 400 in FIG. 4 shows a beam spot 420 at a y position indicating that the objective lens 202 is defocused in a direction away from best focus. Figure 4C The image 406 shows the objective lens 202 in the Figure 4A The beam spot 420 in FIG. 4 is defocused in the y position in the opposite direction away from the best focus. For the purpose of illustration, FIG. 4A to FIG. 4CThe images of three beam spots are shown in FIG. The three beam spots may actually represent a scan of a region of interest (ROI) that may be smaller than the image sensor, centered at the calibrated best focus, and clipping the full dynamic range of measurable defocus of the objective lens. In an exemplary embodiment, more beam spot images may be captured and analyzed to improve resolution, especially for smaller ROIs.

[0070] Can be analyzed FIG. 4A to FIG. 4C The beam spot 420 in the image is used to determine the best focus position and calibration slope based on the scan of the three beam spots. The best focus position can be found by interpolating, fitting or picking the peak focus metric from the three beam spot scans and then comparing it with the original best focus position. The calibration slope found by analyzing the three beam spot scans can also be compared with the original reference calibration slope. The best focus position and the reference calibration slope should be within an acceptable error. If either of the two is not within an acceptable error, the scan can be repeated with more data points or for a larger ROI.

[0071] It should be noted that the reference image may be a beam spot or a patterned image.The light source 206 may be configured to generate white light or any other suitable wavelength of light as a simple beam or as a patterned image. Figure 4D is an example of a patterned image that can be projected onto the reference surface 212a or 212b and detected by the image capture device as a reference image. If a patterned image is used, the best focus can be determined by analyzing the reference image using a preferred data transform (such as a Fourier transform) and then calculating the correlation. Then, similar to the techniques described above for the beam spot, the optical properties of the patterned image corresponding to the best focus can be analyzed. The lateral position can be determined as the centroid or the brightest pixel, and the highest contrast can be the preferred image attribute for determining the best focus. Pattern recognition methods can also be used to analyze the patterned image, where the best pattern that matches the original pattern indicates the best focus.

[0072] The self-calibrating autofocus system can be implemented in various configurations. Figures 5 to 7 is an example of an optional construct. Figures 5 to 7 The construction shown describes Figure 2A Components of the autofocus system 200 in FIG. 1 , wherein any added components are labeled with different numbers. In addition, the controller 220 and the component 221 for sample imaging are not shown either.

[0073] Figure 5 is an example of an autofocus system 500 configured to perform self-calibrating autofocus in a microscope system (not shown), which shows another position for positioning an eccentric aperture 502. The eccentric aperture 502 is disposed between the first imaging lens 208 and the beam splitter 210, rather than after the beam splitter 210. Figure 5 The light path shown in FIG. 5 is only the portion of the total light that passes through the eccentric hole 502 .

[0074] Figure 6 is another example of an autofocus system 600 configured to perform self-calibrating autofocus. The light source 206 and the first imaging lens 208 can be configured to generate a light beam that fills the entrance pupil of the objective lens 202. Using a light beam that fills the entrance pupil of the objective lens 202 can reduce alignment errors and maximize sensitivity. Figure 6 The system 600 in FIG. 6 includes an eccentric aperture 602 inserted above a second imaging lens 604 to control the direction and sensitivity of the beam translation. In an exemplary embodiment, the eccentric aperture 602 is switchable (optically or mechanically in a manner known to those skilled in the art) to provide a means of increasing light throughput and the accuracy of finding the best focus position.

[0075] Figure 7 is another example of an autofocus system 700 configured to perform self-calibrating autofocus. Figure 7 The autofocus system 700 in the embodiment includes an off-axis imaging lens 702 and a beam magnifying lens 704 positioned above an eccentric aperture 706.

[0076] The off-axis imaging lens 702 is positioned off-axis so that only a portion of the light beam passes through the off-axis imaging lens 702. The light passing through the off-axis imaging lens 702 is directed to a beam magnifying lens 704. The light then passes to an off-center aperture 706, which further emphasizes the lateral position of the light beam.

[0077] It is obvious to those skilled in the art that any combination of hardware and / or software may be used to implement the autofocus system 200 described herein. Figure 7 One or more of the processes, sub-processes, and process steps described may be performed by hardware, software, or a combination of hardware and software on one or more electronic or digitally controlled devices. The software may reside in a software memory (not shown) in a suitable electronic processing component or system, such as the one or more electronic processing components or systems of FIGS. 2 to 3. Figure 7One or more of the functional systems, controllers, devices, components, modules or submodules schematically shown in the figure. The software memory may include a command list of executable instructions for implementing logical functions (i.e., "logic" may be implemented in a digital form such as a digital circuit or source code, or in an analog form such as an analog source (such as an analog electrical, sound or video signal)). The instructions may be executed within a processing module or controller (e.g., microscope controller 220 in Figure 2), which includes, for example, one or more microprocessors, general-purpose processors, combinations of processors, digital signal processors (DSPs), field programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs). In addition, the schematic diagrams describe a logical partitioning of functions with physical (hardware and / or software) implementations that are not limited by the architecture or physical layout of the functions. The exemplary systems described in the present application can be implemented in various configurations and operated as hardware / software components in a single hardware / software unit, or as separate hardware / software units.

[0078] Executable instructions can be implemented as a computer program product storing instructions, which, when executed by a processing module of an electronic system, direct the electronic system to execute the instructions. A computer program product can be selectively included in any non-transitory computer-readable storage medium for use by or in conjunction with an instruction execution system, device, or apparatus (such as an electronic computer-based system, a system containing a processor, or other systems that can selectively obtain instructions from and execute instructions on an instruction execution system, device, or apparatus). In the context of the present invention, a computer-readable storage medium is any non-transitory device that can store a program used by or in conjunction with an instruction execution system, device, or apparatus. A non-transitory computer-readable storage medium can be selectively, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus. A non-exhaustive list of more specific examples of non-transitory computer-readable media includes: an electrical connection having one or more conductors (electrical); a portable computer disk (magnetic); random access, i.e., volatile memory (electrical); read-only memory (electrical); erasable programmable read-only memory, such as, for example, flash memory (electrical); optical disk storage, such as CD-ROM, CD-R, CD-RW (optical); and digital versatile disk storage, i.e., DVD (optical).

[0079] It should also be understood that the reception and transmission of the signal as used in the present invention means that two or more systems, equipment, parts, modules or submodules can communicate with each other via the signal traveling on a certain type of signal path. These signals can be communication, electric power, data or energy signals, which can transmit information, electric power or energy from the first system, equipment, parts, modules or submodule to the second system, equipment, parts, modules or submodule along the signal path between the first and second systems, equipment, parts, modules or submodules. The signal path can include physical, electrical, magnetic, electromagnetic, electrochemical, optical, wired or wireless connection. The signal path can also be included in an additional system, equipment, parts, module or submodule between the first and second systems, equipment, parts, modules or submodules.

[0080] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference was individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0081] Unless otherwise specified herein or clearly contradictory to the context, the terms "one" and "an" and "the" and similar references used in the context of describing the present invention (especially in the context of the appended claims) are interpreted as covering the singular and plural. Unless otherwise specified herein, the recording of the numerical range herein is only intended to be used as a shorthand method for individually referring to each individual value falling within the range, and each individual value is incorporated into the specification as if it is individually recorded herein. Unless otherwise specified herein or clearly contradictory to the context, all methods described herein can be performed in any suitable order. Unless otherwise required, the use of any and all examples or exemplary languages ​​(e.g., "such as") provided herein is only intended to better illustrate the present disclosure, rather than to impose limitations on the scope of the present disclosure. The language in the specification should not be interpreted as indicating that any unclaimed element is necessary for the practice of the present disclosure.

[0082] In view of the foregoing description, many modifications to the present disclosure will be apparent to those skilled in the art.It should be understood that the illustrated embodiments are exemplary only and should not be considered to limit the scope of the present disclosure.

Claims

1. A method for automatically focusing an objective lens in a microscope, the method comprising: providing an eccentric aperture in the optical path between the objective lens and the image capture device; generating a reference calibration slope for the objective lens by determining lateral positions of a plurality of images captured by the image capture device at a series of z positions of the objective lens, at least one image of the plurality of images having a specific property corresponding to a best focus position, wherein the optical path formed during the capture of the plurality of images is at least partially blocked by the off-center aperture to provide wavefront sampling at the series of z positions; as well as Automatic focusing of the objective lens is performed based at least in part on the reference calibration slope and the best focus position.

2. The method according to claim 1, wherein: The plurality of images include a beam spot in each image formed by projecting light reflected from a reference surface onto the image capturing device.

3. The method according to claim 1, wherein: The plurality of images includes a pattern in each image formed by projecting patterned light reflected from a reference surface onto an image capture device.

4. The method according to claim 3, wherein: The specific property corresponding to the best focus position is a minimum pattern image size or an optimal contrast.

5. The method according to claim 3, wherein: The specific attribute corresponding to the best focus position is the highest pattern correlation with the original pattern using pattern recognition techniques.

6. The method according to claim 1, wherein: The particular property corresponding to the best focus position is highest contrast.

7. The method according to claim 1, wherein: The specific property corresponding to the best focus position is a minimum beam spot size.

8. The method according to claim 1, wherein: The specific property corresponding to the best focus position is maximum irradiance.

9. The method according to claim 1, wherein: Determining the positions of the plurality of images includes determining a centroid position of each image along the lateral translation axis.

10. The method according to claim 9, wherein: Generating the reference calibration slope includes determining a linear relationship between the centroid position and a corresponding z position of the image.

11. The method according to claim 9, further comprising: positioning a sample aperture in the optical path of the objective lens; and identifying the distance to move the objective lens to the best focus position of the sample hole by the following steps: Capture a reference image, identifying a lateral position on an image plane of the reference image, and The distance to the best focus position is calculated based on the lateral position of the reference image and the reference calibration slope.

12. The method according to claim 1, further comprising: Positioning the objective lens at the optimal focusing position; determining an offset from the best focus position to a sample focus position; as well as The objective lens is positioned at the best focus position plus the offset.

13. The method according to claim 1, further comprising: moving the objective lens to the best focusing position; capturing an image of the sample at each of a plurality of z positions in a direction toward the sample; determining a focus score for each sample image at each corresponding z-position based on any one or a combination of focus metrics including spot diameter, intensity, average intensity, maximum signal measurement, standard deviation, variance, and covariance; plotting the focus score for each sample image at each z position; identifying a highest peak in the focus score; as well as The objective lens is positioned at the z position corresponding to the highest peak.

14. A system for automatically focusing an objective lens in a microscope system, the system comprising: light source; an image capture device configured to capture images of reflections of light from a reference surface while the objective lens is moved along the z-axis through a series of z positions; an off-center aperture disposed in an optical path from the objective to the image capture device, the off-center aperture configured to block at least a portion of the light from the light source to provide wavefront sampling at the series of z positions; A controller configured to control the light source, the image capture device, and the objective lens, the controller configured to perform a self-calibration autofocus process by: generating a reference calibration slope for the objective lens by determining a lateral position of a plurality of images taken at a series of z positions of the objective lens, at least one image in the plurality of images having a particular property corresponding to a best focus position; as well as Automatic focusing of the objective lens is performed based at least in part on the reference calibration slope and the best focus position.

15. The system of claim 14, wherein: The light source is configured to project a beam spot onto the image capture device.

16. The system of claim 14, wherein: The light source is configured to project a pattern image onto the image capture device.

17. The system of claim 16, wherein: The specific property corresponding to best focus is minimum pattern image size or best contrast.

18. The system of claim 16, wherein: The specific attribute corresponding to the best focus position is the highest pattern correlation with the original pattern using pattern recognition techniques.

19. The system of claim 14, wherein: The particular property corresponding to the best focus position is highest contrast.

20. The system of claim 14, wherein: The specific property corresponding to the best focus position is a minimum beam spot size.

21. The system of claim 14, wherein: The specific property corresponding to the best focus position is maximum irradiance.

Images