Systems and methods for microscope optical alignment

Abstract

Methods and systems for microscope systems are provided. In one example, a method for image-based detection of the optical alignment of a microscope system includes: acquiring multiple contrast samples of the field of view of interest; determining the contrast distribution of multiple divided subregions of the field of view based on the multiple contrast samples; and generating an angular shift amount based on the focus position derived from the contrast distribution for each of the multiple divided subregions. In one example, the method further includes adjusting the microscope system based on the angular shift amount.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims priority to U.S. Patent Application No. 18 / 295,206, entitled "SYSTEM AND METHOD FOR MICROSCOPE OPTICAL ALIGNMENT", filed on April 3, 2023. The entire disclosure of the application listed above is incorporated herein by reference for all purposes.

[0002] This specification generally relates to methods and systems for microscope imaging, and more particularly to optical alignment in a microscope.

[0003] Background / Overview In digital microscopy, for whole - slide imaging and high - content analysis, acquisition of high - resolution details of an object is often required, along with wide - field or high - throughput imaging capabilities. Microscope systems configured for high - speed imaging often include features for automatic focusing of an object. Automatic focusing can be based on indirect or direct measurements of imaging parameters. Indirect methods for automatic focusing measure physical quantities other than image quality and rely on a high - precision electromechanical stage with a position encoder, or often utilize specific hardware configurations and components such as decoding optical signals via optical triangulation routines, imaging laser beam convergence, multi - angle adaptation, or low - coherence interferometry. Direct methods for automatic focusing often involve measurement of image contrast and / or image high - frequency components derived from the acquired image content.

[0004] The inventors of this application recognized several potential challenges relating to such methods. For example, in existing microscope systems, direct and indirect methods for automatic focusing generally derive a single evaluation value from either the entire object or a portion of the field of view. Often, the best focus position, where contrast is sharp and the object is most in focus, may be a compromised focus position, or it may produce an image focused only at the center of the image. For example, such methods may not be able to position a high-quality focus across the entire field of view due to possible inclinations between the sample plane and the focal plane. In some cases, the inclination between the sample plane and the focal plane can cause a small focal length mismatch of a few microns; however, the degradation of image quality can be significant, usually at the edges or corners of the field of view. This degradation is particularly important with large numerical aperture objective lenses where the field depth is less than 1 micron. While such image quality degradation can be addressed during the manufacturing process, downstream processes can lead to interference with optical alignment. For example, shipping and handling, improper sample loading, and the selection of accessories such as sample holder substrates regularly raise challenges in maintaining image quality. Effective methodologies for evaluating and further mitigating such image degradation may be desirable to achieve high-performance microscopy.

[0005] For example, the above problem can be addressed by a method for image-based detection of the optical alignment of a microscope system, which includes acquiring multiple contrast samples of the field of view of the subject, determining the contrast distribution of multiple divided subregions of the field of view based on the multiple contrast samples, and generating an angular shift amount based on the focus position derived from the contrast distribution for each of the multiple divided subregions. In this way, the imaging quality across the entire field of view of the subject may be increased, and the imaging throughput may be increased.

[0006] It should be understood that the above summary is provided in a simplified form to introduce a selection of concepts that will be further explained in the detailed description. It is not intended to identify any important or essential features of the claimed subject matter, whose scope is uniquely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to any implementation that solves any defects described above or in any part of this disclosure. [Brief explanation of the drawing]

[0007] [Figure 1] A schematic diagram of a quantitative microscope assembly is shown. [Figure 2] A perspective view of a multi-detector quantitative microscope system is shown. [Figure 3] This shows an exemplary contrast distribution curve across the focused Z coordinate, acquired by the controller of a quantitative microscope assembly. [Figure 4] An example of an object divided into sub-regions is shown. [Figure 5] This shows the first example of a set of contrasts in the distribution curves of multiple divided subregions representing the field of view of the first object. [Figure 6] A second example of a set of contrast distribution curves for multiple divided subregions representing the field of view of the second object is shown. [Figure 7] Figure 6 shows an exemplary tilt map representing the angular displacement between the sample plane and the focal plane for the second target. [Figure 8] This is a schematic diagram illustrating a technique for reducing the angular misalignment between the sample plane and the focal plane via a multi-detector quantitative microscope system. [Figure 9] This schematic diagram illustrates an additional technique for reducing the angular misalignment between the target sample plane and the focal plane via a multi-detector quantitative microscope system. [Figure 10] This flowchart illustrates a method for detecting the angular shift between the sample plane and the focal plane using a multi-detector quantitative microscope system. [Figure 11]This flowchart illustrates a method for reducing the angular misalignment between the sample plane and the focal plane using a multi-detector quantitative microscope system. [Modes for carrying out the invention]

[0008] The following description relates to a system and method for detecting angular misalignment between the sample plane and the focal plane in a high-throughput microscope system. Possible correction systems are also provided to increase focus quality and image quality across the entire field of view. An exemplary microscope imaging system is described with reference to Figures 1 and 2. The microscope imaging system may be a multi-detector quantitative microscope system. An example of a quantitative microscope assembly is depicted as a schematic diagram in Figure 1. The assembly shown in Figure 1 may be included in each of the four blades of a multi-detector quantitative microscope system, as shown in Figure 2. The object for imaging may be placed on the system's stage, and the distance between the object and the system's objective lens may be adjusted to control the focus of the object, but angular misalignment between the sample plane and the focal plane can degrade image quality. This disclosure describes a method for detecting angular misalignment using image-based techniques, including measuring the contrast or Brenner gradient of the sample image in the field of view. Exemplary contrast distributions along various focal lengths are described with reference to Figure 3. By dividing the entire field of view into several sub-regions, the contrast distribution of each sub-region along the Z-axis, including the focal position, can be derived. Figure 4 shows the field of view of an exemplary object divided into sub-regions. Figures 5 and 6 show, respectively, examples of sets of contrast distributions obtained for a field of view divided into sub-regions. Angular shift can be visualized in a 2D array representing the focal position derived in each sub-region of the field of view, as described with respect to Figure 7. Angular shift can be mitigated by mechanical adjustment of the optical alignment of the multi-detector quantitative microscope system based on the derived focal position, as described with respect to Figures 8 and 9. A flowchart illustrating method 1000 for detecting angular shift based on the contrast distribution of sampled images is described with respect to Figure 10. A flowchart illustrating method 1100 for mitigating angular shift is described with respect to Figure 11.

[0009] Referring here to Figure 1, a schematic diagram of a quantitative microscope assembly 100 (hereinafter, assembly 100) is shown. In one example, assembly 100 may be configured as a fluorescence microscope assembly. However, various other types of analytical microscopy imaging techniques are possible, including but not limited to transmission mode microscopy, wide-field microscopy, confocal microscopy, light-sheet microscopy, emission, and colorimetric methods. Assembly 100 in Figure 1 includes a light source 102 that provides incident light to components arranged in the path of the incident light, as indicated by arrow 104. The light source 102 may be a mercury lamp, a xenon arc lamp, a laser, or one or more light-emitting diodes (LEDs). In one example, assembly 100 may be included in a multi-detector microscope system, as will be described later with respect to Figure 2.

[0010] Incident light can be directed to a filter cube 106 (also called a filter block, for example). The filter cube 106 may house components that filter the incident light so that a target wavelength is transmitted to a target to be analyzed, for example, one or more samples supported on a sample holder 108. In one example, the sample holder 108 may be a microplate. In the example in Figure 1, three filtering components, including an excitation filter 110, a dichroic filter 112, and an emission filter 114, are arranged within the filter cube 106. Incident light may first pass through the excitation filter 110, which filters the light so that a selected wavelength, for example, a target wavelength, continues to pass through the excitation filter 110, while blocking light of other wavelengths. The target wavelength may be a wavelength that excites electrons in a particular fluorophore or fluorescent dye, resulting in the emission of photons when the excited electrons relax to the ground state.

[0011] Excitation light, for example, light filtered by the excitation filter 110, then strikes a dichroic filter 112 (or dichroic beam splitter), as indicated by arrow 116. The dichroic filter 112 may be a mirror, for example, arranged at a 45-degree angle to the optical path of assembly 100, or angled at a 45-degree angle to the path of incident light, as indicated by arrow 104. The surface of the dichroic filter 112 may include a coating that reflects the excitation light, for example, light filtered by the excitation filter 110, but allows fluorescence emitted from the sample in the sample holder 108 to pass through it. The reflected excitation light passes through the objective lens 118, as indicated by arrow 116, and illuminates the sample holder 108. If the sample positioned in the sample holder 108 emits fluorescence, light is emitted, generating emitted light, for example, as indicated by arrow 120, which is collected by the objective lens 118. The emitted light passes through a dichroic filter 112 and continues to an emission filter 114, which prevents undesirable excitation wavelengths from passing through. The filtered emitted light is received by a detector 122. In one example, the detector 122 may be a camera, such as a charge-coupled device (CCD) camera. In other examples, the detector 122 may be another type of camera, such as a CMOS camera or a photomultiplier tube.

[0012] The detector 122 can convert emitted light into electronic data. For example, if the detector 122 is a CMOS camera, the detector 122 may include a photosensor configured as a transistor on an integrated circuit. The photons of the emitted light may be incident on the photosensor and generate charges that are converted into electronic data representing the photon pattern of the emitted light captured within the camera's field of view (FOV). The electronic data may be stored in the camera's memory, such as random-access memory, and can be retrieved by the controller 124.

[0013] The controller 124 may be a computer including various components such as a processor, input / output ports, an electronic storage medium for executable programs and calibration values, random access memory, and a data bus. The electronic storage medium can be programmed with computer-readable data representing instructions executable by the processor to perform the methods described below, as well as other expected but not specifically listed variations. The controller 124 may be coupled with various accessory devices, including input devices such as a keyboard and a mouse.

[0014] The controller 124 may be communicatively coupled to components of the assembly 100. For example, the controller 124 may be configured to instruct the activation / deactivation of the light source 102 when prompted based on user input. In another example, the controller 124 may command the adjustment of the position of the sample holder 108 and focus the excitation light onto different areas of the sample holder. The controller 124 may instruct the operation of a motor 126 coupled to the sample holder 108 to vary the position of the sample holder 108 relative to the objective lens 118 and the excitation light, and may provide instructions on how the sample holder position should be corrected. In some examples, a position sensor 128 may monitor the actual position of the sample holder 108 and may be communicatively coupled to the controller 124 to relay the sample holder position to the controller 124.

[0015] The controller 124 can also be communicatively coupled to the detector 122. Thus, the electronic data collected by the detector 122 can be retrieved by the controller 124 for further processing and display on an interface such as a computer monitor. It will be understood that the controller 124 can be further coupled to other sensors and actuators of the assembly 100. In one example, communication between the controller 124 and the sensors and actuators of the assembly 100 can be enabled by various electronic cables, such as hardwiring. In another example, the controller 124 can communicate with the sensors and actuators via wireless protocols such as Wi-Fi, Bluetooth, or Long-Term Evolution (LTE).

[0016] According to the routines and configurations described herein, the quantitative microscope assembly 100 includes image-based angular misalignment detection for increasing focus across the entire FOV. The controller 124 performs image-based angular misalignment detection by acquiring multiple contrast samples of the target FOV. The contrast samples are acquired by the detector 122 at various focal lengths across the focus position of the target (e.g., positioned within the sample holder 108) relative to the objective lens 118. Based on the multiple contrast samples, the controller 124 determines the contrast distribution of multiple divided subregions of the FOV. The amount of angular misalignment is generated by the controller 124 based on the focus position derived from the contrast distribution for each of the multiple divided subregions. In one example, adjustments may be made to the optical alignment of the quantitative microscope assembly 100 in response to the detection of angular misalignment exceeding a threshold. These adjustments may minimize variations in image quality across the entire FOV. Image-based tilt detection and correction methods and systems are discussed in detail below.

[0017] Assembly 100 may further include an autofocus system 130 communicatively coupled to a controller 124. The autofocus system 130 may utilize a sensor configured with a light source and optical elements to correct a light beam from a light source and direct it towards a sample holder 108. Images may be generated based on the reflection of the light beam and used by the controller 124 to determine suitable adjustments for the objective lens and / or the sample holder 108, thereby aligning the focus of the objective lens with the target interface of the sample holder 108. In one example, the autofocus system 130 may rapidly focus the assembly 100 on a desired area of ​​the sample, depending on the laser beam and autofocus algorithm implemented in the controller 124. In one example, the autofocus system 130 may work in conjunction with the controller 124 to provide an initial focus position for the object's FOV during image-based tilt detection. For example, the autofocus system 130 may find the focal length that produces the sharpest image for acquiring multiple contrast samples.

[0018] It will be understood that the assembly 100 depicted in Figure 1 is a non-limiting example of a quantitative microscope assembly. Other examples may include variations in the quantities of individual components, such as the number of dichroic, excitation, and emission filters, the configuration of the light source, and the relative positioning of the components. In one example, a quantitative microscope assembly, such as the assembly 100 in Figure 1, could be used for high-throughput screening of biological samples.

[0019] In one example, as shown in FIG. 2, the multi-detector system 200 can be formed from four individual blades 202 arranged in an x configuration. Each blade 202 of the multi-detector system 200 can include at least the components depicted in the assembly 100 of FIG. 1, and the blades 202 can operate simultaneously to collect image data in parallel. Thus, each blade 202 forms an individual quantitative microscopy assembly. The components arranged on each of the blades 202 can be positioned to optimize both the FOV of the objective lens of each blade and the magnification / resolution of the resulting image. Thus, the components can be arranged in a vertical orientation, for example, as a stack along each blade. A set of reference axes 201 is provided, indicating the y-axis, x-axis, and z-axis. In one example, the z-axis can be parallel to the direction of gravity. Further, the central axis 204 of the multi-detector system 200 can be parallel to the z-axis.

[0020] In the view shown in FIG. 2, the side walls 205 of the housing 220 of the multi-detector system 200 are omitted to show two of the blades 202, and two of the blades 202 are surrounded by the housing 220. The housing 220 includes an upper plate 207 coupled to the upper ends of two pairs of side walls 205 oriented in opposite directions. The upper plate 207 can be a rigid plate having a square shape and can have dimensions similar to those of the base 222. The upper plate 207, the base, and the side walls 205 form a generally rectangular prism for surrounding the components of the multi-detector system 200.

[0021] In one example, each blade 202 of the multi-detector system 200 can be similarly configured to include a vertical plate 206 that supports various components, such as, but not limited to, the components described in the quantitative microscope assembly shown in FIG. 1. Each blade 202 can have a front face 210 and a back face 212. An objective lens 208, which can be an embodiment of the objective lens 118 of FIG. 1, can be disposed on top of each blade 202 along with other components of each blade 202 disposed below the objective lens 208 with respect to the z-axis. In one example, the objective lenses of the blades 202 are clustered around the central axis 204. The objective lens 208 can be attached to the plate 206 by an objective lens module 228. The objective lens module 228 can include an objective lens mover, which can be a motor that adjusts the position of the objective lens 208 along the z-axis, and a position sensor (not shown) that monitors the position of the objective lens along the z-axis. The objective lens mover can be controlled, for example, activated / deactivated, by a controller such as the controller 124 of FIG. 1.

[0022] In one example, the front face 210 of the plate 206 can include a first optical path 232 that extends vertically (e.g., along the z-axis) directly below the objective lens 208. The first optical path 232 can surround microscope components, such as, but not limited to, a tube lens, an emission filter, a selection mirror, etc., and can be an embodiment of the filter cube 106 of FIG. 1. A camera 238, which can be an embodiment of the detector 122 of FIG. 1, can be coupled to the plate 206 directly below the first optical path 232. A second optical path 246 can be coupled to the plate 206 adjacent to the first optical path 232. In one example, the first optical path 232 and the second optical path 246 are optically coupled. The blade 202 can also include a light source 248, which can be an embodiment of the light source 102 of FIG. 1, surrounded by a cover 249 fixed to the front face 210 of the plate 206.

[0023] In one example, the front surface 210 of the plate 206 of the blade 202 may support a laser autofocus sensor (LAF sensor) 242. In one example, the LAF sensor 242 may emit a laser beam into the optical path of the quantitative microscope assembly so that the LAF sensor 242 can use the laser beam as an optical probe to determine the focus of the quantitative microscope assembly. In one example, the LAF sensor 242 may be configured to generate a 785 nm red laser beam from a light source such as a laser diode. The laser beam may be reflected from the surface of the sample or the surface of the sample holder (e.g., a microplate) and returned to the LAF sensor 242 as an optical signal that can be used to evaluate the focus of the quantitative microscope assembly.

[0024] The laser autofocus controller (LAF controller) 260 may be positioned below, for example, the z-axis, the light source 248, and the LAF sensor 242. The LAF controller 260 may include various electronic components for signal processing, LAF sensor operation, monitoring of LAF sensor status, etc., and may be fitted to a connector 262 to allow connection of a cable to the LAF sensor 242. The cable may allow communication between the LAF sensor and the LAF controller 260. The LAF controller 260 may include multiple ports along the axis to allow connection of the LAF controller 260 to a system controller such as controller 124 in Figure 1, for example.

[0025] The connectivity between the LAF controller 260 and the system controller may allow the position of the objective lens 208 to be adjusted based on the alignment of the focus of the quantitative microscope assembly with the target focal plane, as detected by the LAF sensor 242. For example, the LAF controller 260 may include computer-readable instructions for executing an automatic focusing routine to adjust the position of the objective lens 208 with respect to an object placed in the well 221 of the microplate 215. In one example, the automatic focusing routine may include identifying the target focal plane of the FOV of each objective lens of the multi-detector system, adjusting the perpendicular position of each objective lens with respect to the target focal plane based on the shape of the light generated from the laser beam, and aligning the focus of each objective lens according to the power level detected by the LAF sensor. For example, if it is determined that the focus is not aligned with the target focal plane, the LAF controller 260 may notify the system controller of the amount of the focus offset from the target focal plane. In response to information from the LAF controller 260, the system controller may instruct the adjustment of the objective lens 208 by activating the objective lens mover of the objective lens module 228 and, accordingly, correcting the position of the objective lens 208 along the z-axis. The LAF controller 260 and the LAF sensor 242 may be referred to as the LAF system. In one example, the LAF system may be an embodiment of the autofocus system 130 in Figure 1.

[0026] The light source 248 may be powered and controlled by a printed circuit board assembly (PCBA). An example of a PCBA 203 coupled to the back surface 212 of blade 202 is depicted. The PCBA 203 may include various electronic components coupled to a printed circuit board (PCB) 209. For example, the LED of the light source 248 may be directly coupled to the PCB 209 via soldering. The various electronic components may further include diodes, capacitors, resistors, switches, inductors, and the like.

[0027] The multi-detector system 200 may include a sample receiving assembly 211 positioned on and attached to the upper plate 207. In one example, the sample receiving assembly 211 may include a plate holder 213 coupled to a stage 217. The plate holder 213 may include a portion that extends over and covers the stage 217, forming a bracket 219. The bracket 219 may be fixed to the stage 217 via fasteners, thereby fixing the stage 217 to the plate holder 213. The stage 217 is positioned laterally to the plate holder 213 (except for the bracket 219 on the plate holder 213) so that the stage 217 is positioned laterally to the microplate 215 when the microplate 215 is placed in the opening 223 of the plate holder 213. The stage 217, configured to adjust the position of the plate holder 213 along the upper plate 207, is coupled to the plate holder 213.

[0028] In one example, the objective lens 208 may extend upward from the blade 202 and into the space between the upper plate 207 and the plate holder 213, but without contact with the plate holder 213 or the microplate 215 supported by the plate holder. The plate holder 213 can be translated along the xy plane without any obstructions hindering its movement. The distance at which the plate holder 213 is perpendicularly separated from the objective lens 208 may be configured to allow the objective lens 208 to be positioned at a target distance from the microplate 215 located above and a target distance from the tube lens located below. By positioning the objective lens 208 at target distances from the microplate 215 and the tube lens, the maximum FOV and resolution can be obtained from the quantitative microscope assembly of the multi-detector system 200.

[0029] The stage 217 may be a biaxial stage, composed of bearings such as mechanical bearings and pneumatic bearings, to allow the plate holder 213 to translate along the x and y axes, respectively, relative to the objective lens 208. In some embodiments, the movement of the stage, and therefore the movement of the plate holder 213, may be controlled by a motor. In other embodiments, the relative position of the stage 217 may be adjusted manually. Adjusting the position of the stage 217 allows for correction of the FOV of the objective lens 208 relative to the microplate 215, enabling complete imaging of the microplate 215.

[0030] Each blade 202, clustered around the central axis 204, forms a region of space where the central fan 225 may be positioned. The multi-detector system 200 is shown to be mounted on a chassis 227. The chassis 227 may support peripheral components such as electronic devices coupled to the multi-detector system 200. The multi-detector system 200 may also include a vibration isolator 230 coupled to the base 222 of the housing 220. The vibration isolator 230 may be a support on which the housing 220 of the multi-detector system 200 is seated.

[0031] The multi-detector system 200 features image-based angular shift detection for increasing focus across the entire FOV. As is commonly applied, the autofocus strategy may derive a single evaluation value from the entire object or only from the center of the FOV. To increase focus across the entire FOV, the system controller (e.g., controller 124 in Figure 1) includes computer-readable instructions that, when executed, cause the controller to perform a series of actions to detect the degree of inclination between the focal plane of the objective lens 208 and the sample plane of the object positioned in the well 221 of the microplate 215. The actions include acquiring multiple contrast samples of the object, determining the contrast distribution of multiple divided subregions of the field of view based on the contrast samples, and generating an angular shift amount based on the focus position derived from the contrast distribution for each of the multiple divided subregions.

[0032] In one example, multiple contrast samples may be acquired by a camera 238 coupled to the objective lens 208. Multiple contrast samples may be acquired at various focal lengths over the focal position of the central portion of the object's FOV. In one example, the focal position for the central portion of the FOV may be determined using an LAF system that includes an LAF sensor 242, an LAF controller 260, and an automatic focusing algorithm for adjusting the position of the objective lens 208 relative to the object based on the reflection of the light beam programmed thereon. In other examples, an autofocus system may use an alternative method for determining the focal position. The controller may include further commands to mitigate detected angular deviations (e.g., tilt errors) using various strategies that may be selected, for example, based on operating conditions and / or user preferences. Additionally or alternatively, in response to the detection of angular deviations in the tilt information, the user may also manually adjust the microscope assembly. In one example, angular deviation detection may be performed for each detector coupled to each objective lens of multiple objective lenses in a multi-detector system. For example, angular displacement detection may be performed on a microscope assembly including a camera 238 coupled to an objective lens 208 at each of the four individual blades 202 of a multi-detector system 200. In one example, angular displacement detection may be performed after the installation of the multi-detector system or other microscope system. In another example, angular displacement detection may be performed after changing the sample plate. Image-based detection of angular displacement and tilt mitigation strategies increases the focus across the entire imaged sample, enabling faster, more rigorous high-throughput image analysis.

[0033] Referring to Figure 3, plot 300 illustrates a Brenner gradient distribution curve over the focus Z=Z0 coordinate containing a contrast sample acquired by the controller of the microscope assembly (similar to or identical to controller 124 of quantitative microscope assembly 100 described above). The Brenner gradient distribution can be considered an image contrast distribution or contrast distribution and is used to find the focal length (or location) of maximum contrast based on a measurement of the contrast of the sample image. The Brenner gradient distribution is a reliable image-based focusing technique with minimal computational cost. However, commonly applied techniques often derive a single merit figure (such as image contrast or Brenner gradient value) from the entire region or only the central region of the image. Thus, the best focal position where contrast is sharp and the imaged object is clear may only be a compromised focus position, or it may only produce an image focused on the center of the image. The disclosed strategy uses the Brenner gradient distribution to detect the focal length of maximum contrast in each of several divided sub-regions of the object FOV. The contrast sample can be obtained by the controller by adjusting the focal length between the object and the objective lens of the microscope assembly (e.g., between the object in the sample holder 108 in Figure 1 and the objective lens 118, or between the object in the microplate 215 in Figure 2 and the objective lens 208). For example, the distance along the z-axis between the objective lens and the object can be adjusted by adjusting the vertical position of the objective lens, for example, by an automatic or manual mechanism as described above (e.g., via a command to the objective lens module 228 in Figure 2). In the plot, the x-axis represents the object distance Z, where the object distance is the distance between the object and the objective lens, and the y-axis represents the image contrast as the focus increases upward.

[0034] The Brenner gradient is calculated using the pixel intensity of the acquired image and the formula

number

[0035] Various object distances, i.e., B at Z around the focal position where Z=Z0. z The value of is the Lorentz function, i.e.

number

[0036] If the object is not at the actual peak-focused position and Z≠Z0, the calculated Brenner gradient lies on the corresponding Lorentz curve at the off-peak position. For example, the first marker 302, the second marker 304, and the third marker 306 are each located at the off-peak position. The goal of the automatic focusing process is to move the object toward the peak-focused position, which is represented by peak 310 of the Lorentz curve.

[0037] One way to reach the focal direction is to change the object distance Z at fixed intervals toward the desired direction, and the Brenner gradient increases, as indicated by arrow 308. More data points with smaller step sizes across the Brenner gradient distribution curve could increase the accuracy of the focal position, but this is time-consuming and prohibited due to the moving resolution across the mechanical stage. Since the distribution of the Brenner gradient at various object distances is a Lorentz function, it is reasonable to use a curve fitting routine based on several data points to estimate the peak, and therefore the focal position. In some examples, a Lorentz curve fitting routine can be used with only three contrast samples to determine the peak-focused position under conditions where the signal-to-noise ratio of the contrast samples is very high, but the initial position of the object is close to the actual peak-focused position, and the estimated half-width half-maximum of the Lorentz curve is approximately equal to the actual value. However, when the initial position of the object is not close to the actual peak-focused position, the Brenner gradient becomes smaller and its signal-to-noise (SNR) ratio decreases rapidly. Some curve fitting routines, such as those based on the Levenberg-Marquardt (LM) algorithm, utilize initial estimations to model the initial curve, including estimations of the curve's amplitude, peak position, and half-angle half-maximum value. The additional computations in the modeling work can take far less time than the image acquisition process itself, due to the slow mechanical movement.

[0038] Figure 4 shows an example of a subject 400 divided into subregions. The subject 400 has a 2000 × 2000 pixel FOV. In one example, the subject 400 may be divided into multiple subregions to estimate angular shift based on contrast distribution, as described with respect to Figure 3. In one example, the subject 400 may be divided into three columns and three rows for nine subregions. In this example, it is shown that the first column 402 and the first row 410 represent five subregions of the nine exemplary subregions. In one example, at least some portion of each subregion overlaps with one or more adjacent subregions. In another example, each subregion may not overlap with adjacent subregions. In some examples, each pixel representing the FOV, e.g., the entire field of view, is represented in at least one subregion.

[0039] In the example shown for object 400, the first column 402 of the subregion is highlighted, including the first subregion 404 (indicated by a dashed line), the second subregion 406 (indicated by a dashed line), and the third subregion 408 (indicated by a dotted line). The first row 410 of the subregion is highlighted, including the first subregion 404, the fourth subregion 412 (indicated by a dashed line), and the fifth subregion 412 (indicated by a dotted line). In the example, the first subregion 404, the second subregion 406, the third subregion 408, the fourth subregion 412, and the fifth subregion 414 are 1000 × 1000 pixel area regions of object 400.

[0040] For example, the number of subregions into which the field of view is divided may depend on factors including the balance between signal and noise and statistical significance. For instance, if the subregions are too small (e.g., too few pixels) or too numerous, noise in the Brenner value calculation may increase. Noisy signals can increase the difficulty in identifying the focal location of the peak or highest contrast in the Brenner value. Too few or too large subregions may provide insufficient spatial data for detecting angular shifts. For example, the subregion size range may include a lower threshold of 500 x 500 pixels and an upper threshold of 3000 x 3000 pixels. Generally, larger and / or fewer subregions are more statistically stable, but detecting peak Brenner values ​​may be more difficult, and tilt sensitivity may be reduced. Smaller or more subregions increase sensitivity to detecting tilt, but increased signal noise may be a trade-off.

[0041] In one example, depending on the size of the FOV, subregions may overlap with adjacent subregions. For example, subject 400 has a 2000 × 2000 pixel area FOV and a subregion size of 1000 × 1000 pixels. To achieve a desired subregion size, subregions overlap with one or more adjacent subregions. For example, the first subregion 404 overlaps with the second subregion 406 by approximately 500 × 1000 pixels. The first subregion 404 also overlaps with the fourth subregion 412 by approximately 500 × 1000 pixels. The second subregion 406 overlaps with the fourth subregion 412 by approximately 500 × 500 pixels. In one example, this overlap smooths the transition of Brenner peak positions from one region to another, generating higher quality slope information. However, overlapping subregions are arbitrary and can be avoided in some examples. For example, with larger target FOVs and similar subregion sizes such as a 4000×4000 pixel area field of view and a 1000×1000 pixel area subregion, there may be no overlap. As discussed above, balancing signal-to-noise and statistical significance may take precedence over subregion overlap.

[0042] Figures 5 and 6 show the first set 500 and the second set 600 of contrast distribution curves representing the entire FOV of the subject, respectively. As described above with respect to Figures 3 and 4, in order to obtain the first set 500 and the second set 600 of contrast distribution curves, the controller (e.g., controller 124 in Figure 1) can calculate the focal position of the highest contrast for each of the multiple sub-regions of the FOV. For example, the controller can acquire multiple contrast samples at different focal lengths, e.g., 11 images, 21 images, etc., and derive the focal length of the highest image contrast based on a curve fitting algorithm for modeling the images and contrast distribution. Using the resulting set of contrast distribution curves of the subject, the optical alignment of the microscope assembly, i.e., the angular deviation of the degree of inclination between the sample plane and the focal plane, can be estimated. In each curve, the measured image contrast is on the y-axis in arbitrary units, e.g., Brenner gradient values. The relative Z-coordinate, e.g., focal length, is on the x-axis in microns (μm). In this example, the relative Z coordinate on the x-axis is in the range of -10 μm to +10 μm.

[0043] Turning to the first set 500 shown in Figure 5, the subject is divided into nine sub-regions: left column 508, center column 510, right column 512, top column 514, middle column 516, and bottom column 518. For example, the first set 500 includes the first sub-region 502, the second sub-region 504, the third sub-region 506, and so on, from which the contrast distribution curve is acquired. As explained with respect to Figure 4, the size and number of sub-regions may depend on factors including the balance of noise intensity as an effect on the detection of the true Z0 position. For example, sub-regions with a pixel count of 1000 × 1000 may be sufficient for a typical sensor array of visible wavelengths. Multiple sub-regions with higher or lower pixel counts may be preferred for other sensor array configurations or other factors such as assay type.

[0044] In one example, the contrast distribution curve derived for object 500 includes a first curve 502a representing a first sub-region 502, a second curve 504a representing a second sub-region 504, a third curve 506a representing a third sub-region 506, and so on. The contrast distribution curves representing the object, for example, the first curve 502a, the second curve 504a, and the third curve 506a, are positioned relative to locations within the FOV. For example, the first sub-region 502 may represent the upper left region of the FOV, the second sub-region 504 may represent the upper central region, and the third sub-region 506 may represent the lower left region of the FOV. In one example, the first curve 502a, the second curve 504a, and the third curve 506a may represent the contrast distribution curves of the first sub-region 404, the fourth sub-region 412, and the third sub-region 408 of the object 400 illustrated with respect to Figure 4, respectively.

[0045] If the contrast distribution curves peak at the same relative Z0 value, it can be understood that the entire FOV has the best image quality at the same focal position. As can be seen in the first set 500, the peaks occur at the same focal position, which is the relative Z coordinate Z0=0. For example, for each of the nine sub-regions, Z0=0 is present at the first peak 502b of the first curve 502a, the second peak 504b of the second curve 504a, and the third peak 506b of the third curve 506a, and so on. Thus, the sample plane can be understood to be parallel to the focal plane of the optical system. Under such conditions, it is possible to produce a well-focused image across the entire field of view without adjusting the optical alignment. For example, not only is the central region of the image well-focused, but the periphery is also sharp. However, under conditions where the contrast distribution curves peak at Z0≠0 in some regions, the amount of angular misalignment between the sample plane and the focal plane can be identified, and the optical alignment can be corrected to increase the focus across the entire field of view.

[0046] As described above with respect to Figure 4, and referring to the second set 600 shown in Figure 6, the object is divided into sub-regions. The contrast distribution curve is obtained for each sub-region based on the contrast measurement in the sample image acquired over the range of focal positions described above with respect to Figures 2 to 4. In one example, the object may be object 400, and the second set 600 of contrast distribution curves may correspond to the nine sub-regions described with respect to Figure 4.

[0047] The second set 600 includes the left column 620, the center column 622, the right column 624, the top column 626, the middle column 628, and the bottom column 630 of the contrast distribution curves. In the example, the second set 600 includes the first curve 602a representing the first sub-region 602, the second curve 604a representing the second sub-region 604, the third curve 606a representing the third sub-region 606, the fourth curve 608a representing the fourth sub-region 608, the fifth curve 610a representing the fifth sub-region 610, the sixth curve 612a representing the sixth sub-region 612, the seventh curve 614a representing the seventh sub-region 614, the eighth curve 616a representing the eighth sub-region 616, and the ninth curve 618a representing the ninth sub-region 618. Contrast distribution curves representing the object, such as the first curve 602a, the second curve 604a, and the third curve 606a, are positioned relative to locations within the FOV. For example, the first sub-region 602 may be the same as or similar to the first sub-region 404 illustrated with respect to Figure 4.

[0048] The second set 600 illustrates a state in which the contrast distribution curve reaches a peak at Z0≠0 in several regions, and thus the amount of angular displacement between the sample plane and the focal plane can be identified. For example, the curves in the center column 622, e.g., the fourth curve 608a, the fifth curve 610a, and the sixth curve 612a, reach a peak at the same relative Z coordinate, where Z0=0μm. For example, at the fifth peak 610b of the fifth curve 610a, Z0=0. For example, the first curve 602a, the second curve 604a, and the third curve 606a, corresponding to the left column 620, reach a peak at the same relative Z coordinate, where Z0≠0. For example, at the first peak 602b of the first curve 602a, Z0=-1μm. The seventh curve 614a, the eighth curve 616a, and the ninth curve 618a, corresponding to the right column 624, reach their peaks at the same relative Z coordinate, where Z0 ≠ 0. For example, at the ninth peak 618b of the ninth curve 618a, Z0 = +1 μm. Using the center column 622 as the reference point, it can be inferred that the optical system renders the highest contrast on the left side of the field of view, approximately 2 μm ahead on the right.

[0049] A "Z-stack" of images can be acquired to obtain images for calculating contrast distribution curves as shown in Figures 5 and 6. In one example, the Z-stack may be a set of images taken at different focal positions across the field of view. The focal position may be determined using autofocus techniques. For example, the focal position may be determined using a laser autofocus system, as described with respect to Figures 1 and 2. In another example, a merit numerical method such as the Brenner gradient may be used, as described with respect to Figure 3. For each sub-region, the image from the Z-stack is analyzed using the Brenner gradient to measure the contrast in the acquired image. In one example, the Z-stack may be 11 images ranging from Z0+5μm to Z0-5μm, increasing by 1μm between each image. The absolute difference in focal position between subsequent images, also called the step size, may be reduced for higher gradient accuracy. The number of images in the Z-stack and the step size depend on the focusing stage and may be limited to the smallest step size that can be reliably achieved.

[0050] In another example, the Brenner gradient curve can be acquired in situ, and the focusing stage or objective lens module is continuously moved across the focus position while the image is being captured and the Brenner gradient is being calculated sequentially. By rapidly scanning back and forth, peak contrast position and gradient information for all sub-regions can be acquired in semi-real time. Additionally, it is possible to integrate curve fitting algorithms, such as the LM algorithm described with respect to Figure 3, to reduce the number of Brenner gradient values ​​that can be obtained to estimate the curve. Such integration can reduce the processing time for gradient estimation and mitigation.

[0051] Figure 7 shows an example of a 2D intensity map 700. The 2D intensity map 700 illustrates the Z-coordinate (Z=Z0) of the best focus for each of several divided sub-regions relative to the central position. In one example, the 2D intensity map 700 can be derived from a second set 600 of contrast distribution curves illustrated with respect to Figure 6. The key 702 indicates an intensity scale where gray to white represents the range of +1 to 0 μm and white to black represents the range of 0 μm to -1 μm. The checked grid coordinates are shown on the X and Y axes. For the exemplary 3x3 grid, the x and y axis coordinates are 0, 1, and 2.

[0052] In this example, the left intensity region 704 may correspond to the left side of the FOV of the imaged sample, the central intensity region 706 may correspond to the central portion of the FOV, and the right intensity region 708 may correspond to the right side of the FOV. For example, the left intensity region 704 may correspond to the first column 402 of the object 400 illustrated in Figure 4 and represent the contrast distribution curve derived therefrom. Together, the left intensity region 704, the central intensity region 706, and the right intensity region 708 provide a visual representation or summary of the amount of angular shift similarly conveyed by a second set 600 of contrast distribution curves illustrated with respect to Figure 6. The 2D intensity map 700 shows a +1 μm slope to the right, which illustrates that the optical system renders the highest contrast on the left side of the image approximately 1 μm closer to the center of the image and 1 μm closer to the right side of the image.

[0053] Figures 8-9 illustrate optical alignment techniques that may be performed by the controller of a microscope assembly following detection of angular misalignment between the sample plane and the focal plane (similar to, or identical to, the controller 124 of quantitative microscope assembly 100). In one example, the amount of angular misalignment may be detected according to the systems and methods described above with respect to Figures 1-7, and the methods described below with reference to Figure 10. In the examples shown, based on the amount of angular misalignment detected by the controller, the controller may adjust the optical alignment of the microscope in various ways based on operating conditions and / or user preference. A set of reference axes 201 is provided, showing the y-axis, x-axis, and z-axis.

[0054] Once tilt information is obtained, several techniques can be used to adjust the mechanisms of the microscope system to minimize variations in image quality across the entire field of view (FOV). For example, adjustments may include mechanically adjusting the tilt angle of one or a combination of the sample stage, the optical axis including the imaging sensor, or the imaging sensor itself. A first technique for correcting angular misalignment is to adjust the leveling of the sample holder. For example, leveling can be achieved by adjusting the screws using a kinematic mount with fine-pitch adjustment screws. In one example, batch-based Z-stack data, e.g., a set of images taken over a range of focal positions including the focus position or the highest contrast position, can be used to provide feedback between adjustments in an iterative manner. In another example, on-the-fly data, e.g., a rapid scan to obtain peak contrast positions for all sub-regions, and semi-real-time tilt information can be used to provide feedback between adjustments. The kinematic mount can be adjusted manually or by an electric motor or actuator controlled by feedback from the tilt information. However, in certain cases, it may be desirable to adjust the optical axis of the imaging system instead of the sample plane. For example, the user may want to avoid perturbing the sample. In such cases, the entire optical system may be seated on an adjustable leveling mount so that the optical axis, and therefore the focal plane, can be adjusted.

[0055] Figure 8 illustrates schematic diagram 800, which shows a second method for reducing angular misalignment. Schematic diagram 800 includes the sample plane 802 and optical axis 804 of the optical system 806, as well as two possible adjustment angles and corresponding adjustment planes of the optical system 806. In one example, the sample plane 802 may be an object positioned within a sample plate (e.g., an object in well 221 of microplate 215 shown in Figure 2). In one example, the optical system 806 may be the entire optical array of a microscope assembly included in a multi-detector system (e.g., objective lens 208, first optical passage 232, second optical passage 246, camera 238, etc., shown in Figure 2). Furthermore, the optical axis 804 of the optical system may be parallel to the z-axis. The two possible adjustment angles include the first adjustment angle 808 and the corresponding adjustment plane 812, and the second adjustment angle 810 and the corresponding adjustment plane 814. The first adjustment angle 808 and the second adjustment angle 810 may be orthogonal angles. The optical axis 804 may be perpendicular to the sample plane 802 when no tilt is present. Otherwise, the tilt error may be mitigated by adjusting the leveling mount of the optical system 806 based on one or both of the two adjustment angles corresponding to the two adjustment planes.

[0056] The target FOV 818 represents a reduced set of sub-regions located in the north 820, south 822, east 824, west 826, and central 828 portions of the FOV. Angular deviations can be mitigated by following the tilt information provided by these five sub-regions. For example, using the 2D intensity map 700 described with respect to Figure 7, the amount of tilt indicated for the western 826 portion of FOV 818 is +1 μm. The amount of tilt indicated for the north 820, central 828, and south 822 portions of FOV 818 is 0 μm. The amount of tilt indicated for the east 824 portion of FOV 818 is -1 μm. The detected tilt illustrated in Figure 7 can be mitigated by adjusting the corresponding angle by the indicated amount. For example, the first adjustment angle of the leveling component of the optical system 806 may decrease by +1 μm and increase by -1 μm from west to east.

[0057] In some cases, due to the size, weight, and sensitivity of the optical system, mechanically tilting the entire optical axis may be undesirable or not very effective. Therefore, it may be beneficial to adjust the tilt angle of an imaging sensor, such as a camera (e.g., camera 238 in Figure 2), using the Scheimpflug principle. The Scheimpflug principle describes optical imaging conditions that enable imaging of obliquely tilted objects with the maximum possible depth of field and minimal image distortion. In one example, the imaging sensor has a sensing plane that can be tilted. Preferably, the sensing plane is parallel to the sample. However, the tilt between the object and the objective lens can be compensated for by introducing an angle to the camera rather than adjusting the entire optical array. Figure 9 shows a schematic diagram illustrating such a technique for mitigating angular misalignment, e.g., a third technique.

[0058] Schematic Figure 900 includes the sample plane 902 and optical axis 904 of the optical system 906. In one example, the sample plane 902 may be an object positioned within a sample plate (e.g., an object in well 221 of microplate 215 shown in Figure 2). In one example, the optical system 906 may be the entire optical array of a microscope assembly included in a multi-detector system (e.g., the objective lens 208, first optical passage 232, second optical passage 246, camera 238, etc., shown in Figure 2). The camera 908 included in the optical system 906 may be attached to the optical system via a tilt adapter. By controlling the tilt adapter, two orthogonal tilt angles, e.g., a first adjustment angle 910 and a second adjustment angle 912, can be introduced between the optical axis 904 and the sensor plane of the camera 908. The sensor plane may be the active sensing surface of the camera. In one example, the first adjustment angle 910 may correspond to the first adjustment plane 914, and the second adjustment angle 912 may correspond to the second adjustment plane 916 of the FOV 918.

[0059] Similarly, as described above, angular deviations can be mitigated by following tilt information corresponding to a reduced set of sub-regions located in the northern 920, southern 922, eastern 924, western 926, and central 928 portions of the FOV 918. For example, using the 2D intensity map 700 described with respect to Figure 7, the detected tilt can be mitigated by adjusting the corresponding angle by the indicated amount. For example, the first adjustment angle 910 of the leveling adapter of camera 908 can be adjusted to compensate for a tilt of +1 μm to -1 μm from the west to the east of the FOV. The dimensions of the camera are usually much smaller and more robust than the entire optical system, and therefore much easier to operate without interfering with the optical system. In this case as well, such adjustments can be achieved either manually, for example by the user making the adjustment, or by electrical means, for example via an actuator, as described above.

[0060] However, tilting the image sensor may not be very effective in terms of the amount of adjustment needed to achieve the same correction effect, due to the typical magnification values ​​of the microscope system. For example, a tilt angle of about 10 times the sensor plane would be required to achieve the same amount of correction on the sample plane at 10 optical magnification. In addition, a small amount of geometric distortion may be introduced into the resulting image. Fortunately, such perspective distortion can be easily removed with minimal impact on the final image quality, and the tilt amounts common in microscopy applications are small. Perspective distortion can be removed by following common practices in camera calibration procedures. Additional computational costs are a potential penalty of this method.

[0061] The sensor-side advantages of correction include increased image focusing / capture / correction speed, as much smaller and lighter objects are being manipulated. When tilt correction on the sensor plane is implemented via electric actuators and automatic correction algorithms, focus optimization can be implemented during the sample loading step in an on-the-fly and less intrusive manner, which can increase image focus and improve image quality, even in the presence of possible user interference and a higher tolerance for the flatness of the sample holding substrate.

[0062] Furthermore, for multi-detector microscope systems with multiple image trains targeting a common sample, it is advantageous to individually adjust each image train to achieve simultaneous focus optimization. For example, angle detection and mitigation strategies can be implemented independently for each microscope assembly of the multi-detector system 200. Another advantage is that the smaller form factor of the sensor-side mitigation techniques alleviates the physical size constraints of such systems with multiple image trains.

[0063] Regardless of the mitigation scheme chosen, correction procedures may be performed at various frequencies depending on the needs. For example, it may be desirable to implement a mitigation scheme after the shipment and installation of the microscope system. Additionally or alternatively, angular misalignment detection and mitigation may be introduced during routine maintenance procedures. In most demanding applications, such correction procedures may be performed each time a new sample plate is loaded or before a large batch of image acquisition is to be performed.

[0064] Figures 10 and 11 illustrate flowcharts illustrating methods 1000 and 1100, respectively, for detecting and mitigating angular misalignment, i.e., the degree of inclination between the sample plane and the focal plane, in a microscope system. The microscope system may be similar to or identical to the quantitative microscope assembly 100 described above with reference to Figure 1. In another example, the microscope system may be similar to or identical to the multi-detector system 200 described above with reference to Figure 2. Instructions for performing methods 1000 and the remainder of the methods contained herein may be executed by a controller (e.g., controller 124 shown in Figure 1 and described above) based on computer-readable instructions stored in the controller's memory and in conjunction with signals received from system sensors, such as the sensors described above with reference to Figure 1. The controller may use system actuators to adjust system operation according to the methods described herein. As an example, the controller may adjust the position of the object imaged by the microscope (e.g., the object in the microplate 215 shown in Figure 2, described above) relative to the microscope's objective lens (e.g., the objective lens 208 shown in Figure 2, described above) by increasing or decreasing the focal length between the object and the objective lens. Increasing or decreasing the focal length between the object and the objective lens may include adjusting the position of the objective lens relative to the object (e.g., by controlling the objective lens module 228 shown in Figure 2, described above, in the same way as the objective lens). Additionally or alternatively, increasing or decreasing the focal length between the object and the objective lens may include adjusting the position of the stage supporting the object relative to the objective lens. Adjusting the focal length may include sending an electronic signal to an actuator configured to control the focal length, such as a motor (e.g., a motor component to the objective lens module 228). As described herein, both the position of the object and the focal length refer to the distance between the object and the objective lens.

[0065] Referring to Figure 10, method 1000 may include determining the operating conditions of the microscope in 1002. In one example, determining the operating conditions may include determining the initial positions of the object and the objective lens (e.g., determining the initial distance between the object and the objective lens), determining the on / off conditions and / or intensity of the microscope's light source, and receiving service history or usage history stored in a log, etc. In one example, for a multi-detector microscope system, the service history may include angular displacement detection and / or correction history for one or more microscope assemblies. In another example, usage history may be provided indicating whether the microscope assembly is the first, last, or other microscope assembly in a multi-detector system where tilt detection is desired. In one example, determining the operating conditions of the microscope may include receiving the focused position of the field of view of the object obtained by executing an auto-focusing routine. For example, the auto-focusing routine may be stored in the controller's memory. In one example, the auto-focusing routine may rely on an LAF sensor assembly containing a laser beam to quickly focus the microscope assembly to a desired area of ​​the sample, as described with respect to Figure 2. In another example, an autofocus routine might use a direct measurement method, such as the Brenner gradient, to measure the contrast of the acquired image. The Brenner gradient distribution curve can be calculated to derive the focal length of the image with the highest contrast, based on a curve fitting algorithm to model a set of sampled images and contrast distributions.

[0066] In 1004, method 1000 may include acquiring multiple image samples at various focal lengths across the focal position of the FOV. For example, based on the focal position acquired by the autofocus routine described above, the method may include acquiring a Z-stack. In one example, the Z-stack may form a stack containing 11 images from Z0+5μm to Z0-5μm with a 1μm step between images. Acquiring multiple image samples may include imaging the object via the image sensor of the microscope assembly (for example, via the detector 122 as described above with reference to Figure 1, or via the detector of the microscope such as the camera 238 as described above with reference to Figure 2).

[0067] In 1006, method 1000 may include estimating the focus position for subregions of the FOV using a Brenner gradient distribution. This may include dividing the FOV into multiple subregions, as illustrated with reference to Figure 4, and modeling the contrast distribution of the image sample for each subregion, as illustrated with reference to Figures 3, 5-7.

[0068] In 1006a, method 1000 may include dividing the FOV into multiple sub-regions, which may include determining a desired size (e.g., pixel area) and number of sub-regions. In one example, the size and number of sub-regions into which the FOV is divided may depend on factors including the signal-to-noise balance and statistical significance. Generally, larger and / or fewer sub-regions are more statistically stable, but detection of peak Brenner values ​​may be more difficult, and tilt sensitivity may be reduced. Smaller and / or more sub-regions may increase sensitivity, but increased signal noise may be a trade-off. In one example, the pixel area range of the sub-regions may include a lower threshold of 500 × 500 pixels and an upper threshold of 3000 × 3000 pixels. In one example, sub-regions may or may not overlap with adjacent sub-regions.

[0069] In 1006b, method 1000 may include modeling the contrast distribution of sample images within each sub-region to obtain the focal position with the best image quality, in other words, the focal position with the highest contrast. The contrast distribution can be obtained by calculating the Brenner gradient of each sample image (e.g., of the Z-stack) as described above, where the Brenner gradient is

number

[0070] In 1008, method 1000 may include evaluating whether a subregion has the highest image contrast at the same focal position. In one example, the user may obtain a set of Brenner gradient distribution curves representing each subregion of the FOV, as shown with respect to Figures 5-6. In another example, the Brenner peaks and corresponding subregions may be provided in a 2D array representing the FOV. In one example, the focal position may be the relative Z coordinate for a sample having the same focal position with the best image quality (e.g., Brenner peak), where Z0=0. In one example, all subregions may be understood to have the highest image contrast at the same position on the basis that their absolute focus position is less than a threshold absolute focus position. For example, the absolute focus position may be defined by the absolute difference between the relative Z coordinate at Z0=0 and the focus position of the subregion. The threshold absolute focus position may be a calibrated, non-zero angular displacement. If not all subregions have the highest image contrast at the same position, the method proceeds to 1012. If all subregions have the highest image contrast at the same position, the method proceeds to 1010.

[0071] In 1010, method 1000 may include determining that the sample plane is parallel to the focal plane of the optical system. It is possible to generate a well-focused image across the entire FOV. In one example, the method may include acquiring one or more images of the object.

[0072] In 1012, method 1000 may include visualizing tilt or angular displacement information, as described with respect to Figure 7. For example, as described above, the focal position of the highest image contrast and the corresponding sub-regions representing each sub-region of the FOV may be stored in a 2D array. The 2D array may be visualized as a 2D intensity map, similar to a topographic map. The resulting map shows the Z-coordinate of the best focus for each sub-region relative to the central sub-region of the divided FOV (e.g., the fifth sub-region 610 or the central column 622 in Figure 6), where Z=Z0. For example, the Z0 of the central sub-region may be Z0=315μm (e.g., read from a mechanical stage encoder), the left column may have Z0=316μm, and the right column may have Z0=314μm. The resulting 2D intensity map may include relativized Z readings or offsets such that the Z-coordinate of the best focus of the central sub-region has Z0=0μm. Therefore, following the example, the left column for the central subregion may have Z0 = 1 μm, and the right column may have Z0 = -1 μm. In one example, the 2D intensity map may then reveal a simple inclined plane that can be used to increase the focus across the entire field of view.

[0073] In 1014, method 1000 may include mechanically correcting the tilt. Various techniques may be used to mechanically correct the tilt. For example, the method may include continuing to a sub-method for mitigating the detected angular deviation, as described in method 1100 below and with reference to Figures 8-9 above. For example, the sample stage may be adjusted to correct the tilt. In another example, the optical axis of an optical system including a detector or camera may be tilted. In yet another example, the detector itself may be tilted to compensate for the tilt.

[0074] Referring to Figure 11, method 1100 may include receiving tilt information in 1102. In one example, tilt information may be obtained by performing angular shift detection, referring to method 1000 shown in Figure 10, as described above. For example, tilt information may include a 2D array of peak contrast positions of divided subregions of the target FOV.

[0075] In 1104, method 1100 may include determining whether leveling of the sample holder is indicated. For example, using a sample holder that includes a kinematic mount with fine-pitch adjustment screws, leveling may be achieved by adjusting the screws. Such a method may be preferred for simplification. If leveling of the sample holder is indicated, the method may proceed to 1106. If leveling of the sample holder is not indicated, the method may proceed to 1112. In some cases, it may be desirable to adjust the focal plane of the imaging system instead of the sample plane. For example, the user may want to avoid perturbing the sample. In another example, the sample holder may not be configured for leveling or simple adjustment.

[0076] In 1106, method 1100 may include adjusting a screw of a kinematic mount based on tilt information. For example, a 2 μm tilt from north to south can be mitigated by adjusting the screw so that the sample holder plate can tilt accordingly to compensate. For example, the sample holder plate may be seated on the tip of the adjustment screw. The sample holder plate is tilted when the screw is raised or lowered. In one example, the kinematic mount may be adjusted via an electric motor or actuator controlled by feedback of the tilt information.

[0077] In 1108, method 1100 may include receiving tilt information. For example, Z-stack data may be received in batches and provide iterative feedback between adjustments. In one example, method 1100 may revert to method 1000. For example, the method may include deriving a second set of contrast distribution curves for a second set of contrast samples acquired after tilt correction. In another example, on-the-fly data, e.g., a rapid scan to acquire peak contrast positions for all sub-regions, and semi-real-time tilt information can be used to provide feedback between adjustments.

[0078] In 1110, method 1100 may include determining whether all subregions have the highest image contrast at the same focal position. In one example, all subregions may be understood to have the highest image contrast at the same focal position on the basis that their absolute focus position is less than a threshold absolute focus position. For example, the absolute focus position may be defined by the absolute difference between the relative Z coordinate at Z0=0 and the focus position of the central subregion. The threshold absolute focus position may be a calibrated, non-zero angular displacement. If all subregions do not have the highest image contrast at the same position, the method may return to 1104. If all subregions have the highest image contrast at the same position, the method may terminate.

[0079] In 1112, method 1100 may include determining whether it is indicated that the entire optical axis should be adjusted. For example, the entire optical system may be seated on an adjustable leveling mount so that the optical axis, and therefore the focal plane, can be adjusted. If it is indicated that the entire optical axis should be adjusted, the method may proceed to 1114. If it is not indicated that the entire optical axis should be adjusted, the method may proceed to 1116. For example, due to the size, weight, and sensitivity of the optical system, it may not be desirable or very effective to mechanically tilt the entire optical axis. Therefore, it may be beneficial to adjust only the tilt angle of an imaging sensor or detector, such as a camera (e.g., camera 238 in Figure 2, detector 122 in Figure 1).

[0080] In 1114, method 1100 may include adjusting a leveling mount that controls the focal plane of an optical system based on received tilt information using two orthogonal adjustment angles. For example, the entire optical system may be adjusted by adjusting a first adjustment angle corresponding to a first adjustment plane and a second adjustment angle corresponding to a second adjustment plane of the optical system, as described above with reference to Figure 8. Together, the two orthogonal adjustment angles represent north-south tilt control and east-west tilt control that describe possible tilt conditions. In one example, using the 2D intensity map 700 described with reference to Figure 7, the amount of tilt shown for the western part of the FOV may be +1 μm. The amount of tilt shown for the northern, central, and southern parts of the FOV may be 0 μm. The amount of tilt shown for the eastern part of the FOV may be -1 μm. The detected tilt illustrated in the 2D intensity map 700 may be mitigated by adjusting the corresponding angle by the amount shown. For example, the first adjustment angle of the leveling component of the optical system may decrease by +1 μm and increase by -1 μm from west to east. In one example, the leveling mount may be adjusted via an electric motor or actuator controlled by feedback of tilt information.

[0081] In 1108, method 1100 may include receiving tilt information as described above. For example, Z-stack data may be received in batches to provide feedback between adjustments, and the feedback may be provided on the fly to obtain peak contrast positions and semi-real-time tilt information for all sub-regions.

[0082] In 1110, method 1100 may include determining whether all subregions have the highest image contrast at the same focal position, as described above. For example, the method may determine whether the absolute focal position is smaller than the threshold absolute focal position for all subregions of the FOV. If all subregions do not have the highest image contrast at the same position, the method may return to 1104. If all subregions have the highest image contrast at the same position, the method terminates.

[0083] In 1116, method 1100 may determine that adjustment of the imaging sensor is indicated. In one example, the imaging sensor is mounted to the optical system via a leveling adapter that can be tilted to compensate for the inclination between the sample plane and the focal plane.

[0084] In 1118, method 1100 may include adjusting the sensing plane of the imaging sensor via a leveling adapter based on received tilt information using two orthogonal adjustment angles. By adjusting the tilt angle of the leveling adapter, two orthogonal tilt angles can be introduced between the optical axis and the sensing plane of the imaging sensor. The two orthogonal adjustment angles include a first adjustment angle corresponding to a first adjustment plane and a second adjustment angle corresponding to a second adjustment plane of the optical system, as described above with reference to Figure 9. Together, the two orthogonal adjustment angles represent north-south and east-west tilts that describe possible tilt conditions. In one example, using the 2D intensity map 700 described with reference to Figure 7, the amount of tilt shown for the western part of the FOV may be +1 μm. The amount of tilt shown for the northern, central, and southern parts of the FOV may be 0 μm. The amount of tilt shown for the eastern part of the FOV may be -1 μm. The detected tilt illustrated in the 2D intensity map 700 can be mitigated by adjusting the corresponding angle by the indicated amount. For example, the first adjustment angle of the leveling component of an optical system may decrease by +1 μm and increase by -1 μm from west to east. In one example, the leveling adapter may be adjusted via an electric motor or actuator controlled by feedback of tilt information.

[0085] In 1108, method 1100 may include receiving tilt information as described above. For example, Z-stack data may be received in batches to provide feedback between adjustments, and the feedback may be provided on the fly to obtain peak contrast positions and semi-real-time tilt information for all sub-regions.

[0086] In 1110, method 1100 may include determining whether all subregions have the highest image contrast at the same focal position, as described above. For example, the method may determine whether the absolute focal position is smaller than the threshold absolute focal position for all subregions of the FOV. If all subregions do not have the highest image contrast at the same position, the method may return to 1104. If all subregions have the highest image contrast at the same position, the method may terminate.

[0087] In an example where the microscope assemblies described above are included in a multi-detector system, methods 1000 and 1100 may be repeated for each microscope assembly in the system until angular misalignment is detected and mitigated for all microscope assemblies. For example, angular misalignment detection and mitigation may be performed iteratively for each microscope assembly in the system until a non-zero threshold angular misalignment is exceeded. In another example, method 1000 may be performed for all microscope assemblies in a multi-detector system, and then method 1100 may be performed for any microscope assembly that does not exceed the threshold angular misalignment, including iterative correction and tilt detection.

[0088] In this way, by configuring and controlling the microscope assembly as described above, the microscope assembly can increase focus across the entire field of view to capture high-resolution detail and wide-field capabilities more quickly and accurately. By detecting and mitigating the angular misalignment between the focal plane and the sample plane, tilt-related image quality degradation can be reduced, the efficiency of autofocus can be increased, and this can increase productivity. In some examples, the disclosed method can be adapted to semi-real-time tilt estimation to model the contrast distribution with fewer sample images and increase efficiency. In some examples, the disclosed method can be performed independently for each optical axis of a multi-detector microscope system for even higher throughput. Furthermore, the disclosed strategy can be performed at varying frequencies depending on user preference. For example, for some use cases, angular misalignment detection can be performed after the microscope is shipped and installed. In other use cases, angular misalignment detection can be performed more frequently, as part of a regular maintenance strategy or between sample plates, especially for demanding applications.

[0089] This disclosure also provides support for a method for image-based detection of the optical alignment of a microscope system, which includes acquiring multiple contrast samples of a field of view of interest, determining the contrast distribution of multiple divided subregions of the field of view based on the multiple contrast samples, and generating an angular shift amount based on the focus position derived from the contrast distribution for each of the multiple divided subregions. In a first example of the method, the method further includes adjusting the mechanism of the microscope system based on the angular shift amount to minimize variations in image quality across the entire field of view. In a second example of the method, which optionally includes the first example, the adjustment includes mechanically adjusting the tilt angle of one or a combination of the sample stage, the optical axis including the imaging sensor, or the imaging sensor itself. In a third example of the method, which optionally includes one or both of the first and second examples, the multiple contrast samples include multiple image samples acquired at various focal lengths across the focus position of the central portion of the field of view. In a fourth example of the method, optionally including one or more of the first to third examples, the focus position of the field of view is determined using an autofocus system, the autofocus system includes an autofocus controller having computer-readable instructions for executing an automatic focusing routine to adjust the position of the objective lens relative to the object. In a fifth example of the method, optionally including one or more of the first to fourth examples, the object is positioned within a microplate mounted on a stage, the stage is positioned above a plurality of objective lenses, and the detectors are coupled to each objective lens of the plurality of objective lenses. In a sixth example of the method, optionally including one or more of the first to fifth examples, the method further includes generating an angular displacement for each detector coupled to each objective lens of the plurality of objective lenses. In a seventh example of the method, optionally including one or more of the first to sixth examples, the determination includes estimating the focus position using a curve-fitting algorithm having a corresponding Lorentz function.In an optional eighth example of the method, which includes one or more of the first through seventh examples, the method further includes determining that the entire field of view is in focus in response to the fact that the focus position of each sub-region is less than a threshold absolute focus position.

[0090] This disclosure also provides support for a multi-detector microscope system comprising a set of microscope assemblies arranged in an x ​​configuration, each set of microscope assemblies comprising an objective lens, a stage configured to support an object to be imaged, and a controller having computer-readable instructions stored in non-temporary memory, which, when executed, causes the controller to acquire multiple contrast samples of an object for each set of microscope assemblies, to determine the contrast distribution of multiple divided subregions of the field of view based on the multiple contrast samples, and to generate an angular shift amount based on the focus position derived from the contrast distribution for each of the multiple divided subregions. In a first example of the system, the object is positioned in a well of a microplate placed on a stage, and the stage is positioned above the set of microscope assemblies. In a second example of the system, optionally including the first example, the multiple contrast samples are acquired by a camera coupled to the objective lens. In a third example of the system, optionally including one or both of the first and second examples, the multiple contrast samples are acquired at various focal lengths across the focus position of the central portion of the field of view. A fourth example of the system, optionally including one or more of the first to third examples, includes an autofocus controller having computer-readable instructions for the autofocus system to execute an automatic focusing routine to adjust the position of the objective lens relative to the object, wherein the central focus position of the field of view is determined using an autofocus system. A fifth example of the system, optionally including one or more of the first to fourth examples, further includes computer-readable instructions for mechanically adjusting one of the stage, optical axis including camera, and camera, for each set of microscope assemblies, based on an amount of angular misalignment.

[0091] The Disclosure also provides support for a method for a multi-detector quantitative microscope system, wherein the method involves detecting the angular displacement between the focal plane of the objective lens and the sample plane of the object by acquiring the focus position for each of a plurality of divided sub-regions of the field of view of the object, based on the contrast distribution of a plurality of contrast samples acquired by the camera, wherein the object is placed in a well of a microplate and the microplate is positioned on a stage above the objective lens. In a first example of the method, the method further includes mechanically adjusting one of the stage, the optical axis including the camera, and the camera based on the angular displacement. In a second example of the method, optionally including the first example, a plurality of contrast samples are acquired at various focal lengths across the focus position in the central portion of the field of view. In a third example of the method, optionally including one or both of the first and second examples, the focus position for each of the plurality of divided sub-regions is calculated by scanning across the focus position in the central portion of the field of view while simultaneously modeling the contrast distribution of each sub-region, and the angular displacement is estimated in semi-real time. In a fourth example of the method, which optionally includes one or more of the first to third examples, the angular displacement is visualized in a 2D intensity map of the focus position of each sub-region relative to the central position.

[0092] It should be noted that the exemplary control and estimation routines contained herein can be used with a variety of system configurations. The control methods and routines disclosed herein may be stored as executable instructions in non-temporary memory and may be executed by a control system including a controller in combination with various sensors, actuators, and other system hardware. A particular routine described herein may represent one or more of any number of processing strategies, such as event-driven, interrupt-driven, multitasking, and multithreading. Accordingly, the various actions, operations, and / or functions illustrated may be in parallel or, in some cases, omitted in the illustrated sequence. Similarly, the order of processing is not necessarily required to achieve the features and benefits of the exemplary embodiments described herein, but is provided for the sake of illustration and explanation. One or more of the illustrated actions, operations, and / or functions may be executed repeatedly depending on the particular strategy used. Furthermore, the described actions, operations, and / or functions may be graphically represented by code programmed into non-temporary memory of a computer-readable storage medium within the control system, and the described actions may be executed by executing instructions in a system including various hardware components in combination with an electronic controller.

[0093] It will be understood that the configurations and routines disclosed herein are essentially illustrative and numerous variations are possible, and therefore these particular embodiments should not be considered restrictively. For example, the above techniques can be applied to other types of microscopes. Furthermore, unless expressly stated otherwise, terms such as “first,” “second,” and “third” are not intended to indicate any order, position, quantity, or importance, but rather are used solely as labels to distinguish one element from another. The subject matter of this disclosure includes all novel and non-trivial combinations and subcombinations of the various systems and configurations disclosed herein, as well as other features, functions, and / or properties.

[0094] Where used herein, the term “about” shall be interpreted as meaning plus or minus 5 percent of the range unless otherwise specified.

[0095] The following claims specifically point to certain combinations and subcombinations that are considered novel and non-obvious. These claims may refer to “one” element or “first” element or equivalent thereof. Such claims should be understood to include the incorporation of one or more such elements, and do not require or exclude two or more such elements. Other combinations and subcombinations of the disclosed features, functions, elements, and / or properties may be claimed through modifications to these claims or through the presentation of new claims in this application or related applications. Such claims shall also be considered to be included within the subject matter of this disclosure, whether broader, narrower, equal to, or different in scope from the original claims.

Claims

1. A method for image-based detection of optical alignment of a microscope system, Acquiring multiple contrast samples from the target field of view, Based on the plurality of contrast samples, the contrast distribution of a plurality of divided subregions of the field of view is determined. A method comprising generating an angular shift amount based on the focus position derived from the contrast distribution for each of the plurality of divided sub-regions.

2. The method according to claim 1, further comprising adjusting the mechanism of the microscope system based on the amount of angular displacement in order to minimize variations in image quality across the entire field of view.

3. The method according to claim 2, wherein the adjustment includes mechanically adjusting the tilt angle of one or a combination of the sample stage, the optical axis including the imaging sensor, or the imaging sensor itself.

4. The method according to claim 1, wherein the plurality of contrast samples include a plurality of image samples acquired at various focal lengths across the central portion of the field of view at the focus position.

5. The method according to claim 4, wherein the focus position of the field of view is determined using an autofocus system, and the autofocus system includes an autofocus controller having computer-readable instructions for executing an automatic focusing routine to adjust the position of the objective lens relative to the object.

6. The method according to claim 1, wherein the object is positioned within a microplate placed on a stage, the stage is positioned above a plurality of objective lenses, and a detector is coupled to each of the plurality of objective lenses.

7. The method according to claim 6, further comprising generating the angular displacement amount for each detector coupled to each of the plurality of objective lenses.

8. The method according to claim 1, wherein the determination includes estimating the focus position using a curve fitting algorithm having a corresponding Lorentz function.

9. The method according to claim 1, further comprising determining that the entire field of view is in focus in response to the fact that the focus position of each sub-region is less than the threshold absolute focus position.

10. A multi-detector microscope system, It comprises a set of microscope assemblies arranged in an x ​​configuration, and each of the set of microscope assemblies is An objective lens, and a stage configured to support the object to be imaged. A controller having computer-readable instructions stored in non-temporary memory, wherein when executed, the controller: For each of the sets of the aforementioned microscope assemblies, Multiple contrast samples of the aforementioned target are acquired, Based on the aforementioned multiple contrast samples, the contrast distribution of multiple divided subregions of the field of view is determined. A multi-detector microscope system including a controller that generates an angular shift amount based on the focus position derived from the contrast distribution for each of the plurality of divided sub-regions.

11. The multi-detector microscope system according to claim 10, wherein the object is positioned within the wells of a microplate placed on the stage, and the stage is positioned above the set of microscope assemblies.

12. The multi-detector microscope system according to claim 10, wherein the plurality of contrast samples are acquired by a camera coupled to the objective lens.

13. The multi-detector microscope system according to claim 10, wherein the plurality of contrast samples are acquired at various focal lengths across the focal position in the central portion of the field of view.

14. The multi-detector microscope system according to claim 13, wherein the focus position of the central portion of the field of view is determined using an autofocus system, and the autofocus system includes an autofocus controller having computer-readable instructions for executing an automatic focusing routine to adjust the position of the objective lens relative to the object.

15. The multi-detector microscope system according to claim 10, further comprising the computer-readable instruction mechanically adjusting one of the stage, the optical axis including the camera, and the camera, for each of the set of microscope assemblies based on the amount of angular displacement.

Images