Surface Sensing in Automated Sample Analysis
By projecting a two-dimensional light pattern onto the coverslip to determine its position relative to the object plane, the method simplifies and streamlines the localization process in optical microscopy systems, enhancing imaging accuracy and reducing complexity.
Patent Information
- Application Number
- JP2024534372
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-10
- Filing Date
- 2022-12-12
- Publication Date
- 2024-12-05
AI Technical Summary
Existing automated slide scanning systems face challenges in accurately determining the position of a coverslip relative to the object plane of an optical microscopy system, leading to complex and costly setups with imaging artifacts due to dust and debris, especially in fluorescence imaging modes.
A method and system that utilize a two-dimensional light pattern projected onto the coverslip surface to determine its position relative to the object plane, reusing the optical path for imaging and reducing complexity by performing localization and imaging sequentially, without real-time monitoring.
This approach allows for quick and accurate localization of the coverslip, reducing imaging time and artifacts, enabling high-quality sample imaging with fewer system components and lower complexity.
Smart Images

Figure 2024545149000001_ABST
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 288,515, filed December 10, 2021, the entire contents of which are incorporated herein by reference.
[0002] The present disclosure relates to optical microscopes, automated sample handling and imaging systems, and sensing of mounted samples. [Background technology]
[0003] Automated slide scanning systems are used to analyze mounted samples. In particular, such systems are used to digitize, archive, and share images of the samples. By capturing and storing digital images, the images can be retrieved and analyzed by a technician (or an automated analysis system) to grade or classify the sample or to identify pathologies specific to a particular sample. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Bray et al., J. Biomol. Screen 17(2): p. 266-274 (2012) Summary of the Invention [Means for solving the problem]
[0005] The present disclosure features methods and systems for detecting surfaces in optical microscopes, automated slide scanning systems featuring such microscopes, and other optical inspection systems. For example, in a slide scanning system, a sample (e.g., a tissue sample) is typically fixed to the surface of a microscope slide and a coverslip is applied over the mounted sample. Automated slide scanning systems attempt to identify the location of the sample on the surface of the slide. Determining the location of the coverslip surface relative to the object plane of the optical microscope system used to image the sample is an important first step in this process, as it helps reduce the complexity (i.e., dimensionality) of the search process.
[0006] The methods and systems disclosed herein are typically performed before an image of the sample is acquired. Thus, the methods and systems effectively "reuse" the optical path and imaging components to first determine the position of the coverslip and then acquire an image of the sample. Compared to other systems that include a second imaging system dedicated to "distance measurement", the systems disclosed herein are less complex and typically have fewer components. Furthermore, because the position and / or orientation of the coverslip is determined before imaging the sample, high quality images can be acquired without iterating during imaging to determine the position of the coverslip relative to the object plane of the imaging system.
[0007] To determine the relative position of the coverslip and the object plane of the imaging system, a two-dimensional pattern of light is projected onto the surface of the coverslip and the image of the light pattern reflected from the coverslip surface is analyzed to determine the position of the coverslip relative to the object plane of the imaging system. By acquiring images of the pattern corresponding to different distances between the pattern generating optics and the object plane of the imaging system and analyzing the images, the relative position of the coverslip and the object plane can be determined.
[0008] Measuring the position of the coverslip in this way can provide several important advantages. By precisely locating the surface of the coverslip, subsequent scanning operations to localize the tissue of interest can be performed more quickly and accurately, leading to reduced imaging / processing times. This can be particularly important for samples imaged in fluorescence mode, as locating the tissue can be time-consuming and prone to mixed imaging artifacts due to dust and other debris on the coverslip and slide surfaces. These methods can be performed at multiple locations on the coverslip surface to determine if the coverslip surface (i.e., surface height) is changing throughout that sample, and then, by applying appropriate corrections, the shape of the coverslip surface can be taken into account during subsequent slide scans to ensure localization of the tissue of interest. These methods can be performed quickly, and detection of the coverslip surface is independent of (nominally unaffected by) the nature of the sample. Thus, the process of locating the surface is independent of the tissue or other materials that form the sample and can be performed in a fully automated manner.
[0009] In one aspect, the disclosure features a method that includes generating a modulated light pattern using a light modulator; transmitting the modulated light pattern to a surface of a substrate via an objective lens, where the objective lens defines an object plane; acquiring a plurality of images of the modulated light pattern reflected from the surface of the substrate, where each modulated light pattern is acquired at a different relative distance between the objective lens and the substrate surface; analyzing the plurality of images to determine a reference position of the substrate surface relative to the objective lens that corresponds to a position of the image of the modulated light pattern formed by the objective lens at the object plane; and adjusting a position of the substrate relative to the objective lens based on the reference position.
[0010] Embodiments of the method can include any one or more of the following features.
[0011] The substrate may include a coverslip overlaid on the biological sample. The substrate may include a slide supporting the biological sample. The biological sample may include a tissue section.
[0012] Generating the modulated light pattern can include generating illumination light and transmitting the illumination light through a light modulator. The light modulator can be a passive modulator. The light modulator can be an active modulator. The passive modulator can include a reticle.
[0013] Each of the multiple images may be acquired by a detector disposed at an image plane conjugate to the object plane. The objective lens may be an infinite conjugate objective lens.
[0014] Analyzing the plurality of images may include identifying an image of the plurality of images in which the modulated light pattern is in best focus. Determining a reference position may include identifying as a reference position a position of the substrate surface relative to the objective lens at which the identified image was acquired.
[0015] Analyzing the plurality of images may include identifying two or more images from the plurality of images in which the modulated light pattern is at an optimal focus, and determining the reference position may include interpolating between positions of the substrate surface relative to the objective lens at which the identified images were acquired.
[0016] Adjusting the position of the substrate may include adjusting the position of the substrate such that a biological sample disposed below or above the substrate is located at the object plane.
[0017] The method can include acquiring one or more images of the biological sample with the biological sample located at an object plane.
[0018] Method embodiments may also include any of the other features described herein, including combinations of features that are individually described in connection with different embodiments, unless expressly stated otherwise.
[0019] In another aspect, the disclosure features a system that includes an illumination source, an objective lens, a light modulator, a detector, a stage, and a controller connected to the illumination source, the detector, and the stage, where the objective lens defines an object plane of the system, and the controller is configured to activate the illumination source to generate illumination light, direct the illumination light to the light modulator to generate a modulated light pattern, activate the stage to position a surface of a substrate on the stage at a plurality of different distances relative to the objective lens, activate the detector to acquire images of the modulated light pattern at the detector corresponding to each of the plurality of different distances, analyze the plurality of images to determine a reference position of the substrate surface relative to the objective lens that corresponds to a position of the image of the modulated light pattern formed by the objective lens at the object plane, and activate the stage to adjust a position of the substrate relative to the objective lens based on the reference position.
[0020] Embodiments of the system may include any one or more of the following features.
[0021] The substrate may include a coverslip overlaid on the biological sample. The substrate may include a slide supporting the biological sample. The biological sample may include a tissue section.
[0022] The optical modulator may be a passive modulator. The optical modulator may be an active modulator. The passive modulator may be a reticle.
[0023] The detector may be located in an image plane of the system. The image plane may be conjugate with the object plane. The objective lens may be an infinite conjugate objective lens.
[0024] The controller may be configured to analyze the images by identifying an image among the images in which the modulated light pattern is in optimal focus. The controller may be configured to determine a reference position as the position of the substrate surface relative to the objective lens at which the identified image was acquired.
[0025] The controller may be configured to analyze the images by identifying two or more of the images in which the modulated light pattern is at an optimal focus, and to determine a reference position by interpolating between the positions of the substrate surface relative to the objective lens at which the identified images were obtained.
[0026] The controller can be configured to activate the stage to adjust the position of the substrate such that a biological sample disposed below or above the substrate is located at the object plane. The controller can be configured to activate the detector to acquire one or more images of the biological sample with the biological sample located at the object plane.
[0027] Embodiments of the system may also include any of the other features described herein, including combinations of features described individually in connection with different embodiments, unless expressly stated otherwise.
[0028] As used herein, a "coverslip" is a member used to cover a sample (e.g., tissue and / or other biological material) that is attached or fixed to a microscope slide or other support. Coverslips can be made from a wide variety of materials, including, but not limited to, glass, plastic, polymer, quartz, and other transparent and translucent materials. In general, cover slips can be translucent or opaque at one or more wavelengths of light, and at least partially reflect incident light.
[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs.Methods and materials similar or equivalent to those described herein can be used in carrying out or testing the subject matter of this disclosure, and suitable methods and materials are described below.All publications, patent applications, patents and other documents mentioned herein are incorporated by reference in their entirety.In case of conflict, the present specification, including definitions, will take precedence.In addition, materials, methods and examples are illustrative only and are not intended to be limiting.
[0030] In general, the method steps described herein and in the claims may be performed in any order, unless expressly prohibited or logically contradictory. It should be noted that the description of steps in a particular order does not imply that such steps must be performed in the order described. Furthermore, the labeling of steps with an identifier does not impose an order on the steps or imply that the steps must be performed in a particular order. Conversely, steps disclosed herein generally may be performed in any order, unless otherwise noted.
[0031] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]
[0032] [Figure 1] FIG. 1 is a schematic diagram of an example of an optical microscope system. [Diagram 2] FIG. 2 is a schematic diagram of the optical microscope system of FIG. 1 with some additional elements substituted. [Diagram 3] 2 is a flow chart illustrating an exemplary set of steps for performing a calibration of the optical microscope system of FIG. 1. [Figure 4] FIG. 2 is a schematic diagram of an example controller. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0033] Like reference numbers in the various drawings indicate like elements.
[0034] introduction Automated optical microscopes and slide scanning systems typically perform analysis of slide-mounted samples by locating tissues of interest on the slide and acquiring and analyzing detailed tissue images to extract a variety of qualitative and quantitative information and enable classification of the tissue. Localizing a tissue of interest generally involves both locating the tissue and determining its position relative to the object plane of the microscope imaging system.
[0035] For slide-mounted samples imaged in bright-field imaging modes (e.g., absorption or transmission modes), the tissue of interest can be located relatively easily with a reasonable depth of focus, even if the tissue is not located in the object plane of the system. Once the tissue is located in the plane of the slide, the displacement of the tissue from the object plane of the system can be determined in a straightforward manner using methods such as interpolation between in-focus measurements at different "heights" (i.e., object plane positions) orthogonal to the nominal surface of the slide.
[0036] However, samples prepared and imaged in dark-field imaging modes, such as fluorescence, can be more difficult to localize the tissue of interest. In these imaging modes, the optical imaging signal is relatively weak, which can make it difficult to localize the tissue area. Thus, if the integration time is too short while imaging the sample, not enough light intensity is collected to accurately localize the tissue. Conversely, if the integration time is too long, the collected light intensity can saturate the detector.
[0037] Thus, localization of tissue of interest in dark field imaging mode is a complex five-dimensional problem that involves searching not only three spatial coordinates (e.g., x, y, z dimensions) but also the spectral coordinate (i.e., wavelength of illumination or detection) and the temporal coordinate. Along the spectral coordinate, the tissue of interest may not be uniformly stained with a dye or stain that provides a measurable signal at the selected wavelength of illumination or detection. Along the temporal coordinate, as mentioned above, underexposure of the sample will result in a weak or non-measurable signal, while overexposure may saturate the detector.
[0038] Additionally, contaminating effects due to imaging artifacts resulting from dust, excess fluorescent dyes, and other debris on the sample and / or other surfaces (e.g., slide and coverslip surfaces) are usually greater in dark-field imaging modes; dust in particular fluoresces strongly at some wavelengths, obscuring underlying tissues and making them difficult to localize. Dust is not necessarily coplanar with the sample and can be present on the coverslip and / or slide surfaces, so strategies that misidentify dust as sample may yield erroneous results.
[0039] Some variability also occurs due to the sample height and slide profile: Sample heights are typically irregular, and therefore the top surface of the sample (i.e., the surface closest to the objective lens) is not consistently positioned relative to the object plane from sample to sample.
[0040] Furthermore, the thickness of individual slides can vary: for example, an individual microscope slide may have a nominal thickness of about 1 mm, but in practice may be between 0.9 mm and 1.1 mm thick, and thus the thickness variation may be as large or larger than the thickness of the sample on the slide.
[0041] In addition, individual slides are not always perfectly aligned relative to the support structure (e.g., microscope stage) of the imaging system: slide-to-slide variation in the position of the slide relative to the support structure also results in variation in the position of the slide relative to the object plane of the system.
[0042] Typically, a sample (e.g., tissue or other biological sample) is mounted or fixed on the surface of a slide, and then a coverslip is applied over the mounted sample. The coverslip used over the sample in this manner is of relatively uniform thickness. Thus, the position of the sample relative to the object plane of the system can be determined by measuring the position of the coverslip relative to the object plane, more specifically, the position of the top surface of the coverslip (i.e., the surface opposite the surface that contacts the sample).
[0043] Therefore, for illustrative purposes, the following discussion focuses on methods and systems for determining the position of the top surface of a coverslip (simply referred to as the "surface of the coverslip") relative to the object surface of the system. However, it should be understood that the methods and systems disclosed herein can also be applied to determine the position of other surfaces relative to the object surface. For example, when a sample flows through or is placed in a channel of a microfluidic substrate (i.e., chip), the methods and systems disclosed herein can be used to determine the position of the surface of the microfluidic substrate. Therefore, it should be understood that the following discussion is provided for illustrative purposes and is not limited to only determining the position of the surface of the coverslip.
[0044] Some alternative surface localization methods use complex optical systems with separate optical paths for the surface localization light and the imaging light. Separate optical components are used for each path. Some methods employ unconventional detection geometries with tilted detectors (i.e., tilted with respect to the propagation direction of the surface localization light). In addition, some methods involve real-time monitoring of the surface while it is simultaneously imaged, requiring rapid and high-resolution detection of deviations from ideal imaging conditions. Thus, these systems tend to be costly, complex, and have a relatively large number of components.
[0045] In contrast, the methods and systems disclosed herein effectively reuse the optical imaging path of a microscope system to locate a sample relative to the object plane of the system and then image the sample once it is properly positioned. That is, locating the sample and imaging the sample are sequential in time, rather than simultaneous. Because nearly all of the optical components of the system are used for both functions, the complexity of the optical setup is greatly reduced compared to conventional systems.
[0046] Optical Microscope System In general, the methods disclosed herein are performed after a slide-mounted sample is introduced into the system, but before a high-magnification image of the sample is obtained for archiving and analysis. High-magnification sample imaging, such as digital pathology, is greatly aided by first localizing the sample to be imaged relative to the object plane of the imaging system. As described in more detail below, the surface of the coverslip on the mounted sample is first localized by directing a two-dimensional light pattern at the coverslip surface with relatively low optical power. From an image of the reflected pattern from the surface of the coverslip, a system offset is calculated and applied to the sample such that the sample is located at or near the object plane of the system (i.e., the sample is located within 20 μm of the object plane of the system, e.g., within 10 μm, or within 5 μm of the object plane).
[0047] The sample is then translated in a plane parallel to the object plane and imaged to help locate the tissue of interest on the slide surface. Imaging of the sample at this stage can be done at low light power (to determine the position of the coverslip) or high light power. The sample image is analyzed to identify the tissue of interest and the tissue coordinates along all three coordinate directions are stored.
[0048] Before acquiring detailed, high-resolution images of the sample, the system can optionally perform a fine-tuning focus procedure, in which the system determines the optimal focus position at several different locations on the sample, thereby building a focus map for subsequent high-resolution imaging.
[0049] The system then replaces the low magnification objective lens with a high magnification objective lens, if not already replaced, and reuses the same optical components to acquire a high resolution image of the identified target tissue for use in digital pathology. Each of the aforementioned steps is repeated along the same optical path to and from the sample without adjusting the detector position or orientation, and with relatively few adjustments to the optical components of the system. By reusing system components and paths in this manner, the sample is reliably positioned within the object plane of the system for imaging, while at the same time significantly reducing the complexity of the system compared to conventional surface detection systems.
[0050] Thus, the steps of locating and positioning the sample relative to the object plane, locating the tissue of interest at different locations in a plane parallel to the object plane, and acquiring high magnification images of the tissue of interest once it has been found and properly located are all performed sequentially in time. The sequential nature of these operations allows various components of the imaging system to be used for different purposes at different times with little or no reconfiguration of the system between steps. Furthermore, although the operations are sequential in time, the steps of locating and positioning the sample in the object plane and locating the tissue of interest can be performed relatively quickly, such that high resolution images of the sample can be acquired relatively quickly under good imaging conditions despite the sequential nature of the operations.
[0051] FIG. 1 shows a schematic diagram of an example of an optical microscope system 100. The system 100 can be a stand-alone microscope system or can be incorporated into a larger system, such as an automated slide scanning system. The system 100 includes an illumination source 102, an objective lens 104, an excitation filter assembly 152, a beam splitter assembly 154, an emission filter assembly 156, a stage 114, an imaging lens 118, a detector 120, and a controller 122. The excitation filter assembly 152, the beam splitter assembly 154, the emission filter assembly 156, the stage 114, and the detector 120 are electrically connected to and in communication with the controller 122. The controller 122 includes a display interface 124, an input interface 126, and one or more electronic processors 128.
[0052] Stage 114 supports sample 10, which typically comprises a specimen mounted on a substrate with a coverslip placed over the specimen such that the specimen is sandwiched between the substrate and the coverslip.
[0053] During operation, when the system 100 is configured to acquire an image of the sample 10, the illumination source 102 generates illumination light 130 that propagates along the z-direction of the rectangular coordinate system shown in FIG. 1 , passes through the objective lens 104, and is incident on the excitation filter assembly 152. The excitation filter assembly includes one or more excitation filters 106, and the controller 122 can selectively place a desired excitation filter 106 or combination of filters 106 in the path of the illumination light 130 to generate filtered light 132 by sending appropriate control signals to the excitation filter assembly 152. It should be noted that the use of an excitation filter or combination of excitation filters is optional, and in some embodiments, the illumination light 130 does not pass through any excitation filters. Thus, "filtered light 132" can be understood to refer to either the illumination light 130 that has passed through one or more excitation filters 106, or the illumination light 130 that has passed through the region between the objective lens 104 and the beam splitter assembly 154 (and may optionally have passed through other optical elements, such as optical windows).
[0054] The filtered light 132 enters a beam splitter assembly 154. The beam splitter assembly 154 includes a dichroic beam splitter 110, which the controller 122 can selectively place in the path of the filtered light 132 by sending appropriate control signals to the beam splitter assembly 154. During operation of the system 100 to acquire an image of the sample 10, the dichroic beam splitter 110 is typically positioned in the path of the filtered light 132.
[0055] To obtain a fluorescence image of the sample 10, the filtered light 132 is incident on the sample 10 after passing through the dichroic beam splitter 110. Generally, the filtered light 132 excites one or more fluorescent moieties present in the sample. The fluorescent moieties generate fluorescent emission light 134, a portion of which propagates in the -z direction toward the dichroic beam splitter 110. The spectral characteristics of the dichroic beam splitter 110 are generally selected such that the dichroic beam splitter 110 passes light of certain wavelengths (e.g., the filtered light 132) and reflects light in other wavelength bands (e.g., the fluorescent emission light 134). As a result, the fluorescent emission light 134 is reflected by the dichroic beam splitter 110 and incident on the emission filter assembly 156.
[0056] The optional emission filter assembly 156 typically includes one or more emission filters 116. The controller 122 can typically select an emission filter 116 (or a combination of emission filters 116) to be placed in the path of the fluorescent emission light 134. The one or more emission filters 116 are typically used to allow the fluorescent emission light 134 in a particular spectral band or bands to reach the detector 120 and to prevent the fluorescent emission light 134 and other light (e.g., stray light, filtered light 132) from reaching the detector 120.
[0057] The detected light 136 exits the emission filter assembly 156, passes through the imaging lens 118, and forms an image of the sample 10 at an image plane. Typically, the detector 120 includes a sensing element (or multiple sensing elements) located at or near the image plane to capture an image of the sample 10 by detecting the detected light 136. The image may correspond to the entire sample 10 or only a portion of the sample 10, depending on the area of the sample 10 on which the filtered light 132 is incident. Typically, the sample 10 is located at or near an object plane of the system 100, and the detector 120 is located at or near an image plane of the system 100. The combined effect of the objective lens 104, the dichroic beam splitter 110, and the imaging lens 118 is to image light from the object plane of the system (e.g., the fluorescent emission light 134) onto the image plane of the system.
[0058] To acquire an image of an analyte within the sample 10, the sample is generally aligned so that the top surface of the coverslip 150 (i.e., the surface of the coverslip that faces away from the analyte and faces the objective lens 104) is in the system's object plane z obj In this configuration, the object plane of the system is at a distance f along the z direction from the objective lens 104. obj placed in f obj is the nominal focal length of the objective lens 104.
[0059] The illumination source 102 can be implemented as a wide variety of different light sources, including incandescent, fluorescent, diode-based, laser-based, etc. The illumination source 102 can include wavelength-modulating elements, such as filters, to adjust the spectral distribution of the illumination light 130.
[0060] The objective lens 104 may be a single lens or a compound lens and may include one or more spherical and / or aspheric surfaces. Although shown in FIG. 1 as a transmitting lens, the objective lens 104 may also be implemented as a reflective lens. In some embodiments, the objective lens 104 may be an infinity corrected lens, i.e., an infinity conjugate lens. Whether the objective lens 104 includes a single lens element or multiple lens elements, it effectively forms the projection objective of the system 100.
[0061] The imaging lens 118 may be a single lens or a compound lens and may include one or more spherical and / or aspheric surfaces. The imaging lens 118 may be a transmissive lens, as shown in FIG. 1, or may be a reflective lens. In certain embodiments, the imaging lens 118 may be implemented as a tube lens that includes multiple transmissive and / or reflective elements.
[0062] In addition to the elements described above, system 100 can also generally include a wide variety of other optical elements, including, but not limited to, lenses, mirrors, beam splitters, filters, polarizing optics, windows, prisms, and diffraction gratings.
[0063] The detector 120 includes a two-dimensional imaging sensor that captures an image of the sample 10 (and / or an object disposed on the stage 114). The detector 120 can include any one or more of a wide variety of imaging sensors. For example, the detector 120 can include a CCD-based sensor, a CMOS-based sensor, a diode-based sensor, and other imaging sensors.
[0064] The controller 122 includes a display interface 124 capable of displaying images acquired by the detector 120, user interface information, operating parameters, and other information. An input interface 126, which may be implemented as part of the display interface 124, allows a user of the system 100 to enter commands, set operating parameters, adjust or select system configurations, and control other aspects of the operation of the system 100.
[0065] An electronic processor 128 (which may be implemented as a single processor or as multiple processors performing common or different functions) performs a variety of different system control and computational functions. In general, via the electronic processor 128, the controller 122 may perform any of the configuration, operation, imaging, and analysis steps disclosed herein. The controller 122 (and electronic processor 128) are connected to the illumination source 102, the stage 114, the detector 120, the excitation filter assembly 152, the beam splitter assembly 154, and the emission filter assembly 156.
[0066] The controller 122 can send various control signals to the illumination source 102 to adjust various characteristics of the illumination light 130, including the light intensity, wavelength, polarization state, and spatial intensity profile. The controller 122 can send control signals to the stage 114 to translate the stage 114 in any one or more of the x, y, and z directions. Thus, for example, to change the relative distance between the objective lens 104 and the stage 114 along the z direction, the controller 122 can send a control signal to the stage 114, thereby translating the sample 10 in the +z or −z direction as needed.
[0067] As described herein, controller 122 sends control signals to stage 114 to translate stage 114 in a direction parallel to the z-axis to move the stage 114 in an object plane z of the system. objNote that the position of the stage 144 and sample 10 relative to d is adjusted. In some embodiments, in addition to or instead of translating the stage 114, the objective lens 104 can be translated in a direction parallel to the z-axis to adjust the position of the object plane of the system along the z-axis. In fact, by adjusting the position of the objective lens 104, the distance between the patterning modulator 108 and the objective lens 104 can also be changed, and this change is reflected by d 1 (described in more detail below). In the following description, only adjustments to the z coordinate position of stage 114 are described, however, it should be understood that adjustments to the z coordinate position of objective lens 104, and / or adjustments to the positions of both stage 114 and objective lens 104, may be implemented in the methods described herein without loss of functionality or applicability.
[0068] Image information captured by detector 120 is transmitted to controller 122. The image information may be analyzed by electronic processor 128, as described in further detail below, to determine whether the coverslip surface of sample 10 is located in the object plane of system 100, and, if the coverslip surface is not located in the object plane of the system, to determine appropriate adjustments to the position of sample 10 relative to objective lens 104.
[0069] In general, the position of the sample 10 relative to the system's object plane, and corrections to account for the displacement of the sample 10 from the object plane, may be determined at any point in the sequence of operations for acquiring one or more sample images. In some embodiments, the position of the sample 10 relative to the object plane may be determined / corrected, for example, before acquiring the sample image. In certain embodiments, the position of the sample 10 may be determined / corrected after acquiring one or more images and before acquiring additional images. In some embodiments, the position of the sample 10 may be determined / corrected after acquiring one or more low resolution images and before acquiring one or more high resolution images of the sample. In certain embodiments, the position of the sample 10 may be determined / corrected after imaging an initial portion of the specimen and before imaging one or more additional portions of the specimen, for example, after translating the sample 10 in one or both of the x and y directions.
[0070] System object plane z obj To determine the position of the sample 10 relative to the beam splitter 154, and in particular the position of the top surface of the cover slip of the sample 10, the configuration of the optical elements of the system 100 is typically altered. In particular, a patterned modulator 108 is introduced into the path of the illumination light 130 or filtered light 132 upstream from the beam splitter assembly 154, and the dichroic beam splitter 110 is replaced by a partially transmissive and partially reflective beam splitter 112. In general, the patterned modulator 108 can be located anywhere between the illumination source 102 and the beam splitter. In some embodiments, the excitation filter 106 (or the filter 106) is removed from the system 100, and the patterned modulator is inserted in the original position of the excitation filter 106. In certain embodiments, the excitation filter 106 remains in the system 100, and the patterned modulator is inserted either upstream or downstream of the filter, and upstream of the object plane of the system. The insertion of the patterned modulator and / or the removal of the excitation filter 106 can be performed manually or automatically.
[0071] As mentioned above, the partially transmitting beam splitter 112 is typically inserted in the system 100 in the position of the removed dichroic beam splitter 110. As mentioned above, the removal of the dichroic beam splitter 110 and / or the insertion of the partially transmitting beam splitter 112 can typically be performed manually or automatically.
[0072] The patterned modulator 108 typically includes a pattern of features that modulate the intensity of the illumination light 130 or filtered light 132. The patterned modulator can be formed in a variety of ways and can generally be implemented as an active or passive modulator, or can include both active and passive components that modulate the intensity of light incident on the modulator. In some embodiments, for example, the patterned modulator 108 is comprised of a plurality of metallic or other non-transmissive features arranged in a pattern on a substrate, such as an optical window. The features allow a portion of the illumination light 130 or filtered light 132 to pass through the substrate and block a portion of the illumination light or filtered light, thereby providing a modulation in the cross-sectional spatial intensity profile of the illumination light or filtered light.
[0073] In certain embodiments, the patterned modulator 108 includes one or more diffractive elements that impart a modulation to the cross-sectional spatial intensity profile of the illumination light or filtered light. In some embodiments, the patterned modulator 108 includes one or more apertures formed in a light-blocking substrate that collectively impart a modulation to the cross-sectional spatial intensity profile of the illumination light or filtered light before the light reaches an object plane of the system. In certain embodiments, the patterned modulator 108 is a tunable modulator (e.g., a liquid crystal-based spatial light modulator, etc.) that can be configured to impart any of a variety of different modulations to the cross-sectional spatial intensity profile of the illumination light or filtered light.
[0074] As an example, in some embodiments, the patterned modulator is implemented as a reticle, such as a chrome reticle. Chrome reticle R1DS2N (available from Thorlabs, Inc., Newton, NJ) is one example of a chrome reticle suitable for use as the patterned modulator 108.
[0075] Figure 2 is a schematic diagram of the optical microscope system 100 after introducing the patterned modulator 108 and the partially transmissive beam splitter 112. In the example shown in Figure 2, the controller 122 sends a control signal to the excitation filter assembly 152 to remove the excitation filter 106 from the path of the illumination light 130 and introduce the patterned modulator 108 into the path of the illumination light. In addition, the controller 122 sends a control signal to the beam splitter assembly 154 to remove the dichroic beam splitter 110 from the path of the filtered light 132 and introduce the partially transmissive beam splitter 112 into the path of the filtered light.
[0076] In some embodiments, emission filter assembly 156 can include a specialized filter that blocks stray light that is not used to form an image of the patterned modulator on the sensor of detector 120. To block stray light, a specialized filter can be introduced into the path of the light reflected from the partially transmitting beam splitter 112. As with the partially transmitting beam splitter 112 and / or the patterned modulator 108, the specialized filter can be introduced into system 100 manually or in an automated manner via control signals sent from controller 122 to emission filter assembly 156.
[0077] System 100 is an example of a system in which a patterned modulator and a partially transmitting beam splitter (and optionally a stray light filter) are introduced in an automated manner. An excitation filter assembly 152 (which may be implemented, for example, as a motorized or driven filter wheel) includes a patterned modulator 108. The controller 122 can send appropriate control signals to the excitation filter assembly 152 to remove any excitation filter 106 in the path of the illumination light 130 and instead insert the patterned modulator 108 in the position previously occupied by the excitation filter. Thus, the illumination light 130 is no longer spectrally filtered by the excitation filter assembly 152. Instead, the cross-sectional spatial intensity profile of the illumination light 130 is modulated by the patterned modulator 108, and the filtered light 132 carries a spatial pattern in the form of a cross-sectional spatial intensity modulation.
[0078] In general, the image of the patterned modulator 108 is formed in a plane that is a distance ΔD from the object plane of the system. Thus, since the optical path of the system 100 is folded (i.e., the image of the patterned modulator 108 is captured in reflection mode), when the top surface of the cover slip of the sample 10 is located at a distance ΔD / 2 from the object plane of the system, the image of the patterned modulator 108 is formed in the object plane of the system after reflection from the top surface of the cover slip. With the top surface of the cover slip positioned as described above, the spatial pattern carried by the filtered light 132 is imaged at the top surface of the cover slip 150, and a portion of the filtered light 132 reflects from the top surface of the cover slip 150 as if the spatial pattern originated from the top surface of the cover slip. Nominally, the image of the spatial pattern is imaged in focus at the image plane of the system, and therefore the spatial pattern appears in focus in the image captured by the detector 120 at the image plane of the system. However, if the top surface of the cover slip of the sample 10 is displaced from the above-mentioned position, the spatial pattern reflected from the top surface of the cover slip at the location of the top surface of the cover slip will not be perfectly in focus, and as a result, the image of the spatial pattern acquired by the detector 120 at the image plane will also not be perfectly in focus.
[0079] As shown in FIG. 2, with the patterned modulator 108 and the partially transmitting beam splitter 112 positioned in the path of the illumination light 130 and the filtered light 132, the image of the patterned modulator 108 is projected at a distance d from the objective lens 104. 1 where d 1 is the nominal focal length f of the objective lens 104 according to the lens formula obj and the distance d between the patterning modulator 108 and the objective lens 104 along the z direction 2 Related to. 1 / f obj =1 / d 1 +1 / d 2 [1]
[0080] If the objective lens 104 is infinitely conjugate (as described above with respect to the objective lens 104), then the image acquired at the image plane of the system will be in focus if the object (i.e., the image of the patterned modulator 108 formed by the objective lens 104) is in focus at the object plane of the system. For the system shown in Figure 2, the projected image of the patterned modulator 108 is located a distance ΔD from the object plane of the system. d 1 =f obj +ΔD=1 / [1 / f obj -1 / d 2 ]-1 / f obj [2]
[0081] As a result, the projected image of the patterned modulator 108 is reflected by the objective lens 104. 1 If the objective lens 104 is positioned at a distance other than d along the optical path, the image of the patterned modulator 108 captured by the detector 120 located in the image plane of the system will be out of focus. However, as the stage 114 is translated in the z direction, the image of the patterned modulator 108 formed by the objective lens 104 will be shifted to a distance d along the optical path from the objective lens 104. 1 1. If the top surface of the coverslip, onto which the projected image of the patterned modulator 108 is reflected, is positioned such that the image of the patterned modulator 108 acquired by a detector 120 located at the image plane of the system will be in focus. Referring to equation (2) above and noting that the optical path of the system is folded between the partially transmitting beam splitter 112 and the stage 114, the image of the patterned modulator 108 formed by the objective lens 104 will be in focus on the object plane of the system after reflection from the top surface of the coverslip if the top surface of the coverslip is positioned a distance ΔD / 2 from the object plane of the system.
[0082] As a result, by assessing how in-focus or out-of-focus the image of the patterned modulator 108 is in an image acquired at the image plane of the system, the position of the top surface of the coverslip relative to the object plane of the system can be estimated. Furthermore, a corrective displacement can be determined for any particular position of the top surface of the coverslip of the sample 10 along the z-direction to ensure that the top surface of the coverslip is positioned such that the image of the patterned modulator 108 formed by the objective lens 104 is formed at the object plane of the system.
[0083] To position the top surface of the cover slip 150 of the sample 10 at a particular z coordinate position relative to the object plane of the system, multiple different images can be acquired, each with the patterned modulator 108 and the partially transmitting beam splitter 112 positioned in the optical path of the system, as described above in connection with FIG. 1. Each of the different images corresponds to a different relative displacement between the objective lens 104 and the top surface of the cover slip. Before acquiring each image, the controller 122 translates the sample 10 along the z direction to select a different relative displacement by sending appropriate control signals to the stage 114.
[0084] The set of relative displacements is typically such that the optical path length between the objective lens 104 and the object plane of the system for light reflected from the top surface of the coverslip is d 1 Larger than d 1 , is selected to include a sample position smaller than d. For each relative displacement, an image is acquired. The resulting set of images is analyzed, and the image in which the modulation pattern imparted by the patterned modulator 108 is at optimal focus is designated as the in-focus image. As explained above, if the distance along the optical path of the system 100 between the objective lens 104 and the object plane of the system after reflection from the top surface of the coverslip is d, 1 This focused image is taken when the top surface of the coverslip is positioned such that:
[0085] After one or more images of the light modulated by the patterned modulator 108 (or one or more images of features on a coverslip or slide) are acquired, a variety of different methods can be used to analyze the images to determine which image corresponds to an "optimum focus" state. For example, in some embodiments, the images are analyzed by determining the maximum and minimum pixel intensity of each image and then calculating the difference between the maximum and minimum pixel intensities to obtain a contrast metric. The image of the set of images with the highest contrast metric (i.e., the largest difference between the maximum and minimum pixel intensities) may be determined to represent the "optimum focus" state. The maximum and / or minimum pixel intensity may be determined from a single pixel intensity or may be averaged over a group of pixels (e.g., averaged over n maximum or minimum pixel intensities, where n is 2 or more, 3 or more, 5 or more, 10 or more, 20 or more, 30 or more, 50 or more, or more).
[0086] In certain embodiments, the images are analyzed by selecting one or more features that appear in each image and calculating the spatial rate of change of intensity along one or more pixel rows or columns across the one or more features. Generally, in images that are closer to a "best focus" state, the transitions between light and dark regions of the image are more abrupt, i.e., the rate of change of intensity across such transitions is greater. Thus, the rate of change of intensity along one or more common pixel rows or columns of each image can serve as a metric to determine which of the images correspond to the "best focus" state.
[0087] Other methods can be used in addition to or as an alternative to the methods mentioned above. Examples of methods and metrics such as the F-score can be found in "F-score: A New Approach to Classifying Averages in a Class of Averaged Data," in "Classification by Classification ...
[0088] It should be noted that in some embodiments, when no single image of the modulator 108 pattern is perfectly focused in the system's image plane, the z coordinate of the best focus of stage 114 is likely to be a coordinate location that is halfway between two of the z coordinate locations used to generate the two images that are closest to perfect focus. In such a situation, the z coordinate representing best focus can be determined by interpolating between the z coordinate locations used to generate the two images that are closest to perfect focus.
[0089] As indicated by equation (2) above, the controller 122 then translates the stage 114 along the z direction a distance ΔD / 2 so that the top surface of the coverslip is aligned with the object plane of the system (i.e., a distance f from the objective lens 104). obj ) the sample 10.
[0090] After the sample 10 is repositioned so that the top surface of the coverslip is in the object plane of the system, one or more multispectral images (e.g., fluorescence images) of the sample can be acquired. To acquire such images, the controller 122 sends appropriate control signals to the excitation filter assembly 152 to remove the patterned modulator 108 from the path of the illumination light 130 (and, optionally, insert any desired excitation filter 106 into the path). The controller 122 also sends appropriate control signals to the beam splitter assembly 154 to remove the partially transmitting beam splitter 112 from the path of the filtered light 132 and insert the dichroic beam splitter 110 into the path of the filtered light 132.
[0091] As described above, the filtered light 132 is directed towards the sample 10, and the fluorescent emission light 134 generated at the sample 10 propagates in the -z direction, is reflected by the dichroic beam splitter 110 and is incident on the emission filter 116 (or combination of emission filters 116) of the emission filter assembly 156. The detection light 136 exits the emission filter assembly 156, passes through the imaging lens 118, and forms an image of the sample 10 at an image plane of the system 100 where the sensing element of the detector 120 is located. The detector 120 captures image information by detecting the detection light 136, and the image information is transmitted to the controller 122. By using various combinations of the excitation filters 106 and / or emission filters 116, multiple images of the sample 10 can be acquired.
[0092] Calibrating the microscope system In the above example, it is assumed that the distance ΔD is already known, for example, from previously measured calibration information and / or system design information (e.g., calculated from equation (2)). As a result, the controller 122 can use the known value of ΔD to reproducibly reposition the sample 10 so that the top surface of the coverslip is located in the object plane of the system.
[0093] Alternatively, the quantity ΔD can be determined from a calibration procedure performed as part of the sample imaging workflow. Various methods can be used to perform such a calibration. Figure 3 is a flow chart showing a series of exemplary steps for performing a calibration procedure.
[0094] In the first step 302 of the calibration procedure, the patterned modulator 108 and partially transmissive beam splitter 112 are introduced into the system 100 and the sample 10 is placed on the stage 114 as described above.
[0095] Next, in step 304, stage 114 is initiated to determine a particular distance d 1The sample 10 is translated along the z direction to a particular position on the z axis corresponding to: In step 306, an image of the patterned modulator 108 reflected from the surface of the coverslip is acquired.
[0096] Then, in step 308, a suitable distance range d 1 If no image was acquired, control returns to step 304, where the stage 114 is actuated to translate the sample 10 to a new position on the z-axis. Alternatively, the appropriate distance range d 1 If images were acquired at , the set of images is analyzed in step 310 to determine the z-axis coordinate of stage 114 where the image of the patterned modulator is in best focus. This stage position is P 1 and corresponds to the top surface of the coverslip, which is offset from the object plane of the system by a distance ΔD / 2.
[0097] Next, in step 312, stage 114 is activated to calculate a second distance range d 1 The sample 10 is translated along the z direction to a particular location on the z axis that is a portion of the second distance range. The second distance range is typically selected such that the specimen between the coverslip 150 and the slide is located in the object plane of the system at one of the z axis locations within the second distance range.
[0098] An image of the structures on the slide is then obtained in step 314. The structures can be, for example, fiducial marks, debris particles, specimen structures, or other features on or in contact with the slide surface facing the coverslip.
[0099] Next, in step 316, a suitable second distance range d 1 If no image was acquired, control returns to step 312 where stage 114 is actuated to translate sample 10 to a new position on the z-axis. Alternatively, the appropriate distance range d 1If images were acquired at P, then the set of images is analyzed in step 318 to determine the z-axis coordinate of stage 114 where the image of the structures on the slide is in best focus. This stage position is P 2 and corresponds to the surface of the slide that is placed in the object plane of the system.
[0100] Then, in step 320, the stage position (P 2 -P 1 ) and the position of the structure relative to the surface of the slide determines the quantity ΔD. In practice, it is possible to first calibrate the quantity ΔD between the in-focus modulator pattern and the top surface of the coverslip, and then determine the offset between the top surface of the coverslip and the expected sample position separately. Calibrating in this way has the advantage that ΔD is independent of the thickness of the sample. In this way, when ΔD is calibrated independently of the offset between the top surface of the coverslip and the expected sample position, the same calibrated value of ΔD can be used when imaging samples of different thicknesses. When such a sample is introduced into the system, only the offset is calibrated. The procedure shown in FIG. 3 ends with step 322.
[0101] Focus Mapping In some embodiments, system calibration (i.e., measurement of ΔD or a priori knowledge of ΔD) can be used to perform focus mapping of the sample 10. Focus mapping, as used herein, refers to the process of determining a set of z coordinate locations of the sample 10 at which a specimen sandwiched between the cover slip and slide of the sample is at optimal focus as a function of x and y coordinate locations. Since specimens are typically non-uniform in thickness, which can lead to various imaging errors and artifacts, generally not all portions of the specimen in the lateral (x,y) plane will be in focus at a common z coordinate location of the specimen (and stage 114). Instead, at different (x,y) locations, the z coordinate locations that represent optimal focus for portions of the specimen at different (x,y) locations will typically be different. Focus mapping is the procedure of determining a set of z coordinate locations that represent optimal focus of the specimen as a function of (x,y) location.
[0102] To perform focus mapping, as described above, an initial z coordinate value Z is calculated from the position of best focus of the reflected image of the patterned modulator 108 and the value of ΔD. c The z-coordinate value Zc represents the z-coordinate value at which the surface of the coverslip is located in the object plane of the system 100.
[0103] Patterned modulator 108 and partially transmitting beam splitter 112 are then removed from system 100 and dichroic beam splitter 110 is reintroduced into system 100 .
[0104] Next, a range of z coordinate values is determined over which the stage 114 is positioned to determine the location of the best focus at each (x,y) location of the specimen. In some embodiments, the specimen is always positioned at a more positive z coordinate than the surface of the coverslip, so that Z c represents one end of the range of z coordinate values.
[0105] A focus map is then measured. At each of a set of discrete (x,y) locations within the specimen, stage 114 is translated through a range of z coordinate values in discrete steps. After each successive discrete step translation of stage 114, an image of the specimen is acquired.
[0106] The set of images acquired through translation of stage 114 over a range of z coordinate values is then analyzed to determine which image in the set is at the best focus. s The corresponding z coordinate value of stage 114 associated with the best focus image at location (x,y), designated (x,y), may then be stored as an entry in the focus map of sample 10. Alternatively, the z coordinate value Z of stage 114 representing the best focus position may be stored as s (x,y) can be calculated by interpolating between two z coordinate values of stage 114, each of which corresponds to an image that is near optimal focus.
[0107] The aforementioned process is repeated at multiple (x,y) locations within the specimen to obtain the specimen's best focus value Z s You can get a set of (x,y).
[0108] Determining slide position In some embodiments, a procedure similar to that described for coverslips can be used to determine the z coordinate position of stage 114 where the bottom surface of the slide of sample 10 is in the object plane of system 100. As used herein, the "bottom surface" of the slide is the surface of the slide that is closest to and / or in contact with stage 114, and is opposite the surface of the slide that contacts the analyte of sample 10.
[0109] The procedure for determining the z coordinate position of stage 114 where the bottom surface of the sample 10 slide is in the object plane of system 100 can include steps similar to those described above in connection with Figure 3. In a first step, patterned modulator 108 and partially transmitting beam splitter 112 are introduced into system 100 as described above.
[0110] A range of z coordinate values for stage 114 is then determined. Typically, this range includes one or more z coordinate values such that, when stage 114 is positioned at any of the one or more z coordinate values, the bottom surface of the slide is displaced in the +z direction relative to the object plane of the system, and one or more z coordinate values such that, when stage 114 is positioned at any of the one or more z coordinate values, the bottom surface of the slide is displaced in the -z direction relative to the object plane of the system.
[0111] The stage 114 is then translated in sequence to each z coordinate value in the range and an image of the patterned modulator 108 is acquired at each z coordinate value.
[0112] The set of images thus acquired is then analyzed to determine which image in the set provides the best focused image of the patterned modulator 108. The z-coordinate value associated with this image is Z u is the z coordinate value at which the bottom surface of the slide is located a distance ΔD / 2n from the object plane of the system, where n is the refractive index of the material forming the slide.
[0113] In some embodiments, the above procedure can be used to determine whether the sample 10 introduced on the stage 114 is in an expected position. For example, after the sample is placed on the stage 114, the stage 114 can be translated through the range of z coordinate values mentioned above. The projected image of the patterning module 108 can be analyzed to determine the actual z coordinate value Z where the bottom surface of the slide is located in the object plane of the system. u It is possible to determine the measured Z u The value is, for example, Zu may be compared to a calibration or standardization value to determine whether a sample slide is placed in an expected location within the system 100 or deviated from the expected location. The above-described methods may be used, for example, to verify that a slide is properly loaded into the system 100 prior to acquiring an image of the specimen.
[0114] Specimen and sample type The systems and methods described herein can be used to acquire images of a wide variety of different specimens sandwiched between a slide and a coverslip for multispectral imaging, including, but not limited to, core biopsies, cell cultures, tissue sections (e.g., formalin-fixed paraffin-embedded sections), fine needle aspirates, individual cells and groups of cells, and smears of blood and other bodily fluids.
[0115] Hardware and Software Implementation Any step disclosed herein may be performed by the controller 122 (e.g., electronic processor 128 of the controller 122) and / or one or more additional electronic processors (such as a computer or pre-programmed integrated circuits) executing programs based on standard programming techniques. Such programs are designed to run on programmable computing apparatus or specially designed integrated circuits. The controller 122 may include a data storage system (including memory and / or storage elements), at least one input device, such as an input interface 126, and at least one output device, such as a display interface 124 to which the controller 122 is linked.
[0116] 4 is a schematic diagram of an example of a controller 122 that may be present in the systems described herein and that may perform any of the method steps described herein. The controller 122 may include one or more processors 128, a memory 404, a storage device 406, and an interconnect interface 408. The processor 128 may process instructions for execution within the controller, including instructions stored in the memory 404 or the storage device 406. For example, the instructions may direct the processor 128 to perform any of the steps disclosed herein.
[0117] The memory 404 can store executable instructions for the processor 128, information regarding parameters of the system such as excitation and detection wavelengths, measured image information, and / or calibration information. The storage device 406 can be a computer-readable medium such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or an array of devices including devices in a storage area network or other configuration. The storage device 406 can store any of the instructions that can be executed by the processor 128 as described above, and other information that can be stored by the memory 404.
[0118] In some embodiments, the controller 122 may include a graphics processing unit for displaying graphical information (e.g., using a GUI or text interface) on an external input / output device such as the display interface 124. The graphical information may be displayed by a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying any of the information such as the measured and calculated spectra and images disclosed herein. A user may provide input to the controller 122 using an input device (e.g., a keyboard, a pointing device, a touch screen, a voice recognition device, etc.). In some embodiments, one or more such devices may be part of the controller 122.
[0119] A user of the system 100 can provide various types of instructions and information to the controller 122 via an input device. The instructions and information can include, for example, information regarding any of the wavelengths, filters, and physical parameters (e.g., focal length, positions of components of the system 100) associated with any of the workflows described herein, as well as system calibration information. The controller 122 can use any of these various types of information to perform the methods and functions described herein. It should also be noted that any of these types of information can be stored (e.g., in the storage device 406) and recalled as needed by the controller 122.
[0120] The methods disclosed herein can be implemented by the controller 122 executing instructions of one or more computer programs executable and / or interpretable by the controller 122. These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages and / or in assembly / machine languages. For example, a computer program can include instructions that can be stored on the memory 404, storage device 406, and / or tangible computer-readable medium and executed by the processor 128 as described above. As used herein, the term "computer-readable medium" refers to any computer program product, apparatus, and / or device used to provide machine instructions and / or data to a programmable processor (e.g., magnetic disks, optical disks, memories, programmable logic devices (PLDs), ASICs, electronic circuits, etc.), including machine-readable media that receive machine instructions.
[0121] By executing the above-mentioned instructions (which may optionally be part of controller 122), the controller may be configured to perform any one or more of the various steps described in connection with any of the workflows herein. For example, controller 122 may adjust the position of any component of system 100, acquire images as described herein, analyze the acquired images, and adjust system 100 based on the analysis (e.g., calibration information obtained from the analysis).
[0122] Working Example A system similar to that described herein was constructed as follows: an Olympus BX-43 microscope was equipped with BX3 fluorescence illumination optics on an 8 position filter wheel. The objective lens 104 was an Olympus 10x UPLXAPO. The optics used to form an image of the patterned modulator were housed in an epifilter cube located at position 8 of the filter wheel and included a chrome reticle (reticle R1DS2N, obtained from Thorlabs, Inc., Newton, NJ) in the excitation filter position, a partially transmitting neutral density 50 / 50 beam splitter in the dichroic beam splitter position, and a red filter (Semrock FF01-637-7-25) in the emission filter position.
[0123] The system was operated at infinite conjugate and the focal length of the objective was 18 mm. The distance from the objective to the reticle, d 2 was approximately 139 mm. The projected image was formed at a distance ΔD = 2680 μm from the nominal system object plane. Light from the illumination source was reflected off the top of the coverslip to form an image in free space.
[0124] When the coverslip passed the plane that formed the image on the imaging sensor by approximately ΔD / 2, the reflected projected image was formed exactly at that plane. When the coverslip was moved slightly further or closer to the objective, the reflected image moved twice as far away or closer due to the reflective geometry. In the constructed system, an image of the patterned object was formed on the imaging sensor when the coverslip was approximately 1340 μm away from the conventional imaging plane.
[0125] The projected image could be observed by placing a white object 2680 μm away from the conventional imaging plane. Making this type of measurement is one way to measure ΔD. However, in this configuration, the image is formed at a location that is not in a conjugate plane with the imaging sensor, so the imaging sensor cannot assess optimal focus.
[0126] Another approach has been to use a microscope slide to form an image that is projected into space at the image plane of a conventional system.
[0127] A piece of glass with patterned chrome features was placed on the stage, and the epifilter wheel was set to mesh with the patterned object and the partially reflective beam splitter. A set of images was taken with the stage at a range of positions spanning a position that placed the patterned surface 1340 μm away from the normal object plane. This range spanned ±500 μm around that point to allow for uncertainties in the stage position, glass thickness, and other factors. These images were analyzed to determine the optimal focus position Z by measuring the normalized variance of each image. r was determined. Images were taken at 10 μm intervals, and the three positions corresponding to the images where the focus was optimal were fitted to interpolate the position of the optimal focus at finer resolution.
[0128] The epifilter was set to remove all elements from the beam, and transmitted light brightfield images were acquired with the stage set to a range of positions spanning the position that places the pattern plane in the nominal system object plane. This range spanned more than ±500 μm for similar reasons as previously described. These images were inspected and the optimum focus position Z was determined in the same manner as previously described. t Based on these measurements, the system was calibrated according to: ΔD / 2=Z t -Z r [3] In the system herein, ΔD / 2=1340 μm.
[0129] In operation, the epifilter wheel was set to engage the patterned optics, and the system found the stage position corresponding to the optimal focus for the projected image of the patterned object. When the stage was set 1340 μm closer to the objective than that point, the focus was brought to the surface of the coverslip.
[0130] Other embodiments Although specific implementations are described in this disclosure, these should not be construed as limiting the scope of the disclosure, but as describing features in a particular embodiment. Features described in the context of separate embodiments can also generally be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as being present in some combinations and may initially be claimed as such, one or more features from the claimed combinations can generally be excluded from the combination, and the claimed combinations may be directed to subcombinations or variations of subcombinations.
[0131] In addition to the embodiments explicitly disclosed herein, it will be understood that various modifications can be made to the described embodiments without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are within the scope of the following claims. [Explanation of symbols]
[0132] 10 Samples 100 Optical Microscope System 102 Lighting source 104 Objective Lens 106 Excitation Filter 108 Patterned Modulator 110 Dichroic Beam Splitter 112 Partially Transmitting Beam Splitter 114 Stages 118 Imaging Lens 120 Detector 122 Controller 124 Display Interface 126 Input Interface 128 Electronic Processor 130 Illumination 132 Filtered Light 134 Fluorescent Light 152 Excitation Filter Assembly 154 Beam splitter assembly 156 Emission Filter Assembly 404 Memory 406 Storage Devices 408 Interconnect Interface
Claims
1. generating a modulated light pattern using a light modulator; transmitting the modulated light pattern to a surface of a substrate through an objective lens defining an object plane; acquiring a plurality of images of the modulated light pattern reflected from a surface of the substrate, each image being acquired at a different relative distance between the objective lens and the surface of the substrate; analyzing the plurality of images to determine a reference position of a surface of the substrate relative to the objective lens that corresponds to a position of an image of the modulated light pattern formed by the objective lens at the object plane; and adjusting a position of the substrate relative to the objective lens based on the reference position.
2. The method of claim 1 , wherein the substrate comprises a coverslip overlying the biological sample.
3. The method of claim 1 , wherein the substrate comprises a slide supporting a biological sample.
4. The method of claim 2 , wherein the biological sample comprises a tissue section.
5. The method of claim 1 , wherein generating the modulated light pattern comprises generating illumination light and transmitting the illumination light through the light modulator.
6. The method of claim 1 , wherein the optical modulator is a passive modulator.
7. The method of claim 1 , wherein the optical modulator is an active modulator.
8. The method of claim 6 , wherein the passive modulator comprises a reticle.
9. The method of claim 1 , wherein each of the plurality of images is acquired by a detector disposed in an image plane conjugate to the object plane.
10. The method of claim 1 , wherein the objective lens is an infinite conjugate objective lens.
11. The method of claim 1 , wherein analyzing the plurality of images comprises identifying an image of the plurality of images in which the modulated light pattern is in optimal focus.
12. The method of claim 11 , wherein determining the reference position comprises identifying as the reference position a position of the surface of the substrate relative to the objective lens at which the identified image was acquired.
13. 2. The method of claim 1, wherein analyzing the plurality of images comprises identifying two or more images from the plurality of images in which the modulated light pattern is at an optimal focus, and determining the reference position comprises interpolating between positions of the surface of the substrate relative to the objective lens at which the identified images were acquired.
14. The method of claim 1 , wherein adjusting the position of the substrate comprises adjusting the position of the substrate such that a biological sample disposed below or above the substrate is located at the object plane.
15. The method of claim 14 , further comprising acquiring one or more images of the biological sample while the biological sample is located at the object plane.
16. An illumination source; An objective lens; An optical modulator; A detector; Stage and a controller connected to the illumination source, the detector, and the stage, the objective lens defines an object plane of the system; The controller: activating the illumination light source to generate illumination light and directing the illumination light to the light modulator to generate a modulated light pattern; activating the stage to position a surface of the substrate on the stage at a plurality of different distances relative to the objective lens; activating the detector to obtain a plurality of images of the modulated light pattern at the detector corresponding to each of the plurality of different distances; analyzing the images to determine a reference position of the surface of the substrate relative to the objective lens that corresponds to a position of an image of the modulated light pattern formed by the objective lens at the object plane; The system is configured to actuate the stage to adjust a position of the substrate relative to the objective lens based on the reference position.
17. The system of claim 16 , wherein the substrate comprises a coverslip overlying the biological sample.
18. The system of claim 16 , wherein the substrate comprises a slide that supports a biological sample.
19. The system of claim 17 , wherein the biological sample comprises a tissue section.
20. The system of claim 16 , wherein the optical modulator is a passive modulator.
21. The system of claim 16 , wherein the optical modulator is an active modulator.
22. The system of claim 20 , wherein the passive modulator comprises a reticle.
23. The system of claim 16 , wherein the detector is disposed in an image plane of the system.
24. The system of claim 23 , wherein the image plane is conjugate with the object plane.
25. The system of claim 16 , wherein the objective lens is an infinite conjugate objective lens.
26. 17. The system of claim 16, wherein the controller is configured to analyze the plurality of images by identifying an image of the plurality of images in which the modulated light pattern is at an optimal focus.
27. 27. The system of claim 26, wherein the controller is configured to determine the reference position as a position of the surface of the substrate relative to the objective lens at which the identified image was acquired.
28. 17. The system of claim 16, wherein the controller is configured to analyze the plurality of images by identifying two or more images among the plurality of images in which the modulated light pattern is at an optimal focus, and to determine the reference position by interpolating between positions of the surface of the substrate relative to the objective lens at which the identified images were acquired.
29. 17. The system of claim 16, wherein the controller is configured to actuate the stage to adjust the position of the substrate such that a biological sample disposed below or above the substrate is located at the object plane.
30. 30. The system of claim 29, wherein the controller is configured to activate the detector to acquire one or more images of the biological sample while the biological sample is located at the object plane.
