Digital imaging system and method

Abstract

To solve the problem of conventional systems and methods suffering from slow image acquisition because of the time required to stop and focus while scanning an entire specimen.SOLUTION: The invention provides a system and method for acquiring images of objects distributed within a specimen affixed to a surface of a slide, the specimen having an uneven height relative to the slide surface using a camera having an objective lens with an optical axis that forms a non-orthogonal angle with the surface of the slide, the method including: acquiring a first plurality of images of a first linear portion of the specimen; evaluating a focus of objects within the linear portion of the specimen captured in the first plurality of images; and acquiring a second plurality of images of the first linear portion or of a second linear portion of the specimen different from the first linear portion; where a height of the objective lens relative to the slide surface is varied during the acquisition of the second plurality of images based on the evaluated focus of the objects captured in the first plurality of images.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] This application claims the benefit under 35 U.S.C. § 119 of U.S. Provisional Patent Application No. 62 / 940,163, filed November 25, 2019, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates generally to systems and methods for acquiring digital images of specimens affixed to the surface of a slide, such as cytological (cellular) specimens having a thickness that exceeds the depth of field of an imaging device.

[0003] All US and PCT patents and patent publications identified herein for any purpose are incorporated by reference in their entirety. [Background technology]

[0004] Cytology is a branch of biology that studies the formation, structure, and function of cells. In laboratory settings, cytologists, cytotechnologists, and other medical professionals make medical diagnoses of patient conditions based on the visual examination of a patient's cell sample. Such samples are referred to herein as "cytological" specimens. A typical cytological procedure is the "Pap smear" test, in which cells are scraped from a woman's cervix and analyzed to detect the presence of abnormal cells that may be precursors to the development of cervical cancer. Cytological procedures are also used to detect abnormal cells and disease in other parts of the human body.

[0005] Cytological techniques are widely adopted because obtaining cell specimens for analysis is generally less invasive than traditional surgical pathology procedures, such as biopsies, in which a solid tissue sample, referred to herein as a "pathological" specimen, is removed from a patient using a specialized biopsy needle with a spring-loaded, translatable stylet or fixed cannula. Cell samples can be obtained from a patient by a variety of techniques, including, for example, scraping or swabbing an area or using a needle to aspirate bodily fluids from the chest cavity, bladder, spinal canal, or other suitable area. Once collected, the cell sample is typically placed in a preservative solution and then extracted from the solution and transferred to a glass slide. To facilitate subsequent staining and examination, a fixative is applied to the cell sample to ensure that the cells are fixed to the slide.

[0006] Generally, it is desirable for cells on a slide to have an appropriate spatial distribution so that individual cells can be examined. A monolayer of cells is usually preferred. Therefore, preparing a cytological specimen from a liquid sample containing many cells (e.g., tens of thousands) typically requires that the cells be first separated from one another by mechanical dispersion, liquid shearing, or other techniques so that a thin monolayer of cells can be collected and deposited on the slide. In this way, the cytotechnologist can more easily identify the presence of any abnormal cells in the patient sample. Cells can also be counted to ensure that an appropriate number of cells have been evaluated.

[0007] Specific methods and apparatus for producing a thin monolayer of cells from a liquid sample container and then transferring this thin layer to a "specimen slide" suitable for visual inspection are disclosed in U.S. Patent Nos. 5,143,627, 5,240,606, 5,269,918, 5,282,978, 6,562,299, 6,572,824, and 7,579,190. In one method disclosed in these patents, patient cells, suspended in a preservative solution and stored in a sample container, are dispersed by a rotating sample collector inserted into the container. A controlled vacuum is applied to the sample collector, drawing the liquid through the filter until the desired amount and spatial distribution of cells are captured on the screen filter of the sample collector. The sample collector is then removed from the sample container, and the filter is pressed against a glass slide, transferring the collected cells to the slide in approximately the same spatial distribution as when they were collected. Devices manufactured according to the teachings of one or more of these patents have been commercially successful, such as the ThinPrep® 2000 Processor (for processing single specimen slides from patient samples) and the ThinPrep® 5000 Processor (for processing batches of specimen slides from patient samples), manufactured and sold by Hologic, Inc., based in Marlborough, Massachusetts. See also U.S. Patent Nos. 7,556,777 and 7,771,662.

[0008] Once the specimen slide is prepared, the specimen may be visually inspected by a cytotechnologist, typically under magnification, with or without various illumination sources. Additionally or alternatively, automated slide imaging systems are also used to assist the cytotechnologist in the cytologic examination process. For example, automated slide imaging systems capture images of all or substantially all of the individual cells within a cytological specimen mounted on a slide, and perform a preliminary evaluation of the cells using image processing techniques, guiding the cytotechnologist to the potentially most relevant cells on the slide for further examination. Examples of such imaging systems are disclosed in U.S. Patent Nos. 7,587,078, 6,665,060, 7,006,674, 7,369,304, and 7,590,492. Whether examining the actual specimen slide under magnification or a magnified image of the specimen, the specimen is typically classified by the cytotechnologist as either "normal" or "abnormal," with abnormal samples typically falling into one of the major categories defined in the Bethesda System for Reporting Cervical / Vaginal Cytological Diagnosis.

[0009] However, conventional systems and methods for acquiring digital images of biological specimens have many drawbacks. For example, conventional systems and methods require time to stop and focus as the entire specimen is scanned, resulting in slow image capture times. Furthermore, conventional systems and methods that do not stop to focus typically provide only a single focal plane across the entire specimen. Biological specimens, both cytological and pathological, are three-dimensional in nature (i.e., have depth). Therefore, the high magnification and focal length required to acquire a digital image of a biological specimen results in a very limited depth of field for the image. As a result, portions of the specimen outside the depth of field within the focal plane are out of focus or are not visible in the image. To obtain focused digital images at multiple depths of the specimen, the focal plane must be adjusted by moving the specimen or the camera or by adjusting the focusing lens. However, this requires additional scanning of the specimen for each focal plane or requires stopping periodically to refocus, further increasing acquisition time.

[0010] Many of the aforementioned problems of prior art imaging systems are addressed and solved by the imaging system and method disclosed and described in PCT Publication WO 2020 / 091965(A2) (International Application PCT / US19 / 55458, filed October 9, 2019). Key aspects of the imaging system and method disclosed in PCT Publication WO 2020 / 091965(A2), as described and discussed below, include the ability to capture images of cells at different depths within a focused specimen using a single scanning pass of a camera with an objective lens tilted relative to the slide surface to capture images within a focal range that encompasses the entire thickness of the specimen. However, some specimen slides present specific challenges not addressed in PCT Publication WO 2020 / 091965(A2). For example, uneven coverslips or other aspects of initial slide preparation can result in uneven specimen thickness, i.e., specimen height relative to the slide surface. In this case, a portion (perhaps a significant amount) of the cells in the specimen will be out of focus. Therefore, further improvements to the imaging systems and techniques disclosed in WO 2020 / 091965 A2 would be useful in solving this out-of-focus cell problem. Summary of the Invention

[0011] According to a first aspect of the disclosed invention, a method is disclosed for acquiring images of objects distributed within a specimen attached to a surface of a slide using a camera having an objective lens with an optical axis that forms a non-orthogonal angle with the surface of the slide. The specimen has a non-uniform height relative to the slide surface, the method including: (i) acquiring a first plurality of images of a first linear portion of the specimen; (ii) evaluating the focus of the objects within the linear portion of the specimen imaged in the first plurality of images; and (iii) acquiring a second plurality of images of the first linear portion or a second linear portion of the specimen different from the first linear portion, wherein during acquisition of the second plurality of images, the height of the objective lens relative to the slide surface is varied based on the evaluated focus of the objects imaged in the first plurality of images. Optionally, during acquisition of the first plurality of images, the height of the objective lens relative to the slide surface is approximately constant. Optionally, the second linear portion is directly adjacent to the first linear portion.

[0012] In various embodiments, evaluating the focus of the objects captured in the first plurality of images includes determining whether a total number of out-of-focus objects exceeds a threshold number, and either (i) acquiring a second plurality of images from the first linear portion if the total number of out-of-focus objects in the first plurality of images exceeds the threshold number, or (ii) acquiring a second plurality of images from the second linear portion if the total number of out-of-focus objects in the first plurality of images does not exceed the threshold number.

[0013] In various embodiments, assessing the focus of the objects imaged in the first plurality of images includes determining a height of each of the out-of-focus objects relative to the slide surface and determining whether, during acquisition of the first plurality of images, each out-of-focus object was located at a height relative to the slide surface that was outside the focus range of the objective lens. Preferably, assessing the focus of the objects in the first plurality of images includes determining whether, during acquisition of the first plurality of images, each out-of-focus object was located at a height relative to the slide surface that was higher than a maximum height or lower than a minimum height of the focus range of the objective lens.

[0014] In various embodiments, assessing the focus of the objects in the first plurality of images includes determining a position of each of the out-of-focus objects within the first linear portion.

[0015] In various embodiments, one or both of the camera and the slide are moved relative to the other during image capture, and the height of the objective lens relative to the slide surface is varied by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the linear position of the camera relative to the longitudinal position of each of the first or second linear portions. In particular, the height of the objective lens relative to the slide surface may be varied during image capture by moving the slide perpendicular to the camera, or by moving the camera perpendicular to the slide, or both.

[0016] In various embodiments, when a second plurality of images is acquired from the first linear portion, the method further includes evaluating a focus of the object imaged in the second plurality of images and acquiring a third plurality of images of the second linear portion, wherein during acquisition of the third plurality of images, the height of the objective lens relative to the slide surface is changed based on the evaluated focus of the object imaged in the second plurality of images.

[0017] According to another aspect of the invention disclosed herein, a method for acquiring images of objects distributed within a specimen affixed to a surface of a slide using a camera having an objective lens with an optical axis that forms a non-orthogonal angle with the surface of the slide, the specimen having a non-uniform height relative to the slide surface, the method comprising: (a) acquiring a first plurality of images of a linear portion of the specimen; (b) assessing the focus of the objects imaged in the first plurality of images; (c) acquiring a second plurality of images of the same or a different linear portion of the specimen, varying the height of the objective lens relative to the slide surface during acquisition of the second plurality of images based on the assessed focus of the objects imaged in the first plurality of images; and (d) repeating steps (a)-(c) until images of substantially the entire specimen have been acquired. If the second plurality of images are acquired from a different linear portion of the specimen, such different linear portion may be immediately adjacent to the linear portion from which the first plurality of images were acquired.

[0018] In various embodiments, assessing the focus of the objects imaged in the first plurality of images may include determining whether the total number of out-of-focus objects exceeds a threshold number.

[0019] In a preferred embodiment, evaluating the focus of the objects in the first plurality of images includes determining whether, during acquisition of the first plurality of images, each out-of-focus object is located at a height relative to the slide surface that is higher than the maximum height or lower than the minimum height of the objective lens's focus range, and determining each position within a linear portion of the out-of-focus object.

[0020] In a preferred embodiment, during acquisition of each of the first and second pluralities of images, one or both of the camera and the slide are moved laterally relative to the other, and the height of the objective lens relative to the slide surface is varied by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the linear position of the camera relative to the longitudinal position of each linear portion. Again, the height of the objective lens relative to the slide surface can be varied by moving the slide vertically relative to the camera during image acquisition, or by moving the camera vertically relative to the slide, or both.

[0021] According to a further aspect of the disclosed invention, there is provided a system for acquiring images of objects distributed within a specimen attached to a surface of a slide, the specimen having a non-uniform height relative to the slide surface, the system including a camera having an objective lens with an optical axis positioned such that the optical axis forms a non-orthogonal angle with the surface of the slide, the system further including an image processor operatively connected to the camera, the image processor configured to receive a first plurality of images of a first linear portion of the specimen acquired by the camera, evaluate focus of objects within the linear portion of the specimen imaged in the first plurality of images, and cause the camera to acquire a second plurality of images from the first linear portion or a second linear portion of the specimen different from the first linear portion, and vary the height of the objective lens relative to the slide surface during acquisition of the second plurality of images based on the evaluated focus of the objects imaged in the first plurality of images.

[0022] In one embodiment, the height of the objective lens relative to the slide surface remains approximately constant during acquisition of the first plurality of images.

[0023] In one embodiment, the image processor evaluates the focus of the objects captured in the first plurality of images by at least partially determining whether a total number of out-of-focus objects exceeds a threshold number.

[0024] In one embodiment, the image processor evaluates the focus of the objects captured in the first plurality of images by at least partially determining the height of each of the out-of-focus objects relative to the slide surface.

[0025] In one embodiment, the image processor evaluates the focus of the objects in the first plurality of images by, at least in part, determining whether each out-of-focus object is located at a height relative to the slide surface that is outside the focus range of the objective lens during acquisition of the first plurality of images.

[0026] In one embodiment, the image processor evaluates the focus of the objects in the first plurality of images by at least partially determining whether each out-of-focus object is located at a height relative to the slide surface that is higher than the maximum height or lower than the minimum height of the objective lens's focus range during acquisition of the first plurality of images.

[0027] In one embodiment, the image processor assesses the focus of the objects in the first plurality of images by at least partially determining the position of each out-of-focus object within the first linear portion.

[0028] In various embodiments, preferably, one or both of the camera and slide are configured to move laterally relative to the other during image capture.

[0029] In one non-limiting embodiment, the height of the objective lens relative to the slide surface is varied by increasing and / or decreasing the height of the slide surface relative to the camera as a function of the linear position of the camera relative to the longitudinal position of each linear segment. In the same or another embodiment, the height of the objective lens relative to the slide surface is varied by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the linear position of the camera relative to the longitudinal position of each of the first or second linear segments.

[0030] The second plurality of images may be acquired from the first or second linear portion. In one embodiment, the second plurality of images is acquired from the first linear portion, and the image processor is further configured to evaluate the focus of the object imaged in the second plurality of images and cause the camera to acquire a third plurality of images of the second linear portion, wherein during acquisition of the third plurality of images, the height of the objective lens relative to the slide surface is varied based on the evaluated focus of the object imaged in the second plurality of images. Without limitation, the second linear portion may be directly adjacent to the first linear portion.

[0031] Other and further aspects and features of the disclosed embodiments will become apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0032] The foregoing and other aspects of the embodiments will be described in further detail with reference to the accompanying drawings, in which like reference numerals refer to like elements and the descriptions of like elements apply to all relevant embodiments described. [Brief explanation of the drawings]

[0033] [Figure 1] FIG. 1 shows a specimen slide used in an automated digital imaging system.

[0034] [Figure 2] FIG. 2 is a cross-sectional view of the specimen area of ​​the slide of FIG.

[0035] [Figure 3] Figure 3 is a table of the results of the relative height survey of cell contents.

[0036] [Figure 4A] Figure 4A is a 3D focus map showing the relative height of cellular content across the specimen. [Figure 4B] Figure 4B is a heat map showing the relative height of the cellular content across the specimen.

[0037] [Figure 5] FIG. 5 is a schematic diagram illustrating a serpentine scan pattern used by a digital imaging device to scan a specimen slide, according to one embodiment.

[0038] [Figure 6] FIG. 6 is a schematic diagram for explaining the tilt angle of the imaging device with respect to the slide.

[0039] [Figure 7] FIG. 7 is a graph illustrating the focus merge operation.

[0040] [Figure 8A] FIG. 8A is a cross-sectional view of a specimen area showing the cell path, the area covered by the volume scan, and the area covered by the Z-curve following volume scan. [Figure 8B] FIG. 8B is a cross-sectional view of the specimen area showing the cell path, the area covered by the volume scan, and the area covered by the Z-curve following volume scan. [Figure 8C] FIG. 8C is a cross-sectional view of the specimen area showing the cell path, the area covered by the volume scan, and the area covered by the Z-curve following volume scan.

[0041] [Figure 9A] FIG. 9A is an image of the specimen acquired without Z-curve tracking. [Figure 9B] FIG. 9B is an image of the specimen acquired with Z-curve tracking.

[0042] [Figure 10] FIG. 10 is a flow diagram of a method for acquiring an image of a specimen by Z-curve following. DETAILED DESCRIPTION OF THE INVENTION

[0043] The emergence of digital whole slide imaging (WSI) systems is revolutionizing the fields of pathology and cytology. The ability to rapidly acquire high-quality whole slide images will be an essential step for successful clinical workflow, especially in high-volume screening applications such as Pap tests. Liquid-based cytology slides appear visually nearly monolayered to the laboratory technician, but cytology is inherently three-dimensional. These slides can present challenges for WSI, as the depth of focus of closely apposed material can be orders of magnitude greater than the depth of field (DOF) of a high-magnification microscope objective. Therefore, cytology slides are more challenging to image than histology slides. Additionally, slides with film coverslips can add scanning depth requirements due to the curvature of the cell spot area on the slide. Therefore, the distance between the target object in a cytology specimen and the glass slide on which the specimen is attached can vary significantly relative to the DOF of the microscope objective. Many current WSI systems require repeated scans to cover multiple focal planes to acquire high-quality images, significantly increasing imaging time. As a result, it is difficult to efficiently obtain high-quality, focused images of cytology slides. Disclosed herein are systems and methods for efficiently scanning such slides to obtain high-quality, focused images.

[0044] Referring to FIG. 1, an exemplary embodiment of a microscope slide 102 is shown. The microscope slide 102 is a rectangular glass plate 110 (or other suitable material) having a slide identification area 112, a specimen area 114, and a fiducial mark 116. The microscope slide 102 may be a standard-sized microscope slide, approximately 75 mm by 25 mm, or may be any other suitable size. The microscope slide 102 may have chamfered corners to facilitate handling and positioning of the slide 102. The specimen area 114 may be a circle with a diameter of up to approximately 22 mm. The entire specimen area 114 on the microscope slide 102 may be imaged. The slide identification area 112 may be up to approximately 25-28 mm in length. The slide identification area 112 may be printed with a bar code, ID number, and / or other information. The specimen area 114 remains as a transparent area on the glass slide 102. The fiducial marks 116 may be used by the imaging device as reference points on the slide 102 to determine the position and / or orientation of the slide 102 and its features relative to the imaging device. A specimen 119, including multiple objects distributed within a three-dimensional volume, is affixed to the slide 102, typically within the specimen region 114, although in some cases the specimen may extend beyond the specimen region 114. The three-dimensional volume of the specimen 119 has a length (l), a width (w), and a thickness or depth (d). The thickness (d) defines a z-axis relative to the plane of the slide 102. The specimen 119 may be any suitable specimen, such as a cytology specimen, in which the objects are cells, or a solid tissue specimen, in which the objects are tissue structures.

[0045] As shown in Figure 1, a coverslip 115 may be used to cover the specimen 119 in the specimen region 114. The specimen coverslip 115 is sufficiently transparent to allow an image of the specimen 119 to be acquired through the coverslip 115; that is, the coverslip 115 does not prevent an imaging device from acquiring an image through the coverslip 115. The coverslip 115 functions to preserve the specimen 119, protect it from contamination and from contaminating other objects, and keep the specimen 119 flat and in place. The coverslip 115 has a thickness 117.

[0046] As shown in FIG. 2 , a specimen 119, positioned between a glass slide 110 and a coverslip 115, has a thickness 120. Cells 122 are distributed throughout the specimen layer at various depths, i.e., along the z-axis. The specimen may have a thickness throughout the specimen that exceeds the depth of field of the optical system used to capture images of objects (e.g., cells 122) within the specimen. This is particularly true for liquid-based cytology specimens, where individual cells are collected. The coverslip 115, which may be made of glass or plastic and is attached to the slide 110 with a thin layer of adhesive, is placed over the specimen 119. Cells have been observed to tend to float within the adhesive, lifting above the glass 110. Furthermore, the coverslip 115 is not always perfectly flat and often exhibits undulations, hills, and valleys. This results in uneven adhesive thickness, which in turn varies the distance of cells from the glass substrate 110 and from the objective lens, sometimes exceeding the depth of field. Disclosed herein is a digital imaging method and system for efficiently scanning cytology slides that solves the problem of digitizing specimens that are thicker than the depth of field of the objective lens in a given field of view.

[0047] Cytology slides are three-dimensional in nature, with cells suspended in the medium and potentially stacked on top of each other. Microscope objectives have a very small depth of field (DOF), making it impossible to capture all cells in focus in a single image. In fact, individual cells may be thicker than a single DOF. A 40x microscope objective with a 0.75 NA (numerical aperture) has a depth of field of less than 2 microns (μm). Cytology slides with flexible film (plastic) coverslips may require a deeper scanning (i.e., image acquisition) depth due to the curved nature of the entire cell spot area. A wider range of focus is required to capture all cells on a cytology slide (e.g., ThinPrep slide) with high image quality.

[0048] As an example, 23 ThinPrep Pap slides were scanned using a computer-controlled microscope with a digital camera (Hologic ThinPrep Integrated Imager) to collect cell preparation depth data. First, the cell spot area was scanned using a computer-driven XY stage. At each position, a stack of images was taken over a wide range of Z-heights (>40 μm). All images were divided into small regions (35 μm squares) and evaluated using the Brenner focus score criteria for all levels of the Z-stack. The optimal focus for that tile was determined. By focusing on fiducial marks printed on the slide, the entire slide plane was determined and subtracted from the focus data to determine the relative height of the cellular contents.

[0049] The table in Figure 3 summarizes the data obtained from these 23 slides. From this table, we can see that the average cell depth for ThinPrep slides was 11.09 μm for glass-coverslip slides and 23.6 μm for film-coverslip slides. In some cases, cell depths could exceed 40 μm. Note that even with glass coverslips, local variations in cell height within a slide can be as much as seven times the depth of field of the microscope objective. Surface plot focus maps were created and examined for each slide. Figures 4A and 4B show an example of a film-coverslip slide, demonstrating the effect of curvature across the range of cell spot areas. Notably, in the example shown in Figures 4A and 4B, cells in the center of the specimen are closer to the glass slide, while cells around the edges of the specimen are closer to the coverslip.

[0050] As shown in FIG. 5 , during imaging, the slide 102 is moved by an XY slide stage to scan the camera's field of view across the entire specimen area 114 (or a predetermined area across the entire specimen 119, for example, if the actual specimen 119 covers an area different from the specimen area 114 and the boundaries of the actual specimen 119 are predetermined). The XY slide stage moves the slide 102 back and forth in a serpentine path, capturing tiny images of a swath, row, or linear portion of the specimen 119 along each path. To capture a swath, the XY slide stage continuously moves the slide 102, triggering the camera to capture an image at each trigger point according to the stage's encoder position. A very high-speed camera is used to ensure continuous relative motion between the slide and the camera. This high-speed image capture along a linear portion of the slide may be referred to as "scanning" the row. To minimize the time required to scan the entire specimen 119, a serpentine path is used, with the beginning of each successive swath approaching the end of the previous swath. As the slide 102 is moved along the swath, the camera captures small images of the specimen 119. That is, the imaging device captures multiple images covering the entire specimen area according to a scan pattern such as that shown in FIG. 5. The scan pattern includes multiple linear segments, represented by horizontal arrows 130 in FIG. 5. The specimen is divided into multiple focal zones, represented by squares 132 in FIG. 5. Thus, each linear segment 130 along which an image is acquired includes multiple focal zones 132. Each linear segment 130 may include between 30 and 70 focal zones. In another example, rather than moving the slide 102, the slide 102 remains stationary while the camera moves along the scan pattern. Whether the slide 102 or the camera moves, the slide 102 moves relative to the camera.

[0051] Most current WSI systems scan a single focal plane at a time. A typical scanner can complete a 15 × 15 mm scan in 1 minute. At this speed, scanning a circular ThinPrep cell spot area to a depth of 14 focal planes would take at least 26 minutes. To dramatically increase the throughput of slide digitizers, some systems are designed with an inclined angle between the objective and the glass slide. This method takes advantage of the depth of field to simultaneously digitize different layers at slightly different depths. Then, by selecting the most in-focus layer (or a portion of a layer) and stitching the in-focus areas together into a single, in-focus composite layer, a process known as focus merging, the layers can be moved continuously under the objective without stopping to adjust the focus. This allows for a system that digitizes and merges various layers, essentially performing post-acquisition focusing. The only limitation of such systems is the depth of field of the objective, which translates to the maximum specimen thickness that can be imaged. A clear advantage of such systems is that they do not require stopping and focusing.

[0052] Using tilted plane volumetric scanning, acquisition time to scan the entire cellular content area is significantly reduced. A ThinPrep Pap slide can be completed in approximately 2.5 minutes. As shown in FIG. 6, the imaging optics and camera 202 are tilted relative to the slide 102. Regions of the image at one edge of the camera frame are imaged closer to the glass slide than regions at the other edge of the camera frame. In one example, a tilt angle of 48 milliradians and an image frame width for a slide of 0.5 mm provides a scan depth of 0.5 × sin(0.048), or 24 microns (μm).

[0053] As best shown in FIG. 6 , the optical axis 204 of the objective lens is tilted at a tilt angle 206 with respect to a normal to the scanning direction in the plane of the slide 102 (see FIG. 5 ). This means that the optical axis 204 of the camera and optics in the plane of the slide is non-orthogonal to the plane of the slide 102. As explained in more detail in WO 2020 / 091965 A2, the tilt angle 206 enables the imaging device to acquire volumetric images of the specimen 119 on the slide 102 (i.e., images that span the depth of the specimen 119). That is, the microimage includes focused images of features at different depths of the specimen 119 on the slide 102, rather than just a single focal plane as in the case of an image captured at an orthogonal angle of the specimen 119. The imaging station may be configured to acquire microimages, each of which includes at least a portion of the slide 102 depth below the plane of the slide 102. If a coverslip 115 is used on the slide 102 , the imaging station may be configured to acquire micro-images, each of which includes at least a portion of the depth of the coverslip 115 .

[0054] As the camera moves continuously, a new image is triggered every time the camera moves 1 / 14 of its own width. A very high-speed camera (>100 fps) is used. These overlapping images can be sliced ​​and reconstructed to obtain 14 focal plane images, as shown in Figure 7 and described in more detail in WO 2020 / 091965. To optimize storage space, the focal planes are combined into a single extended depth-of-field image by selecting in-focus pixels from the various planes. Image processing is performed in real time using GPU hardware acceleration.

[0055] While coverslips are not perfectly flat, the variations in flatness are essentially gradual. This means that the coverslip may form a ridge in the center of the slide, or a ridge may form from multiple waves across the surface of the slide, but there are no abrupt changes in the distance between the coverslip and the glass underneath. For example, a perfectly flat coverslip 115 is shown in Figure 8A, while a coverslip with a wavy or uneven surface is shown in Figures 8B and 8C.

[0056] This gradual change in gap allows imaging systems that focus after capture to achieve a much greater effective depth of field than that achievable by optics alone. Essentially, as a slide is scanned (as shown in Figure 5), typically using a raster back-and-forth pattern, at the end of each pass (before reversing the scan direction to examine the next row or swath), the collected layers are analyzed to determine the best focus during the focus merge operation. As the biological specimen rises, following the undulations of the coverslip, the best-focused layer in the previous swath can be used to predict the best focus pattern for the next swath to be scanned. During focus merge, individual objects of interest are found and their depths are recorded. Each column is then used to predict the best focus for the next row.

[0057] When an out-of-focus region is detected at the edge of a swath, the swath can be rescanned at a different height that brings the out-of-focus region into focus. The resulting focus height is used to inform the next swath, thereby minimizing the number of times the swath needs to be rescanned.

[0058] The advantage of such a system is that it can achieve higher throughput (shorter scan times) because it does not need to stop to focus and rarely needs to return to out-of-focus regions to rescan. Instead, the system simply scans back and forth until the entire specimen is digitized (focus merge). During the traverse, the objective lens is driven up and down according to a focus map provided by the previous swath, designed to keep the object within the depth of field so that it remains in focus even as its distance from the slide surface changes.

[0059] To cover a larger total cell depth range, such as that found on a slide with a film coverslip, the imaging optics can be driven in the Z-axis to follow the curvature, as shown in Figures 8A-8C. The cell path 802 is undulating and generally tilted. A volumetric scan using a tilted camera, as described above, provides a thicker scan area 804 than a scan performed at a single focal depth, but cannot provide in-focus images of the bottom 806 and top 808 of the cell path 802. A Z-curve-following volumetric scan 810 images the entire cell path 802 of the specimen. While local cell depths within the camera's field of view fit within the scanning depth of the tilted surface, greater variations in focal depth may be required at longer distances.

[0060] The curve-following scanning method minimizes local focus errors and provides higher quality WSI images. The slide used to generate the images in Figures 9A and 9B has a Z focal depth of over 40 microns (μm). A volume scan Z without Z-curve following is shown in Figure 9A. A volume scan with Z-curve following is shown in Figure 9B. The image in Figure 9B is sharper and more focused than the image in Figure 9A.

[0061] Referring now to FIG. 10 , a method 500 for acquiring images of objects distributed within a specimen attached to the surface of a slide will be described. In particular, images are acquired using a Z-curve volumetric scan of a specimen at a non-uniform height relative to the surface of the slide and with a thickness exceeding the depth of field of the optical system. The first step 502 of method 500 involves sampling discrete locations on the specimen slide to determine an initial focal height (Z). Next, in step 504, for each focal zone in the first linear portion, several images are acquired along the first linear portion of the specimen using the initial focal height determined in step 502. The initial focal height remains constant along the first linear portion, meaning that the height of the objective lens relative to the slide surface is approximately constant during acquisition of images of the first linear portion of the specimen in step 504. Each row (linear portion) of the specimen includes multiple focal zones.

[0062] For example, referring to FIG. 5, each square along a linear segment may be a focal zone. Each linear segment scanned by the imaging device may include, for example, 30 to 70 focal zones. Alternatively, rather than using a substantially constant focal height as in step 504, the discrete positions sampled in step 502 may be used to determine a z-curve for the first linear segment, and images along the first linear segment may be acquired by following that z-curve. In this manner, the objective lens follows the z-curve during image acquisition along the first linear segment, moving up and down along the z-axis relative to the slide.

[0063] Next, in step 506, optimal focus is determined by evaluating whether the imaged object is in focus for each focal zone in the linear portion just scanned. This evaluation includes determining whether out-of-focus objects exist within the specimen at heights relative to the slide surface that are outside the objective's focus range. The evaluation may also include identifying out-of-focus objects based on their relative position along the linear portion of the specimen. That is, images are evaluated for each focal zone along the linear portion to determine which focal plane is most in focus. As shown in FIG. 7 , acquiring oblique-angle images captures images at 12 focal planes at a time. For example, if optimal focus is found to be at the top or bottom focal plane (focal plane 1 or 12), it may be necessary to move the z-axis position of the objective lens relative to the slide up or down, respectively, to obtain an image with better focus. Furthermore, if optimal focus is found to be at the top or bottom focal plane, it may be necessary to move the z-axis position of the objective lens relative to the slide up or down, respectively, during image acquisition of the next adjacent linear portion of the specimen.

[0064] If too many objects are out of focus (i.e., the number of out-of-focus objects exceeds a predetermined threshold number), the focal height (Z) of each of the out-of-focus objects is adjusted, and the linear portion is rescanned according to the new focal height curve in step 508. As the camera follows the focal height curve, the z-axis position of the objective lens relative to the slide surface is varied by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the camera's relative position along the linear portion of the specimen. The z-axis position of the objective lens relative to the slide may vary from one focal zone to the next along the linear portion of the specimen. In one embodiment, the slide 110 moves up and down while the z-axis position of the objective lens remains stationary. In an alternative embodiment, the objective lens moves up and down while the z-axis position of the slide remains constant.

[0065] If most or all of the focal zones are in focus, then in step 510, the next value for each focal zone is calculated based on the best focus for each focal zone in the current row. Some of the focal zones in the next row may be moved up to track objects of interest closer to the coverslip (i.e., z-position closer to the coverslip and farther from the slide), while some of the focal zones in the next row may be moved down to track objects of interest closer to the slide. Next, in step 512, the next linear segment is scanned using the focus curve calculated in step 510. Steps 506-512 are repeated for each linear segment within the specimen until an image of the entire specimen has been acquired. That is, after an image is acquired for each linear segment of the specimen, the focus of the objects in those images is evaluated (step 506). Based on the evaluation, the focus curve for the next linear segment of the specimen is determined (step 510) and used to acquire an image of the object in the next linear segment immediately adjacent to the linear segment just scanned (step 512). Alternatively, if the focus evaluation reveals that there are too many out-of-focus objects, the focus curve is adjusted and the same straight line segment is scanned again using the adjusted focus curve (step 508).

[0066] While particular embodiments have been shown and described, it should be understood that the above description is not intended to limit the scope of these embodiments, and that such disclosure is provided for purposes of illustration and example only. Accordingly, various changes and modifications can be made to the disclosed embodiments without departing from the scope of the following claims.

Claims

1. 1. A method of acquiring an image of a specimen fixed to a surface of a slide using a camera having an objective lens with an optical axis at a non-orthogonal angle to the surface of the slide, the specimen having a non-uniform height relative to the surface of the slide, the method comprising: acquiring a first plurality of images of a first swath of the specimen fixed to the surface of the slide using the camera having an objective lens with an optical axis at a non-orthogonal angle to the surface of the slide, the first plurality of images being volumetric images spanning the depth of the specimen; Dividing the first swath into a plurality of focal zones; processing the first plurality of images and reconstructing the first plurality of images into a first plurality of focal plane images; evaluating each focal zone of the first swath to determine which focal plane within each focal zone has an optimal focus of an analyte contained in each focal zone; calculating a Z focus height curve along the first swath based on each of the focal planes determined to have an optimal focus of the specimen contained within each of the focal zones of the first swath; using the camera to (i) acquire a second plurality of images of the first swath, or (ii) acquire a second plurality of images of a second swath of the specimen different from the first swath, wherein a height of the objective lens relative to a plane of the slide is varied during acquisition of the second plurality of images based on the Z focus height curve; A method comprising:

2. assessing focus of objects in the first plurality of images and determining whether a total number of out-of-focus objects exceeds a threshold number; if the total number of out-of-focus objects in the first plurality of images exceeds the threshold number, the second plurality of images is a re-image of the first swath of the specimen; if the total number of out-of-focus objects in the first plurality of images does not exceed the threshold number, the second plurality of images are images of a second swath immediately adjacent to the first swath; The method of claim 1 further comprising:

3. The method described in claim 1, wherein the first plurality of images and the second plurality of images are obtained by continuously moving the slide along each region of the specimen, and the camera is triggered at discrete intervals to capture individual images comprising the first plurality of images and the second plurality of images, such that the movement of the slide relative to the camera is continuous.

4. The method described in claim 1, wherein the height of the objective lens relative to the surface of the slide is substantially constant during acquisition of the first plurality of images.

5. A method described in any one of claims 1 to 4, wherein evaluating each focal zone to determine which focal plane within the focal zone has the best focus for the specimen includes determining the height of each out-of-focus object relative to the plane of the slide.

6. The method of claim 2, wherein evaluating each focal zone to determine which focal plane in each focal zone has the best focus for the specimen includes determining whether each of the out-of-focus objects is at a height position relative to the surface of the slide that is outside the focal range of the objective lens during acquisition of the first plurality of images.

7. The method of claim 2, wherein evaluating each focal zone to determine which focal plane in each focal zone has the best focus of the specimen includes determining whether each of the out-of-focus objects is located at a height relative to the surface of the slide that is higher than a maximum height or lower than a minimum height in the focal range of the objective lens during acquisition of the first plurality of images.

8. The method described in claim 2, wherein evaluating the focus of the object in the first plurality of images includes determining the position of each out-of-focus object within the first swath.

9. A method described in any one of claims 1 to 4, wherein one or both of the camera and slide move laterally relative to the other during acquisition of the image.

10. A method as described in claim 9, wherein the height of the objective lens relative to the surface of the slide is varied by increasing and / or decreasing the height of the surface of the slide relative to the camera as a function of the linear position of the camera relative to the longitudinal position of each of the linear portions.

11. A method as described in claim 9, wherein the height of the objective lens relative to the surface of the slide is varied by increasing and / or decreasing the height of the surface of the slide relative to the camera as a function of the linear position of the camera relative to the longitudinal position of each of the first or second linear portions.

12. The method described in claim 1, wherein the second plurality of images are acquired for the first swath.

13. The second plurality of images are obtained from the second swath, and the method further comprises: Dividing the second swath into a plurality of focal zones; processing the second plurality of images and reconstructing the second plurality of images into a second plurality of focal plane images; evaluating each focal zone of the second swath to determine which focal plane within each focal zone has an optimal focus of the analytes contained in each focal zone; calculating a second Z focus height curve along the second swath based on each of the focal planes determined to have an optimal focus of the specimen contained within each of the focal zones of the second swath; using the camera to (i) acquire a third plurality of images of the second swath, or (ii) acquire a third plurality of images of a third region of the specimen different from the first swath and the second swath, wherein a height of the objective lens relative to a surface of the slide is varied during acquisition of the third plurality of images based on the second Z focal height curve; The method of claim 1 further comprising:

14. A method as described in claim 12 or 13, wherein the second swath is directly adjacent to the first swath.

15. The method of claim 1, wherein the object comprises a cell.

16. The method of claim 1, wherein the object comprises a tissue sample.

17. A system for acquiring images of objects distributed within a specimen fixed to a surface of a slide, the specimen having a non-uniform height relative to the surface of the slide, the system comprising: a camera having an objective lens with an optical axis, the camera being positioned such that the optical axis forms a non-orthogonal angle with respect to the plane of the slide; an image processor operatively connected to the camera; Equipped with The system is configured to perform the method of any one of claims 1 to 16.

Images