Digital imaging system and method

Abstract

A system and method for acquiring images of objects distributed within a specimen attached to the surface of a slide, the system comprising: acquiring a first plurality of images of a first linear portion of the specimen from the specimen having a non-uniform height relative to the slide surface using a camera having an objective lens with an optical axis that forms a non-orthogonal angle with the surface of the slide; evaluating the focus of the objects within the linear portion of the specimen imaged in the first plurality of images; and acquiring a second plurality of images of the first linear portion or a second linear portion of the specimen different from the first linear portion, wherein during acquisition of the second plurality of images, the height of the objective lens relative to the slide surface is changed based on the evaluated focus of the objects imaged in the first plurality of images.

Description

Technical Field

[0001] This application claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application No. 62 / 940,163, filed on November 25, 2019, and the content thereof is incorporated herein by reference.

[0002] The present disclosure generally relates to systems and methods for acquiring digital images of specimens affixed to the surface of a slide, such as cytological (cellular) specimens having a thickness exceeding the depth of field of an imaging device.

[0003] All U.S. and PCT patents and patent publications identified herein for any purpose are hereby incorporated by reference in their entirety.

Background Art

[0004] Cytology is a branch of biology that studies the formation, structure, and function of cells. As an application in a laboratory setting, cytologists, cytotechnologists, and other medical professionals perform medical diagnoses of a patient's condition based on a visual examination of the patient's cell sample. Such samples are referred to herein as "cytological" specimens. A typical cytological technique is the "Pap smear" test, in which cells are scraped from a woman's cervix and analyzed to detect the presence of abnormal cells that are precursors to the development of cervical cancer. Cytological techniques are also used to detect abnormal cells and diseases in other parts of the human body.

[0005] Cytological techniques are widely adopted because collecting cell specimens for analysis is generally less invasive compared to traditional surgical pathological procedures such as biopsies where solid tissue samples, referred to herein as "pathological" specimens, are excised from patients using special biopsy needles with spring-loaded translatable stylets, fixed cannulas, etc. Cell samples can be collected from patients by various techniques including, for example, scraping or swabbing an area, or aspirating body fluids from the chest cavity, bladder, spinal canal, or other appropriate areas using a needle. The collected cell samples are typically placed in a preservation fluid and subsequently extracted from the preservation fluid and transferred to a slide glass. To facilitate subsequent staining and examination, a fixative is applied to the cell sample to ensure that the cells are securely fixed on the slide glass.

[0006] Generally, it is desirable for the cells on the slide to have an appropriate spatial distribution so that individual cells can be examined. Usually, a single layer of cells is preferred. Thus, to prepare a cytological specimen from a liquid sample containing many cells (e.g., tens of thousands), it is typically necessary to first separate the cells from each other by mechanical dispersion, liquid shear, or other techniques and recover a thin single layer of cells that can be deposited on a slide. By doing so, the cytotechnologist can more easily identify the presence of any abnormal cells in the patient sample. Also, the cells can be counted to confirm that an appropriate number of cells have been evaluated.

[0007] Specific methods and apparatuses for creating a thin monolayer of cells from a liquid sample container and then transferring this thin layer to a "specimen slide" that is advantageous for visual inspection are disclosed in U.S. Patent Nos. 5,143,627; 5,240,606; 5,269,918; 5,282,978; 6,562,299; 6,572,824; and 7,579,190. One of the methods disclosed in these patents involves dispersing the patient's cells stored in the sample container, suspended in a preservation solution, with a rotary sample collector inserted into the container. A controlled vacuum is applied to the sample collector, and the liquid is suctioned through the filter until the desired amount and spatial distribution of cells are collected on the screen filter of the sample collector. The sample collector is then removed from the sample container, and the filter portion is pressed against a slide glass to transfer the collected cells to the slide glass in substantially the same spatial distribution as when they were collected. Apparatuses manufactured in accordance with the teachings of one or more of these patents, such as the ThinPrep® 2000 Processor (specimen slides processed one by one from patient samples) and the ThinPrep® 5000 Processor (specimen slides batch processed from patient samples), which are manufactured and sold by Hologic, Inc., based in Marlborough, Massachusetts, have achieved commercial success. See also U.S. Patent Nos. 7,556,777 and 7,771,662.

[0008] Once a specimen slide is prepared, the specimen can typically be visually inspected by a cytotechnologist under magnification and with or without various light sources. Additionally or alternatively, automated slide imaging systems that assist in the cytology process are also used. For example, an automated slide imaging system can image all or substantially all of the individual cells within a cytological specimen fixed to a slide and perform a preliminary evaluation of the cells using image processing techniques so as to guide the cytotechnologist to the potentially most relevant cells on the slide for a thorough examination. Examples of such imaging systems are disclosed in U.S. Patent Nos. 7587078, 6665060, 7006674, 7369304, and 7590492. Whether examining an actual specimen slide magnified or examining a magnified image of the specimen, the specimen is typically classified by a cytotechnologist as either "normal" or "abnormal", and abnormal samples typically fall into one of the major classifications defined by the Bethesda system for reporting cervical / vaginal cytological diagnoses.

[0009] However, conventional systems and methods for acquiring digital images of biological specimens have many drawbacks. For example, in conventional systems and methods, it takes time to stop and focus when scanning the entire specimen, so imaging takes time. Furthermore, conventional systems and methods that do not stop to focus typically provide only a single focal plane across the entire specimen. Biological specimens, both cytological and pathological specimens, are actually three-dimensional (i.e., have depth). Therefore, since high magnification and focal distance are required to obtain a digital image of a biological specimen, the depth of field of the image is very limited. As a result, portions of the specimen outside the depth of field within the focal plane are out of focus or become invisible in the image. To obtain digital images in focus at multiple different depths of the specimen, it is necessary to adjust the focal plane by moving the specimen or the camera, or by adjusting the focus lens. However, this requires additional scanning of the specimen for each focal plane or stopping at regular intervals to refocus, which further increases the acquisition time.

[0010] Many of the aforementioned problems of conventional imaging systems are addressed and solved by the imaging systems and methods disclosed and described in PCT International Publication No. 2020 / 091965 (A2) (International Application PCT / US19 / 55458, filed October 9, 2019). The main aspects of the imaging systems and methods disclosed in International Publication No. 2020 / 091965 (A2) are described and discussed below, and include the ability to image cells at different depths within a focused specimen by a single scanning path of a camera having an objective lens tilted with respect to the slide surface to image an image within a focus range that includes the entire thickness of the specimen. However, in some specimen slides, certain problems arise that are not addressed in International Publication No. 2020 / 091965 (A2). For example, when the coverslip is non-uniform or due to other aspects when initially preparing the slide, the thickness of the specimen, i.e., the height of the specimen with respect to the slide surface, is non-uniform. In this case, some (possibly a significant amount) of the cells in the specimen are out of the focus range. Therefore, further improvements to the imaging systems and techniques disclosed in International Publication No. 2020 / 091965 (A2) would be useful in solving the problem of out-of-focus cells.

Summary of the Invention

[0011] According to a first aspect of the present invention disclosed, there is disclosed a method for acquiring an image of an object distributed in a specimen attached to a surface of a slide, using a camera having an objective lens with an optical axis forming a non-orthogonal angle with the surface of the slide. The specimen has a non-uniform height with respect to the slide surface, and the method includes: (i) acquiring a first plurality of images of a first linear portion of the specimen; (ii) evaluating the focus of an object within the linear portion of the specimen imaged in the first plurality of images; and (iii) acquiring a second plurality of images of the first linear portion or a second linear portion of the specimen different from the first linear portion, wherein during the acquisition of the second plurality of images, the height of the objective lens with respect to the slide surface is changed based on the evaluated focus of the object imaged in the first plurality of images. Optionally, during the acquisition of the first plurality of images, the height of the objective lens with respect to the slide surface is substantially constant. Optionally, the second linear portion is directly adjacent to the first linear portion.

[0012] In various embodiments, evaluating the focus of an object imaged in the first plurality of images includes determining whether the total number of out-of-focus objects exceeds a threshold number, and (i) if the total number of out-of-focus objects in the first plurality of images exceeds the threshold number, acquiring the second plurality of images from the first linear portion, or (ii) if the total number of out-of-focus objects in the first plurality of images does not exceed the threshold number, acquiring the second plurality of images from the second linear portion.

[0013] In various embodiments, evaluating the focus of an object imaged in the first plurality of images includes determining the height of each out-of-focus object with respect to the slide surface, and during the acquisition in the first plurality of images, determining whether each out-of-focus object is located at a height with respect to the slide surface outside the in-focus range of the objective lens. Preferably, evaluating the focus of an object in the first plurality of images includes, during the acquisition in the first plurality of images, determining whether each out-of-focus object is located at a height with respect to the slide surface higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens.

[0014] In various embodiments, evaluating the focus of an object in a first plurality of images includes determining the position of each out-of-focus object within a first straight-line portion.

[0015] In various embodiments, one or both of the camera and the slide move relative to the other during acquisition of the image, and the height of the objective lens with respect to the slide surface varies by increasing and / or decreasing the height of the camera with respect to the slide surface as a function of the linear position of the camera with respect to the longitudinal position of each of the first or second straight-line portions. In particular, during acquisition of the image, the height of the objective lens with respect to the slide surface can be varied by moving the slide perpendicular to the camera, moving the camera perpendicular to the slide, or doing both.

[0016] In various embodiments, when a second plurality of images are acquired from the first straight-line portion, the method further includes evaluating the focus of the object imaged in the second plurality of images and acquiring a third plurality of images of a second straight-line portion, and during acquisition of the third plurality of images, the height of the objective lens with respect to the slide surface varies based on the evaluated focus of the object imaged in the second plurality of images.

[0017] According to another aspect of the invention disclosed herein, a method of acquiring an image of an object distributed within a specimen affixed to a surface of a slide using a camera having an objective lens with an optical axis that forms a non-orthogonal angle with the surface of the slide, wherein the specimen has a non-uniform height with respect to the slide surface, the method includes: (a) acquiring a first plurality of images of a straight-line portion of the specimen; (b) evaluating the focus of the object imaged in the first plurality of images; (c) acquiring a second plurality of images of the same or a different straight-line portion of the specimen, and during acquisition of the second plurality of images, varying the height of the objective lens with respect to the slide surface based on the evaluated focus of the object imaged in the first plurality of images; and (d) repeating (a) through (c) until an image of substantially the entire specimen is acquired. When the second plurality of images are acquired from a different straight-line portion of the specimen, such different straight-line portions can be directly adjacent to the straight-line portion from which the first plurality of images were acquired.

[0018] In various embodiments, evaluating the focus of an object imaged in a first plurality of images may include determining whether the total number of out-of-focus objects exceeds a threshold number.

[0019] In a preferred embodiment, evaluating the focus of an object in the first plurality of images includes, during acquisition of the first plurality of images, determining whether each out-of-focus object is located at a height relative to a slide surface that is higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens, and determining each position within the straight-line portion of the out-of-focus object.

[0020] In a preferred embodiment, during acquisition of each of the first and second pluralities of images, one or both of the camera and the slide move laterally relative to the other, and the height of the objective lens relative to the slide surface changes by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the linear position of the camera relative to the longitudinal position of each straight-line portion. Again, the height of the objective lens relative to the slide surface can be changed by vertically moving the slide relative to the camera, or vertically moving the camera relative to the slide, or both, during acquisition of the image.

[0021] According to a further aspect of the disclosed invention, a system is provided for acquiring an image of an object distributed within a specimen affixed to a surface of a slide, the specimen having a non-uniform height relative to the slide surface, the system including a camera having an objective lens with an optical axis, the camera being positioned such that the optical axis forms a non-orthogonal angle with the surface of the slide. The system further includes an image processing unit operably connected to the camera, the image processing unit receiving a first plurality of images of a first straight-line portion of the specimen acquired by the camera, evaluating the focus of an object within the straight-line portion of the specimen imaged in the first plurality of images, and configured to cause the camera to acquire a second plurality of images from a second straight-line portion of the specimen that is the first straight-line portion or different from the first straight-line portion, and to change the height of the objective lens relative to the slide surface based on the evaluated focus of the object imaged in the first plurality of images during acquisition of the second plurality of images.

[0022] In one embodiment, during the acquisition of the first plurality of images, the height of the objective lens with respect to the slide surface is substantially constant.

[0023] In one embodiment, the image processing unit evaluates the focus of the objects imaged in the first plurality of images by at least partially determining whether the total number of out-of-focus objects exceeds a threshold number.

[0024] In one embodiment, the image processing unit evaluates the focus of the objects photographed in the first plurality of images by at least partially determining the height of each out-of-focus object with respect to the slide surface.

[0025] In one embodiment, the image processing unit evaluates the focus of the objects in the first plurality of images by at least partially determining whether each out-of-focus object is located at a height with respect to the slide surface outside the in-focus range of the objective lens during the acquisition of the first plurality of images.

[0026] In one embodiment, the image processing unit evaluates the focus of the objects in the first plurality of images by at least partially determining whether each out-of-focus object is located at a height with respect to the slide surface higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens during the acquisition of the first plurality of images.

[0027] In one embodiment, the image processing unit evaluates the focus of the objects in the first plurality of images by at least partially determining the position of each out-of-focus object within the first straight portion.

[0028] In various embodiments, preferably, during the acquisition of the image, one or both of the camera and the slide are configured to move in a lateral direction with respect to the other.

[0029] Although not limited, in one embodiment, the height of the objective lens with respect to the slide surface varies by increasing and / or decreasing the height of the slide surface with respect to the camera as a function of the linear position of the camera with respect to the lengthwise position of each straight portion. In the same or another embodiment, the height of the objective lens with respect to the slide surface varies by increasing and / or decreasing the height of the camera with respect to the slide surface as a function of the linear position of the camera with respect to the lengthwise position of each of the first or second straight portions.

[0030] The second plurality of images may be acquired from the first or second straight portion. In one embodiment, the second plurality of images are acquired from the first straight portion, and the image processing unit is further configured to evaluate the focus of the object imaged in the second plurality of images and to cause the camera to acquire a third plurality of images of the second straight portion, and during the acquisition of the third plurality of images, the height of the objective lens with respect to the slide surface changes based on the evaluated focus of the object imaged in the second plurality of images. Although not limiting, the second straight portion may be directly adjacent to the first straight portion.

[0031] Other and further aspects and features of the disclosed embodiments will become apparent from the detailed description set forth hereinafter in conjunction with the accompanying drawings.

[0032] The foregoing and other aspects of the embodiments are described in more detail with reference to the accompanying drawings, where like reference numerals mean like components, and the description of like components applies to all related embodiments described.

Brief Description of the Drawings

[0033]

Figure 1

[0034]

Figure 2

[0035]

Figure 3

[0036]

Figure 4A

Figure 4B

[0037]

Figure 5

[0038]

Figure 6

[0039]

Figure 7

[0040]

Figure 8A

Figure 8B

Figure 8C

[0041]

Figure 9A

Figure 9B

[0042]

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0043] The advent of digital whole slide imaging (WSI) systems has revolutionized the fields of pathology and cytology. The ability to rapidly acquire high-quality whole slide images is an essential step in enabling successful clinical workflows, particularly for high-volume screening applications such as pap tests. Liquid-based cytology slides may appear visually almost monolayered to the technician, but cytology is inherently three-dimensional. These slides can have a significantly higher depth of focus of closely juxtaposed material than the depth of field (DOF) of the objective lens of a high-magnification microscope, which can pose difficulties for WSI. For this reason, cytology slides are more difficult to image than tissue diagnosis slides. Also, slides using film coverslips can have additional requirements for scanning depth due to the curvature of the cell spot regions of the slides. Therefore, the distance between the object of interest in a cytology specimen and the glass slide to which the specimen is attached can vary significantly with respect to the DOF of the objective lens of the microscope. Many current WSI systems need to repeat scanning to cover multiple focal planes in order to acquire high-quality images, which significantly increases the imaging time. Therefore, it is difficult to efficiently obtain high-focus-quality images of cytology slides. This specification discloses a system and method for efficiently scanning such slides to obtain high-quality and in-focus images.

[0044] Referring to FIG. 1, an exemplary embodiment of a microscope slide 102 is shown. The microscope slide 102 is a rectangular glass plate 110 (or other suitable material) having a slide identification region 112, a specimen region 114, and a reference mark 116. The microscope slide 102 may be a standard-sized microscope slide that is approximately 75 mm × 25 mm, or it may be of other suitable sizes. The microscope slide 102 may have chamfered corners to facilitate handling and positioning of the slide 102. The specimen region 114 may be a circle having a diameter of up to about 22 mm. The entire specimen region 114 on the microscope slide 102 may be imaged. The slide identification region 112 may be up to about 25-28 mm in length. A barcode, ID number, and / or other information may be printed on the slide identification portion 112. The specimen region 114 is left as a transparent region of the slide glass 102. The reference mark 116 can be used as a reference point on the slide 102 by an imaging device to determine the position and / or orientation of the slide 102 and its features relative to the imaging device. A specimen 119 including a plurality of objects distributed within a three-dimensional volume is usually affixed within the specimen region 114 on the slide 102, but in some cases, the specimen may extend outside the specimen region 114. The three-dimensional volume of the specimen 119 has a length (l), width (w), and thickness or depth (d). The thickness (d) defines the z-axis relative to the surface of the slide 102. The specimen 119 may be any suitable specimen, such as a cytology specimen where the object is a cell, a solid tissue specimen where the object is a tissue structure, and the like.

[0045] As shown in FIG. 1, a cover slip 115 may be used to cover the specimen 119 in the specimen region 114. The cover slip 115 for the specimen has sufficient transparency to acquire an image of the specimen 119 through the cover slip 115. That is, the cover slip 115 does not prevent the imaging device from acquiring an image through the cover slip 115. The cover slip 115 has the function of protecting the specimen 119 so as to preserve it and not contaminate it, and also not contaminate other objects, and holding the specimen 119 flat and in a predetermined position. The cover slip 115 has a thickness 117.

[0046] As shown in FIG. 2, the specimen 119 disposed between the slide glass 110 and the cover slip 115 has a thickness 120. The cells 122 are dispersed throughout the specimen layer at various depths, i.e., positions in the z-axis direction. The specimen can have a thickness that exceeds the depth of field of the optical system used to acquire an image of an object (such as the cell 122) in the specimen throughout the specimen. This can occur particularly in a liquid-based cytology specimen from which individual cells are collected. The cover slip 115 placed on top of the specimen 119 may be made of glass or plastic and is adhered to the slide 110 with a thin adhesive layer. It has been observed that the cells tend to float within the adhesive and rise above the glass 110. Further, the cover slip 115 is not necessarily completely flat and often exhibits undulations, hills, and valleys. This causes the thickness of the adhesive to be non-uniform, and as a result, the distance of the cells from the glass substrate 110 and from the objective lens changes, and sometimes the distance becomes greater than the depth of field. Disclosed herein is a digital imaging method and system for efficiently scanning a cytology slide that solves the problem of digitizing a specimen that is thicker than the depth of field of an objective lens in a given field of view.

[0047] A cytology slide is inherently three-dimensional, with cells potentially floating or piling up in a medium. Since the depth of field (DOF) of a microscope objective lens is very small, it is not possible to image all cells in focus in a single image. In fact, individual cells can be thicker than a single DOF. The depth of field of a 40x microscope objective lens with a numerical aperture (NA) of 0.75 is less than 2 micrometers (μm). A cytology slide using a flexible film (plastic) cover slip may require an even deeper scanning (i.e., image acquisition) depth because the entire cell spot area is curved. A wider range of focus is required to image all cells on a cytology slide (such as a ThinPrep slide) with high quality.

[0048] As an example, 23 ThinPrep Pap slides were scanned with a computer-controlled microscope having a digital camera (ThinPrep Integrate Imager from Hologic) to collect cell preparation depth data. First, the cell spot areas were scanned using a computer-driven XY stage. At each position, image stacks were taken over a wide range of Z heights (more than 40 micrometers (μm)). All images were divided into small areas (35 micrometers (μm) square) and evaluated for all levels of the Z stack according to the Brenner focus score criterion. The optimal focus for that tile was determined. By focusing on the reference marks printed on the slide, the entire slide glass plane was determined and subtracted from the focus data to determine the relative height of the cell content.

[0049] The table in Figure 3 shows an overview of the data obtained from these 23 slides. From this table, it can be seen that the average cell depth of the ThinPrep slides is 11.09 micrometers (μm) for slides with glass coverslips and 23.6 micrometers (μm) for slides with film coverslips. In some cases, the depth of the cells can exceed 40 micrometers (μm). Note that even when using a glass coverslip, the local variation in the height of the cells within the slide can be up to 7 times the depth of field of the microscope objective lens. Surface plot focus maps were created and examined for each slide. Figures 4A and 4B show an example of a film coverslip slide and illustrate the effect of curvature across the cell spot area. In particular, in the examples shown in Figures 4A and 4B, the cells in the center of the specimen are closer to the slide glass, while the cells around the edge of the specimen are closer to the coverslip.

[0050] As shown in FIG. 5, during imaging, in order to scan the entire specimen area 114 (or, for example, a predetermined area of the entire actual specimen 119 if the actual specimen 119 covers an area different from the specimen area 114 and the boundary of the actual specimen 119 is predetermined), the slide 102 is moved by an XY slide stage. The XY slide stage reciprocates the slide 102 along a serpentine path to image microscopic images of the swaths, columns, or straight portions of the specimen 119 on each path. To image a swath, the XY slide stage continuously moves the slide 102 and triggers the camera to take an image when the XY slide stage reaches each trigger point according to the encoder position of the stage. A very fast camera is used so that the relative movement between the slide and the camera is continuous. Acquiring images along the straight portions of the slide at such high speed may be referred to as "scanning" of the columns. At this time, a serpentine path is used such that the starting point of each successive swath approaches the ending point of the previous swath in order to minimize the time required to scan the entire specimen 119. When the slide 102 is moved along a swath, the camera takes a microscopic image of the specimen 119. That is, the imaging device images a plurality of images covering the entire specimen area according to a scanning pattern as shown in FIG. 5. The scanning pattern includes a plurality of straight portions represented by the horizontal arrows 130 in FIG. 5. The specimen is divided into a plurality of focus zones, represented by the squares 132 in FIG. 5. Thus, each straight portion 130 along which an image is acquired includes a plurality of focus zones 132. Each straight portion 130 may include 30 to 70 focus zones. In another example, instead of the slide 102 moving, the slide 102 remains stationary while the camera moves along the scanning pattern. Whether the slide 102 moves or the camera moves, the slide 102 moves relative to the camera.

[0051] Most current WSI systems scan a single focal plane at a time. A typical scanner can complete a 15×15 mm scan in one minute. At this speed, it would take at least 26 minutes to scan a circular ThinPrep cell spot area to a depth of 14 focal planes. To dramatically improve the throughput of slide digitizers, some systems are designed to have an inclination angle between the objective lens and the slide glass. In this method, the depth of field can be utilized to simultaneously digitize different layers at different depths little by little. At this time, by performing an operation called focus merge, in which the most focused layer (or part of the layer) is selected and the focused areas are joined together, the layers can be made into one composite layer that is in focus. In this method, the slide can be continuously moved under the objective lens without stopping for focusing. A system can be constructed that digitizes and merges various layers and basically performs post-acquisition focusing. The only limitation of such a system is the depth of field of the objective lens, which is interpreted as the maximum thickness of the specimen that can be imaged. The obvious advantage of such a system is that stopping and focusing are not required.

[0052] By using the inclined plane volume scanning method, the acquisition time for scanning the entire cell content area is significantly shortened. A ThinPrep pap slide can be completed in about 2.5 minutes. As shown in FIG. 6, the imaging optical system and the camera 202 are inclined with respect to the slide 102. In the area of the image at one end of the camera frame, an image closer to the slide glass is acquired than in the area at the other end of the camera frame. In one example, an inclination angle of 48 milliradians and a slide image frame width of 0.5 mm provide a scanning depth of 0.5×sin(0.048), that is, 24 micrometers (μm).

[0053] As best shown in FIG. 6, the optical axis 204 of the objective lens is inclined at an inclination angle 206 with respect to the perpendicular to the scanning direction of the plane of the slide 102 (see FIG. 5). That is, as a result, the optical axis 204 of the camera and the optical system on the plane of the slide is non-orthogonal to the plane of the slide 102. As described in more detail in WO 2020 / 091965 (A2) pamphlet, the inclination angle 206 enables the imaging device to acquire a volume image of the specimen 119 on the slide 102 (i.e., an image that extends to the depth of the specimen 119). That is, the micro-images include in-focus images of features at different depths of the specimen 119 on the slide 102, rather than just a single focal plane as in the case of an image taken at an orthogonal angle to the specimen 119. The imaging station may be configured to acquire micro-images such that each micro-image includes at least a portion of the depth of the slide 102 below the plane of the slide 102. When a coverslip 115 is used on the slide 102, the imaging station may be configured to acquire micro-images such that each micro-image includes at least a portion of the depth of the coverslip 115.

[0054] When the camera moves continuously, it is triggered to acquire a new image each time it moves 1 / 14 of its width. A very high-speed (>100 fps) camera is used. As shown in FIG. 7 and as described in more detail in WO 2020 / 091965 pamphlet, these overlapping images can be sliced and reconstructed to obtain 14 focal plane images. To optimize the memory space, the focal planes are combined into a single extended depth-of-field image by selecting in-focus pixels from various planes. The image processing is performed in real time using GPU hardware acceleration.

[0055] The cover slip is not perfectly flat, but the variations in flatness are essentially gradual. This means that while the cover slip may form a bulge at the center of the slide or form bulges from a plurality of waves across the slide surface, there is no abrupt change in the distance between the cover slip and the glass beneath it. For example, a perfectly flat cover slip 115 is shown in FIG. 8A, and cover slips having wavy or uneven surfaces are shown in FIGS. 8B and 8C.

[0056] By this gradual change in the gap, an imaging system that focuses after imaging can obtain an effective depth of field that is much larger than the depth of field of the subject obtained by the optical system alone. Basically, when the slide is scanned (as shown in FIG. 5) using a raster reciprocating pattern (usually) at the end of each path (before reversing the scanning direction to examine the next row or swath), the collected layer is analyzed and the optimal focus is determined during the focus merge operation. When the biological specimen rises following the undulations of the cover slip, the optimal focus pattern for the next swath to be scanned can be predicted from the layer that was most in focus in the previous swath. Individual target objects are found during the focus merge and their depths are recorded. Next, each column is used to predict the optimal focus for the next row.

[0057] When an out-of-focus region is detected at the end of a swath, the swath can be resampled at a different height focused on that out-of-focus region. The resulting focus height is used to notify the next swath, thus minimizing the number of times the swath needs to be resampled.

[0058] The advantage of such a system is that it can achieve higher throughput (short scanning time) because there is no need to stop and focus, and there is also little need to return to the out-of-focus area for rescan. Instead, this system simply performs reciprocating scans until the entire specimen is digitized (focus merge). During the traverse, the objective lens is driven up and down according to the focus map given from the previous swath, and the object is designed to be maintained within the depth of field so that all objects are in focus even when the distance from the slide glass surface changes.

[0059] To handle a larger total cell depth range as seen in slides with film cover slips, as shown in FIGS. 8A - 8C, the imaging optics can be driven in the Z-axis direction to follow the curvature. The cell path 802 is undulating and overall tilted. Volume scanning using a tilted camera as described above provides a thicker scanning region 804 than scanning at a single depth of focus, but cannot provide in-focus images of the bottom 806 and top 808 of the cell path 802. The Z-curve following volume scan 810 images the entire cell path 802 of the specimen. The local cell depth within the field of view of the camera fits within the scanning depth of the inclined plane, but larger variations in depth of focus may be required at longer distances.

[0060] The curve following scanning method minimizes local focus errors and provides higher quality WSI images. The slides used to create the images in FIGS. 9A and 9B have a Z depth of focus exceeding 40 microns (μm). A volume scan Z that does not follow the Z-curve is shown in FIG. 9A. A Z-curve following volume scan is shown in FIG. 9B. The image in FIG. 9B is sharper and more in focus than the image in FIG. 9A.

[0061] Next, with reference to FIG. 10, a method 500 for obtaining an image of an object distributed in a specimen attached to the surface of a slide will be described. In particular, for a specimen having a thickness exceeding the depth of field of the optical system with a non-uniform height with respect to the surface of the slide, an image is obtained by Z-curve following volume scanning. In a first step 502 of the method 500, discrete positions on the specimen slide are sampled to determine an initial focus height (Z). Next, in step 504, for each focus zone of the first linear portion, several images are obtained along the first linear portion of the specimen using the initial focus height determined in step 502. The initial focus height remains constant along the first linear portion, which means that the height of the objective lens with respect to the slide surface is substantially constant during the acquisition of the images in the first linear portion of the specimen in step 504. Each column (linear portion) of the specimen includes a plurality of focus zones.

[0062] For example, referring to FIG. 5, each square along the linear portion may be used as a focus zone. Each linear portion scanned by the imaging device may include, for example, 30 to 70 focus zones. Alternatively, instead of using a substantially constant focus height as in step 504, the discrete positions sampled in step 502 can be used to determine the z-curve of the first linear portion, and the images along the first linear portion can be obtained by following that z-curve. Thus, the objective lens moves up and down along the z-axis with respect to the slide following the z-curve during the acquisition of the images along the first linear portion.

[0063] Next, in step 506, for each focus zone in the straight line portion that has just been scanned, by evaluating whether the object imaged in the image is in focus, the optimal focus is determined. This evaluation includes determining whether an out-of-focus object is within the specimen at a height relative to the slide surface outside the focusing range of the objective lens. The evaluation may also include identifying out-of-focus objects based on the relative position of the out-of-focus objects along the straight line portion of the specimen. That is, the image is evaluated for each focus zone along the straight line portion, and it is determined which focal plane is the most in focus. As shown in FIG. 7, by acquiring the tilt angle image, images are taken at 12 focal planes at a time. For example, if it is found that the optimal focus is at the uppermost or lowermost focal plane (focal plane 1 or 12), it may be necessary to move the z-axis position of the objective lens relative to the slide up and down respectively to obtain an image with a better focus. Further, if it is found that the optimal focus is at the uppermost or lowermost focal plane, it may be necessary to move the z-axis position of the objective lens relative to the slide up and down respectively during image acquisition in the next adjacent straight line portion of the specimen.

[0064] If there are too many out-of-focus objects (i.e., the number of out-of-focus objects exceeds a predetermined threshold number), the focus height (Z) of each out-of-focus object is adjusted, and in step 508, the straight line portion is rescanned according to the new focus height curve. When the camera follows the focus height curve, the z-axis position of the objective lens relative to the slide surface changes by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the relative position of the camera along the straight line portion of the specimen. The z-axis position of the objective lens relative to the slide can change from one focus zone to the next along the straight line portion of the specimen. In one embodiment, the slide 110 moves up and down while keeping the z-axis position of the objective lens stationary. In an alternative embodiment, the objective lens moves up and down while the z-axis position of the slide remains constant.

[0065] When most or all of the focus zones are in focus, at step 510, based on the optimal focus of each focus zone in the current column, the next value of each focus zone is calculated. Some of the focus zones in the next column may move up (i.e., the z position is closer to the cover slip and farther from the slide glass) to track the target object close to the cover slip, while some of the focus zones in the next column may move down to track the target object close to the slide glass. Next, at step 512, the next straight line portion is scanned using the focus curve calculated at step 510. Steps 506 - 512 are repeated for each straight line portion in the specimen until an image of the entire specimen is obtained. That is, after an image is obtained for each straight line portion of the specimen, the focus of the objects in those images is evaluated (step 506). Based on the evaluation, the focus curve for the next straight line portion of the specimen is determined (step 510) and used to obtain an image of the objects in the next straight line portion that is directly adjacent to the straight line portion that has just been scanned (step 512). Alternatively, if it is determined that there are too many out-of-focus objects based on the focus evaluation, the focus curve is adjusted and the same straight line portion is scanned again using the adjusted focus curve (step 508).

[0066] Although specific embodiments have been shown and described, it should be understood that the above description is not intended to limit the scope of these embodiments, and such disclosure is provided for illustrative and exemplary purposes only. Accordingly, various changes and modifications can be made to the disclosed embodiments without departing from the scope of the following claims. The above-described embodiments may be described as follows, but are not limited thereto. [Configuration 1] A method of acquiring an image of an object distributed in a specimen attached to a surface of a slide using a camera having an objective lens having an optical axis forming a non-orthogonal angle with the surface of the slide, wherein the specimen has a non-uniform height with respect to the slide surface, acquiring a first plurality of images of a first linear portion of the specimen; evaluating the focus of an object within the linear portion of the specimen imaged in the first plurality of images; acquiring a second plurality of images of the first linear portion or a second linear portion of the specimen different from the first linear portion, and during acquisition of the second plurality of images, changing a height of the objective lens with respect to the slide surface based on the evaluated focus of the object imaged in the first plurality of images. [Configuration 2] The method according to Configuration 1, wherein a height of the objective lens with respect to the slide surface is substantially constant during acquisition of the first plurality of images. [Configuration 3] The method according to Configuration 1 or 2, wherein evaluating the focus of the object imaged in the first plurality of images includes determining whether a total number of out-of-focus objects exceeds a threshold number. [Configuration 4] The method according to any one of Configurations 1 to 3, wherein evaluating the focus of the object imaged in the first plurality of images includes determining a height of each of the out-of-focus objects with respect to the slide surface. [Configuration 5] The method according to Configuration 4, wherein evaluating the focus of the object in the first plurality of images includes determining whether each out-of-focus object is located at a height with respect to the slide surface outside a focusing range of the objective lens during acquisition in the first plurality of images. [Configuration 6] The method according to Configuration 4 or 5, wherein evaluating the focus of the object in the first plurality of images includes determining whether each out-of-focus object is located at a height with respect to the slide surface higher than a maximum height or lower than a minimum height of the focusing range of the objective lens during acquisition in the first plurality of images. [Configuration 7] The method according to any one of Configurations 1 to 6, wherein evaluating the focus of the object in the first plurality of images includes determining a position of each of the out-of-focus objects within the first linear portion. [Configuration 8] The method according to any one of Configurations 1 to 7, wherein during the image acquisition, one or both of the camera and the slide move in a lateral direction with respect to the other. [Configuration 9] The method according to Configuration 8, wherein the height of the objective lens with respect to the slide surface changes by increasing and / or decreasing the height of the slide surface with respect to the camera as a function of the linear position of the camera with respect to the lengthwise position of each of the straight portions. [Configuration 10] The method according to Configuration 8, wherein the height of the objective lens with respect to the slide surface changes by increasing and / or decreasing the height of the camera with respect to the slide surface as a function of the linear position of the camera with respect to the lengthwise position of each of the first or second straight portions. [Configuration 11] The method according to any one of Configurations 1 to 10, wherein a plurality of second images are acquired from the second straight portion. [Configuration 12] A plurality of second images are acquired from the first straight portion, and the method evaluates the focus of the object imaged in the plurality of second images; further includes acquiring a plurality of third images from the second straight portion, and changing the height of the objective lens with respect to the slide surface based on the evaluated focus of the object imaged in the plurality of second images during the acquisition of the plurality of third images, according to the method of any one of Configurations 1 to 10. [Configuration 13] The method according to any one of Configurations 1 to 12, wherein the second straight portion is directly adjacent to the first straight portion. [Configuration 14] When the total number of out-of-focus objects in the plurality of first images exceeds the threshold number, acquiring a plurality of second images from the first straight portion; The method according to Configuration 3, further including, when the total number of out-of-focus objects in the plurality of first images does not exceed the threshold number, acquiring a plurality of second images from the second straight portion. [Configuration 15] A method of acquiring an image of an object distributed in a specimen attached to a surface of a slide, using a camera having an objective lens with an optical axis forming a non-orthogonal angle with the surface of the slide, wherein the specimen has a non-uniform height with respect to the slide surface, and the method (a) acquires a first plurality of images of a straight portion of the specimen; (b) evaluates the focus of the object imaged in the first plurality of images; (c) obtaining a second plurality of images of the same or different linear portions of the specimen, and during the obtaining of the second plurality of images, changing the height of the objective lens with respect to the slide surface based on the evaluated focus of the object imaged in the first plurality of images, and (d) a method including repeating (a) to (c) until an image of substantially the entire specimen is obtained. [Configuration 16] The method according to configuration 15, wherein evaluating the focus of the object in the first plurality of images includes determining whether the total number of out-of-focus objects exceeds a threshold number. [Configuration 17] The method according to configuration 15 or 16, wherein evaluating the focus of the object in the first plurality of images includes determining whether each out-of-focus object in the first plurality of images is located at a height with respect to the slide surface that is higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens during acquisition in the first plurality of images. [Configuration 18] The method according to any one of configurations 15 to 17, wherein evaluating the focus of the object in the first plurality of images includes determining the position of each out-of-focus object within each linear portion. [Configuration 19] The method according to any one of configurations 15 to 18, wherein during the acquisition of each of the first and second pluralities of images, one of the camera and the slide moves in a lateral direction with respect to the other. [Configuration 20] The method according to configuration 19, wherein the height of the objective lens with respect to the slide surface changes by increasing and / or decreasing the height of the slide surface with respect to the camera as a function of the linear position of the camera with respect to the longitudinal position of each linear portion. [Configuration 21] The method according to configuration 19, wherein the height of the objective lens with respect to the slide surface changes by increasing and / or decreasing the height of the camera with respect to the slide surface as a function of the linear position of the camera with respect to the longitudinal position of each linear portion. [Configuration 22] The method according to any one of configurations 15 to 21, wherein the second plurality of images are obtained from the different linear portions, and the different linear portions are directly adjacent to the linear portions from which the first plurality of images are obtained. [Configuration 23] A system for obtaining an image of an object distributed within a specimen affixed to a surface of a slide, the specimen having a non-uniform height with respect to the slide surface, the system comprising: a camera having an objective lens with an optical axis, the camera positioned such that the optical axis forms a non-orthogonal angle with the surface of the slide; A system comprising an image processing unit operably connected to the camera, the image processing unit receiving a plurality of first images of a first straight portion of the specimen acquired by the camera, evaluating the focus of an object within the straight portion of the specimen imaged in the plurality of first images, and causing the camera to acquire a plurality of second images of a second straight portion of the specimen that is the first straight portion or different from the first straight portion, and during acquisition of the plurality of second images, changing a height of the objective lens with respect to the slide surface based on the evaluated focus of the object imaged in the plurality of first images. [Configuration 24] The system according to configuration 23, wherein during acquisition of the plurality of first images, the height of the objective lens with respect to the slide surface is substantially constant. [Configuration 25] The system according to configuration 23 or 24, wherein the image processing unit evaluates the focus of the object imaged in the plurality of first images by at least partially determining whether the total number of out-of-focus objects exceeds a threshold number. [Configuration 26] The system according to any one of configurations 23 to 25, wherein the image processing unit evaluates the focus of the object imaged in the plurality of first images by at least partially determining a height of each of the out-of-focus objects with respect to the slide surface. [Configuration 27] The system according to configuration 26, wherein during acquisition of the plurality of first images, the image processing unit evaluates the focus of the object in the plurality of first images by at least partially determining whether each out-of-focus object is located at a height with respect to the slide surface outside the in-focus range of the objective lens. [Configuration 28] The system according to configuration 27 or 28, wherein during acquisition of the plurality of first images, the image processing unit evaluates the focus of the object in the plurality of first images by at least partially determining whether each out-of-focus object is located at a height with respect to the slide surface higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens. [Configuration 29] The system according to any one of configurations 23 to 28, wherein the image processing unit evaluates the focus of the object in the plurality of first images by at least partially determining a position of each of the out-of-focus objects within the first straight portion. [Configuration 30] The system according to any one of configurations 23 to 29, wherein during image acquisition, one or both of the camera and the slide move in a lateral direction with respect to the other. [Configuration 31] The system according to configuration 30, wherein the height of the objective lens with respect to the slide surface changes by increasing and / or decreasing the height of the slide surface with respect to the camera as a function of the linear position of the camera with respect to the lengthwise position of each of the linear portions of the straight line portion. [Configuration 32] The system according to configuration 30, wherein the height of the objective lens with respect to the slide surface changes by increasing and / or decreasing the height of the camera with respect to the slide surface as a function of the linear position of the camera with respect to the lengthwise position of the first or second linear portion of the straight line portion. [Configuration 33] The system according to any one of configurations 23 to 32, characterized in that a plurality of second images are acquired from the second linear portion. [Configuration 34] A plurality of second images are acquired from the first linear portion, and the image processing unit evaluates the focus of the object imaged in the plurality of second images, and is configured to cause the camera to acquire a third plurality of images of the second linear portion, and during the acquisition of the third plurality of images, the height of the objective lens with respect to the slide surface changes based on the evaluated focus of the object imaged in the plurality of second images. The system according to any one of configurations 23 to 32. [Configuration 35] The system according to any one of configurations 23 to 34, wherein the second linear portion is directly adjacent to the first linear portion.

Claims

Claim 1 A method for acquiring an image of an object distributed in a specimen attached to a slide surface using a camera having an objective lens with an optical axis forming a non-orthogonal angle with the slide surface, wherein the specimen has a non-uniform height with respect to the slide surface, acquiring a first plurality of images of a first linear portion of the specimen, evaluating the focus of an object within the first linear portion of the specimen imaged in the first plurality of images, including determining whether the total number of out-of-focus objects exceeds a threshold number and determining an optimal focus height of the specimen along the first linear portion, calculating a z-focus height curve of the optimal focus height of the specimen along the first linear portion, when the total number of out-of-focus objects in the first plurality of images exceeds the threshold number, re-photographing the first linear portion of the specimen to acquire a second plurality of images, and changing the height of the objective lens with respect to the slide surface during acquisition of the second plurality of images based on the z-focus height curve, when the total number of out-of-focus objects in the first plurality of images does not exceed the threshold number, acquiring a second plurality of images of a second linear portion of the specimen that is directly adjacent to the first linear portion, and changing the height of the objective lens with respect to the slide surface during acquisition of the second plurality of images based on the z-focus height curve. A method comprising: Claim 2 The method according to claim 1, wherein during acquisition of the first plurality of images, the height of the objective lens with respect to the slide surface is substantially constant. Claim 3 The method according to claim 1 or 2, wherein evaluating the focus of the object imaged in the first plurality of images includes determining the height of each out-of-focus object with respect to the slide surface. Claim 4 The method according to claim 3, wherein evaluating the focus of the object in the first plurality of images includes determining whether each out-of-focus object is located at a height with respect to the slide surface outside the in-focus range of the objective lens during acquisition in the first plurality of images. Claim 5 Evaluating the focus of the object in the first plurality of images includes, during acquisition of the first plurality of images, determining whether each out-of-focus object is located at a height relative to the slide surface that is higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens. The method according to claim 3.

6. Evaluating the focus of the object in the first plurality of images includes determining the position of each out-of-focus object within the first straight portion. The method according to claim 1 or 2.

7. During acquisition of the image, one or both of the camera and the slide move in a lateral direction relative to the other. The method according to claim 1 or 2.

8. The height of the objective lens relative to the slide surface varies by increasing and / or decreasing the height of the slide surface relative to the camera as a function of the linear position of the camera relative to the longitudinal position of each of the straight portions. The method according to claim 7.

9. The height of the objective lens relative to the slide surface varies by increasing and / or decreasing the height of the camera relative to the slide surface as a function of the linear position of the camera relative to the longitudinal position of each of the first or second straight portions. The method according to claim 7.

10. The second plurality of images are acquired from the second straight portion. The method according to claim 1.

11. The second plurality of images are acquired from the first straight portion, and the method includes: evaluating the focus of the object imaged in the second plurality of images, including determining whether the total number of out-of-focus objects exceeds a threshold number, and determining the optimal focus height of the specimen along the next straight portion; calculating a second z-focus height curve of the optimal focus height of the specimen along the second straight portion; acquiring a third plurality of images of a third straight portion, the third straight portion being directly adjacent to the second straight portion, and changing the height of the objective lens relative to the slide surface during acquisition of the third plurality of images based on the second z-focus height curve. The method according to claim 1.

12. The second straight portion is directly adjacent to the first straight portion. The method according to claim 10 or 11.

13. A method of obtaining an image of an object distributed in a specimen attached to a slide surface using a camera having an objective lens with an optical axis forming a non-orthogonal angle with the slide surface, wherein the specimen has a non-uniform height with respect to the slide surface, and the method comprises: (a) obtaining a first plurality of images of a first straight portion of the specimen; (b) evaluating the focus of the object imaged in the first plurality of images, including determining whether the total number of out-of-focus objects exceeds a threshold number, and determining an optimal focus height of the specimen along the first straight portion; (c) calculating a z-focus height curve of the optimal focus height of the specimen along the first straight portion; (d) when the total number of out-of-focus objects in the first plurality of images exceeds the threshold number, re-photographing the first straight portion of the specimen to obtain a second plurality of images, including changing the height of the objective lens with respect to the slide surface during the acquisition of the second plurality of images based on the z-focus height curve; and when the total number of out-of-focus objects in the first plurality of images does not exceed the threshold number, obtaining a second plurality of images of a second straight portion of the specimen immediately adjacent to the first straight portion, including changing the height of the objective lens with respect to the slide surface during the acquisition of the second plurality of images based on the z-focus height curve; (e) repeating (a) to (d) until images of substantially the entire specimen are obtained.

14. Evaluating the focus of the object in the first plurality of images includes determining, during acquisition in the first plurality of images, whether each out-of-focus object is located at a height with respect to the slide surface that is higher than the maximum height or lower than the minimum height of the in-focus range of the objective lens. The method according to claim 13.

15. Evaluating the focus of the object in the first plurality of images includes determining the position of each out-of-focus object within each straight portion. The method according to claim 13.

16. During the acquisition of each of the first and second plurality of images, one of the camera and the slide moves in a lateral direction with respect to the other. The method according to claim 13.

17. The method according to claim 16, wherein the height of the objective lens with respect to the slide surface varies by increasing and / or decreasing the height of the slide surface with respect to the camera as a function of the linear position of the camera with respect to the lengthwise position of each linear portion.

18. A system for acquiring an image of an object distributed within a specimen affixed to a slide surface, the specimen having a non-uniform height with respect to the slide surface, the system comprising: a camera having an objective lens with an optical axis, the camera positioned such that the optical axis forms a non-orthogonal angle with the slide surface; an image processing unit operably connected to the camera; A system configured to execute the method according to any one of claims 1 to 12.

Images