Real-time focusing in slide-scanning system

Abstract

To provide a system and method for overcoming significant problems with real-time focusing found in conventional systems.SOLUTION: Focus points are added to an initialized focus map while acquiring a plurality of image stripes of a sample on a glass slide. For each image stripe, a plurality of frames collectively representing the image stripe may be acquired using both an imaging line-scan camera and a tilted focusing line-scan camera. Focus points representing positions of best focus for trusted frames are added to the focus map. Outlying focus points are removed from the focus map. In some cases, one or more image stripes may be reacquired. Finally, the image stripes are assembled into one composite image of the sample.SELECTED DRAWING: Figure 5A

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 883,525, filed August 6, 2019, the contents of which are incorporated by reference herein as if fully set forth herein.

[0002] This application is further related to the following applications, all of which are incorporated by reference herein as if fully set forth herein: No. PCT / US2016 / 053581, filed September 23, 2016; International Patent Application No. PCT / US2017 / 028532, filed April 20, 2017; International Patent Application No. PCT / US2018 / 063456, filed November 30, 2018; International Patent Application No. PCT / US2018 / 063460, filed November 30, 2018; International Patent Application No. PCT / US2018 / 063450, filed November 30, 2018; International Patent Application No. PCT / US2018 / 063461, filed November 30, 2018; No. PCT / US2018 / 062659, filed November 27, 2018; International Patent Application No. PCT / US2018 / 063464, filed November 30, 2018; International Patent Application No. PCT / US2018 / 054460, filed October 4, 2018; International Patent Application No. PCT / US2018 / 063465, filed November 30, 2018; International Patent Application No. PCT / US2018 / 054462, filed October 4, 2018; International Patent Application No. PCT / US2018 / 063469, filed November 30, 2018; No. PCT / US2018 / 054464, filed October 4, 2018; International Patent Application No. PCT / US2018 / 046944, filed August 17, 2018; No. PCT / US2018 / 054470, filed October 4, 2018; International Patent Application No. PCT / US2018 / 053632, filed September 28, 2018; International Patent Application No. PCT / US2018 / 053629, filed September 28, 2018; International Patent Application No. PCT / US2018 / 053637, filed September 28, 2018; International Patent Application No. PCT / US2018 / 062905, filed November 28, 2018; International Patent Application No. PCT / US2018 / 063163, filed November 29, 2018; International Patent Application No. PCT / US2017 / 068963, filed December 29, 2017; International Patent Application No. PCT / US2019 / 020411, filed March 1, 2019; U.S. Patent Application No. 29 / 631,492, filed December 29, 2017; U.S. Patent Application No. 29 / 631,495, filed December 29, 2017; U.S. Patent Application No. 29 / 631,499, filed December 29, 2017; U.S. Patent Application No. 29 / 631,501, filed December 29, 2017.

[0003] FIELD OF THEINVENTION FIELD OF THE DISCLOSURE The embodiments described herein relate generally to controlling a slide scanning system, and more specifically, to real-time focusing in a slide scanning system. [Background technology]

[0004] Digital pathology is an image-based information environment enabled by computer technology that allows for the management of information generated from physical slides. Digital pathology is made possible in part by virtual microscopy, a technique in which specimens on physical glass slides are scanned to produce digital slide images that can be stored, viewed, managed, and analyzed on a computer monitor. With the ability to image entire glass slides, the field of digital pathology has exploded and is now viewed as one of diagnostic medicine's most promising avenues for better, faster, and cheaper diagnosis, prognosis, and prediction of important diseases such as cancer.

[0005] A major objective for the digital pathology industry is to reduce scan times. By switching to real-time focusing during the actual scan, a reduced scan time can be achieved. To achieve focused, high-quality image data using real-time focusing during the actual scan, the scanning device must be capable of determining the next Z value for the objective lens (e.g., the distance between the objective lens and the sample). Therefore, what is needed is a system and method that overcomes the significant problems of real-time focusing found in conventional systems. Summary of the Invention [Means for solving the problem]

[0006] A system, method, and non-transitory computer-readable medium for real-time focusing in a slide scanning system are disclosed.

[0007] In one embodiment, a method is disclosed that includes initializing a focus map using at least one hardware processor of a scanning system; while acquiring a plurality of image stripes of at least a portion of a sample on a glass slide, for each of the plurality of image stripes, acquiring each of a plurality of frames collectively representing the image stripe using both the imaging line scan camera and the tilted focusing line scan camera, and adding a focus to the focus map by adding a focus representing a best focus position for a trusted frame of the plurality of frames to the focus map; removing all outlier foci from the focus map; for each of a plurality of frames in the plurality of image stripes, determining whether to restrip one or more image stripes of the plurality of image stripes based on a focus error; if determining to restrip one or more image stripes, reacquiring the one or more image stripes; and assembling the plurality of image stripes into one composite image of at least a portion of the sample.

[0008] Adding focus to the focus map while acquiring the plurality of image stripes may further include, for each of the plurality of image stripes other than a last one of the plurality of image stripes to be acquired, determining from the image stripe after acquiring the image stripe a direction of a next one of the plurality of image stripes to be acquired. The plurality of image stripes may be acquired, in order, by acquiring a reference stripe, sequentially acquiring image stripes from a first side of the reference stripe to a first edge of a scanned area of ​​the sample, and sequentially acquiring image stripes from a second side of the reference stripe opposite the first side of the reference stripe to a second edge of the scanned area opposite the first edge of the scanned area.

[0009] The method may further include adding a plurality of macro focal points to the focus map prior to beginning acquisition of the plurality of image stripes. The method may include adding one or more macro focal points to the focus map after acquiring one or more of the plurality of image stripes. It may further include:

[0010] Adding focus to the focus map while acquiring the plurality of image stripes may further include determining, for each of a plurality of frames in each of the plurality of image stripes, whether the frame is trusted. Determining whether the frame is trusted can include calculating a principal gradient vector including an average gradient vector for each column in the frame acquired by the imaging line scan camera, calculating a gradient gradient vector including an average gradient vector for each column in the frame acquired by the gradient-focus line scan camera, identifying a number of analyzable columns in the principal gradient vector, calculating a ratio vector based on the principal gradient vector and the gradient gradient vector, determining whether the frame is analyzable based on the number of analyzable columns and the ratio vector, and if determining that the frame is not analyzable, determining that the frame is not trusted, and if determining that the frame is analyzable, fitting at least one Gaussian function to a ratio curve represented by the ratio vector, identifying a peak of the Gaussian function as a best focus position, identifying an amplitude of the ratio vector at the best focus position as a fitted maximum, determining whether the frame is trustworthy based on the best focus position and the fitted maximum, and if determining that the frame is not trustworthy, determining that the frame is not trusted, and if determining that the frame is trustworthy, adding the best focus position to a focus map. Identifying the number of analyzable columns can include identifying a number of columns in the principal gradient vector that exceed a threshold. Calculating the ratio vector can include dividing the gradient slope vector by the principal gradient vector.Determining whether the frame is analyzable can include determining whether a number of analyzable columns exceeds a predefined threshold percentage, determining whether a value of a ratio vector at a parfocal position is within a predefined range, where the parfocal position is a point on the tilted focus line scan camera that is parfocal with the imaging line scan camera, determining that the frame is not analyzable if it is determined that the number of analyzable columns does not exceed a predefined threshold percentage or that the value of the ratio vector at the parfocal position is not within the predefined range, and determining that the frame is analyzable if it is determined that the number of analyzable columns exceeds a predefined threshold percentage and that the value of the ratio vector at the parfocal position is within the predefined range. Fitting at least one Gaussian function to the ratio curve can include sampling a plurality of possible Gaussian functions within a range of average values ​​and within a range of sigma values, and selecting one Gaussian function from the plurality of possible Gaussian functions that has a minimum difference from the ratio curve to be used to identify the best focus position.

[0011] Removing all outlier foci from the focus map may include calculating, for one or more sample points in the focus map, a slope in each of four directions away from the sample point within the focus map, and removing the sample point from the focus map if the minimum of the calculated slopes exceeds a predefined threshold.

[0012] Determining whether to restrip one or more of the plurality of image stripes may include, after removing all outlier foci from the focus map, calculating a focus error for the frame for each of a plurality of frames in each of the plurality of image stripes by subtracting an actual position of the objective lens during acquisition of the frame from a best focus position for that frame within the focus map, and determining to restrip the image stripe if, for each of the plurality of image stripes, the number of frames having a focus error above a predefined threshold exceeds a predefined threshold percentage.

[0013] The method may be embodied in the form of an executable software module for a processor-based system, such as a server, and / or in the form of executable instructions stored on a non-transitory computer-readable medium.

[0014] The details of the present invention, both as to its structure and operation, can be obtained in part by studying the accompanying drawings, in which like reference characters refer to like parts and in which: [Brief description of the drawings]

[0015] [Figure 1A] FIG. 1 illustrates an exemplary processor-enabled device that can be used in connection with various embodiments described herein, according to one embodiment. [Figure 1B] FIG. 2 illustrates an exemplary line scan camera having a single linear array, according to one embodiment. [Figure 1C] FIG. 2 illustrates an exemplary line scan camera having three linear arrays, according to one embodiment. [Figure 1D] FIG. 2 illustrates an exemplary line scan camera having multiple linear arrays, according to one embodiment. [Figure 1E] FIG. 2 is an exemplary side view schematic diagram of a line scan camera in a scanning system, according to one embodiment. [Figure 1F] 2 is an exemplary top view diagram of an imaging sensor relative to an imaging optical path, according to one embodiment. [Figure 1G] FIG. 2 is an exemplary top view diagram of a focus sensor relative to a focusing optical path, according to one embodiment. [Figure 1H] FIG. 2 illustrates an exemplary focus sensor, according to one embodiment. [Diagram 2] 1A-1C show exemplary focus errors before and after applying offset correction using a calculated macro focus offset, according to one embodiment; [Diagram 3] FIG. 1 illustrates an exemplary graph of a low-pass filtered signal and fitting according to one embodiment. [Figure 4] FIG. 1 illustrates parfocal calculations according to one embodiment. [Figure 5A] FIG. 1 illustrates a process for scanning an image of a sample on a glass slide, according to one embodiment. [Figure 5B] FIG. 1 illustrates a process for scanning an image of a sample on a glass slide, according to one embodiment. [Figure 5C] FIG. 1 illustrates a process for scanning an image of a sample on a glass slide, according to one embodiment. [Figure 6A] 2 illustrates an exemplary frame of image data captured by the main imaging sensor, according to one embodiment. [Figure 6B] 2A-2C illustrate example frames of image data captured by a tilt focus sensor, according to one embodiment. [Figure 7] FIG. 2 illustrates a process for calculating a ratio vector according to one embodiment. [Figure 8A] FIG. 2 illustrates example gradient vectors for a main imaging sensor and a gradient focus sensor, according to one embodiment. [Figure 8B] FIG. 8B illustrates an example ratio vector for the two gradient vectors of FIG. 8A according to one embodiment. [Figure 9]FIG. 13 shows ratio curves for a tissue sample scanned at a fixed offset from parfocality, according to one embodiment. [Figure 10] FIG. 1 illustrates an example of a Gaussian fitting process, according to one embodiment. [Figure 11] FIG. 13 illustrates a partial set of Gaussian test functions for different mean values ​​and one fixed width, according to one embodiment. [Figure 12] 1 illustrates an example of a Gaussian fitting process where the ratio curve has two peaks, according to one embodiment. [Figure 13] FIG. 2 illustrates an exemplary set of Gaussian functions, according to one embodiment. [Figure 14] FIG. 1 illustrates an exemplary minimum RMS difference value and location of the best-fit Gaussian function, according to one embodiment. [Figure 15] FIG. 13 illustrates the calculation of error slope according to one embodiment. [Figure 16] FIG. 2 illustrates an example of outlier detection, according to one embodiment. [Figure 17A] FIG. 1 illustrates an exemplary heatmap depicting focus error, according to one embodiment. [Figure 17B] FIG. 1 illustrates an exemplary heatmap depicting focus error, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] In one embodiment, a system, method, and non-transitory computer-readable medium for real-time focusing in a slide scanning system are disclosed. After reading this specification, it will be apparent to one skilled in the art how to implement the present invention in various alternative embodiments and alternative applications. However, although various embodiments of the present invention are described herein, it should be understood that these embodiments are presented only by way of example and illustration, and not by way of limitation. Thus, this detailed description of various embodiments should not be construed as limiting the scope or breadth of the present invention as set forth in the appended claims.

[0017] 1. Exemplary Scanning System

[0018] 1A is a block diagram illustrating an exemplary processor-enabled slide scanning system 100 that can be used in connection with various embodiments described herein. As will be appreciated by those skilled in the art, alternative forms of scanning system 100 may be used. In the illustrated embodiment, the scanning system 100 is presented as a digital imaging device including one or more processors 104, one or more memories 106, one or more motion controllers 108, one or more interface systems 110, one or more movable stages 112 supporting one or more glass slides 114 with one or more samples 116, one or more illumination systems 118 illuminating the samples 116, one or more objective lenses 120 each defining an optical path 122 traveling along an optical axis, one or more objective lens positioners 124, one or more optional epi-illumination systems 126 (e.g., included in a fluorescent scanning embodiment), one or more focusing optics 128, one or more line scan cameras 130, and / or one or more area scan cameras 132 each defining a separate field of view 134 on the sample 116 and / or on the glass slide 114. The various elements of the scanning system 100 are communicatively coupled via one or more communication buses 102. Although each of the various elements of the scanning system 100 may be plural, for purposes of simplifying the following description, these elements will be described in the singular unless a plural description is necessary to convey the appropriate information.

[0019] The processor 104 may include, for example, a central processing unit (CPU) and a separate graphics processing unit (GPU) capable of processing instructions in parallel, or may include a multi-core processor capable of processing instructions in parallel. Additional separate processors may be provided to control specific components or to perform specific functions, such as image processing. For example, the additional processors may include auxiliary processors for managing data input, auxiliary processors for performing floating-point mathematical operations, dedicated processors (e.g., digital signal processors) having an architecture suitable for fast execution of signal processing algorithms, slave processors (e.g., back-end processors) subordinate to the main processor, additional processors for controlling the line scan camera 130, the stage 112, the objective lens 120, and / or a display (e.g., a console including a touch panel display integrated into the scanning system 100). Such additional processors may be separate discrete processors or may be integrated into a single processor.

[0020] The memory 106 provides storage of data and instructions for programs executable by the processor 104. The memory 106 may include one or more volatile and / or non-volatile computer-readable storage media for storing data and instructions. These media may include, for example, random access memory (RAM), read-only memory (ROM), hard disk drives, and / or removable storage drives (including, for example, flash memory), etc. The processor 104 is configured to execute the instructions stored in the memory 106 and communicate with various elements of the scanning system 100 via the communication bus 102 to perform the overall functions of the scanning system 100.

[0021] The communication bus 102 can be configured to carry analog electrical signals and / or digital data. Thus, communications from the processor 104, the motion controller 108, and / or the interface system 110 via the communication bus 102 can include both electrical signals and digital data. The processor 104, the motion controller 108, and / or the interface system 110 can also be configured to communicate with one or more of the various elements of the scanning system 100 via wireless communication links.

[0022] Motion control system 108 is configured to precisely control and coordinate the X, Y, and / or Z movement of stage 112 (e.g., in the XY plane), the X, Y, and / or Z movement of objective lens 120 (e.g., along a Z axis orthogonal to the XY plane, via objective lens positioner 124), the rotational movement of the carousel described elsewhere herein, the lateral movement of the push / pull assembly described elsewhere herein, and / or any other movable components of scanning system 100. For example, in a fluorescent scanning embodiment that includes epi-illumination system 126, motion control system 108 can be configured to coordinate the movement of optical filters, etc. in epi-illumination system 126.

[0023] The interface system 110 enables the scanning system 100 to communicate with other systems and with a human operator. For example, the interface system 110 may include a console (e.g., a touch panel display) for providing information directly to the operator via a graphical user interface and / or for allowing direct input from the operator via a touch sensor. The interface system 110 may also be configured to facilitate communication and data transfer between the scanning system 100 and one or more external devices (e.g., printers, removable storage media, etc.) directly connected to the scanning system 100 and / or one or more external devices (e.g., image storage systems, Scanner Administration Manager (SAM) servers, and / or other management servers, operator stations, user stations, etc.) indirectly connected to the scanning system 100, for example, via one or more networks.

[0024] The illumination system 118 is configured to illuminate at least a portion of the sample 116. The illumination system 118 can include, for example, one or more light sources and illumination optics. The light source can include a variable intensity halogen light source with a concave reflector to maximize light output and a KG-1 filter to suppress heat. The light source can include any type of arc lamp, laser, or other light source. In one embodiment, the illumination system 118 illuminates the sample 116 in a transmission mode such that the line scan camera 130 and / or the area scan camera 132 sense light energy transmitted through the sample 116. Alternatively or additionally, the illumination system 118 can be configured to illuminate the sample 116 in a reflection mode such that the line scan camera 130 and / or the area scan camera 132 sense light energy reflected from the sample 116. The illumination system 118 can be configured to be suitable for examining the sample 116 in any known mode of optical microscopy.

[0025] In one embodiment, the scanning system 100 includes an epi-illumination system 126 to optimize the scanning system 100 for fluorescent scanning. It should be understood that if fluorescent scanning is not supported by the scanning system 100, the epi-illumination system 126 may be omitted. Fluorescent scanning is the scanning of a sample 116 that contains fluorescent molecules, which are photon-sensitive molecules that can absorb light of a specific wavelength (i.e., excitation light). These photon-sensitive molecules also emit light of a higher wavelength (i.e., emission light). Because the efficiency of this photoluminescence phenomenon is very low, the amount of emitted light is often very small. This small amount of emitted light typically causes conventional techniques for scanning and digitizing the sample 116 (e.g., transmission mode microscopy) to fail.

[0026] In an embodiment of the scanning system 100 that utilizes fluorescent scanning, it is advantageous to use a line scan camera 130 that includes multiple linear sensor arrays (e.g., a time-delay integration (TDI) line scan camera) to increase the sensitivity of the line scan camera 130 to light by exposing the same single area of ​​the sample 116 to each of the multiple linear sensor arrays of the line scan camera 130. This is particularly beneficial when scanning weakly fluorescent samples that have low levels of emitted light. Thus, in an embodiment of fluorescent scanning, the line scan camera 130 is preferably a monochrome TDI line scan camera. A monochrome image is ideal for fluorescent microscopy because it more accurately represents the actual signals from the various channels present on the sample 116. As will be appreciated by those skilled in the art, a fluorescent sample can be labeled with multiple fluorescent dyes that each emit light at a different wavelength, also called a "channel."

[0027] Furthermore, because the low and high signal levels of various fluorescent samples present a wide spectrum of wavelengths for the line scan camera 130 to sense, it is desirable for the low and high signal levels that can be sensed by the line scan camera 130 to be similarly wide. Thus, in a fluorescent scanning embodiment, the line scan camera 130 may include a monochrome 10-bit 64 linear array TDI line scan camera. It should be noted that various bit depths for the line scan camera 130 may be used for use with such embodiments.

[0028] The movable stage 112 is configured for precise XY movement under the control of the processor 104 or the motion controller 108. The movable stage 112 can also be configured for Z movement under the control of the processor 104 or the motion controller 108. The movable stage 112 is configured to position the sample 116 at a desired location during capture of image data by the line scan camera 130 and / or the area scan camera 132. The movable stage 112 is also configured to accelerate the sample 116 to a substantially constant velocity in the scan direction and then maintain this substantially constant velocity during capture of image data by the line scan camera 130. In one embodiment, the scanning system 100 can use a highly accurate, tightly coordinated XY grid to assist with positioning of the sample 116 on the movable stage 112. In one embodiment, the movable stage 112 is a linear motor-based XY stage with high precision encoders used in both the X and Y axes. For example, highly accurate nanometer encoders can be used on the axis in the scan direction, as well as on an axis perpendicular to and in the same plane as the scan direction. Stage 112 is also configured to support a glass slide 114 on which sample 116 is placed.

[0029] The sample 116 can be anything that can be investigated by optical microscopy. For example, a microscope glass slide 114 is often used as a viewing substrate for specimens that contain tissues and cells, chromosomes, deoxyribonucleic acid (DNA), proteins, blood, bone marrow, urine, bacteria, beads, biopsy material, or any other type of biological material or substance, dead, alive, stained, unstained, labeled, or unlabeled. The sample 116 can be an array of any type of DNA or DNA-related material, such as complementary DNA (cDNA) or ribonucleic acid (RNA), or protein, deposited on any type of slide or other substrate, including any and all samples commonly known as microarrays. The sample 116 can be a microtiter plate (e.g., a 96-well plate). Other examples of samples 116 include integrated circuit boards, electrophoretic recordings, Petri dishes, films, semiconductor materials, forensic materials, and machined parts.

[0030] The objective lens 120 is mounted on an objective lens positioner 124, which in one embodiment uses highly accurate linear motors to move the objective lens 120 along an optical axis defined by the objective lens 120. For example, the linear motors of the objective lens positioner 124 may include 50 nanometer encoders. The relative position between the stage 112 and the objective lens 120 in the X, Y, and / or Z axes is adjusted and controlled in a closed-loop manner using a motion controller 108 under the control of a processor 104, which uses a memory 106 to store information and instructions, including computer-executable programmed steps for the overall operation of the scanning system 100.

[0031] In one embodiment, the objective lens 120 is a plan-apochromatic ("APO") infinity-corrected objective lens (e.g., Olympus 40X with 0.75 NA, or Olympus 20X with 0.75 NA) suitable for a transmitted-mode illumination microscope, a reflected-mode illumination microscope, and / or an epi-illumination mode fluorescence microscope. Advantageously, the objective lens 120 can be corrected for chromatic and spherical aberrations. Because the objective lens 120 is infinity-corrected, a focusing optic 128 can be placed in the optical path 122 above the objective lens 120, where the light beam passing through the objective lens 120 becomes a collimated light beam. The focusing optic 128 focuses the optical signal captured by the objective lens 120 onto the photoresponsive elements of the line scan camera 130 and / or the area scan camera 132, and can include optical components such as filters and / or magnification lenses. The objective lens 120 in combination with the focusing optics 128 provides the total magnification for the scanning system 100. In one embodiment, the focusing optics 128 can include a tube lens and an optional 2X magnification changer. Advantageously, the 2X magnification changer allows the original 20X objective lens 120 to scan the sample 116 at a magnification of 40X.

[0032] The line scan camera 130 includes at least one linear array of picture elements 142 ("pixels"). The line scan camera 130 can be monochrome or color. A color line scan camera typically has at least three linear arrays, while a monochrome line scan camera can have a single linear array or multiple linear arrays. Any type of linear array or arrays can be used, whether packaged as part of the camera or custom built into the imaging electronics module. For example, a three linear array ("red-green-blue" or "RGB") color line scan camera, or a 96 linear array monochrome TDI can also be used. TDI line scan cameras typically provide a much better signal-to-noise ratio ("SNR") in the output signal by summing intensity data from previously imaged areas of the specimen, resulting in an increase in SNR proportional to the square root of the number of integration stages. TDI line scan cameras include multiple linear arrays. For example, TDI line scan cameras are available with 24, 32, 48, 64, 96, or even more linear arrays. The scanning system 100 also supports linear arrays manufactured in a variety of formats, including 512 pixel formats, 1024 pixel formats, and 4096 pixel formats. Similarly, linear arrays having a variety of pixel sizes can also be used with the scanning system 100. A notable requirement for selecting any type of line scan camera 130 is that the movement of the stage 112 be capable of being synchronized with the line rate of the line scan camera 130 such that the stage 112 can move relative to the line scan camera 130 during capture of the digital image of the sample 116.

[0033] In one embodiment, the image data generated by the line scan camera 130 is stored in a portion of the memory 106 and processed by the processor 104 to generate a successive digital image of at least a portion of the sample 116. The successive digital images may be further processed by the processor 104 and the processed successive digital images may also be stored in the memory 106.

[0034] In an embodiment having two or more line scan cameras 130, at least one of these line scan cameras 130 can be configured to function as a focus sensor operating in combination with at least one other line scan camera 130 configured to function as an imaging sensor 130A. The focus sensor can be logically positioned on the same optical axis as the imaging sensor 130A, or the focus sensor can be logically positioned upstream or downstream of the imaging sensor 130A with respect to the scanning direction of the scanning system 100. In such an embodiment having at least one line scan camera 130 functioning as a focus sensor, image data generated by the focus sensor can be stored in a portion of the memory 106 and processed by the processor 104 to generate focus information that enables the scanning system 100 to adjust the relative distance between the sample 116 and the objective lens 120 to maintain focus on the sample 116 during scanning. Furthermore, in one embodiment, the at least one line scan camera 130 functioning as a focus sensor can be oriented such that each of a plurality of individual pixels 142 of the focus sensor is positioned at a different logical height along the optical path 122.

[0035] In operation, the various components of the scanning system 100 and programmed modules stored in the memory 106 enable the automated scanning and digitization of a sample 116 disposed on a glass slide 114. The glass slide 114 is securely positioned on a movable stage 112 of the scanning system 100 for scanning the sample 116. Under the control of the processor 104, the movable stage 112 accelerates the sample 116 to a substantially constant velocity for sensing by the line scan camera 130, where the velocity of the stage 112 is synchronized with the line rate of the line scan camera 130. After scanning a stripe of image data, the movable stage 112 decelerates and brings the sample 116 to a substantially complete stop. The movable stage 112 then moves orthogonally to the scan direction to position the sample 116 for scanning a subsequent stripe of image data (e.g., an adjacent stripe). Additional stripes are then scanned until an entire portion of the sample 116 or the entire sample 116 has been scanned.

[0036] For example, during digital scanning of the sample 116, successive digital images of the sample 116 are acquired as multiple successive fields of view that are combined together to form an image stripe. Multiple adjacent image stripes are similarly combined together to form a continuous digital image of a portion or the entire sample 116. Scanning of the sample 116 may include acquiring vertical image stripes or horizontal image stripes. Scanning of the sample 116 may be from top to bottom, bottom to top, or both (i.e., bidirectional) and may begin at any point on the sample 116. Alternatively, scanning of the sample 116 may be from left to right, right to left, or both (i.e., bidirectional) and may begin at any point on the sample 116. The image stripes need not be acquired in an adjacent or contiguous manner. Furthermore, the resulting image of the sample 116 may be an image of the entire sample 116 or may be an image of only a portion of the sample 116.

[0037] In one embodiment, computer-executable instructions (e.g., programming modules and software) are stored in memory 106 that, when executed, enable scanning system 100 to perform various functions described herein (e.g., displaying a graphical user interface, executing the disclosed processes, controlling components of scanning system 100, etc.). The term "computer-readable storage medium" is used herein to refer to any medium used to store and provide computer-executable instructions to scanning system 100 for execution by processor 104. Examples of these media include memory 106 and any removable or external storage medium (not shown) communicatively coupled to scanning system 100 directly (e.g., via a Universal Serial Bus (USB), wireless communication protocols, etc.) or indirectly (e.g., via a wired and / or wireless network).

[0038] 1B illustrates a line scan camera 130 having a single linear array 140, which may be implemented as a charge-coupled device ("CCD") array. The single linear array 140 includes a plurality of individual pixels 142. In the illustrated embodiment, the single linear array 140 has 4096 pixels 142. In alternative embodiments, the linear array 140 may have a greater or lesser number of pixels. For example, common forms of linear arrays include 512, 1024, and 4096 pixels. The pixels 142 are arranged linearly to define a field of view 134 of the linear array 140. The size of the field of view 134 varies depending on the magnification of the scanning system 100.

[0039] 1C illustrates a line scan camera 130 having three linear arrays 140, each of which may be implemented as a CCD array. The three linear arrays 140 are combined to form a color array 150. In one embodiment, each individual linear array in the color array 150 detects a different color intensity, including, for example, red, green, or blue. The color image data from each individual linear array 140 in the color array 150 is combined to form a single field of view 134 of color image data.

[0040] FIG. 1D shows a line scan camera 130 with multiple linear arrays 140, each of which can be implemented as a CCD array. The multiple linear arrays 140 are combined to form a TDI array 160. Advantageously, a TDI line scan camera can provide a much better SNR in the output signal by summing intensity data from previously imaged areas of the specimen, resulting in an increase in SNR proportional to the square root of the number of linear arrays 140 (also called integration stages). A TDI line scan camera can include a more diverse number of linear arrays 140. For example, common forms of TDI line scan cameras include 24, 32, 48, 64, 96, 120, and even more linear arrays 140.

[0041] 1E shows an exemplary side view schematic of a line scan camera 130 in a scanning system 100, according to one embodiment. In the illustrated embodiment, the scanning system 100 includes a glass slide 114 with a tissue sample 116, which is positioned on a motorized stage 112, illuminated by an illumination system 118, and moved in a scan direction 170. An objective lens 120 has an optical field 134 that is trained on the slide 114 and provides an optical path 122 for light from the illumination system 118, which may be light that has passed through, reflected from, fluoresced from, or otherwise passed through the objective lens 120. The light travels on optical path 122 to beam splitter 174, which allows a portion of the light to pass through lens 176 to main imaging sensor 130A. The light may optionally be bent by mirror 178, as shown in the illustrated embodiment. Imaging sensor 130A may be, for example, a linear charge-coupled device (CCD).

[0042] The other light travels from beam splitter 174 through lens 180 to focus sensor 130B, which may also be, for example, a linear CCD. The light traveling to imaging sensor 130A and the light traveling to focus sensor 130B preferably each represent the complete optical field 134 from objective lens 120. Based on this configuration of scanning system 100, the scan direction 170 of slide 114 is logically oriented relative to imaging sensor 130A and focus sensor 130B such that logical scan direction 172 causes optical field 134 of objective lens 120 to pass through each of imaging sensor 130A and focus sensor 130B.

[0043] FIGURE 1F illustrates an example top-down configuration of imaging sensor 130A relative to imaging optical path 122A, according to one embodiment. Similarly, FIGURE 1G illustrates an example top-down configuration of focus sensor 130B relative to focusing optical path 122B, according to one embodiment. As can be seen in FIGURE 1G, focus sensor 130B is tilted at an angle θ with respect to a direction perpendicular to focusing optical path 122B.

[0044] FIG. 1H illustrates an exemplary focus sensor 130B, according to one embodiment. In the illustrated embodiment, the focus sensor 130B includes multiple sensor pixels 142 within a focus range (d) (e.g., 20 μm) on the tissue sample. As illustrated, the focus sensor 130B can be positioned such that the entire focus range (d) in the Z axis is transferred by the optical system to the entire array of focus sensors 130B in the Y axis (orthogonal to the X axis, i.e., the scan direction 170). The location of each sensor pixel 142 is directly correlated to the Z position of the objective lens 120. As illustrated in FIG. 1H, each dashed line (i.e., p1, p2,..., p3) that intersects the projected focus range (d) is a pixel. i ,···p n ) each represent a different focus value and correspond to a respective focus height (i.e., Z height) of the objective lens 120. The p i can be used by the scanning system 100 to identify the optimal focal height for that portion of the sample 116.

[0045] The relationship between the focus range (d) projected on the focus sensor 130B and the focus range (z) on the sample 116 is given by: d=z×M focusing 2 where M focusing is the optical magnification of the focusing optical path. For example, z=20 μm and M focusing If = 20, then d = 8 mm.

[0046] In order for the entire projected focus range (d) to be covered by the tilted focus sensor 130B including the linear array 140, the tilt angle θ should satisfy the following relationship: sinθ=d / L where L is the length of the linear array 140 of focus sensor 130B. With d=8 mm and L=20.48 mm, θ=23.0°. θ and L can be varied as long as tilted focus sensor 130B covers the entire focus range (d).

[0047] The focusing resolution, or the minimum increment of the objective lens height movement Δz, is a function of the size of the sensor pixel 142, i.e., e=min(ΔL). From the above equation, Δz=e×z / L For example, if e=10 μm, L=20.48 mm, and z=20 μm, then Δz=0.0097 μm<10 nm.

[0048] Objective lens height Z i and the focus position L of the focus i on the focus sensor 130B. i The relationship between L i =Z i ×M focusing 2 / sinθ It is.

[0049] If the focus height is determined by the average of L1 to L2, then the height of the objective lens 120 may be determined according to an analysis of the data from the focus sensor 130B as: Z2 = Z1 + (L2 - L1) × sinθ / M focusing 2 Based on this, we need to move from Z1 to Z2.

[0050] The fields of view (FOV) 134 in the Y-axis of the focus sensor 130B and the imaging sensor 130A may be different, but the centers of both sensors 130A and 130B are preferably aligned with each other along the Y-axis.

[0051] 2. Process Overview

[0052] An embodiment of a process for real-time focusing in a slide scanning system is described in detail below. It should be understood that the described process can be embodied in one or more software modules executed by one or more hardware processors 104 within the scanning system 100. The described process can be implemented as instructions expressed in source code, object code, and / or machine code. These instructions may be executed directly by the hardware processor, or alternatively, may be executed by a virtual machine operating between the object code and the hardware processor.

[0053] Alternatively, the described processes may be implemented as hardware components (e.g., general purpose processors, integrated circuits (ICs), application specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates, or transistor logic, etc.), a combination of hardware components, or a combination of hardware and software components. To clearly illustrate the interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described herein generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in different manners for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention. Furthermore, the grouping of functions within a component, block, module, circuit, or step is for ease of description. Certain functions or steps can be moved from one component, block, module, circuit, or step to another component, block, module, circuit, or step without departing from the invention.

[0054] Additionally, while the processes described herein are shown with a particular arrangement and order of steps, each process may be implemented with fewer, more, or different steps, and with different arrangements and / or orders of steps. Additionally, it should be understood that any step that is not dependent on the completion of another step may be performed before, after, or in parallel with other independent steps, even if those steps are described or illustrated in a particular order.

[0055] In one embodiment, the scanning system 100 uses a focus map to predict the trajectory of the objective lens 120 during the scanning of each image stripe. The focus values ​​for the focus map can be measured using two techniques: (1) a macro focus point (MFP) technique; and (2) a real-time focus (RTF) technique. The focus values ​​for the MFP technique are calculated before scanning the image stripe and / or between acquisitions of the image stripe, whereas the focus values ​​for the RTF technique are calculated while acquiring the image stripe. Both techniques can be used in combination to create a focus map that is used to predict the focus position of the objective lens 120 during the scan. Advantageously, the RTF technique provides many more focus values ​​to the focus map than using only the MFP technique, but adds little or no time to the scanning process.

[0056] The RTF method also provides real-time measurements of focus errors that can be analyzed during the scanning of the sample 116 in order to correct the trajectory of the objective lens 120 as the image stripe is scanned, thereby minimizing the focus error at the focus height predicted from the focus map.

[0057] 2.1. MFP method

[0058] In one embodiment, the MFP method involves capturing image data along the entire Z axis (e.g., by moving the objective lens 120) using the line scan camera 130 while the stage 112 moves at a constant velocity. The image row with the greatest contrast within the image data is then identified, and a timing formula is used to calculate the encoder count (e.g., for the objective lens 120) that corresponds to that row. A minimum contrast threshold, representing the noise threshold, can be used to ensure that the focus is above the noise threshold. Historically, this minimum contrast threshold has been around 350.

[0059] Historically, the MFP method requires a macro focus offset to provide good image quality. Therefore, in one embodiment, the macro focus offset is calculated by testing the focus value in a closed-loop measurement to ensure that the MFP method is performed accurately. By design, the macro focus offset is required to be zero. However, in practice, there is a systematic error in the Z position of the objective lens 120, as calculated from a given formula and Z stage adjustment.

[0060] The macro focus offset can be experimentally determined by macro focusing on a location on the tissue and recording the contrast curve and the encoder count of maximum contrast. The objective lens 120 can then be moved (i.e., in the Z axis) to the recorded encoder count and a second buffer of image data can be recorded along with the corresponding average contrast value. The average contrast value can be compared to the recorded contrast curve and the distance from the recorded maximum contrast value to the average contrast value can be measured to provide the Z offset to be used as the macro focus offset.

[0061] FIG. 2 shows an example focus error before and after applying offset correction using a calculated macro focus offset, according to one embodiment. As shown, the average error was reduced from 0.6 to 0.0 microns, and the maximum error was reduced from 1.0 to 0.4 microns. The impact on image quality was a reduction in the restriping rate (i.e., the rate at which an image stripe had to be reacquired due to lack of focus). This offset correction is important because it allows for additional focus when and where it is needed, providing a ground truth for evaluating RTF methods.

[0062] In one embodiment, the MFP parameters, along with exemplary nominal values, are defined as follows and can be stored in a configuration file (e.g., a “scanner.xml” file defined using Extensible Markup Language (XML)) used by the scanning system 100 for configuration:

[0063] Good_Focus_Threshold=350. If the maximum contrast value of a macro focus is less than this threshold, the focusing attempt for that macro focus is considered to have failed and the macro focus value is discarded.

[0064] Retry_Count=1. This is the number of times to try to refocus if a focusing attempt fails due to the maximum macro focus contrast value being less than the value of Good_Focus_Threshold.

[0065] Macrofocus_Pos_Offset=+0.0006. This is the calculated macrofocus offset value (in millimeters) due to errors in the Z motion of the objective lens 120.

[0066] 2.2. RTF method

[0067] In one embodiment, the RTF method utilizes two line scan cameras 130: a primary imaging sensor 130A (e.g., a tri-linear RGB camera) and a single channel focus sensor 130B (e.g., a monochromatic camera). Both line scan cameras 130 are aligned such that their respective linear arrays 140 image the same portion of the sample 116 (which may include tissue, for example). For example, the primary imaging sensor 130A may be parallel to the plane of the sample 116 and function similarly to the tri-linear camera used in Aperio ScanScope® products. On the other hand, the focus sensor 130B is tiltable in the optical Z-axis (e.g., perpendicular to the scan direction 170) along the linear array 140 of the focus sensor 130B.

[0068] 2.2.1. design

[0069] The line scan cameras 130A and 130B can be aligned with each other such that when the main imaging sensor 130A is at best focus, the point of maximum contrast of the tilted focus sensor 130B is near the center of the tilted linear array 140. The pixel 142 at this point in the tilted linear array 140 of the focus sensor 130B is called the parfocal position. As the objective lens 120 moves up or down relative to the best focus position, the point of maximum contrast on the tilted focus sensor 130B moves respectively to the left and right of the parfocal position. This allows the tilted focus sensor 130B to be used to dynamically determine the direction and amount of focus error of the main imaging sensor 130A. The measured focus error can be used to adjust the position of the objective lens 120 in real time so that the main imaging sensor 130A is always in focus.

[0070] An embodiment of the RTF approach may address one or more, preferably all, of the following issues:

[0071] Variation in tissue contrast across the linear array. In one embodiment, not only the gradient focus sensor 130B is used to identify the best focus position, since contrast variations also arise from variations in tissue characteristics across the linear array 140. Data from the focus sensor 130B can be combined with data from the main imaging sensor 130A to remove tissue effects using a ratio method. The ratio method divides the contrast function of the gradient focus sensor 130B by the contrast function of the main imaging sensor 130A. This normalized function peaks at the best focus position and removes tissue dependent effects.

[0072] Noise and camera alignment errors. In one embodiment, the linear array 140 is 1×4096 pixels, with a nominal image pixel dimension of 0.25 microns. With this level of precision, it is not possible to precisely align the main imaging sensor 130A and the tilted focus sensor 130B. In addition, there is a slight magnification change across the tilted focus sensor 130B due to the tilt of the tilted focus sensor 130B relative to the optical axis. The contrast values ​​of individual pixels 142 are also noisy because they are calculated by taking the difference in values ​​between adjacent pixels 142. To mitigate these effects, spatial averaging can be used. For example, in one embodiment, the output of each line scan camera 130 is grouped into consecutive frames as it scans the image stripe. A frame is 1000 lines by 4096 pixels. After the pixel contrast values ​​for a frame are calculated, the lines are averaged into a single line. A box car filter (e.g., 100 pixels wide) is then applied across this single line. These operations are performed on frames from both the main imaging sensor 130 A and the tilted focus sensor 130 B. These two averaging operations significantly reduce the effects of noise and alignment errors in the ratio method.

[0073] Parfocal position. The value of the parfocal position is known and can be separately calibrated. Any error in the parfocal position will cause the RTF method to move to a position that is not the best focus. Methods for calculating the parfocal position are described elsewhere in this specification.

[0074] Skipped frames. In one embodiment, all other frames are "skipped" frames because data for these frames is not analyzed for focus error. During acquisition of a skipped frame, data for the previous frame is analyzed and the objective lens 120 is moved to the best focus position for the next non-skipped frame. In other words, the objective lens 120 moves only during acquisition of a skipped frame and remains stationary during acquisition of a non-skipped frame. In normal operation, the frame lag is one frame, which means that the current frame under objective lens 120 (CFUO) is the frame following the frame being analyzed.

[0075] Over-framing. Over-framing is the term used when the frame lag is greater than one frame. In one embodiment, since the scanning of the image stripe proceeds at a constant rate, a software timer is implemented, which allows the CFUO to be calculated. The RTF method checks the CFUO when identifying the next best focus position. Typically, the CFUO is a skipped frame following the frame being analyzed. If not, the RTF method predicts the best focus position for the first non-skipped frame following the CFUO and instructs the objective lens 120 to move to that position. Using this strategy, the objective lens 120 is always moved to the best position for the current actual position. As long as the frame lag is not too large, the RTF method works well. A configuration parameter Frame_Lag_Stripe_Abort_Threshold can be provided that causes the scanning of the image stripe to be aborted and rescanned if the frame lag becomes too large (i.e., exceeds the value of the Frame_Lag_Stripe_Abort_Threshold parameter).

[0076] Fitting a ratio curve. The ratio method produces a curve that peaks at the best focus position. In one embodiment, a Gaussian function is fitted to the ratio curve, and the peak of the Gaussian function is identified as the best focus position. The goodness of the fit of the Gaussian function can be evaluated using a set of metrics to qualify the focus estimate as reliable. Only reliable focus values ​​are added to the focus map.

[0077] Focusing on tissue only. Points added to the focus map are expected to correspond to actual tissue locations in the image data. In previous designs, a tissue-finding algorithm was used to calculate a probable tissue map from the macro image and MFP values ​​were added only to probable tissue locations. However, various artifacts on the glass slide 114 (e.g., the edge of the cover slip, plus marks, dirt on the cover slip, etc.) can result in undesired focus outside the plane of the tissue. In one embodiment, the probable tissue map from the tissue-finding algorithm can be input to the RTF method and can optionally be used to constrain the allowable locations for focus used by the RTF method. The RTF method can also analyze each frame to determine if tissue is present and allow focus only on frames where tissue is present.

[0078] Tissue Gaps. In order to reliably estimate focus error, sufficient tissue contrast must be available across the linear array 140. In one embodiment, a contrast vector for the main imaging sensor 130A is used to determine whether sufficient signal is available to calculate focus error. Details of this implementation are described elsewhere herein.

[0079] Adding MFP values. The RTF method does not always return reliable focus values ​​for each tissue frame. This may be due to insufficient tissue in the frame, as well as the results being classified as unreliable by the Gaussian fitting process. In one embodiment, after an image stripe is scanned, the tissue frames are considered sequentially, and frames that are more than a predefined distance (e.g., stored as a value in the form of an Rmin parameter) from the nearest reliable focus value are identified for focus by the MFP. These points can be focused and added to the focus map before scanning the next image stripe.

[0080] Outlier Rejection: In one embodiment, after all image stripes are scanned, the focus map is examined to determine if any of the foci in the focus map are outliers. Before evaluating the scan quality, outliers can be removed as discussed elsewhere herein.

[0081] Rescanning Image Stripes. In one embodiment, after all image stripes have been scanned and the focus map is complete, the actual focus position for each tissue frame is compared to the best focus position from the focus map, as described in the restriping process discussed elsewhere herein. Image stripes in which 5% of the frames have focus errors above a predefined threshold may be rescanned to create the final focus map.

[0082] Scanning initialization. In one embodiment, a focus map for scanning is initialized. A tissue finding algorithm can identify a reference stripe and three or more MFP values ​​to start the scan. The image stripe with the most tissue can be selected as the reference stripe. The MFP focus can be measured at the beginning of the reference stripe and at two or more other locations selected to provide good spacing of the tissue. The reference stripe is then scanned first using the starting MFP focus value of the stripe, and the focus position is updated using the RTF method. Scanning the reference stripe typically generates a number of focus values, which are added to the focus map along with the initial MFP focus value.

[0083] Scanning Order. In one embodiment, after the focus map is initialized, the reference stripe is rescanned. Scanning then proceeds to the right or left of the reference stripe until that side of the scan area is completed. Scanning then proceeds from the other side of the reference stripe to the opposite edge of the scan area. This order is selected to ensure that the image stripe being scanned is as close as possible to the focus value in the focus map to maximize the chances of obtaining acceptable focus on the first pass.

[0084] 2.2.2. parfocal position

[0085] In one embodiment, to identify the parfocal position, a vertical Ronchi slide is imaged simultaneously by both the main imaging sensor 130A and the tilted focus sensor 130B by sweeping at a constant speed through the Z range of the objective lens 120. In this manner, a buffer pair of image data is acquired for sensors 130A and 130B.

[0086] For the primary imaging sensor 130A, an average contrast value is calculated for each row of the buffer, and the row with the maximum contrast is taken as the best focus index. An additional check can be added to qualify the buffer pair based on the gradient of the contrast of the image data from the primary imaging sensor 130A. This can be done by dividing the buffer of image data from the primary imaging sensor 130A into 40 segments (e.g., about 100 columns each) and calculating the best focus index for each segment. If the difference in index between the leftmost segment and the rightmost segment is less than a threshold value (e.g., 4), the buffer pair is accepted as having the least gradient and is used to estimate the parfocal position. The threshold value of 4 corresponds to a gradient of 0.5 microns / millimeter, which may be a system requirement for flatness.

[0087] The tilted focus sensor 130B does not have a single best focus index (i.e., row). A column in the buffer of image data from the focus sensor 130B has the maximum contrast when the pixel 142 corresponding to that column is at the best focus. The processing of the focus buffer proceeds to calculate the gradient of each row, and then finds the row index corresponding to the maximum value for each column. This data can be very noisy, as shown in the example graph of the low-pass filtered signal and fitting in FIG. 3. Therefore, due to the asymmetry of the noise, a median filter can be used rather than a mean filter to reduce the noise. A linear fit is then constructed to the filtered index values, as shown by the fit line in FIG. 3. The column where the linear fit intersects with the maximum index value from the image data acquired by the main imaging camera is the point of parfocality in the tilted focus sensor 130B.

[0088] The slope of the linear fit corresponds to the change in Z distance per pixel and is needed to calculate the actual distance to move the objective lens 120 to the parfocal point. However, the left side of the parfocal position appears to have a larger slope than the right side. Therefore, as shown in Figure 3, a linear fit is calculated separately for the data for the left side of the parfocal position and the data for the right side of the parfocal position. These two slopes provide the left and right scale factors in microns / pixel.

[0089] In one embodiment, the output of the parfocal calculation includes a parfocal position, a left scale factor, and a right scale factor. The parfocal position is the position of the parfocal pixel 142 on the tilted focus sensor 130B. The left scale factor is used to convert pixels 142 to the left of the parfocal position into microns. Similarly, the right scale factor is used to convert pixels 142 to the right of the parfocal position into microns.

[0090] 4 illustrates a parfocal calculation 400 according to one embodiment. The parfocal calculation 400 can be implemented in the form of software instructions executed by the processor 104 of the scanning system or an external system. In steps 405A and 405B, synchronized buffers are received from the tilt focus sensor 130B and the main imaging sensor 130A, respectively. In one embodiment, each buffer contains image data acquired by the respective sensor during scanning of a vertical Ronchi slide.

[0091] In steps 410A and 410B, the contrast gradient for each row in each buffer is calculated according to a predefined step. By default, the step for both buffers may be one row so that no rows are skipped. Alternatively, the step may be greater than one row.

[0092] In step 415, the maximum value of the contrast gradient is found for the focus sensor 130B and the column index corresponding to that maximum is identified. A median filter (e.g., default=500) is then applied to the contrast gradient points in step 420. In step 425, a linear fit is found to the median filtered points. A line representing this linear fit is sent to step 470, with the assumption that a line will also be found for the buffer of image data from the main imaging sensor 130A in step 465.

[0093] The contrast gradients calculated from the buffer of the main imaging sensor 130A in step 410B are averaged across each row in step 430. The R, G, and B color channels are then averaged in step 435. The contrast gradients are segmented in step 440. By default, the number of segments used in step 440 may be, for example, 40.

[0094] In step 445, the leftmost segment from step 440 is compared with the rightmost segment from step 440. Specifically, the rightmost segment can be subtracted from the leftmost segment or vice versa. If the absolute value of the difference is greater than or equal to a predefined threshold T (i.e., "no" in step 450), the buffer pair received in step 405 can be discarded in step 455 and the process 400 can be restarted using a new buffer pair. Otherwise, i.e., if the absolute value of the difference is less than the predefined threshold T (i.e., "yes" in step 450), the process 400 proceeds to step 460. In one embodiment, the predefined threshold T is equal to 4.

[0095] In step 460, the maximum of the averages calculated in steps 430 and 435 is found and the row index that corresponds to that maximum is identified. Then, in step 465, the line is calculated at this point.

[0096] Once a line is found for focus sensor 130B in step 425 and a line is found for image sensor 130A in step 465, the intersection or crossing point of these two lines is found in step 470. This crossing point is the point of parfocality. Further, in step 475, a linear fit is found independently for both the segment to the left of the parfocal point and the segment to the right of the parfocal point. Finally, in step 480, the slope of the left fit line is converted to a left scale factor and the slope of the right fit line is converted to a right scale factor.

[0097] 2.2.3. RTF-based workflow

[0098] 5A-5C show a process 500 for scanning an image of a sample 116 on a glass slide 114, according to one embodiment. Many of the illustrated steps are described in further detail elsewhere herein. It should be understood that the process 500 can be implemented in the form of software instructions executed by the processor 104 of the scanning system 100.

[0099] The process 500 begins in step 510 by acquiring an image stripe representing image data of a portion of the sample 116. In step 590, it is determined whether the last image stripe has been acquired. If image stripes remain to be acquired (i.e., "no" at step 590), the next image stripe is acquired in a further iteration step 510. Otherwise, i.e., if no image stripes remain to be acquired (i.e., "yes" at step 590), the process 500 proceeds to step 592. Notably, the image stripes may be acquired in any order. Advantageously, the ability to acquire image stripes in any order allows the processor 104 of the scanning system 100 to more effectively build a focus map during the scan using focus values ​​acquired by the RTF method.

[0100] In step 592, outliers are removed, as discussed in more detail elsewhere herein. In step 594, it is determined whether any of the image stripes need to be restriped, as discussed in more detail elsewhere herein. If the image stripes do not need to be restriped (i.e., "no" at step 594), the process 500 ends with a complete set of image stripes for at least a portion of the sample 116. Otherwise, if at least one image stripe needs to be restriped (i.e., "yes" at step 594), then in step 596, those image stripes are rescanned, after which the process 500 ends with a complete set of image stripes for at least a portion of the sample 116. Once a complete set of image stripes has been acquired, the processor 104 of the scanning system 100 can align and combine the image stripes into one complete composite image of the entire scanned portion of the sample 116. Additionally, the processor 104 can compress the composite image using any known compression technique.

[0101] An embodiment of step 510 is shown in more detail in Figure 5B. Specifically, in step 512, a frame of the image stripe is captured at the calculated Z position. Then, in step 520, the captured frame is processed to determine the Z position of the next frame. If frames remain to be acquired (i.e., "no" at step 580), then in a further iteration step 512, the next frame is captured at the Z position determined in step 520. Otherwise, i.e., if no frames remain to be acquired for that image stripe (i.e., "yes" at step 580), process 500 proceeds to step 582.

[0102] In step 582, the Z position with the best focus for the analyzable and reliable frame is added to the focus map. In step 584, additional macro focus is requested. In step 586, the scan direction is set; that is, process 500 determines whether to move to the left or right of the current image stripe to capture the next image stripe.

[0103] An embodiment of step 520 is shown in further detail in Figure 5C. Buffers of image data captured by both the primary imaging sensor 130A and the tilted focus sensor 130B, each representing a frame, are processed in step 522 to generate the location of the maximum value of the tilted focus sensor 130B buffer, the root mean square gradient of the primary imaging sensor 130A buffer, the total number of analyzable columns in the primary imaging sensor 130A buffer, a weight vector for the primary imaging sensor 130A, and a ratio vector (parfocal ratio).

[0104] At step 524, it is determined whether the captured frame is analyzable. For example, a frame is determined to be analyzable if it has enough tissue to perform a Gaussian fitting process, as described elsewhere herein. If the frame is analyzable (i.e., "yes" at step 524), process 500 proceeds to step 525. Otherwise, i.e., if the frame is not analyzable (i.e., "no" at step 524), process 500 proceeds to step 530.

[0105] In step 525, Gaussian fitting is performed. In step 526, it is determined whether the Gaussian fitting process is reliable (i.e., a good fit) or unreliable (i.e., a poor fit). If the Gaussian fitting result is reliable (i.e., "yes" at step 526), ​​process 500 provides the resulting predicted delta-Z (i.e., the predicted change in Z-value to maintain focus) to step 528. Otherwise, i.e., if the Gaussian fitting result is unreliable (i.e., "no" at step 526), ​​process 500 sets the frame as unanalyzable and unreliable and proceeds to step 530.

[0106] In step 528, the best focus position is calculated as the sum of the Z position of the current frame and the predicted delta Z from the Gaussian fitting process in step 526. Alternatively, in step 530, the best focus position is simply set to the Z position of the current frame. In either case, in step 532, the Z position for the next frame is calculated.

[0107] 2.2.4. ratio method

[0108] As discussed with respect to process 500, in one embodiment, image data from imaging sensor 130A and focus sensor 130B are captured in a frame. A single frame includes two buffers, one corresponding to data from main imaging sensor 130A and another corresponding to data from tilted focus sensor 130B. Figure 6A shows an exemplary buffer of image data acquired by main imaging sensor 130A, and Figure 6B shows an exemplary buffer of image data acquired by tilted focus sensor 130B, according to one embodiment. In the illustrated embodiment, each frame consists of two buffers, each 1000 lines by 4096 pixels wide.

[0109] 7 illustrates a process 700 for calculating a ratio vector, according to one embodiment. It should be understood that the process 700 can be implemented in the form of software instructions executed by the processor 104 of the scanning system 100. Furthermore, the process 700 can represent at least a portion of steps 522 and / or 524 in the process 500.

[0110] In steps 705A and 705B, frames of image data are received from the main imaging sensor 130A and the tilted focus sensor 130B, respectively. Illumination correction is then applied to the image pixels within each frame in step 710. For example, the illumination correction may utilize sensitivity non-uniformity (PRNU) and / or fixed pattern noise (FPN) techniques. In step 715, the RGB channels in the frame from the main imaging sensor 130A are each corrected separately and then averaged with equal weighting into a grayscale frame.

[0111] In step 720, a squared gradient operator is applied to each grayscale frame, i.e., the primary imaging frame corrected in step 710A and transformed in step 715, and the tilted focus frame corrected in step 710B. Central differencing in both horizontal dimensions can be used, with a difference interval of 8 pixels (D8).

[0112] In step 725, the gradient images are averaged along the columns. This converts each frame into a vector. Then, in step 730, a boxcar filter (e.g., 101 pixels) is applied to each vector to reduce residual noise. Finally, in step 735, a ratio vector is calculated by dividing the pixel values ​​for the two vectors.

[0113] In one embodiment, process 700 is performed for the entire set of frames captured by imaging sensor 130A and focus sensor 130B, including skipped frames, in which case only non-skipped frames are analyzed for optimal focus position, but it is still useful to know whether tissue is present in each frame.

[0114] In addition to the gradient and ratio vectors for each set of frames, the pixel location of the maximum value in the gradient gradient vector and the total number of analyzable columns in the main imaging gradient vector can also be calculated. The number of analyzable columns can be used (e.g., in step 524 of process 500) to determine whether sufficient signal is present to allow further analysis by a Gaussian fitting process.

[0115] 8A shows an example gradient vector for the main imaging sensor 130A (the darker line graph starting and ending at the top) and the gradient focus sensor 130B (the lighter line graph starting and ending at the bottom), according to one embodiment. The structure in the gradient focus sensor 130B is irregular due to tissue variations across frames. Therefore, no single peak can be identified in the gradient vector for the focus sensor 130B.

[0116] 8B shows an example ratio vector from the two gradient vectors of FIG. 8A, according to one embodiment. Notably, the ratio vector is much smoother than either gradient vector. As shown, a Gaussian function can be fitted to the ratio vector to allow for accurate identification of a peak 800. The peak 800 in the Gaussian function represents the best focus position for this frame.

[0117] In one embodiment, the output from the ratio method exemplified by process 700 is input to a Gaussian fitting process, which may include one or more of the following:

[0118] Main Gradient Vector The main gradient vector is the average gradient signal for each column of a frame captured by the main imaging sensor 130A.

[0119] Gradient gradient vector: The gradient gradient vector is the average gradient signal for each column of the frame captured by the gradient focus sensor 130B.

[0120] Ratio Vector: The ratio vector is the ratio of each column of the gradient vector divided by the principal gradient vector.

[0121] Parfocal Ratio. Parfocal ratio is the value of the ratio curve represented by the ratio vector at the parfocal position.

[0122] Baseline Ratio: The baseline ratio is the average value of the ratio curve represented by the ratio vector near the ends of the ratio vector.

[0123] Number of analyzable columns. The number of analyzable columns is the number of columns in the main gradient vector whose main gradient vector exceeds a threshold (e.g., MAIN_IMG_COLM_ANALYZABLE_THRESH=50).

[0124] Weight Vector. The principal gradient vector is normalized to unit area and used as the weight vector for the Gaussian fitting. Columns (i.e. pixels) that have little tissue (i.e. small gradient values) are less important in fitting the Gaussian function to the ratios because these ratio values ​​are very noisy. The larger the gradient values ​​are, the more signal there is and the correspondingly less noise in the ratios.

[0125] 2.2.5. Gaussian fitting

[0126] The Gaussian fitting process is represented as step 525 of process 500. The purpose of the Gaussian fitting process is to fit a smooth Gaussian function to the ratio curve and then identify the peak of this Gaussian function (e.g., peak 800 in FIG. 8B). This peak represents the calculated best focus position. In one embodiment, the Gaussian fitting process is attempted only if there is sufficient tissue and the ratio curve has an acceptable value at parfocal. For example, if the ratio at parfocal is between 0.5 and 1.5 and the number of analyzable columns is greater than 85% of the total columns (e.g., "yes" at step 524 of process 500), the Gaussian fitting process is performed.

[0127] Fitting a Gaussian function to the ratio curve is a non-linear problem, and in one embodiment, an approach to solving this non-linear problem is to sample a set of possible Gaussian functions and select the Gaussian function that has the smallest root-mean-square (RMS) difference from the ratio curve.

[0128] The Gaussian curve for each sample has four parameters: amplitude (peak), center (mean), width (sigma), and baseline (offset). The amplitude is parameterized as a function of distance from the parfocal point. Figure 9 shows the ratio curve for a tissue sample scanned at a fixed offset from the parfocal point. In this example, the parfocal point is at column (i.e., pixel) 1590. In particular, as the offset from the parfocal point increases, the peak increases. The size of the peak is also tissue dependent. In this way, the rate of increase of the peak from the parfocal point can be estimated for each frame.

[0129] In one embodiment, to scale the Gaussian test function, the location of the maximum of the gradient vector is identified, as shown in the leftmost graph of Figure 10. The ratio vector value at this location and the distance of this location from the parfocal point can be used to define the slope of the fit, as shown in the rightmost graph of Figure 10.

[0130] In one embodiment, the Gaussian test function is scaled by a ratio equal to the slope of the fit so that the peak increases with distance from the parfocal point. One of these Gaussian test functions is shown in the rightmost graph of FIG. 10 as a smooth line approximating the ratio curve. In particular, it is not necessary to center the best Gaussian function at the peak location obtained from the ratio curve, since this peak is only used for scaling, not necessarily centered. The ratio at the baseline of the leftmost graph of FIG. 10 is about 3.5. This value is extrapolated from the end of the ratio curve and used as an offset applied to the Gaussian test function to raise it to an appropriate level.

[0131] In one embodiment, a set of means and sigmas are used to generate a Gaussian test function. The mean values ​​can range from 100 to 3996 (i.e., 4096-100) in increments of 25. The sigma values ​​can range from 500 to 1200 in increments of 100. Figure 11 shows a partial set of Gaussian test functions for different mean values ​​and one fixed width.

[0132] A second set of modified Gaussian functions (only one-sided) is also added to this set. The reason is that the ratio curve may not have a single well-defined shape corresponding to a single peak. For example, the ratio curve may be wider when there are two peaks as shown in FIG. 12. Experience has shown that the subjective image quality is best when focusing on the right-most peak. In the case shown in FIG. 12, this right-most peak is also the largest and most prominent peak. The problem with symmetric Gaussian functions is that the result may be a broad peak centered between the two peaks. In this case, neither set of features is in focus and the image will look "sweet" and will be defocused throughout the frame. The right-most peak represents the direction away from the glass slide 114, and therefore this set of Gaussian functions will prefer to focus towards the upper features in the tissue section when there are two possible depths of focus.

[0133] FIG. 13 shows two sets of Gaussian functions (i.e., symmetric and one-sided) and the means corresponding to the parfocal positions. The full complement of Gaussian test functions, including possible solutions, is found in the set of mean and sigma values ​​described above. Each of these Gaussian functions is normalized for amplitude and baseline as described above. The RMS difference is then calculated for the ratio vector for each Gaussian test function. The Gaussian function associated with the smallest RMS difference value is selected as the best fitting Gaussian function. Point 1400 in FIG. 14 shows the location of the smallest RMS difference value and the best fitting Gaussian function.

[0134] In one embodiment, the Gaussian fitting process returns two numbers: the best focus position and the fit maximum. The best focus position is the column (i.e., pixel) that corresponds to the best focus. The focus error is proportional to the difference between this value and the isofocus point. The fit maximum is the amplitude of the ratio vector at the best focus position.

[0135] In one embodiment, the returned values ​​from the Gaussian fitting process are analyzed to determine if they are trustworthy. Only the trustworthy values ​​are added to a focus map to be used to scan subsequent image stripes. For example, the slope of the error is calculated and compared to the slope of the fit defined above for Gaussian fitting. For the returned value to be trusted, these two slopes should be equal. If not, the returned value should not be trusted. The slope of the error can be calculated as follows and is shown in FIG. 15:

[0136] Error slope = (max fit - ratio at parfocal point) / (best focus - parfocal point)

[0137] There are two possible cases:

[0138] Case 1: The error slope and the fitting slope have different signs (i.e., on either side of the parfocal). In this case, the returned values ​​are reliable if the absolute value of (fit max-parfocal ratio) / (parfocal ratio) is less than 0.20. These slopes are noisy near the parfocal point. Therefore, they are not used to invalidate the Gaussian result.

[0139] Case 2: The error slope and the fitting slope have the same sign (i.e., on the same side of parfocality). In this case, the returned value is reliable if (fitting slope - error slope) / (fitting slope + error slope) is less than 0.5. The difference of these slopes is equal to the average of these two slopes.

[0140] 2.2.6. Frame Analysis Possible Score

[0141] In one embodiment, each frame of image data receives (e.g., in step 524 of process 500) one of the following status scores:

[0142] NonAnalyzable (eg, =-2): The frame has no organization.

[0143] NonAnalyzableButHasTissue (e.g., =-1): The frame has tissue but not enough for the Gaussian fitting process. In one embodiment, a frame has tissue if the average of all columns for the main gradient vector is greater than the value of the MAIN_IMG_COLM_ANALYZABLE_THRESH parameter (e.g., 50).

[0144] AnalyzableButUntrustable (e.g., =0): The results of the Gaussian fitting process are not trustworthy.

[0145] AnalyzableButSkipped (eg, =1): The frame has enough tissue for the Gaussian fitting process, but is a skipped frame.

[0146] AnalyzableAndTrustable (e.g., =2): The Gaussian fitting process returned a reliable result. The point is added to the focus map to be utilized for focusing when scanning subsequent image stripes.

[0147] MFPFrame (e.g., =3): The frame has macro focus. In one embodiment, only one focus value is allowed per frame, so frames that receive macro focus before scanning begins do not receive an RTF value.

[0148] 2.2.7. Outlier rejection

[0149] After all image stripes have been scanned, the focus map is complete. At this point, in one embodiment, the focus map is analyzed to determine if any points in the focus map (either RTF points or MFP points) are outliers. This determination is represented as step 592 in process 500.

[0150] Outliers can be identified by considering the slope of the surface away from the sample points. The slope can be calculated in four directions on the surface away from each sample point: up, down, left, and right. If the minimum slope exceeds a threshold, the point can be designated as an outlier.

[0151] An example is shown in Figure 16 looking sideways along the focal plane. Two points 1610 and 1620 are identified as outliers. These two points 1610 and 1620 are therefore removed from the focus map (e.g., in step 592 of process 500) before testing the stripes for possible restriping (e.g., in step 594 of process 500).

[0152] 2.2.8. Restriping

[0153] In one embodiment, the focus error for each frame containing tissue is calculated by subtracting the actual position of the objective lens 120 during the scan of that frame from the best focus position for that frame as identified from the final focus map. This calculation (e.g., represented as step 594 of process 500) is performed after potential outliers have been removed from the focus map (e.g., in step 592 of process 500). FIG. 17A shows the focus error after all image stripes have been scanned in a first pass. The red and dark blue frames represent positive and negative errors, respectively, whereas the green frames represent very small errors. Stripe seven is shown to have a dark red area, and therefore this image stripe can be selected for restriping. FIG. 17B shows the focus error after stripe seven has been restriped. After restriping, most of stripe seven is green indicating that the objective lens 120 is substantially aligned with the focus map.

[0154] Since the objective lens 120 moves in steps only for non-skipped frames, the skipped frames generally have a small focus error after restriping. If a large focus error remains after restriping, this indicates a large tilt along the scan axis, i.e., generally poor focus. A final image quality assessment can be made from the heatmap after restriping (shown in the example of FIG. 17B).

[0155] In one embodiment, the decision to restrip (e.g., in step 594 of process 500) is made for each image stripe based on the number and size of focus errors for those image stripes. For example, an image stripe is restriped if 5% of the frames in the image stripe exceed a defined threshold (e.g., stored as a Focus_Quality_Restripe_Threshold parameter). This threshold may be a setting that can be adjusted to a level consistent with user preferences. Of course, restriping all image stripes will provide the best image quality. However, this would also double the scan time.

[0156] 2.2.9. Image Quality Score

[0157] Image quality is primarily a function of focus accuracy, provided that focus values ​​are measured on actual tissue and not on artifacts (e.g., the edge of the coverslip, air bubbles, dirt on the coverslip, etc.). Some tissues may have a large tilt, which makes it difficult to focus the entire frame. This can result in poor image quality, but there is not much that can be done to solve this problem.

[0158] In one embodiment, a binary image quality score is given to each scanned image: pass or fail. A "fail" equates to very poor focus over a significant portion of the slide 114. A "fail" image quality score can be based on two indicators:

[0159] Percentage of bad frames. If the percentage of bad frames exceeds the Image_Quality_Bad_Frames_Threshold parameter, a failure is reported. In one embodiment, the percentage of bad frames is typically less than 5% since individual image stripes are restriped if the percentage is higher than 5%.

[0160] Average tilt. The average tilt is calculated from the focus map. Slides with tissue tilt less than 1 micron per millimeter can be expected to have good image quality. If the tilt exceeds the Image_Quality_Tilt_Threshold parameter, a failure can be reported.

[0161] In one embodiment, images that receive a passing image quality score may still be determined to be unacceptable by an operator of the scanning system 100. Improved quality may be achieved by reducing the Focus_Quality_Restripe_Threshold parameter, which results in more image stripes being rescanned to improve quality. Rejected slides and slides determined by the operator to have insufficient image quality may be rescanned with the focus scheme set to ReScan. The ReScan workflow adds additional MFP points at the start of the scan and restripes all image stripes, which of course is relatively time consuming.

[0162] 2.2.10. RTF Parameters

[0163] In one embodiment, the RTF method utilizes a set of parameters, some of which can be adjusted for improved performance and to accommodate different samples 116 and / or glass slides 114. Many of these parameters may be configurable at run time from a configuration file (e.g., "scanner.xml" or other file stored in memory 106), while other parameters may be fixed in the software. Examples of both types of parameters are identified and described below.

[0164] 2.2.10.1. Fixed parameters

[0165] Certain parameter values ​​can be fixed in the software code. These parameter values ​​can be determined based on scans of test slides and the desired algorithm performance. Illustrative and non-limiting examples of such fixed parameters are described below:

[0166] MAIN_IMG_COLM_ANALYZABLE_THRESH. An average main gradient vector value greater than this threshold indicates the presence of tissue. An exemplary value for this parameter is 50. In one embodiment, 85% of the columns of the main gradient vector must exceed this value to proceed to the Gaussian fitting process.

[0167] Parfocal Threshold: In one embodiment, if the parfocal ratio is greater than 0.5 and less than 1.5 (i.e., 0.5<parfocal ratio<1.5), the frame may proceed to the Gaussian fitting process.

[0168] Sample values ​​for Gaussian fitting. This value is the center point (mean) and width (sigma) to test as a candidate for the best fitting Gaussian function. In one embodiment, the mean values ​​range from 100 to 3996 (4096-100) in increments of 25, and the sigma values ​​range from 500 to 1200 in increments of 100.

[0169] 2.2.10.2. Configurable parameters

[0170] The configurable parameter values ​​can be stored in an XML file (e.g., “scanner.xml” or other file stored in memory 106) that is used to hold various parameters needed to configure scanning system 100 for operation. Illustrative, non-limiting examples of such configurable parameters, along with their nominal values, are described below:

[0171] Focusing_Method. This parameter can be set to one of the following values:

[0172] RTF: This is the default method. When the RTF method is set as the focusing method, a predetermined number (e.g., three) of macro focuses are used to start the scan, and then the RTF method and additional macro focus values ​​are used to create a focus map during the scan. A likely texture map from the texture finding algorithm can be used to identify acceptable frames for focus. Additionally, outlier focus values ​​are identified and removed, and restriping is performed on image stripes that are above a restriping threshold.

[0173] PointFocus: When the focusing method is set to MFP, only macro focus is used to create the focus map. These points are evaluated before the image stripes are acquired, so no restriping is performed.

[0174] ReScan: This method is time consuming but is intended to provide good image quality for slides 114 that did not pass the default RTF method. The entire available scan area is scanned. Therefore, tissue finding algorithms are not used to constrain the focus position during scanning. The scan starts with a full complement of macro focus (equivalent to PointFocus) and all image stripes are restriped.

[0175] Parfocal. The value of this parameter is the pixel location on the tilted focus sensor 130B that corresponds to parfocality with the main imaging sensor 130A. By default, the value of this parameter can be set to 1766.

[0176] Stage_Tilt_Scan. This may be a parameter available for future use and may have a default value of 0.0.

[0177] Stage_Tilt_Index. This may be a parameter available for future use and may have a default value greater than or equal to 0.0.

[0178] Image_Data_Number_Rows. The value of this parameter is the number of rows that make up a single frame. By default, the value of this parameter can be set to 1000.

[0179] Image_Data_Number_Columns. The value of this parameter is the number of columns that make up a single frame and must be less than or equal to the number of pixels 142 of the line scan camera 130. By default, the value of this parameter can be set to 4096.

[0180] Image_Data_Number_Color_Channels. The value of this parameter is the number of color channels. For RGB data, the value of this parameter will always be 3, representing three color channels: red, green, and blue. Thus, by default, the value of this parameter can be set to 3.

[0181] Z_Offset. The value of this parameter is the offset for the focus value when scanning in X / Y mode (i.e., both MFP and RTF modes). This value can be adjusted experimentally by scanning a tissue sample 116 on a slide 114 with a series of values ​​to see which value provides the sharpest image. Alternatively, a test script can be used to scan a small area (e.g., 1 mm x 1 mm) with a sequence of offset values ​​and calculate the average contrast for each offset value in the sequence. The offset value with the maximum contrast corresponds to the optimal value for this parameter. By design, this offset value is required to be zero. However, it has been found that there can be a systematic error in the Z position when the stage 112 is moving compared to when the stage 112 is stationary. By default, the value of this parameter can be set to 0.0005 millimeters.

[0182] Z_Scaling_Factor_Left. The value of this parameter is a scale factor multiplied by 0.01 for units of microns per pixel. The scale factor has units of encoder counts (relative to the objective lens 120) per pixel, where 100 counts per micron. This parameter is used to convert pixels to microns along the tilt focus sensor 130B and is applied to the Z distance in the downward direction (i.e., toward the glass slide 114). By default, the value of this parameter can be set to 0.38832.

[0183] Z_Scaling_Factor_Right. The value of this parameter is a scale factor multiplied by 0.01 for units of microns per pixel. The scale factor has units of encoder counts (relative to the objective lens 120) per pixel, where 100 counts per micron. This parameter is used to convert pixels to microns along the tilt focus sensor 130B and is applied to the Z distance in the upward direction (i.e., away from the glass slide 114). By default, the value of this parameter can be set to 0.31943.

[0184] Issue_Move_Every_X_Frames_Modulo. The scanning workflow involves moving the objective lens 120 only on certain frames. Frames where no movement of the objective lens 120 occurs are analyzed by the RTF method. By default, the value of this parameter can be set to 2, which means that every other frame is skipped.

[0185] Frame_Lag_Stripe_Abort_Threshold. The value of this parameter is the number of frames the RTF process is allowed to lag behind the objective lens 120 position. A large value effectively disables this feature. If the frame lag exceeds the value of this parameter, the image stripe scan is aborted and a rescan of the image stripe is initiated. By default, the value of this parameter can be set to 300.

[0186] Debugging. The value of this parameter is a Boolean value. By default, the value of this parameter can be set to false. When the value of this parameter is set to true, the actual frame image data is output.

[0187] Debug_Frame_Number. The value of this parameter is the number of frames to print if the Debugging parameter is set to true. By default, the value of this parameter can be set to 1.

[0188] Debug_Stripe_Number. The value of this parameter is the number of image stripes that should be output if the Debugging parameter is set to true. Only a single frame may be output to avoid interference with subsequent RTF processing and modification of data from normal operation that may be caused by writing information to memory. By default, the value of this parameter may be set to 3.

[0189] Focus_Quality_Restripe_Threshold. The value of this parameter is a number of encoder counts. If a certain percentage of frames containing tissue within an image stripe (e.g., 5%) have a focus error that exceeds this value, the image stripe will be restriped. The focus error can be calculated by comparing the actual Z position of the objective lens 120 to the final focus map. Setting this parameter to a high value will effectively disable restriping, whereas setting this parameter to a low value will result in all image stripes containing tissue being rescanned. By default, the value of this parameter can be set to 90.

[0190] Do_Restripe. The value of this parameter is a Boolean value. If the value of this parameter is true, restriping is enabled. Otherwise, i.e., if the value of this parameter is false, restriping is disabled and thus no restriping is performed (e.g., steps 594 and 596 of process 500 are skipped). Furthermore, if the value of this parameter is false, other related functions (e.g., outlier rejection in step 592 of process 500) may also be skipped. By default, the value of this parameter may be set to true.

[0191] Do_Outlier_Rejection. The value of this parameter is a Boolean value. If the value of this parameter is true, then the focus map is analyzed for outliers (e.g., in step 592 of process 500). All detected outliers are discarded and the focus map is recalculated without the discarded outliers. This is performed before determining whether to perform restriping (e.g., in step 594 of process 500). If the value of this parameter is false, then the focus map is not analyzed for outliers and no outlier rejection is performed (e.g., step 592 of process 500 is skipped). By default, the value of this parameter can be set to true.

[0192] Outlier_Rejection_Threshold. The value of this parameter is in microns per millimeter. In an embodiment of step 592 of process 500, for each point in the focus map, four slopes are calculated along the surface: up, down, right, and left. If the minimum slope is greater than the value of this parameter, the point is marked as an outlier and is not used in the focus map thereafter. The idea is that an outlier point has a large slope in all directions away from the point. By default, the value of this parameter can be set to 2.0.

[0193] Focus_On_Probable_Tissue. The value of this parameter is a Boolean value. If the value of this parameter is true, the probable tissue map from the tissue finding algorithm is used as a mask for acceptable focus positions in the RTF method. The RTF method is very sensitive to detecting frames that contain tissue, and as a result, focus values ​​can be added to the focus map even if they are not really on tissue. Using the probable tissue mask prevents this. If the value of this parameter is false, the probable tissue map is not used as a mask. By default, the value of this parameter can be set to true.

[0194] Request_Additional_MFPs. The value of this parameter is a Boolean value. If the value of this parameter is true, additional macro focuses can be requested (e.g., in step 584 of process 500) after each image stripe is scanned. These additional macro focuses are used to provide additional focus values ​​to the focus map if the RTF method cannot reliably focus. If the value of this parameter is false, additional macro focuses are not requested after each image stripe is scanned. By default, the value of this parameter can be set to true.

[0195] Focus_Point_R_Min. The value of this parameter is in millimeters. If the value of the Request_Additional_MFPs parameter is true and the frame containing the tissue is farther away from the valid focus value than the value of this Focus_Point_R_Min parameter, then macro foci are requested for the center of the frame (e.g., in step 584 of process 500). These additional macro foci are focused and added to the focus map before scanning the next image stripe. By default, the value of this parameter can be set to 2.

[0196] Enable_PredictedZ_Offset. The value of this parameter is a Boolean value. If the value of this parameter is true, the predicted value from the focus map is subtracted from the best focus value calculated by the RTF method. The resulting difference is added to the focus value in the focus map predicted for the next frame. This feature aims to improve focusing accuracy by detrending focus errors and to reduce the restriping rate. If the value of this parameter is false, the feature is disabled. By default, the value of this parameter can be set to true.

[0197] Image_Quality_Bad_Frames_Threshold. The value of this parameter is a percentage. If the percentage of bad frames exceeds the value of this parameter, the image quality score is set to "Fail". By default, the value of this parameter can be set to 10%.

[0198] Image_Quality_Tilt_Threshold. The value of this parameter is in microns per millimeter. If the average tilt for the tissue exceeds the value of this parameter, the image quality score is set to "Fail". By default, the value of this parameter can be set to 5 microns / mm.

[0199] The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles described herein may be applied to other embodiments without departing from the spirit or scope of the present invention. It should therefore be understood that the description and drawings presented herein represent presently preferred embodiments of the present invention, and are therefore representative of the subject matter broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that will be apparent to those skilled in the art, and is therefore not limited.

[0200] Combinations described herein, such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof," include any combination of A, B, and / or C, and can include multiple As, multiple Bs, or multiple Cs. Specifically, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" can be A only, B only, C only, A and B, A and C, B and C, or A and B and C, and any such combination can contain one or more members of its constituent A, B, and / or C. For example, a combination of A and B can include one A and multiple Bs, multiple A and one B, or multiple A and multiple Bs.

Claims

1. 1. A method comprising: The method includes using at least one hardware processor of the scanning system: initializing a focal map; While acquiring a plurality of image stripes of at least a portion of a sample on a glass slide, for each of the plurality of image stripes, acquiring each of a plurality of frames collectively representative of the image stripe using both an imaging line scan camera and a tilted focus line scan camera; A focus point representing a best focus position for a trusted frame of the plurality of frames is added to the focus map. adding a focus to the focus map by removing all outlier foci from the focus map; determining, for each of the plurality of frames within the plurality of image stripes, whether to restrip one or more image stripes of the plurality of image stripes based on a focus error; reacquiring one or more image stripes if it is determined to restrip the one or more image stripes; assembling the plurality of image stripes into a composite image of the at least a portion of the sample; A method comprising:

2. Adding focus to the focus map while acquiring the plurality of image stripes includes: for each of the plurality of image stripes other than a last one of the plurality of image stripes to be acquired, after acquiring the image stripe, determining from the image stripe a direction of a next one of the plurality of image stripes to be acquired. Further comprising: The method of claim 1.

3. The plurality of image stripes, in order, Obtaining a reference stripe; sequentially acquiring image stripes from a first side of the reference stripe to a first edge of a scan area of ​​the specimen; sequentially acquiring image stripes from a second side of the reference stripe opposite the first side of the reference stripe to a second edge of the scan area opposite the first edge of the scan area; Obtained by The method of claim 2.

4. The method comprises: adding a plurality of macro foci to the focus map before starting acquisition of the plurality of image stripes; Further comprising: The method of claim 1.

5. The method comprises: adding one or more macro-focal points to the focus map after acquiring one or more of the plurality of image stripes. Further comprising: The method of claim 1.

6. Adding focus to the focus map while acquiring the plurality of image stripes includes: for each of the plurality of frames in each of the plurality of image stripes, determining whether the frame is trusted; Further comprising: The method of claim 1.

7. Determining whether the frame is trusted includes: calculating a principal gradient vector, including an average gradient vector, for each column in the frame acquired by the imaging line scan camera; calculating a gradient gradient vector, including an average gradient vector, for each column in the frame acquired by the gradient-focused line scan camera; determining a number of analyzable columns in the principal gradient vector; calculating a ratio vector based on the principal gradient vector and the gradient gradient vector; determining whether the frame is analyzable based on the number of analyzable columns and the ratio vector; determining that the frame is not trusted if the frame is determined to be not analyzable; If it is determined that the frame is analyzable, fitting at least one Gaussian function to a ratio curve represented by said ratio vector; identifying a peak of the Gaussian function as a best focus position; identifying the amplitude of the ratio vector at the best focus position as a fitted maximum; determining whether the frame is reliable based on the best focus position and the fit maximum; determining that the frame is not trusted if the frame is determined to be untrusted; adding the best focus position to the focus map if the frame is determined to be reliable; Including, The method of claim 6.

8. Identifying the number of analyzable columns includes: determining a number of columns in the principal gradient vector that exceed a threshold; The method of claim 7.

9. Calculating the ratio vector comprises: Dividing the gradient vector by the principal gradient vector. Including, The method of claim 7.

10. Determining whether the frame is analyzable includes: determining whether the number of analyzable columns exceeds a predefined threshold percentage; determining whether a value of the ratio vector at a parfocal position is within a predefined range, the parfocal position being a point on the gradient-focused line scan camera that is parfocal with the imaging line scan camera; and determining that the frame is not analyzable if it is determined that the number of analyzable columns does not exceed the predefined threshold percentage or that the value of the ratio vector at the parfocal position is not within the predefined range; determining that the frame is analyzable if it is determined that the number of analyzable columns exceeds the predefined threshold percentage and that the value of the ratio vector at the parfocal position is within the predefined range; Including, The method of claim 7.

11. Fitting at least one Gaussian function to the ratio curve comprises: Sampling a number of possible Gaussian functions within a range of mean values ​​and within a range of sigma values; selecting one of the plurality of possible Gaussian functions having a minimum difference from the ratio curve to be used to identify the best focus position; Including, The method of claim 7.

12. Removing all outlier foci from the focus map comprises, for one or more sample points in the focus map: calculating a gradient in each of four directions away from the sample point within the focal map; removing the sample point from the focus map if the minimum of the calculated slope exceeds a predefined threshold; Including, The method of claim 1.

13. Determining whether to restrip one or more image stripes of the plurality of image stripes includes, after removing all outlier foci from the focus map, calculating, for each of the plurality of frames in each of the plurality of image stripes, a focus error for the frame by subtracting an actual position of an objective lens during acquisition of the frame from a best focus position for that frame within the focus map; determining to restrip the image stripe if, for each of the plurality of image stripes, a number of frames having a focus error exceeding a predefined threshold exceeds a predefined threshold percentage; Including, The method of claim 1.

14. 1. A scanning system comprising: The scanning system comprises: An imaging line scan camera; a tilted focusing line scan camera; at least one hardware processor; One or more software modules; Including, The one or more software modules, when executed by the at least one hardware processor, Initialize the focal map, While acquiring a plurality of image stripes of at least a portion of a sample on a glass slide, for each of the plurality of image stripes, acquiring each of a plurality of frames collectively representative of the image stripe using both the imaging line scan camera and the tilted focus line scan camera; A focus point representing a best focus position for a trusted frame of the plurality of frames is added to the focus map. adding a focus to the focus map by removing all outlier foci from the focus map; determining, for each of the plurality of frames within the plurality of image stripes, whether to restrip one or more image stripes of the plurality of image stripes based on a focus error; reacquiring one or more image stripes if it is determined that one or more image stripes should be restriped; Assembling the plurality of image stripes into a composite image of the at least a portion of the sample. The scanning system is configured to:

15. A non-transitory computer-readable medium having instructions stored thereon, comprising: The instructions, when executed by a processor of a scanning system, cause the processor to: Initialize the focal map, While acquiring a plurality of image stripes of at least a portion of a sample on a glass slide, for each of the plurality of image stripes, acquiring each of a plurality of frames collectively representative of the image stripe using both an imaging line scan camera and a tilted focusing line scan camera; A focus point representing a best focus position for a trusted frame of the plurality of frames is added to the focus map. adding a focus to the focus map by removing all outlier foci from the focus map; determining, for each of the plurality of frames within the plurality of image stripes, whether one or more image stripes of the plurality of image stripes should be restriped based on a focus error; if it is determined that one or more image stripes should be restriped, then reacquiring the one or more image stripes; assembling the plurality of image stripes into a composite image of the at least a portion of the sample. Non-transitory computer-readable medium.

Images