How to Analyze Scanning Effectiveness

Abstract

A method for operating a microscope scanner is provided. A first imaging scan of one or more areas of interest (AOI) is performed on a target (6) comprising a sample. This involves moving a detector array (3) along an image scan path relative to the target (6) and acquiring images of the target at each of a plurality of locations along the image scan path. Focus control data is generated during the imaging scan, where generating the focus control data includes calculating a focus merit value at each location along the image scan path. The focus height of the detector array (3) is then adjusted along the image scan path based on the focus merit value. The effectiveness of the first imaging scan is analyzed using the focus control data, and changes to one or more scanning parameters from the first imaging scan are determined for performance of a second imaging scan based on this analysis.

Description

[Technical field]

[0001] The present invention relates to a method for analyzing the validity of imaging scans obtained using a microscope scanner. [Background technology]

[0002] For pathology slides, the biopsy tissue is mounted on a block of paraffin as a support. A thin layer of about 4 μm is taken from the block and placed on a slide. The slide is then stained to show the tissue of interest and background tissue. The paraffin is removed by evaporation and a protective cover slip is glued over the tissue. Using a whole slide imaging scanner, the tissue is then scanned to a digital file where either a pathologist or a software algorithm checks and shows if disease is present. Typically, the pixel size for digitizing the image is around 0.25 μm, and the natural aperture of the imaging limits the depth of focus to around 1 μm. The slide with the tissue mounted on it is not flat within 1 μm. The standard equation for a perfectly manufactured slide indicates that the slide will bend under its own weight by around 2 μm, and distortions from the manufacture and mounting of the slide can give much larger errors than that. The tissue itself is typically not always of uniform thickness, some parts of the tissue are sliced ​​thinner than other types, or the tissue is sliced ​​into a wedge shape.

[0003] If a whole slide imaging scanner does not scan the tissue within the focal depth, the area outside the focal depth may be unsuitable for diagnosis or may cause errors in diagnosis. Therefore, whole slide imaging scanners typically have a focal tracking mechanism, such as that described by US9116035, to achieve correct focus throughout the tissue. However, for several reasons, the tracking of the tissue focal point may not be perfect throughout the tissue.

[0004] Once a slide has been scanned, it is possible to analyze the scanned image to determine several aspects of how good the quality of the scan is. For example, it is possible to analyze how well the tissue is in focus by examining the gradient at the edges of tissue items such as cells. For the entire slide, this analysis requires significant processing power. A typical 15x15mm area with 0.25μm pixels would have 3.9 gigapixels. An RGB image before compression would be 11Gb of data. Inspecting the complete image in detail, as is done for example in US2019 / 0266726A1, would be slow. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] US 9116035 B2 [Patent Document 2] US 2019 / 0266726 A1 Summary of the Invention [Problem to be solved by the invention]

[0006] It is desirable to be able to quickly inspect the image immediately after the imaging scan is completed. If defects in the scan are identified while the slide is still in the scanner, it is possible to change the scanning conditions and perform a second scan that does not have the defects of the first scan. This second scan may have control parameters adjusted only for this slide type or it may have different starting conditions, but typically the control parameters can be adjusted to improve the scan. It may not be practical to inspect the entire 3.9 gigapixel image while the slide is still in the scanner. The present invention is concerned with finding faster and more effective ways to analyze the validity of a scanned image so that a second scan can be performed if necessary to correct any defects found in the first scan. [Means for solving the problem]

[0007] A first aspect of the present invention provides a method of operating a microscope scanner comprising a detector array, the method comprising: performing a first imaging scan of one or more areas of interest AOI on a target comprising a sample, moving the detector array along an image scan path relative to the target and acquiring images of the target at each of a plurality of locations along the image scan path using the detector array; generating focus control data during an imaging scan, said generating comprising calculating a focus merit value at each said location along an image scan path; and said performing step comprising adjusting a focal height of a detector array along an image scan path based on said focus merit value; and analyzing the effectiveness of the first imaging scan using the focus control data, and determining changes to one or more scanning parameters from the first imaging scan for performance of the second imaging scan based on the analysis.

[0008] Thus, an improved method is provided for analyzing the effectiveness of an imaging scan using focus control data generated during the imaging scan. Conventionally, focus control data is generated solely for the purpose of performing the imaging scan, and it is the images generated from the imaging scan that are considered to be looked at to assess whether the scan was successful. In contrast, the above-described method uses focus control data to analyze the effectiveness of the imaging scan. This can be accomplished through a number of different techniques (as discussed below), each of which facilitates rapid and effective results without placing high demands on computer processing resources. The result can be a determination of changes to one or more scanning parameters from the first imaging scan such that corrective action can be taken in the second imaging scan. This ensures that the user can obtain high quality scanned images of the target with minimal delay.

[0009] In a particularly preferred implementation, the first imaging scan is performed over a first scanning area comprising one or more AOIs, and the step of determining the change of one or more scanning parameters comprises selecting a second scanning area on the target different from the first scanning area. This may be the case, for example, when the focus tracking data indicates that the initially selected one or more AOIs were too small to completely capture the target area occupied by the sample. Typically, a pathologist will attempt to capture the entire sample in the scanned image, and therefore, when the analysis suggests that the first scanning area was too small, it is advantageous to enlarge the scanning area accordingly. However, imaging scans are time consuming, and therefore, in order to reduce the time required for the second imaging scan, it is desirable to enlarge the scanning area for the second scan only by the amount required to encompass the sample. Thus, analysing the effectiveness of the first imaging scan preferably comprises identifying one or more boundary locations on the target when the focus control data indicates that the detector array was primarily in focus at the boundaries of the first scan area, and selecting the second scan area preferably comprises adjusting the first scan area to incorporate an area of ​​the target outside the first scan area and surrounding the one or more boundary locations. The first scan area may be expanded a preset amount surrounding the boundary locations to encompass the contour of the sample outside of it. Alternatively, a mathematical prediction of the shape of the sample on the target may be made. In particular, analysing the effectiveness of the first imaging scan may comprise identifying one or more sample focal regions on the target from the focus control data indicating that the detector array was primarily in focus through the one or more sample focal regions, and selecting the second scan area may comprise interpolating an area of ​​the target over which the sample extends outside the one or more sample focal regions imaged during the first imaging scan. The shape of the interpolated region may be determined by applying a shape fit to the outline of the sample focal region, for example by fitting a circle, arc, or spline curve to the tissue edges determined from the focus control data.This allows a more intelligent estimation of where the sample will extend outside the first scan area, such that the scan area is extended only the amount necessary to cover the sample, and does not extend over areas of the target where no sample is present.

[0010] The step of evaluating the validity of the first imaging scan preferably further comprises identifying one or more poor focus areas on the target, the focus control data indicating that the detector array was mainly out of focus through the one or more poor focus areas during the first imaging scan. The poor focus areas can be used in the process of determining the second scan area. Alternatively or additionally, the step of determining a change in one or more scanning parameters can comprise adjusting a focusing procedure so as to obtain images of the target at a different focus height during the second imaging scan in one or more target areas, such as the one or more poor focus areas, compared to corresponding images from the first imaging scan. Such a scenario may occur when an analysis of the focus control data indicates that the detector array was not properly focused on the sample in one or more AOIs during the first imaging scan. A change in the scanning parameters in the form of a new focusing procedure is then determined so that the detector array has a different focus in these areas during the second imaging scan.

[0011] It is common to precede the first imaging scan with a pre-scan performed with a much larger depth of focus to obtain a low-resolution image of the target that allows one or more AOIs to be selected. In particular, the method preferably further comprises the steps of performing a pre-scan to generate a pre-scan image of the target, the depth of focus for the pre-scan being greater than 500 micrometers and less than 3 micrometers for each image obtained during the first imaging scan, and performing an automated image analysis of the pre-scan image to generate a first probability map indicating where samples are expected to be present on the target, where one or more AOIs are selected from the first probability map. This process of performing a pre-scan to identify one or more AOIs has been found to be particularly fast and effective in allowing the parameters that allow the first imaging scan to be set.

[0012] The detector array typically comprises an array of photodetectors arranged as a narrow CCD strip. Thus, the detector array is typically a line detector, but could be a 2D camera. Typically, the first imaging scan further comprises generating a first image of the target from images acquired along the image scan path. Such an image could correspond to a combination of each of the individual images or "lines" acquired by the detector array along the image scan path. Thus, the first image has a relatively high resolution, which may require loading significant computer processing resources. Therefore, it is preferable to derive a second image of the target from the first image, the second image having a resolution lower than that of the first image. The second image can be viewed by a user to obtain a "snapshot" of the target without the need for the computer system to load additional data present in the first image. Advantageously, this second image can be further used to assist in identifying one or more of the poor focus areas mentioned above, by having only a significantly smaller depth of focus compared to the pre-scan image used to select one or more AOIs. Thus, surface details such as stains or scratches on the target that are visible in the pre-scan image and may be erroneously identified as corresponding to a sample area are more likely to be invisible in the first or second image. It is therefore advantageous to identify this surface detail so that a second scan is not unnecessarily performed over target areas that are occluded by the surface detail and over target areas where no sample may be present. Thus, the step of analysing the validity of the first imaging scan may further comprise the steps of performing an automated image analysis of the second image to generate a second probability map for areas of the target where sample is expected to be present, and identifying surface details in the pre-scan image that correspond to locations where the first probability map indicates that sample is expected to be present on the target and where the second probability map indicates that no sample is present, wherein the one or more poor focus areas do not include locations having said surface details on the target.The resolution of the second image is preferably equal or nearly equal to the resolution of the pre-scan image, which allows the same or similar algorithms to be used to generate the first probability map without requiring significant processing resources to generate the second probability map, and further allows for an effective comparison between the first and second probability maps for identifying surface details.

[0013] The focus control data may further comprise an estimated focal plane (also referred to as a "target height") for the target and / or an actual focal height of the detector array. From these data, a difference between where the detector array was and where it was desired to be for any image obtained during the first imaging scan may then be identified. This difference is a known position error. There may be a tolerance for the position error, but when this tolerance is exceeded, a corresponding location may be identified for the second imaging scan. For example, the focus control data may comprise an estimated focal plane of the target at each location along the image scan path, and the actual focal height of the detector array may be monitored for each location along the image scan path where an image is obtained. Identifying one or more poor focus regions on the target may then comprise identifying one or more position error locations where the difference between the estimated focal plane and the actual focal height of the detector array exceeds an error threshold. Typically, the error threshold is approximately equal to or exceeds the depth of focus for the detector array.

[0014] As discussed in the background section, automated image analysis techniques are known for analyzing the quality of an image, in particular whether a sample is sufficiently focused in an image. However, such techniques are typically computationally intensive and therefore time consuming. It is therefore desirable to limit the image region to which such techniques are applied so that image regions known to be in focus are excluded. In this case, the process of analyzing the image can be limited to the remaining regions where the focus is considered to be poor or unknown. This limitation can be advantageously achieved by using focus control data. In particular, the first imaging scan can further comprise the generation of a first image of the target from images acquired along the image scan path, as discussed above. The step of identifying one or more poor focus regions then further comprises the performance of an automated image analysis of one or more sub-regions of the first image to identify whether the focus of the first image is acceptable, and designating each sub-region of the target where the focus is unacceptable as a "poor focus region", the one or more sub-regions being selected based on the focus control data. Preferably, the one or more sub-regions include a position error location. The one or more sub-regions can be determined based on a target area occupied by the position error location.

[0015] Determining the modification of one or more scanning parameters may comprise adjusting a focus procedure for the second imaging scan, as discussed above. The focus procedure may be based on measuring a focus position for at least one seed location on the target. In particular, the method further comprises measuring a focus position for at least one seed location, typically within one or more AOIs, prior to the first imaging scan, and setting a focus height at one or more locations along the image scan path based on the measured focus position for the seed location. Thus, the one or more scanning parameters to be modified may be selected from the group comprising: an area of ​​the target to be imaged, a number of seed locations on the target, and a location of the seed locations on the target. Furthermore, analyzing the validity of the first imaging scan may comprise identifying an error in a measured focus position for the one or more seed locations, which may be performed by comparing focus control data from the first imaging scan with a measured focus position for the one or more seed locations prior to the first imaging scan. For example, comparing focus control data from the first imaging scan with a measured focus position for one or more seed locations prior to the first imaging scan may comprise comparing focus control data from the seed locations during the first imaging scan with a measured focus position for one or more seed locations prior to the first imaging scan. Typically, the focus height of the detector array is adjusted along the image scan path to track the estimated focal plane of the target based on the obtained focus control data. Thus, the focus height of the detector array will not necessarily arrive at the measured focus position for the seed locations during the first imaging scan, but the focus position may be identified as erroneous if the focus heights are significantly different (e.g., the difference is greater than a threshold separation). The focus height at one or more locations along the image scan path is based on the measured focus position for the seed locations, and thus, if the focus position is found to be erroneous, will typically affect the validity of the imaging scan.Determining a change in one or more scan parameters from the first imaging scan may then include selecting a new seed location on the target at a location where the focus control data indicates that the detector array was out of focus during the first imaging scan.

[0016] The first imaging scan may be performed across multiple AOIs, and the focus position is measured by adjusting the focus height of the detector array at each of the seed locations through a range of positions perpendicular to the surface of the target. If the average focus of each of these AOIs is not similar or not within an expected tolerance due to slide tilt, slide distortion, and tissue thickness variations, it may be considered that the detector array was not properly focused on the sample in at least one of the AOIs during the imaging scan. Thus, analyzing the validity of the first imaging scan may comprise calculating the average focus height of the sample in each AOI from the focus control data. The step of determining the change of one or more scan parameters from the first imaging scan may then comprise, in response to determining that the separation between the average focus height for the first said AOI and the average focus height for the second said AOI exceeds a threshold distance, modifying the position range in which the focus height is adjusted when monitoring the focus position at the seed location in the first AOI to exclude average focus heights for the first AOI calculated from the first imaging scan that exceed the average focus height calculated for the second AOI. The lowest focus level of an AOI will typically give a good indication of the correct focus level. Thus, the average focus height calculated for the second AOI is preferably the lowest focus height from the group comprising the average focus heights calculated for each said AOI. This technique allows determining parameters of the second imaging scan such that each AOI is correctly imaged.

[0017] Once the parameters of the second imaging scan have been set, in particular once the changes in one or more scanning parameters from the first imaging scan have been determined, the second imaging scan can be performed. Thus, the method typically further comprises performing a second imaging scan of the target using the detector array with the changes in one or more scanning parameters determined compared to the performed first imaging scan.

[0018] A second aspect of the present invention provides a computer program product comprising instructions which, when executed using a microscope scanner comprising a detector array and a target comprising a sample, cause the microscope scanner to carry out the method of the first aspect. The second aspect shares similar features and advantages as discussed with respect to the first aspect.

[0019] The microscope scanner is typically a bright field microscope scanner, and more preferably a virtual microscope, and the computer program product typically corresponds to a non-transitory computer readable medium such as a hard drive. The target can take several forms, but preferably comprises a biological tissue sample. Preferably, the target does not exhibit substantial variation in surface height and is therefore substantially flat. For example, the target could be a stained tissue extract that has been sliced ​​and held between flat glass slides.

[0020] Embodiments of the present invention will now be discussed with reference to the following figures. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram of a microscope scanner that may be operated in accordance with an embodiment of the present invention. [Diagram 2] 1 is a flow diagram illustrating an embodiment of the present invention. [Diagram 3] 4 is a flow diagram illustrating a process for analyzing the effectiveness of an imaging scan according to an embodiment of the present invention. [Figure 4A] FIG. 4 shows an example of a pre-scan image obtained in accordance with an embodiment of the present invention. [Figure 4B] FIG. 4 shows an example of a second image obtained according to an embodiment of the present invention. [Figure 5A] FIG. 2 illustrates a first example of a poor focus area on a target identified in accordance with an embodiment of the present invention. [Figure 5B] FIG. 2 illustrates a second example of a poor focus area on a target identified in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] An example of a brightfield microscope scanner 1 is shown in FIG. 1. The microscope scanner comprises a scan head 2 with a line scanner 3 in the form of an array of linear photodetectors. The scan head 2 further comprises an adjustable focus system 4 and a pre-scan camera 12. A platen 5 is provided on which a target 6 in the form of a pathology slide with a sample is placed. A drive mechanism 7 mounts the scan head 2 on a track 8 so that the scan head can be moved relative to the target 6 as shown by the arrow 9. The track 8 may also allow the scan head 2 to be displaced laterally (perpendicular to the optical axis 11 of the scan head 2 and the scanning direction 9) in order to image adjacent scan widths. The microscope scanner 1 is controlled by means of an electronic controller in the form of a computer system 10. FIG. 1 also provides coordinate axes for reference in the following examples. The ordinate z-axis is aligned with the optical axis of the scan head 2 and the focal height of the target 6, whereas the abscissa x-axis represents the scanning direction (parallel to the arrow 9 in FIG. 1). The face of the target 6 is aligned in the xy-plane and the lens of the adjustable focus system 4 is movable along the z-axis. In the alternative, the scan head 2 remains stationary and the target 6 is moved in the xy-plane to achieve relative motion between the scan head 2 and the target 6 during the scanning process. The pre-scan camera 12 may form a separate unit from the scan head 2 and be operable independently of the scan head 2.

[0023] The operation of the microscope scanner 1 when performing a method according to an embodiment of the invention will now be described with reference to Figures 2 to 5. The method starts at step 100 (Figure 2), where a pre-scan is performed to acquire a pre-scan image of the target 6. The scan head 2 is moved along a trajectory 8, when an image of the target 6 is acquired using a pre-scan camera 12. The pre-scan camera 12 is configured to acquire an image of the target with a focal depth of 1 mm, which is significantly larger than the thickness of the sample (typically 2 μm to 10 μm). Thus, the focal height of the pre-scan camera 12 is not adjusted during the pre-scan, and the pre-scan image is generated quickly with a relatively low resolution, for example 1200 dpi (47 lines per mm). The pre-scan image provides an overview of the outline of the sample on the target 6, from which one or more areas of interest AOI can be selected for imaging during a first imaging scan. This imaging scan proceeds significantly slower compared to the pre-scan, and therefore it is desirable to only image areas on the target 6 where a sample is present during the imaging scan. "Blank" or "free" areas of the target can be excluded. In step 101, the pre-scan image is processed using conventional software such that each area in the pre-scan image where a sample is visible is identified as an AOI. More specifically, an automated image analysis of the pre-scan image is used to generate a first probability map that identifies where the sample is expected to be present on the target, and one or more AOIs are selected from the first probability map. Typically, the boundaries of the AOI can be selected to be slightly outside the outer edge of the sample (as identified from the pre-scan image) so that the sample is completely contained within the AOI. The pre-scan image with the one or more AOIs demarcated can then be displayed to a user using the computer system 10. Optionally, the user can then use the computer system to manually adjust the boundaries of the one or more AOIs as needed.

[0024] The method then proceeds to step 102, at which a seed location is selected at each AOI range in the xy plane (typically at the center of the AOI) and the focal height of the sample at this location is measured using the line scanner 3. The focal height of the adjustable focus system 4 is varied through a predefined focal height range to obtain image information from the target 6 at a focal depth of around 1 μm at several different focus positions along the z axis. A focus position is then calculated from this image information at each focus position by using a focus parameter in the form of a focus merit value. Typically, the focus merit value is a numerical measure of the complexity in an image, the higher the value, the more detail there is in the image. A better focused image has a corresponding high merit value. The focus height with the highest merit value is then stored as the measured focus position for the seed location and the above process is repeated for any remaining identified seed locations.

[0025] An image scan path is calculated along which the scan head 2 is moved so that the line scanner 3 images the entire surface of one or more AOIs. The line scanner 3 is moved along the image scan path to a starting location and focused at a focus position for each AOI. Then, in step 103, a first imaging scan is started, at which point the line scan head 2 is moved across an image scan width to obtain images of multiple adjacent locations on the target 6. For each location, focus control data is recorded comprising a focus merit value for the line scanner 3, an estimate of where the focus level is, and an actual focus height of the line scanner 3. The estimated focus level can be calculated using techniques known in the art, such as those described in US9116035B2. The focus height of the line scanner 3 is adjusted between each xy location along the image scan path towards or to the estimated focus position. In this way, the target 6 is imaged within one or more scan widths until each AOI has been completely imaged. Then, in step 104, the individual images obtained by the line scanner 3 are combined with each other to form a first image of the target.

[0026] It is desirable for the first image to include the entire sample across the xy plane and to be in focus throughout. However, some errors may occur in the above method steps that prevent in-focus from occurring. Next, in step 105, the effectiveness of the first imaging scan (which can be thought of as the extent to which the first imaging scan produced the intended result) is analyzed. This analysis may involve any number of individual sub-processes to check respective criteria, examples of which are further discussed with reference to FIG. 3. Typically, however, this analysis involves an assessment of whether the imaged area on the target sufficiently covers the area of ​​the target 6 occupied by the sample and / or whether the sample is correctly focused throughout the image. Importantly, this processing is performed on the focus control data obtained during the first imaging scan. Such an analysis is not computationally demanding, and therefore errors in the first imaging scan can be substantially identified in a time-efficient manner. If any errors are identified, changes to the scanning parameters are determined in step 106 so that these errors can be corrected in the second imaging scan. Then, in step 107, a second imaging scan is performed with appropriate changes in the scanning parameters (using focusing procedure or imaging target area of ​​target 6) to improve the effectiveness of the imaging scan.

[0027] The embodiment according to Fig. 2 will now be discussed in more detail with reference to the flow chart according to Fig. 3. In step 201, a first imaging scan is performed such that each AOI is imaged using the line scanner 3. This step is equivalent to step 103 according to Fig. 2. The method then proceeds to step 202, in which the computer system 10 performs the first of a series of tests to analyze the validity of the first imaging scan. It is asked whether or not more than one AOI has been imaged during the first imaging scan, and if the answer is "yes", the method proceeds to step 203, otherwise it proceeds to step 205.

[0028] At least one of the AOIs can be considered to not have a correct focus if the average focus heights of the AOIs are not similar to each other or are not within the expected tolerance due to slide tilt, slide distortion, changes in tissue thickness. Typically, the line scanner 3 will have misfocused on something on the top of the coverslip relative to the target 6, and the scan will appear to be complete if there is sufficient detail present on the top of the coverslip. In this case, there will typically be a coverslip thickness difference of around 170 μm between the focus levels. Thus, in step 203, an evaluation is made as to whether the average focus of each AOI satisfies the tolerance range. The AOI with the lowest average focus height is found and the tolerance range is set within a threshold difference from this lowest average focus height. This threshold can be set depending on the application, but is typically larger than the thickness of the sample, in this case 20 μm. More typically, it is believed that the threshold can be selected within 10-30 μm. If the average focus of each AOI falls below this threshold, it can be considered that the line scanner 3 did not focus on the area of ​​the coverslip and the method proceeds to step 205. If, on the other hand, the average focus of one of the AOIs exceeds the threshold, the method proceeds to step 204. In step 204, changes in the scanning parameters from the first imaging scan are determined for the performance of the second imaging scan. In particular, changes in the focusing procedure are determined. The lowest average focus height of the AOIs will give a good indication of the correct tissue focus level. This allows a new scan to be performed in step 204, but since the approximate focus level is now known, the focus height range searched for each seed location in step 102 can be limited, for example, to the previously found threshold focus range. This ensures that the line scanner 3 does not focus on the coverslip during the second imaging scan, thus improving the effectiveness of the scan.

[0029] In step 205, a focus merit tissue map is generated showing the focus merit values ​​calculated for each location along the image scan path during the previous imaging scan. From these focus merit values, it is possible to determine where the line scanner 3 was in focus. Where tissue is present, the focus merit value will be large even in areas that are slightly out of focus. This gives an alternative map of where tissue is present. If high focus merit values ​​are recorded at the edge of the first scan area during the first imaging scan, it can be assumed that the AOI was set too small to capture all of the tissue. In step 206, any "border locations" where the focus merit value is greater than a threshold value at the border of the AOI are identified. If a set of adjacent border locations is identified indicating that the first scan area is too small to capture the sample, the method proceeds to step 207.

[0030] In step 207, the change in the scanning parameters is determined by calculating a second scan area that covers a portion of the target that is larger than the first scan area. The second scan area can simply be a small extension of the boundary if the first scan area was not large enough, or it can be a mathematical prediction of the sample shape, such as fitting a circle, an arc, or a spline curve to the tissue edge determined from the focal merit tissue map. It is desirable not to unnecessarily extend the scan area to cover areas of the target where the sample can be considered absent, as this would unnecessarily lengthen the scan time. Optionally, it can be assumed that the user does not want to scan all of the tissue if it is known that the user has changed the AOI boundary at the boundary location. Otherwise, a second imaging scan is then performed over the new scan area to image the sample at the location not included in the first imaging scan.

[0031] If the focus merit map does not indicate that the first scan area was too small to capture the sample sufficiently, the method proceeds from step 206 to step 208. In step 208, any surface details visible in the pre-scan image are identified. This identification is achieved by comparison of the pre-scan image with the imaging data obtained from the first imaging scan. It will be recalled from step 104 (FIG. 2) that a first image of the target was generated, which corresponds to a combination of a number of individual images obtained during the imaging scan. This image will have a completely different numerical aperture and therefore a completely different depth of focus than the pre-scan image. The pre-scan image will have a larger depth of focus, which means that defects such as stains and scratches on the surfaces of the slide and cover slip are in focus, whereas the small depth of focus of the imaging scan means that stains and scratches on the surfaces of the slide and cover slip are not in focus. Meanwhile, before the imaging data from the first imaging scan can be compared to the pre-scan image, the resolution of the first image is reduced to a similar or the same resolution as the pre-scan image.

[0032] A typical full slide capture image file contains multiple image layers, each at a different resolution. This allows an image viewing application to zoom out and move around the image without having to access large amounts of data. For example, if an image file has 0.25 μm pixels and the viewing application shows 2000×3000 pixels to display this, it would require reading 6 Mpixels which is 18 Mbytes of data uncompressed. With compression, this data could be as little as 2 Mbytes of data. If the viewing application were asked to display a larger area at a lower resolution, for example "zoomed out" to 10 times the area, it would require 60 Mpixels or 20 Mbytes of compressed data to read this area from the 0.25 μm pixel data. To avoid this condition, it is common for the file to include additional images that are closer to the required resolution. For example, the file could have an image that is one-eighth the resolution of the maximum 0.25 μm resolution data that the viewing application is able to read and display. This would allow the viewing application to read only about 8 Mpixels or 3 Mbytes of data. Although it is not important how this arrangement works, the presence of this lower resolution data derived from the high resolution (HR) data allows for rapid access to image data at a resolution lower than the maximum scan resolution. This lower resolution data is referred to herein as the "second image" of the target, and is analyzed to identify surface details in step 208. Typically, the second image has a similar or the same resolution as the pre-scan image. For example, the resolution of these images may be comparable within a factor of two. In this case, the second image can resolve features of similar size to the pre-scan image, and can be processed much faster (e.g., about 4000 times faster) than the HR "first image."

[0033] FIG. 4A shows an example of a pre-scan image of a target, showing two scratches in the upper central portion of the image. A corresponding second image of similar or equal resolution to the pre-scan image derived from this first image is shown in FIG. 4B, which does not show these scratches. It will be recalled that in step 101 a first probability map of the pre-scan image was generated to identify one or more AOIs. In step 208, a probability map (herein called the "second probability map") is generated in a similar manner, but using a second image of the target, to identify areas of the image that are likely to represent a sample. The second probability map is then subtracted from the first probability map to indicate target areas where surface details are visible in the first image.

[0034] The method then proceeds from step 208 to step 209, where the focus control data from the first imaging scan is analysed to identify one or more "bad focus areas" where the line scanner 3 was not focused on the sample. This analysis can be achieved by identifying locations on the target where the focus merit value fell below a threshold during the first imaging scan. It can be expected that the focus merit values ​​recorded for locations on the target where surface details were found may be less than this threshold. Such locations may have been imaged because they were incorrectly identified as having tissue in step 101, and therefore there is no need to repeat the imaging scan at these locations. However, if one or more bad focus areas (excluding the locations of the surface details) remain, the method proceeds to step 210. Optionally, the method proceeds from step 209 to step 210 only if the bad focus areas have a minimum size, for example a length of at least 0.5 mm in either direction, otherwise it proceeds to step 211. Optionally, the minimum size can be set by a user of the computer system 10. In step 210, a new focusing procedure is determined for the performance of the second imaging scan. In this case, it is determined that the first imaging scan should be repeated, but if in-focus positions for additional seed locations within one or more poor focus regions have been found in step 101. If in-focus positions are measured at these additional locations, the second imaging scan will proceed with the line scanner 3 imaging the target at different focus heights across the one or more poor focus regions.

[0035] If the outcome of step 209 was that no poor focus regions remain, the method proceeds instead to step 211. For each location along the image scan path, the focus control data provides an estimate of the correct focus height for the sample (also called the "focal plane") and a record of the actual focus height for the line scanner 3 when the image was acquired. From these data, it is determined where the scanner focus was one focus depth outside the tissue focus, which can make it possible to identify tissue areas where the focus is known to be in error. Since the scanner must predict where the tissue focus will be when approaching the tissue, the focus error is more significant near the edge of the tissue. Due to several factors such as the tissue being thinner or thicker at the edge or the slide being distorted, this prediction is in error, and when the scanner passes the transparent slide and reaches the tissue, the focus mechanism identifies this predicted error, but may take a short distance to correct the predicted error. Other conditions that may occur that give rise to this focus tracking error may include abrupt changes in focus such as when the tissue folds or bends on itself.

[0036] In step 211, a position error map is generated in which all tissue areas where the actual focal height of the line scanner 3 deviates from the actual focal plane of the sample by at least 1 μm or where the correct focal height is unknown are identified. FIG. 5A shows an image of the target with the transparent slide shown in white and the areas of the sample in focus shown in grey. Areas of poor focus (where the position error is greater than 1 μm or the focus is unknown) are shown as black. There will be a tolerance for the position error, but this error may be deemed unacceptable when it is excessively large. Once the areas shown in black have been identified, in step 213 an automated analysis of the quality of the first image limited to these areas can be performed, for example using an algorithm similar to that described in US2019 / 0266726 (discussed above). Other automated analysis techniques can be used instead, including 2D standard deviation, gradient analysis, and edge filtering (known in the art). The result of this analysis is that the focus of the image is either acceptable or unacceptable for the user's purpose (typically for performing a pathology evaluation). Image analysis processes are typically computationally intensive, but because only a small portion of the first image is required for analysis here, the analysis can be completed relatively quickly. In the example of FIG. 5A, less than 1% of the image needs to be examined. Alternatively, it is possible to examine the HR "first image" only for tissue areas with unknown focus, i.e., in this way only a smaller portion of the image is analyzed.

[0037] FIG. 5B provides a second example of a target image, which is the same as FIG. 5A, except that the areas now identified as having acceptable focus quality (following implementation of the image analysis techniques described above) are also gray. If any poor focus areas (corresponding to the areas marked in black) remain in the first image, or if such areas exceed a threshold amount in the first image, a change in the scanning parameters for the performance of the second imaging scan can be determined. For example, it can be determined that the first imaging scan must be repeated if focus measurements are obtained for additional seed locations inside the remaining poor focus areas. Optionally, the resulting error map can be displayed to the user instead as a guide to which image areas to inspect, which would improve the speed and accuracy of the task.

[0038] If it is determined in step 212 that the position error does not exceed the depth of focus or that bad / unknown focus zones / areas of unacceptable quality do not occupy a threshold amount of the first image, the method proceeds to step 214. The focus positions initially calculated for one or more seed locations before the first imaging scan are compared with the estimated focus planes identified during the first imaging scan for said locations. If they are not similar, for example within a threshold separation to each other, an error can be considered to have occurred and the method proceeds to step 215. In step 215, a change in the focusing procedure for the performance of the second imaging scan is determined. In particular, the positions of the seed locations are moved on the target. The second imaging scan can then proceed as described above in FIG. 2 but with the new seed locations. If the result of step 215 was that each seed focus matches the tracked focus found from the focus control data, it is determined that the first imaging scan is acceptable. The scanning parameters would then not require modification for performance of the second imaging scan, and the first image of the target generated in step 104 can be displayed to the user for analysis.

[0039] It will be appreciated that each of steps 203, 206, 209, 212, and 214 relate to an example of a process for analyzing the effectiveness of a first imaging scan using focus control data and determining, based on this analysis, changes to one or more scanning parameters from the first imaging scan for the performance of a second imaging scan. After the second imaging scan is performed, the quality control process discussed with reference to FIG. 3 will be repeated in its entirety until the imaging data generated by the previous imaging scan is determined to be acceptable or an iteration threshold is reached. In alternative embodiments, any of these processes may be performed alone or in any combination before starting the second imaging scan. Thus, to further reduce the overall time that elapses between starting the first imaging scan and outputting a target image suitable for analysis by a user, multiple changes to scanning parameters may be identified before starting the second imaging scan.

[0040] In summary, there has therefore been provided an improved method for operating a microscope scanner which includes a method for analysing data generated by an imaging scan to ensure that a target has been properly imaged. The analysis is advantageously carried out using focus control data generated during the imaging scan. This allows results to be obtained in a fast and efficient manner. If any errors are identified in the first imaging scan as a result of this analysis, changes to the scanning parameters are identified so that a second imaging scan can be performed which corrects the errors. [Explanation of symbols]

[0041] 100 performing a preliminary scan. 101 AOI Identification Stage 102 measuring the focal height at the seed location 103 Acquiring an image along an image scanning path 104 generating a first image of the target

Claims

1. 1. A method of operating a microscope scanner having a detector array, comprising: performing a pre-scan to generate a pre-scan image of a target, the target comprising a sample, the pre-scan having a depth of focus for the pre-scan exceeding 500 micrometers; performing automated image analysis of the pre-scan image to generate a first probability map indicating where on the target the sample is expected to be present; performing a first imaging scan of one or more areas of interest (AOI) on the target, the one or more AOIs being selected from the first probability map; moving the detector array along an image scan path relative to the target and acquiring an image of the target using the detector array at each of a plurality of locations along the image scan path, wherein the depth of focus for each image acquired during the first imaging scan is less than 3 micrometers; generating focus control data during the imaging scan, said generating comprising calculating a focus merit value at each of the locations along the image scan path; adjusting a focal height of the detector array along the image scan path based on the focus merit value; and combining images of the target acquired at each of the plurality of locations along the image scan path to form a first image of the target; performing the first imaging scan comprising: after performing the first imaging scan, analyzing the effectiveness of the first imaging scan using the focus control data, and determining changes to one or more scanning parameters from the first imaging scan for performance of a second imaging scan based on the analysis; 23. A method comprising:

2. the first imaging scan is performed across a first scan area comprising the one or more AOIs; The method of claim 1 , wherein determining to alter one or more scan parameters comprises selecting a second scan area on the target that is different from the first scan area.

3. analyzing the validity of the first imaging scan comprises identifying one or more boundary locations on the target where the focus control data indicates that the detector array was in focus at a boundary of the first scan area; 3. The method of claim 2, wherein selecting the second scan area comprises adjusting the first scan area to incorporate an area of ​​the target outside the first scan area surrounding the one or more boundary locations.

4. analyzing the effectiveness of the first imaging scan comprises identifying one or more sample focal regions on the target from the focus control data indicating that the detector array was in focus throughout the one or more sample focal regions; 3. The method of claim 2, wherein selecting the second scan region comprises interpolating an area of ​​the target through which the sample extends outside the one or more sample focal regions imaged during the first imaging scan.

5. The method of claim 4 , wherein the shape of the interpolated region is found by applying a shape fit to the contour of the sample focal region.

6. evaluating the validity of the first imaging scan further comprises identifying one or more poor focus regions on the target, the focus control data indicating that the detector array was out of focus throughout the one or more poor focus regions during the first imaging scan; 6. The method of claim 2, wherein determining a change to the one or more scanning parameters comprises adjusting a focusing procedure to obtain images of the target at a different focus height during the second imaging scan in the one or more poor focus regions compared to corresponding images from the first imaging scan.

7. evaluating the validity of the first imaging scan further comprises identifying one or more poor focus regions on the target, the focus control data indicating that the detector array was out of focus throughout the one or more poor focus regions during the first imaging scan; 2. The method of claim 1, wherein determining a change to the one or more scanning parameters comprises adjusting a focusing procedure to obtain an image of the target at a different focus height during the second imaging scan in the one or more poor focus regions compared to corresponding images from the first imaging scan.

8. The first imaging scan further comprises deriving a second image of the target from the first image, the second image having a lower resolution than the first image; The step of analyzing the validity of the first imaging scan includes: performing automated image analysis of the second image to generate a second probability map for areas of the target where the sample is expected to be present; identifying surface details in the pre-scan image that correspond to locations on the target where the first probability map indicates that the sample is expected to be present and the second probability map indicates that the sample is not present; Further comprising: The method of claim 7 , wherein the one or more poor focus regions exclude locations on the target having the surface detail.

9. The method of claim 8 , wherein the resolution of the second image is approximately equal to the resolution of the pre-scan image.

10. the focus control data comprises an estimated focal plane of the target at each location along the image scan path, and an actual focal height of the detector array is monitored for each location along the image scan path at which an image is acquired; 10. The method of claim 7, wherein identifying one or more poor focus regions on the target comprises identifying one or more position error locations where a difference between the estimated focal plane and the actual focal height of the detector array exceeds an error threshold.

11. the first imaging scan further comprises generating a first image of the target from the images acquired along the image scan path; A method according to any one of claims 7 to 9, wherein identifying the one or more poor focus regions further comprises: performing an automated image analysis of one or more subregions of the first image to identify whether the focus of the first image is acceptable; and designating each subregion of the target where the focus is unacceptable as a poor focus region, wherein the one or more subregions are selected based on the focus control data.

12. The method of claim 11 , wherein the one or more sub-regions include position error locations.

13. 2. The method of claim 1, further comprising, prior to the first imaging scan, measuring a focus position for at least one seed location within the one or more AOIs, and setting the focus height at one or more locations along the image scan path based on the measured focus position for the seed location.

14. The method of claim 13 , wherein the one or more scanning parameters are selected from a group including an area of ​​the target imaged, a number of seed locations on the target, and positions of the seed locations on the target.

15. The method of claim 13 , wherein analyzing the validity of the first imaging scan comprises identifying an error in the measured focus position relative to one or more of the seed locations.

16. 16. The method of claim 15, wherein identifying an error in the focus position comprises comparing the focus control data from the first imaging scan to a measured focus position for the one or more seed locations prior to the first imaging scan.

17. 17. The method of claim 16, wherein comparing the focus control data from the first imaging scan to the measured focus positions for the one or more seed locations prior to the first imaging scan comprises comparing the focus control data from the seed locations during the first imaging scan to the measured focus positions for the one or more seed locations prior to the first imaging scan.

18. 14. The method of claim 13, wherein determining a change in one or more scan parameters from the first imaging scan comprises selecting a new seed location on the target at a location where the focus control data indicates that the detector array was out of focus during the first imaging scan.

19. the first imaging scan is performed across a plurality of AOIs, and the focus position is measured by adjusting the focal height of the detector array through a range of positions perpendicular to a face of the target at each of the seed locations; analyzing the effectiveness of the first imaging scan comprises calculating an average focus height of the samples within each of the plurality of AOIs from the focus control data; 14. The method of claim 13, wherein in response to finding a separation between average focus heights for a first AOI and a second AOI of the plurality of AOIs exceeding a threshold distance, determining a change to one or more scan parameters from the first imaging scan comprises modifying a range of positions over which the focus height is adjusted when monitoring the focus position at the seed location within the first AOI to exclude the average focus height for the first AOI calculated from the first imaging scan, wherein the average focus height calculated for the first AOI exceeds the average focus height calculated for the second AOI.

20. 20. The method of claim 19, wherein the average focus height calculated for the second AOI is the lowest focus height from a group that includes the average focus heights calculated for each of the multiple AOIs.

21. 14. The method of claim 13, further comprising performing the second imaging scan of the target using the detector array with the determined changes in the one or more scanning parameters relative to the first imaging scan performed.

22. The method of claim 1, wherein the step of analyzing the effectiveness of the first imaging scan using the focus control data includes evaluating whether the imaged area on the target sufficiently covers the area on the target occupied by the sample and / or whether the sample is correctly focused throughout the image.

23. 13. A computer program product containing instructions that when executed using a microscope scanner comprising a detector array and a target comprising a sample causes the microscope scanner to perform the method of claim 1.

Images