Method for determining boundaries of z-stack of images of object, corresponding optical apparatus, and computer program therefor

Abstract

To provide a method of automatically determining boundaries of a z-stack of images of an object.SOLUTION: The z-stack of the images is acquired by imaging the object at different focal positions by an optical apparatus. The optical apparatus includes an apparatus optical system and a focus adjusting unit for imaging the object at different focal positions through the apparatus optical system. The method includes the steps of: generating a set of images of the object, each image being captured at different focal positions; applying a blurriness-W distance function to each image of the set of images, calculating a distance value for each image by the blurriness-W distance function, and associating respectively different distance values with different image sharpness or blurriness of the corresponding image; and determining the z-stack boundaries according to the focal positions allocated to two secondary extrema when the blurriness-W distance function having the focal position as a variable shows a primary extremum and the two secondary extrema adjacent to the primary extremum.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a method for automatically determining the boundaries of a z-stack of an image of an object, wherein the z-stack of the image is acquired by imaging the object at different focal positions using an optical instrument. The present invention further relates to the corresponding optical instrument and a computer program for performing the method. In particular, the present invention is in the field of microscopy in which the z-stack of an image of an object is acquired for three-dimensional imaging of the object by image processing of the z-stack image in particular. [Background technology]

[0002] In microscopy imaging, a z-stack is a set of multiple images captured sequentially at the same x-y position but at different depths of focus / z positions of a sample / object. In this application, the focal direction or z-axis direction is perpendicular to the x-y plane or "sample plane" of the sample or object. Images captured within a z-stack are typically image-processed to generate a three-dimensional representation of the sample / object within the boundaries of the z-stack.

[0003] Microscope users typically spend a considerable amount of time observing a sample until they define the limits or boundaries of the z-stack. Initially, they may manually or automatically determine, for example, the optimal focal point, e.g., the z-position of the central plane at the center of the sample. Then, they may expand the z-stack boundaries, starting symmetrically from the central plane, by defining the z-stack range and the number of images / slice within the z-stack. However, such methods are often impractical for samples themselves, especially those that are very time-consuming, particularly with respect to the z-stack boundaries and / or especially when done manually. Furthermore, such methods can be problematic for automated acquisition processes, such as large-scale screening. When manually determining acquisition limits, users need to observe the sample beforehand to learn the approximate degree of object depth. If the sample has different regions with large variations in the focal plane, setting corresponding z-stack limits is time-consuming and laborious. On the other hand, setting fixed z-stack limits in such cases may result in empty images or z-stacks that do not include the sample boundaries, depending on the sample region, potentially leading to a loss of image information. [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] An object of the present invention is to provide an improved method for automatically determining the boundary / border / limit of the z-stack of an image of an object / sample, particularly in microscopy. This z-stack of the image is obtained by imaging the object at different focal positions using an optical instrument, such as a microscope. Another object of the present invention is to provide an improved corresponding optical instrument and a computer program having program code for carrying out the method. [Means for solving the problem]

[0005] This objective is achieved by the methods, optical instruments, and computer programs described in the independent claims. Embodiments of the present invention are the subject of each dependent claim and the description herein.

[0006] To automatically determine the appropriate z-stack boundaries at the x-y positions of a sample / object, a set of images of the object is generated, and each image is captured at a different focal position. For this purpose, the optical instrument includes an instrument optics system and a focusing unit for imaging the object at different focal positions via the instrument optics system. That is, the z-stack or set of images of the object is acquired to automatically determine the boundaries of another z-stack of images acquired by imaging the object at different focal positions within the determined z-stack boundaries. The images acquired to determine the z-stack boundaries may or may not be used as images of the z-stack itself at a later point.

[0007] The z-stack acquisition method is often used for autofocus in microscopy. Images are acquired while the microscope stage or objective lens is scanned at different z-positions in the focal direction at a predetermined x-y position of the sample. For the obtained z-stack, a specific distance function applied to each image can be used, along with the focal position as a variable. Such a distance function allows for the extraction of information about the image contrast, entropy, spatial frequency content, etc., of the z-stack images. Various distance functions have been evaluated, particularly in terms of different microscope imaging modes and noise tolerance (see, for example, “Analysis of focus measure operators for shape-from-focus”, Said Pertuz et al., in Pattern Recognition 46 (2013) 1415-1432).

[0008] The inventors have found that several distance functions used in autofocus can be used for the automatic determination of z-stack boundaries. The inventors have found that two types of distance functions can be identified from among the distance functions used in autofocus: the "blurriness distance function" and the "blurriness-W distance function".

[0009] By using a so-called "blur distance function," its global minimum value can be used to obtain the focal position of the minimum blurriness in the z-stack, i.e., the focal position of the maximum sharpness, and therefore typically the focal plane of the object being imaged. The best results are achieved for reference objects that can be defined as spherical or elliptical objects with defined boundaries, such as cells or cell clusters. The "sharpness distance function" should be encompassed within the term "blur distance function" as it corresponds to the "blur distance function" with the opposite sign. When an object is in focus, the captured image should exhibit high image contrast, a wide range of intensity values, and sharp edges. Therefore, a suitable blur (or sharpness) distance function is particularly sensitive to the focal point, decreases monotonically on both sides of the focal point, is optimally symmetric with respect to the peak, and does not exhibit local extrema. A group of distance functions that can be used as blur distance functions is given below.

[0010] The “blurriness-W distance function” can be considered a subfamily of blurriness or sharpness distance functions, which represent global extremes, e.g., the global maximum value for maximum sharpness at the focal point, and further, local extremes, e.g., two local minimums adjacent to the global maximum value (the distance function in this example has the form “W,” which is why the name blurriness-W distance function is chosen. On the other hand, if such a distance function with the opposite sign is considered, it can be called the “blurriness-M distance function” due to the resulting “M” form. For this reason, the blurriness-M distance function should be encompassed within the term “blurriness-W distance function”). Again, the best results are achieved for spherical or elliptical objects of the defined boundary, e.g., reference objects that can be defined as cells or cell clusters. Using such a “blurriness-W distance function,” the inventors have found that the focal positions assigned to two local extrema / minims represent or indicate the boundary of an object that is sharp enough to still distinguish the corresponding image from its background. That is, the corresponding focal positions can be used to determine the appropriate boundary of the z-stack. The z-stack boundary can be determined depending on or largely at the focal positions assigned to the two local extrema. In this specification, “primary extrema” is another term for global maximum or global minimum, while the term “secondary extrema” refers to a local maximum or local minimum.

[0011] In summary, according to embodiments of the present invention, the blurriness-W distance function defined above is applied to each image in a set of images acquired by an optical instrument at different focal positions, a distance value is calculated for each image using the blurriness-W distance function, and each different distance value is associated with different image sharpness or blurriness of the corresponding image, and if the blurriness-W distance function, which has focal position as a variable, indicates a first-order extremum and two second-order extremums adjacent to that first-order extremum, the z-stack boundary is determined according to the focal positions assigned to the two second-order extremums. If the first-order extremum is the maximum value, the two second-order extremums are typically the minimum values, and vice versa.

[0012] The present invention makes it possible to determine the z-stack boundary purely automatically, without requiring the user to observe the sample / object beforehand, infer the extent of the sample / object, or define a z-stack starting symmetrically from the center of the sample. By using a computer program having program code for carrying out the method according to the present invention, the user can simply select z-stack imaging at a given location on the sample, and the method according to the present invention will automatically determine the limits or boundaries of the z-stack without further user input. This greatly improves the usability for users operating optical instruments, especially microscopes. When performing z-stack imaging, the user only needs to specify the z-step interval and / or the number of images / slice in a given z-stack. Alternatively, it may be convenient to set such step intervals or the number of images / slices by default, based on experience with a particular sample and / or imaging mode.

[0013] It should be noted that, in practice, there is not just one blurriness-W distance function suitable for application to the set of images according to the method of the present invention. For this reason, two or more blurriness-W distance functions can be selected from the group of blurriness distance functions defined above. Then, it may be convenient to apply two or more such blurriness-W distance functions to the set of images and select a focal position having at least several, for example, at least two, quadratic extrema where the blurriness-W distance functions coincide.

[0014] Furthermore, it may be advantageous to limit the range from which the set of images is acquired to a region around the focal plane. The focal plane can be found by applying the blurring distance function described above to a set of images that can be acquired in a larger step size in the focal direction. The applied blurring distance function must have an absolute extremum at the focal plane and, optimally, should not have any further extrema adjacent to this absolute extremum. By applying such a blurring distance function, the focal plane can be easily identified. Alternatively, the user can visually identify the focal plane. According to the present invention, the step of generating each set of images captured at different focal positions of the object should cover a range of focal positions including the focal plane. Note that the first extremum of the blurring-W distance function is typically at the same focal position as the absolute extremum of the blurring distance function.

[0015] For example, there may be cases, depending on the object and / or imaging method, where the selected blur-W distance function does not or does not clearly show two quadratic extremes. In this case, as already explained above, another blur-W distance function can be selected from the appropriate set of blur-W distance functions.

[0016] Alternatively, or for redundancy testing, the blur distance function defined above is applied to each image in the set of images; the blur distance function calculates a distance value for each image; each different distance value is associated with a different image sharpness or blurriness of the corresponding image; and if the blur distance function, with focal position as a variable, shows an absolute extremum, the integral of the blur distance function is applied starting from the focal position of the absolute extremum, and is integrated equally on both sides until a predetermined portion of the total integral over all focal positions is reached. Then, the z-stack boundary is determined according to the focal positions assigned to the two limits of the predetermined portion of the total integral.

[0017] In embodiments, this alternative method can be applied either to verify the results of the first method mentioned using the blur-W distance function, or, contrary to expectations, when the blur-W distance function does not show or does not show two quadratic extrema sufficiently clearly (in each experimental setting). However, it should be noted that an integral method based on the blur distance function can be used without first applying the blur-W distance function.

[0018] In this embodiment, typically, a blurring distance function having an absolute extremum at the optimal focal position or focal plane of the sample / object is calculated by starting from the focal position of this absolute extremum and integrating equally on both sides until a predetermined proportion of the total integral is reached. The total integral over all focal positions in the set of images is calculated beforehand, in parallel, or afterward. It is an advantage, but not a requirement, that the blurring distance function be symmetric with respect to this absolute extremum focal position. If it is symmetric, the distances of the two limits of a given portion of the total integral with respect to the absolute extremum focal position are (essentially) the same. In particular, in this case, the z-stack boundary can be determined as the focal position assigned to the two limits of a given portion of the total integral. In general, and especially in the case of an asymmetric blurring distance function, the z-stack boundary is determined according to the discovered focal positions assigned to the two integration limits. An example will be described in more detail below.

[0019] In embodiments where the blurriness-W distance function indicates a first extremum and two adjacent second extremums, it may still be meaningful to apply the blurriness distance function to verify whether the first extremum of the blurriness-W distance function corresponds to the absolute extremum of the blurriness distance function. This verification serves the purpose of applying both distance functions to a set of images acquired near the focal plane or optimal focal value, but not to a set of images acquired near the focal positions of the second extremums.

[0020] In a similar embodiment where the blur - W distance function is applied, before the step of applying the blur - W distance function to each image of the set of images, the blur distance function can be (additionally) applied to each image of the set of images. By the blur distance function defined above, a distance value for each image is calculated, and different distance values are associated with different image sharpness or blur of the corresponding images. The blur distance function having the focal position as a variable shows an absolute extremum and is symmetric with respect to the corresponding focal position of the absolute extremum. In particular, when it does not show two secondary extrema adjacent to the absolute extremum or more generally does not show any secondary extrema, the blur - W distance function is applied to each image of the set of images to determine the z - stack boundary. This embodiment of first applying the blur distance function by the above method and then applying the blur - W distance function to determine the z - stack boundary has proven to be beneficial and yields good results when determining the z - stack boundary. This embodiment will be examined in more detail below.

[0021] Regarding an alternative or additional method of determining the boundary of the z - stack by integrating the blur distance function, a predetermined portion of the total integral of the blur distance function can be selected to be in the range of 60 - 80% or 60 - 70% or 63% (corresponding to 1 / e) or 66% (2 / 3). The latter two values are preferred, but can be replaced with other values depending on the experimental setup.

[0022] In practice, depending on the type of sample and imaging mode, the blur distance function having the focal position as a variable may show an absolute extremum but may not be symmetric with respect to the corresponding focal position of the absolute extremum. In such a case, the integration of the blur distance function can still be applied to determine the z - stack boundary. On the other hand, when an absolute extremum - showing blur distance function cannot be found, it is usually shown that the z - stack boundary cannot be determined.

[0023] In an embodiment, the blur-W distance function is selected from a group of distance functions based on a gradient method or a distance function based on a wavelet decomposition method.

[0024] In an embodiment, the blur distance function is selected from a group of distance functions based on an autocorrelation method.

[0025] In another embodiment, the optimal focus for imaging an object is defined as the focus at which the blur distance function has its absolute extreme value.

[0026] In another embodiment, the step of generating each set of images captured at different focus positions of the object includes generating a first stack of images using a first focus step size over a first range of focus positions and generating a second stack of images using a second focus step size over a second range of focus positions. In an embodiment, the second step size is smaller than the first step size. In an embodiment, the second range of focus is smaller than the first range of focus. This helps to increase the speed of image acquisition by concentrating on the area around the focal plane, and the focal plane can be quickly found after generating the first stack of images. Thereafter, the area around the focal plane is scanned with higher accuracy.

[0027] In another embodiment, the boundary of the object is determined for each set of images in the first and second stacks of images. Thus, the process can be optimized in two or more granularity steps as follows.

[0028] In the first step of the embodiment, a stack of about 30 slices / image is acquired with a larger step size within the limits of the objective lens, i.e., within a range of 600 μm, for example 20 μm. According to an embodiment of the present invention, a z-range finder is applied to obtain a first z-stack limit. The z-range finder provides, for example, slices 3 and 18 as two z-stack boundaries, and the z-stack is 300 μm wide.

[0029] In the second step, a thinner step size is applied (here, for example, 3.6 μm over a range of 300 μm, resulting in a total of 84 images). The z-rangefinder is applied a second time, and the z-stack boundaries are provided, for example, as slices 25 and 60.

[0030] Ultimately, the output is both the lower and upper boundaries of the z-stack for the object. It should be noted that, generally, instead of slice numbers, the position can also be obtained in microscope stage coordinates in the z-direction. This can also involve interpolating the distance between two slices and finding the optimal equivalent z-value in micrometers.

[0031] In the embodiment, after the step of generating a set of images of the object, an indistinct distance function is applied to each image in the set of images. If the indistinct distance function has an absolute extremum and is symmetric with respect to the corresponding focal position of that absolute extremum, and in particular does not show any quadratic extremums, then an indistinct-W distance function is applied to each image in the set of images. On the other hand, if the indistinct distance function has an absolute extremum but is not symmetric, then the integral of the indistinct distance function is applied to determine the z-stack boundary. If the indistinct-W distance function is applied, it is verified whether the indistinct-W distance function has two quadratic extremums adjacent to a linear extremum. In that case, the z-stack boundary is determined according to (or assuming) the focal positions assigned to the two quadratic extremums. If the indistinct-W distance function does not have such two quadratic extremums, the z-stack boundary is determined by applying an integral over the indistinct distance function.

[0032] Furthermore, the present invention relates to an optical device for imaging an object. The optical device comprises an instrument optical system for imaging an object, a focusing unit for adjusting the focal position at which the object is imaged, and a processor operably connected to the instrument optical system and the focusing unit and configured to perform any one of the embodiments of the method according to the present invention described above, in order to generate sets of images captured at different focal positions of the object.

[0033] In an embodiment, the optical instrument is a microscope, and the instrument optics include a microscope optics, in particular a microscope objective lens configured to produce a magnified image of an object.

[0034] Furthermore, the present invention relates to a computer program having program code for carrying out embodiments of the above-described method according to the present invention when the computer program is executed on a processor, particularly on a processor of an optical device according to the present invention.

[0035] It should be noted that the above-described features of the embodiments can be combined, in whole or in part, to achieve other embodiments that still fall within the scope of the concept of the present invention as defined in the appended claims.

[0036] As used herein, the term "and / or" includes all possible combinations of one or more of the related items and may be abbreviated as " / ".

[0037] While several embodiments have been described in the context of methods, it is clear that these embodiments also represent descriptions of apparatus or devices configured to operate according to such methods.

[0038] Further embodiments and their advantages are described below in relation to the following figures. [Brief explanation of the drawing]

[0039] [Figure 1] This diagram schematically shows an example of a sample carrier for holding samples to acquire a z-stack of images obtained using a microscope. [Figure 2] This figure schematically shows the blur-W distance function and blur distance function used to determine the z-stack boundary within a set of images. [Figure 3] The flowchart of the workflow (left side of Figure 3) and the corresponding graph of the distance function (right side) are schematically shown. [Figure 4] A schematic flowchart of another workflow for an embodiment of the method according to the present invention is shown. [Figure 5] A schematic flowchart (left) and a graph of the corresponding distance function (right) are shown for another embodiment of the method according to the present invention. [Figure 6] A flowchart (left) and a corresponding graph (right) of the corresponding distance function of another embodiment of the method according to the present invention are schematically shown. [Figure 7] This figure schematically illustrates an example of determining the z-stack boundary by integrating an exemplary blurred distance function. [Figure 8] This figure schematically shows a microscope as an example of an optical instrument according to an embodiment of the present invention. [Modes for carrying out the invention]

[0040] The following drawings provide a comprehensive explanation, and the same reference numerals indicate the same or at least functionally the same elements. The detailed explanation pertains to imaging objects with a microscope, but other optical instruments may be considered.

[0041] Figure 1 schematically shows a tilted sample carrier 140 for holding the sample. The tilted focal plane / level 150 (see "F") is determined by an autofocusing method. In this embodiment, the sample has a spread that varies in the focal direction, and therefore, typically, a focus map can be generated by an autofocusing method for microscopic imaging. The microscope is used to image the sample and has an objective lens range limit 160. The sample from which the z-stack of images is acquired is designated 130 (object of interest). The desired z-stack boundaries may be 120 as the lower boundary and 110 as the upper boundary. In practice, the object boundary of the object of interest 130 can coincide with the z-stack boundaries or be either outside or inside such z-stack boundaries 110,120. In the schematic diagram of Figure 1, the object boundary is within the z-stack boundaries 110,120.

[0042] The following description will examine a method for automatically determining such z-stack boundaries 110,120 according to embodiments of the present invention.

[0043] Figure 2 shows exemplary images of object 130 horizontally at the bottom. Object 130 represents a typical reference object discussed above, which is spherical and has clearly defined boundaries, and is photographed at different focal positions in the z-direction shown in Figure 1 through a microscope optical system configured to generate magnified images of the object. Visual inspection reveals that the focal plane with the sharpest image is at approximately the focal position of the 10th image (although it may also be the 11th or 12th image, not shown here). It should be noted that Figure 2 shows only eight of a total of 31 images (images 0 to 30). While a user can easily identify the optimal focal plane for imaging object 130 by visually inspecting images 0 to 30 in Figure 2, only through a thorough inspection of the object boundaries can the user determine the z-stack boundaries 110, 120 shown in Figure 1. On the other hand, the Z-stack boundaries 110, 120 can be found by applying the following embodiments of the method of the present invention.

[0044] After a set of images 231, 230, is generated, each image 231 is captured at a different focal position in the z-direction (see Figure 1), and the blur-W distance function 210 is applied to each image 231 in the set of images 230. The blur-W distance function 210 calculates a distance value (y-axis) for each image 231, and each different distance value is associated with a different image sharpness or blurriness of the corresponding image, as can be seen from the set of images 231, 230, below the graph. The graph in Figure 2 shows the blur-W distance function 210 with focal position as the variable (x-axis). The blur-W distance function 210 clearly shows a first-order extremum or maximum value 214 and two second-order extremums or minimum values ​​212 and 216 adjacent to this first-order extremum 214, resulting in a typical "W" shape. The inventors have found that the z-stack boundaries 110 and 120 in Figure 1 can be determined according to the focal positions assigned to the two quadratic extrema 212 and 216.

[0045] For example, the focal positions assigned to the two minimum values ​​212 and 216 can be used directly as z-stack boundaries 120 and 110. Note that, depending on the experimental setup, the shape and topology of another object 130 may deviate from the examples considered in relation to Figures 1 and 2. Also, the microscope imaging mode affects the image 231 captured in set 230 of images 231. Thus, the behavior of the blurriness-W distance function 210 may actually differ when applied to different objects 130 and / or under different microscope imaging modes. Such different experimental setups may result in a blurriness-W distance function 210 that no longer clearly shows the quadratic and / or primary extremes. Similar considerations apply to the blurriness distance function 220 shown in Figure 2. Embodiments of the present invention even make it possible to address such different experimental setups, as described below.

[0046] Furthermore, an appropriate blurriness distance function 220 is also applied to each image 231 in the set of images 230, and the blurriness distance function 220 also calculates a distance value for each image 231, associating each different distance value with different image sharpness or blurriness of the corresponding image 231, as can be seen from Figure 2. The blurriness distance function 220, which has the focal position as a variable (x axis), indicates an absolute extremum 224 at the focal position, and that focal position represents the optimal focus. For the purpose of autofocusing, the optimal focus can be easily determined by applying the blurriness (or sharpness) distance function 220 to the set of images 231 230. On the other hand, the blurriness distance function 220 itself does not provide any indication regarding the z-stack boundary of the image of the object. Nevertheless, the inventors have found an alternative method for determining the z-stack boundaries 110,120 by applying only the blurriness-W distance function 220, as shown in Figure 2, particularly when the blurriness-W distance function 210 is not applicable for the experimental setup in this case. Such cases will be discussed in more detail below.

[0047] As is best seen in Figure 2, the blurriness distance function 220 is rather a "sharpness distance function" because its global maximum value is obtained from the focal position of maximum sharpness. It is particularly sensitive to the focal point, decreases monotonically on both sides of the focal point, is perfectly symmetrical with respect to the peak, and shows no local extrema. For this reason, the blurriness distance function 220 in the embodiment of Figure 2 is a good representative of a certain type of distance function that can be used as a "blurriness distance function" in the sense of the present invention.

[0048] The blur-W distance function 210 exhibits a global maximum value for maximum sharpness at approximately the same focal position as the blur distance function 220. In addition, the blur-W distance function 210 exhibits two local minimums 212,216 adjacent to the global maximum value 214. These local minimums 212,216 indicate the edges of the object 130 where the corresponding image 231 is still sufficiently sharp, and as a result, the image of the object can be distinguished from its background. Therefore, the blur-W distance function 210 in the embodiment of Figure 2 is a good candidate for some kind of distance function that can be used as a blur-W distance function.

[0049] In embodiments, the blur-W distance function 210 is selected from a group of distance functions based on gradient methods or wavelet decomposition methods. In particular, among the gradient method-based distance functions, distance functions that use the first derivatives of the image in the x and y directions, respectively, and therefore spatial frequencies, have been proven to be useful. In embodiments, the blur-W distance function is the function described in Equation A.13 on page 1427 of “Analysis of focus measure operators for shape-from-focus”, Said Pertuz et al., in Pattern Recognition 46 (2013) 1415-1432. Distance functions based on wavelet decomposition methods are mainly based on the statistical properties of discrete wavelet transform (DWT) coefficients. In a first level DWT, the image is decomposed into four subimages. Distance functions based on a first level DWT have been proven to be useful for use as the blur-W distance function. In such embodiments, the blurriness-W distance function is the function described in Equation A.40 on page 1430 of “Analysis of focus measure operators for shape-from-focus”, Said Pertuz et al., in Pattern Recognition 46 (2013) 1415-1432. Furthermore, in embodiments, the blurriness distance function is selected from a group of distance functions based on autocorrelation. In such autocorrelation, image autocorrelation is calculated, for example, by autocorrelating the pixel values ​​of adjacent columns or rows. In embodiments, the blurriness distance function is the function described in Equation A.14 on page 1428 of “Analysis of focus measure operators for shape-from-focus”, Said Pertuz et al., in Pattern Recognition 46 (2013) 1415-1432.

[0050] Figure 3 shows a flowchart of an embodiment of the method according to the present invention. S1 specifies the first step. In the first step, a set 230 of images 231 of the object 130 is generated (see Figures 1 and 2). Each image is captured at a different focal point or z position. In step S2, before step S3 in which the blurriness-W distance function is applied, the blurriness distance function 220 (see Figure 2) is applied to each image 231 of the set of images 230. Another example of the blurriness distance function used in step S2 in the embodiment of Figure 3 is shown on the right side of Figure 3 by reference numeral 320. In the next step, it is evaluated whether the blurriness distance function exhibits an absolute extremum. In this case, the absolute extremum is the maximum value of the function 320. Next, it is evaluated whether the blurriness distance function 320 is symmetric with respect to a good approximation. Again, if this evaluation yields a positive response, the method proceeds to step S3. Before proceeding, it may be beneficial to further evaluate whether the blurriness distance function 320 shows any quadratic extrema adjacent to its absolute extremum. This is because, if it does not, the selected blurriness distance function would rather fall into the category of a blurriness-W distance function.

[0051] In step S3, by applying the blurriness-W distance function 310, the corresponding graph shown on the right side of Figure 3 can be obtained in (S3). This graph shows the blurriness distance function 320 and the superimposed blurriness-W distance function 310.

[0052] Next, the blur-W distance function 310 is evaluated to determine whether there are two quadratic extrema adjacent to the primary extremum. Since the blur-W distance function 310 indicates the primary maximum value and the two local minimums adjacent to the maximum value, the method proceeds to step S4, which determines the z-stack boundary according to the focal positions assigned to the two minimums of the distance function 310 shown on the right side of Figure 3 for step S3 (assuming they are the same here). The focal positions of the z-stack boundaries 110 and 120 are f, respectively. min and f max That is the case.

[0053] The reason for applying the blurriness-W distance function 320 to the set of images 230 before applying the blurriness-W distance function 310 can be considered as follows: First, by applying the blurriness-W distance function 320, the focal position of the focal plane can be easily found as the focal position of the maximum value of the blurriness-W distance function 320. The set of images to be evaluated to determine the z-stack boundary should be acquired in the focal region around the focal position of the maximum value of the blurriness-W distance function 320. Therefore, a shift in the range of focal positions for acquiring the set of images is possible after the blurriness-W distance function 320 has been initially applied. Furthermore, the inventors have found that applying the blurriness-W distance function 310, as shown in step S3 of Figure 3, yields faster and more reliable results for z-stack boundary determination. Thirdly, as shown in step S2 of Figure 3, once the blurriness distance function 320 is derived from the set of images, it is most likely that a blurriness-W distance function, such as function 310 in Figure 3, which can be considered to belong to a subgroup of blurriness distance functions, will satisfy the requirement that it has a first extremum and two distinct second extrema adjacent to that first extremum.

[0054] Figure 4 schematically shows a flowchart of another embodiment of the method according to the present invention, in the case where the evaluation of the blurriness distance function in step S2 results in the blurriness distance function not being symmetric with respect to its peak. Steps S1 and S2 correspond to steps S1 and S2 considered in the embodiment of Figure 3. However, when the blurriness distance function is applied to a set 230 of images 231 of the object, in this case a graph is obtained, shown on the right side of Figure 4 in (S2), showing the blurriness distance function 420 for this experimental setup. The inventors have found that, in particular in such cases, it is almost impossible to find a blurriness-W distance function suitable for determining the z-stack boundary. Instead, an existing blurriness distance function 420, which shows the absolute extremum, here the maximum value, is used for determining the z-stack boundary. Since the symmetry evaluation returned a negative response, the method proceeds to step S31 by integrating the blurriness distance function 420, starting from the focal position f0 of the absolute extremum / maximum value, and integrating equally on both sides until it reaches a predetermined part / proportion of the total integral over all focal positions of the set of images. It has been proven beneficial to define a portion of the total integral as 63% or 66%, but other values, preferably in the range of 60% to 80% or 60% to 70%, may be taken depending on the experimental setup. Then, the z-stack boundary f min and f max However, in step S4, it is determined either according to the focal position or as the focal position assigned to two limits of a given part of the total integral. The integration method will be described in more detail below in relation to Figure 7.

[0055] It should also be noted that the integral method considered in relation to the embodiment in Figure 4 can be applied immediately after step S2, instead of applying the blurriness-W distance function (see Figure 3). From here, we will consider another case in which the integral method is applied in relation to Figure 5.

[0056] Figure 5 shows a flowchart of another embodiment of the method according to the present invention. The flowchart from step S1 to step S3 is identical to the flowchart in Figure 3. Therefore, the consideration of steps S1 to S3 related to the embodiment in Figure 3 is explicitly referenced. When the blurriness distance function is applied to the set 230 of images 231, a function 520 is obtained as shown in the graph on the right side of Figure 5 in (S3). The blurriness distance function 520 has an absolute extremum and is symmetric with respect to its maximum value, and the blurriness-W distance function is applied in step S3. However, in this experimental setup, the obtained blurriness-W distance function 510 does not show two quadratic extrema adjacent to the first extremum / maximum value. Although one could try selecting another suitable blurriness-W distance function, it is easier and quicker to proceed to step S31, i.e., to proceed to the integral method described in step S31 of the embodiment in Figure 4. For this reason, this embodiment is referenced for the explanation of the integral method of the blurriness distance function 520 shown in Figure 5. As a result of the integration method, in step S4, the z-stack boundary f min and f max This is determined.

[0057] Figure 6 schematically shows a flowchart of yet another embodiment of the method according to the present invention. The flowchart in Figure 6 is the complete workflow of the combined flowcharts in Figures 3 to 5. For details of the workflow shown in Figure 6, please refer to the above discussion. Below, only a brief description of the overall workflow is given. After the step of generating a set 230 of images 231 of the object 130 (not shown in Figure 6), the blurriness distance function 220 is first applied to each image in the set of images. If the blurriness distance function 220 has an absolute extremum 224 (see Figure 2), is symmetric with respect to the corresponding focus of this absolute extremum 224, and does not show a quadratic extremum adjacent to the absolute extremum, the workflow proceeds to step S3. On the other hand, if the blurriness distance function cannot show an absolute extremum, the workflow terminates in step S5, stating that the boundary of the z-stack cannot be found. On the other hand, if the blurriness distance function is not symmetric with respect to the focus to which its absolute extremum belongs, as in the blurriness distance function 420 shown in Figure 4, the workflow moves to step S31, where the integral method is applied to the blurriness distance function, as discussed in detail in relation to the embodiment in Figure 4.

[0058] If, in step S3, a blur-W distance function is applied again that does not have two quadratic extrema adjacent to the primary extrema, the workflow proceeds to step S31, as already discussed in the embodiment of Figure 5. On the other hand, if the blur-W distance function 210 applied in step S3 indicates two quadratic extrema adjacent to the primary extrema, the workflow proceeds to step S4, where the z-stack boundary is determined according to or as these focal positions assigned to the two quadratic extrema.

[0059] FIG. 7 shows the integration method applied to the blurriness distance function, which was described above in connection with the embodiments of FIGS. 4 (step S31) and 5 (also step S31). FIG. 7 shows the steps of an integration algorithm applied to the blurriness distance function 720, and the maximum value 724 is shown as its absolute extreme value. The graph (1) in FIG. 7 shows the blurriness distance function 720 after the signal is smoothed, normalized in the range of values 0 to 1 (y-axis).

[0060] First, the total integration of the function 720 over the entire range of the focus positions of the set of images is calculated. Then, starting from the global maximum value 724, integration is performed by moving stepwise left and right until the covered area first becomes 66% or more. Also in this case, 66% is just an exemplary value as considered above. The corresponding limit of the 66% integration is the potential position of the z-stack (see graph (2) in FIG. 7). The actual z-stack boundary is determined according to these two potential focus positions, as shown in graph (3) of FIG. 7.

[0061] As a redundant check, as shown in graph (2) of FIG. 7, starting from the potential f min position determined in the previous step, it is evaluated whether there is a higher focus value before the potential limit f min , and it is inspected whether there is a point with a higher focus value behind the potential limit f max . This clearly cannot be the case for the function 720 that monotonically decreases on both sides of the peak 724. On the other hand, if such a higher focus exists, the corresponding focus position becomes the new limit. Then, if the potential limits f min and f max shown in graph (2) of FIG. 7 are not symmetric, at least one of these limits is set to a more restrictive value so that the final limits f min and f max shown in graph (3) of FIG. 7 are symmetric around the peak 724. In the example shown in graphs (2) and (3) of FIG. 7, the potential upper limit f max is set to the final limit f maxI set it to a lower number, or a more restrictive number.

[0062] Figure 8 schematically shows a microscope 800 as one embodiment of the optical instrument according to the present invention. The microscope 800 comprises instrument optics 810, 820 for imaging an object 130. The instrument optics include, in particular, an objective lens 810 having a front lens 812 and an image detector 820. In this embodiment, the sample or object 130 can be placed on a microscope stage 860 and illuminated by a light source 852 and an illumination optics 850. The objective lens 810 defines an optical axis 802 for transmitted light imaging of the sample 130.

[0063] The microscope 800 further comprises a processor 840 in the form of a controller and a focusing unit 830. The focusing unit 830 in this embodiment includes a z-drive for moving the microscope stage 860 in the direction of arrow 832. In principle, the objective lens 810 can also be moved in the z-direction. By moving the z-drive, the focal position from which the object 130 is imaged can be adjusted.

[0064] The controller / processor 840 is operably connected to the instrument optics, in this case the image detector 820 and the focusing unit 830, to generate a set 230 of images 231. Each image 231 is captured at a different focal point or z position (see Figure 2). The controller / processor 840 is further configured to process this set 230 of images 231 according to an embodiment of the method according to the present invention in order to automatically determine the lower boundary 120 and upper boundary 110 of the z-stack range 115, as illustrated in Figure 8.

[0065] After the z-stack boundaries 110 and 120 are determined, z-stack image acquisition is started, and for example, a 3D representation of a portion of the object 130 captured within the z-stack range 115 is generated.

[0066] The computer program according to the present invention is executed on a controller / processor 840, and generally on a computer system 840, in particular to carry out embodiments of the method according to the present invention.

[0067] The computer system 840 is configured to carry out at least part of the methods described herein. The computer system 840 may also be configured to run machine learning algorithms. The computer system 840 and the microscope 800 may be separate entities, or they may be integrated within a single common housing. The computer system 840 may be part of the central processing system of the microscope 800, and / or the computer system 840 may be part of a dependent component of the microscope 800, such as a sensor, actor, camera, or illumination unit of the microscope 800.

[0068] The computer system 840 may be a local computer device (e.g., a personal computer, laptop, tablet computer, or mobile phone) comprising one or more processors and one or more storage devices, or it may be a distributed computer system (e.g., a cloud computing system comprising one or more processors and one or more storage devices distributed to various locations such as local clients and / or one or more remote server farms and / or data centers). The computer system 840 may include any circuit or combination of circuits. In one embodiment, the computer system 840 may include one or more processors, which can be of any kind. As used herein, the processor may be intended to be any kind of computing circuit, such as a microprocessor for a microscope or microscopic component (e.g., a camera), a microcontroller, a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multicore processor, a field-programmable gate array (FPGA), or any other kind of processor or processing circuit. Other types of circuits that may be included in the computer system 840 may be custom circuits, application-specific integrated circuits (ASICs), etc., such as one or more circuits (communication circuits, etc.) used in wireless devices such as mobile phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 840 may also include one or more storage devices that may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives that handle removable media such as compact discs (CDs), flash memory cards, digital video discs (DVDs), etc.The computer system 840 may also include a display device, one or more speakers and a controller which may include a keyboard and / or mouse, trackball, touchscreen, voice recognition device, or any other device which enables a user of the system to input information into and receive information from the computer system 840.

[0069] Some or all of the steps may be performed by a hardware device (or by using a hardware device), such as a processor, microprocessor, programmable computer, or electronic circuit. In some embodiments, one or more of the most critical steps may be performed by such a device.

[0070] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation is feasible using a non-transient recording medium, which is a digital recording medium, etc., that stores electronically readable control signals and cooperates (or can cooperate) with a programmable computer system to carry out each method. Examples include floppy disks, DVDs, Blu-rays, CDs, ROMs, PROMs and EPROMs, EEPROMs, or FLASH memory. Thus, the digital recording medium may be computer-readable.

[0071] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system so as to carry out any of the methods described herein.

[0072] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code, which operates to perform one of the methods when the computer program product is executed on a computer. This program code may be stored, for example, on a machine-readable carrier.

[0073] Another embodiment includes a computer program stored in a machine-readable carrier for carrying out any of the methods described herein.

[0074] Therefore, in other words, embodiments of the present invention are computer programs having program code for carrying out any of the methods described herein when the computer program is executed on a computer.

[0075] Accordingly, another embodiment of the present invention is a recording medium (or data carrier or computer-readable medium) containing a stored computer program for carrying out any of the methods described herein when executed by a processor. The data carrier, digital recording medium, or recording medium is typically tangible and / or non-transient. Another embodiment of the present invention is an apparatus, such as those described herein, comprising a processor and a recording medium.

[0076] Therefore, another embodiment of the present invention is a data stream or signal sequence representing a computer program for carrying out any of the methods described herein. The data stream or signal sequence may be configured to be transmitted, for example, over a data communication connection, such as the Internet.

[0077] Another embodiment includes processing means, for example, a computer or programmable logic device configured or adapted to carry out any of the methods described herein.

[0078] Another embodiment includes a computer having an installed computer program for carrying out any of the methods described herein.

[0079] Another embodiment of the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for carrying out any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0080] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to carry out any of the methods described herein. Generally, the methods are advantageously carried out by any hardware device. [Explanation of symbols]

[0081] 110,f min SESTAC upper boundary 115 z stack range 120,f max Lower boundary of z stack 130 Target object, sample 140 sample carrier 150 focal plane 160 Objective lens range limit 210 Blurryness-W Distance Function 212 Second-order extreme values 214 Primary extreme values 216 Second-order extreme values 220 Blurry Distance Function 224 Absolute Extrema Set of 230 images 231 images 310,510 Blurriness - W distance function 320,520 Blurredness Distance Function 720 Blurry Distance Function 724 Absolute extremum 800 Microscopes, optical instruments 802 Optical axis 810 Objective Lens 812 Front Lens 820 Image Detector 830 Focusing Unit 832 Direction of movement 840 Processor, Controller, Computer System 850 Illumination optical system 852 Light source 860 Microscope Stage Steps of the method: S1-S5, S31

Claims

1. 1. A method for automatically determining boundaries (110, 120) of a z-stack of images of an object (130), comprising: The z-stack of images is acquired by imaging the object at different focus positions with an optical instrument, the optical instrument comprising instrument optics and a focusing unit for imaging the object at different focus positions via instrument optics; The method comprises: generating a set (230) of images (231) captured at different focus positions of the object (130); a step (S3) of applying a blur-W distance function (210) to each image (231) of the set of images (230), the blur-W distance function (210) being a blur or sharpness distance function having the focus position as a variable and exhibiting a global extremum of maximum or minimum sharpness at the focus position and additionally exhibiting secondary extrema adjacent to the global extremum, the blur-W distance function (210) calculating a distance value for each image (231), each different distance value being associated with a different image sharpness or blur of the corresponding image (231); a step (S4) of determining the boundaries (110, 120) of the z-stack according to the focus positions assigned to the two secondary extrema (212, 216) when the blur-W distance function (210) having the focus position as a variable indicates a primary extrema (214) and two secondary extrema (212, 216) adjacent to the primary extrema (214); A method comprising:

2. 1. A method for automatically determining boundaries (110, 120) of a z-stack of images of an object (130), comprising: The z-stack of images is acquired by imaging the object at different focus positions with an optical instrument, the optical instrument comprising instrument optics and a focusing unit for imaging the object at different focus positions via instrument optics; The method comprises: generating a set (230) of images (231) captured at different focus positions of the object (130); applying (S2) a blur distance function (220) to each image (231) of the set of images (230), wherein the blur distance function (220) calculates a distance value for each image, each different distance value being associated with a different image sharpness or blurriness of the corresponding image (231), the blur distance function exhibiting a first-order extremum for maximum or minimum sharpness or blurriness at a focus position; If the blur distance function (220) having the focus position as a variable exhibits an absolute extremum (224), applying (S31) an integral of the blur distance function starting from the focus position of the absolute extremum and integrating equally on both sides until a predetermined portion of the total integral over all of the focus positions is reached; determining (S4) the boundaries of said z-stack according to focus positions assigned to two limits of a predetermined portion of said total integral; A method comprising:

3. Prior to step (S3) of applying the blur-W distance function (210) to each image (231) of the set of images (230), a blur distance function (220) is applied (S2) to each image (231) of the set of images (230), and the blur distance function (220) calculates a distance value for each image, each different distance value being associated with a different image sharpness or blurriness of the corresponding image (231); If the blur distance function (220) having the focus position as a variable exhibits absolute extrema (224) and is symmetric about the corresponding focus position of the absolute extrema, the blur-W distance function (210) is applied (S3) to each image (231) of the set of images (230) to determine (S4) the boundaries of the z-stack; The method of claim 1.

4. If the blur-W distance function (210) exhibits a primary extremum (214) and two secondary extrema (212, 216) adjacent to the primary extremum (214), the method comprises the further step of verifying whether the primary extremum (214) of the blur-W distance function (210) corresponds to the absolute extremum (224) of the blur-W distance function (220). The method of claim 2.

5. selecting the predetermined portion of the total integral of the blur distance function to be in the range of 60-80%, or in the range of 60-70%, or 63% or 66%; The method of claim 2.

6. If the blur distance function (220) having the focus position as a variable exhibits absolute extrema (224) and is not symmetric about the corresponding focus position of the absolute extrema (224), an integral of the blur distance function (220) is applied (S31) to determine the boundary of the z-stack (S4); The method of claim 2.

7. the blur-W distance function (210) is selected from the group of gradient-based distance functions or wavelet decomposition-based distance functions; The method of claim 1.

8. the blur distance function (220) is selected from a group of distance functions based on autocorrelation methods; The method of claim 2.

9. The optimum focus for imaging the object is defined as the focus position where the blur distance function (220) has its absolute extreme value (224). The method of claim 2.

10. generating a set (230) of images (231) captured at different focus positions of the object to determine the boundaries (110, 120) of the z-stack; and generating a first set of images (231) using a first focus step size over a first range of focus positions and a second set of images using a second focus step size over a second range of focus positions. The method of claim 1.

11. the second focus step size is smaller than the first focus step size; The method of claim 10.

12. the second range of focus positions is smaller than the first range of focus positions; The method of claim 10.

13. determining a boundary of the z-stack for each set of images of the first set of images and the second set of images; The method of claim 10.

14. After the step (S1) of generating a set (230) of images (231) of the object, first, the blur distance function (220) is applied (S2) to each image (231) of the set (230), and if the blur distance function (220) has an absolute extremum (224) and is symmetric with respect to a corresponding focus position of the absolute extremum (224), the blur-W distance function (210) is applied (S3) to each image (231) of the set (230); If the blur distance function (220) has an absolute extremum (224) but is not symmetric, an integral of the blur distance function (220) is applied (S31) to determine the boundary of the z-stack; The blur-W distance function (210) is applied (S3) and, if the blur-W distance function (210) has two secondary extrema (212, 216) adjacent to a primary extrema (214), the boundaries of the z-stack are determined (S4) according to focus positions assigned to the two secondary extrema (212, 216); If the blur-W distance function (210) lacks two such quadratic extrema, an integral is applied (S31) over the blur-W distance function (220) to determine the boundaries of the z-stack (S4); The method of claim 2.

15. An optical instrument (800) for imaging an object (130), said optical instrument (800) comprising: instrument optics (810, 820) for imaging the object (130); a focus adjustment unit (830) for adjusting a focus position at which the object (130) is imaged; a processor (840) operatively connected to the instrument optics (820) and the focus adjustment unit (830) to generate a set of images captured at different focus positions of the object (130), and configured to perform the method of any one of claims 1 to 14; An optical instrument (800) comprising:

16. the optical instrument (800) is a microscope; The instrument optics (810, 820) include a microscope objective (810) configured to generate a magnified image of the object. The optical instrument (800) of claim 15.

17. A computer program comprising: comprising program code for carrying out the method of any one of claims 1 to 14 when the program code is executed on a processor, Computer program.

Images