MICROSCOPE, METHOD FOR DETERMINING FIELD INHOMOGENY IN A FIELD OF VIEW OF A MICROSCOPE AND MICROSCOPY METHOD

Abstract

The present invention relates to a microscope comprising a light source, an illumination beam path for guiding excitation light to a sample, a detector for detecting emission light emitted by a sample, a detection beam path with a microscope objective for guiding the emission light to the detector, a mechanical drive for setting a relative lateral position between the sample and the microscope objective, and a control unit.The microscope is characterized in that the control unit is configured to perform a setting step, wherein the mechanical drive is sequentially adjusted to at least three different relative lateral positions; a detection step, wherein measurement data from the detector are acquired for at least a subset of points in the sample within a field of view of the detection beam path, wherein for each point of the subset measurement data are acquired for at least two different lateral positions of the mechanical drive; and an evaluation step, wherein field inhomogeneity in the field of view and microscopic sample information are extracted from the measurement data. In further aspects, the invention relates to a method for determining field inhomogeneity in a field of view of a microscope and to a microscopy method.

Description

[0001] In a first aspect, the present invention relates to a microscope according to the preamble of claim 1. In a second aspect, the invention relates to a method for determining a field inhomogeneity in a field of view of a microscope according to the preamble of claim 27. In a third aspect, the invention relates to a microscopy method.

[0002] A generic microscope comprises a light source for supplying excitation light, an illumination beam path for guiding the excitation light to a sample chamber, a detector for detecting emission light emitted by a sample in the sample chamber, a detection beam path with a microscope objective for guiding the emission light to the detector, a mechanical drive for setting a relative lateral position between the sample and the microscope objective with respect to an optical axis of the microscope objective, and a control unit configured to control the mechanical drive and to acquire and evaluate measurement data from the detector.

[0003] A generic method for determining field inhomogeneity in a microscope's field of view comprises the following steps: illuminating a sample with excitation light through an illumination beam path of the microscope, guiding emission light emitted by the sample through a detection beam path, which includes a microscope objective, to a detector, detecting the emission light with the detector, and setting certain lateral positions between the sample and the microscope objective with respect to an optical axis of the microscope objective by means of a mechanical drive.

[0004] Modern microscopes have seen a massive improvement in the available spatial bandwidth product (SBP), aiming to deliver a larger field of view while maintaining high spatial resolution, often requiring throughput in the megapixel to gigapixel range. The need for large SBPs poses challenges to available hardware, such as larger image sensors and the necessity of aberration-corrected objectives across large fields of view. Imaging across large fields of view, in turn, carries the risk of field inhomogeneity. A prime example of inhomogeneity is vignetting, which can have several causes: 1) Light emanating from off-axis points within the field of view tends to be more blocked by obstacles such as apertures and lens holders than light points on-axis.2) The illumination profile itself is already inhomogeneous, for example, because it is generated from a single-mode fiber or the condenser optics exhibit vignetting. Other effects, such as angle-dependent sensitivity of the LED illumination or detector sensitivity, can lead to vignetting. A second example of field inhomogeneity is caused by field obstacles that block a plane conjugate to the sample plane, such as field stops or dust particles on a camera sensor. While vignetting can be described by a function that changes slowly or uniformly across the field of view, dust spots, field stops, and edges are described by abrupt changes and cannot be easily described by elementary functions, making the modeling, detection, and distance of the latter by parametric regression difficult.

[0005] To correct illumination inhomogeneities, a fundamental distinction is made between prospective and retrospective methods [4]. Prospective correction strategies use calibration targets to measure the shading caused by the illumination and the variable detector sensitivity [1]. This step is performed before measuring the sample of interest. In some cases, prospective shading methods fail to correct the shading effects caused by the sample [1], for example, when using immersion objectives or when the sample is embedded in a surrounding aqueous medium, creating a wavefront due to the cohesion of the liquid and the sample container (e.g., a multi-well plate), which deforms the meniscus. Therefore, it is not always possible to acquire the sample of interest data under the same conditions under which the calibration targets were measured.Furthermore, calibration targets can fade and wear down over time during daily laboratory use (accumulation of scratches and dust). Finally, calibration targets incur additional costs that must be passed on to the customer.

[0006] Several retrospective methods involving signal post-processing strategies have been proposed to correct inhomogeneities. For example, Preibisch et al. [2] published an image stitching module as part of the open-source software package Fiji. The authors mitigate shading between adjacent tiles by nonlinearly weighting pixels within the overlap region. While this technique produces visually appealing images and, in particular, softens sharp edges between adjacent tiles, the nonlinear weights used for edge blending destroy the linearity between multiple fluorescence channels. This is undesirable for further post-processing, such as fluorescence unmixing or color deconvolution, which rely on the linearity of the results. The same shortcoming applies to the method proposed in [3], which also uses nonlinear blending.Furthermore, the method described in [3] is a poorly posed method, meaning that less data is available than the sum of the unknown pixels of the illumination and shading profile. Therefore, the method described in [3] can only work if certain a priori assumptions about the shape and smoothness of the shading profile are satisfied.

[0007] Another method for estimating field inhomogeneity is based on averaging many tiles. The underlying assumption of this retrospective statistical approach is that the occurrence of pattern features is uniformly distributed across the tiles. Under this assumption, averaging many tiles should compensate for the sample contribution (which is tile-dependent) and converge to a useful estimate of field inhomogeneity (which is tile-independent) at each sensor pixel location. However, a statistical argument shows that this averaging strategy is inefficient: it can be shown that several hundred samples (corresponding to the number of tiles in this case) are required for a uniformly distributed random variable to converge to its mean with an accuracy better than 1%. Refined approaches to this averaging strategy have been reported in the literature. Smith et al.[4] described a statistical approach for a retrospective method called CIDRE (Corrected Intensity Distributions using Regularized Energy-Minimization), the aim of which is to extract robust mean estimates from an image sequence. In contrast to the naive averaging approach described above, the approach of Smith et al. [4] removes outliers by using robust statistics.

[0008] Another recently published method is BaSiC [5], an algorithm for background and shading correction. The algorithm assumes that the set of all tiles in the sample 1) has a low-rank structure and 2) that the shading function is smooth across the viewing field.

[0009] The object of the present invention can be considered to be to provide a microscope, a method for determining a field inhomogeneity in a field of view of a microscope and a microscopy method in which improvements can be achieved at least with regard to the precision of a certain field inhomogeneity.

[0010] This problem is solved according to the invention by the microscope with the features of claim 1, by the method with the features of claim 27 and by the method with the features of claim 28.

[0011] The generic microscope described above is further developed according to the invention in that the control unit is configured to perform the following steps: a setting step, wherein the mechanical drive is sequentially set to at least three different relative lateral positions; a acquisition step, wherein in each of the different relative lateral positions measurement data from the detector are acquired for at least a subset of points in the sample in a field of view of the detection beam path, wherein for each of the points of the subset measurement data are acquired for at least two different lateral positions of the mechanical drive; and an evaluation step, wherein, based on the measurement data acquired in the acquisition step, the following steps are performed: extracting a field inhomogeneity in the field of view from the measurement data, assuming thatthat the field inhomogeneity is independent of the respective set relative lateral position, and extracting microscopic sample information from the measurement data, which represents a sample-induced portion of the measurement data.

[0012] According to the invention, the generic method described above is further characterized by a setting step, wherein the mechanical drive is sequentially set to at least three different relative lateral positions; a acquisition step, wherein in each of the different relative lateral positions measurement data are acquired by the detector for at least a subset of points in the sample in a field of view of the detection beam path, wherein for each of the points in the subset measurement data are acquired for at least two different lateral positions of the mechanical drive; and an evaluation step, wherein, based on the measurement data acquired in the acquisition step, the following steps are performed: extracting a field inhomogeneity in the field of view from the measurement data, assuming that the field inhomogeneity is not dependent on the respective set relative lateral position.and extracting microscopic sample information from the measurement data, representing a sample-induced portion of the measurement data.

[0013] The present invention further claims a microscopy method comprising the following steps: a determination step, wherein a field inhomogeneity in a field of view of a microscope is determined using the method according to the invention for determining a field inhomogeneity in a field of view of a microscope using a sample; a microscopy step, wherein the steps of the method according to the invention for determining a field inhomogeneity in a field of view of a microscope are carried out for a sample that differs from the sample used in the determination step; and wherein the field inhomogeneity determined in the determination step is used as an initial estimate for the field inhomogeneity.

[0014] The microscope can be any type of light microscope. For example, the microscope can be at least one of the following: a wide-field microscope, a scanning microscope, a confocal microscope, a light-field microscope, a light-sheet microscope, a TIRF microscope, a SIM microscope.

[0015] The light source can be any light source capable of providing excitation light with a desired wavelength(s) and a suitable intensity. For example, the light source could be a laser or an LED module. The excitation light can be coherent or at least partially coherent.

[0016] The excitation light is electromagnetic radiation, particularly in the visible spectral range and adjacent regions. This excitation light can also be called illumination light, and in this description, these two terms are largely used synonymously. Emission light is the light emitted by the sample under investigation as a result of irradiation with illumination or excitation light, which travels through the detection beam path to the detector, e.g., a camera. For light to be considered emission light, it is only necessary that the light originates from the illuminated sample. Typically, the emission light is fluorescent light, emitted by the sample, particularly by dye molecules present within it, or emitted as a result of irradiation with the excitation light. The emission light can be reflected, transmitted, or scattered illumination light.The only requirement of the present invention for the contrasting principle is that the sample emits emission light as a result of irradiation with the excitation light. Typically, the emission light is fluorescence light, which the sample, in particular any dye molecules present therein, emits as a result of irradiation with the excitation light. The sample can also be referred to as the sample.

[0017] The term illumination beam path encompasses all optical beam-guiding and beam-modifying components, such as lenses, mirrors, prisms, gratings, filters, apertures, and beam splitters, through which the excitation light from a light source is directed to the sample under investigation. The illumination beam path may include an illumination objective. Both the illumination objective and the microscope objective can be microscope objectives of a known type. In principle, the illumination objective and the microscope objective can be separate objectives. However, in preferred embodiments, the illumination objective and the microscope objective are the same objective.

[0018] The term sample space refers to the spatial area in which a sample to be examined can be arranged. In typical designs, a sample can be placed or mounted on a cross stage, for example, using a sample holder or sample frame. This stage can be manipulated laterally with respect to an optical axis. A Z-axis drive may be provided to vary the distance between the sample stage and the illumination objective or between the sample and the microscope objective.

[0019] The sample can be any type of sample. The microscope and the methods of the invention are particularly suitable for examining biological samples.

[0020] Emission light is the light emitted by the sample under investigation as a result of irradiation with the excitation light, which reaches the detector, e.g. a camera, via the detection beam path.

[0021] The term detection beam path encompasses all beam-guiding and beam-modifying optical components, such as lenses, mirrors, prisms, gratings, filters, apertures, and beam splitters, through which the emitted light is directed from the sample under investigation to the camera. Advantageously, a sensor plane of the detector can be arranged in a plane that is optically conjugate to a focal plane of the microscope objective.

[0022] The field of view of the detection beam path refers to the lateral spatial area in a sample plane from which emission light can be captured and propagated to the detector. The size of a field of view is usually determined by the optical parameters, e.g., numerical aperture and magnification, of the microscope objective and other components in the detection beam path, e.g., a tube lens.

[0023] The type of detector used to detect the emitted light generally depends on the type of microscope. In embodiments of the invention, the detector can comprise a two-dimensional spatially resolved photodetector, e.g., a camera; a one-dimensional spatially resolved detector, e.g., a linear detector array; or a single photodetector, e.g., a point-like photodetector. More specifically, the detector can comprise at least one of: a CCD element, a CMOS element, a SPAD element, or a PMT.

[0024] The mechanical drive for adjusting the relative lateral position between the sample and the microscope objective can be configured to move the sample relative to an optical axis of the microscope objective and / or to move the microscope objective, and thus its optical axis, relative to the sample. The mechanical drive can be mounted on a microscope stand. The microscope stand can be either upright or inverted.

[0025] The term control unit encompasses all hardware and software components that interact with the components of the microscope according to the invention to ensure its intended functionality. The control unit may, in particular, include a computing device, for example a PC, and a camera controller capable of reading measurement signals. Detector measurement data refers to the measurement data generated by the detector when irradiated with emission light.

[0026] The subset of points in the sample for which measurement data is obtained can be defined by a subset of the pixels of a camera when a camera is used as a detector. In cases where the microscope is a scanning microscope, the subset of points can be defined by a subset of the scan positions of a scan unit and additionally by a subset of the pixels of a pixelated detector.

[0027] The term field inhomogeneity is generally to be understood as explained above, i.e., it encompasses inhomogeneities caused by vignetting and inhomogeneities caused by field obstacles, particularly those obstructing a plane conjugate to the sample plane. Field inhomogeneity also includes inhomogeneities in a detection intensity distribution, i.e., a detection profile. The latter would be relevant, for example, in a camera where the pixels have slightly different detection properties.

[0028] Vignetting can be caused by light emanating from off-axis points within the field of view being blocked more readily by obstructions such as apertures and lens holders than by points on the axis. Vignetting can also result from intrinsic inhomogeneities in the illumination profile, such as those caused by single-mode fiber optics and / or the vignetting inherent in condenser optics. Finally, vignetting can be caused by other effects, such as angle-dependent sensitivity of the LED illumination or detector sensitivity. Field obstructions that occlude any plane conjugate to the sample plane could include, for example, field stops or dust particles on a camera sensor.

[0029] The assumption that the field inhomogeneity is independent of the respective set relative lateral position is physically meaningful, since none of the described constraints causing the inhomogeneities depend on a relative position of the sample with respect to the optical axis of the microscope objective.

[0030] The term "tile" refers to a single image acquired for a specific relative lateral position of the sample with respect to the microscope objective. A stitched image is the combination of multiple tiles, each acquired as the sample is moved laterally and / or axially. Stitching the tiles significantly increases the field of view. In this context, stitching means that the measurement data of each tile are recorded based on their relative positions. Adjacent tiles are typically scanned with overlap to allow for the recording and alignment of common features.

[0031] A fluorescence channel is defined as a microscope configuration that includes a specific wavelength of excitation light entering the sample chamber and a specific emission spectral filter characteristic of the detection beam path. As is generally known in the art, these parameters can be adapted, for example, to the specific fluorophores used and the samples under investigation.

[0032] The method described here can be considered diametrically opposed to statistical methods based on averaging many tiles and differs fundamentally, in particular, from the approach reported by Smith et al. [4]. Instead of assuming statistical independence between the tiles, the microscope and the method according to the invention require a high degree of overlap and thus a certain level of correlation between the individual tiles. Instead of hoping that naive averaging or robust statistics will converge to the true mean of the inhomogeneity profile, the method according to the invention explicitly models a precise translation of the sample and uses the assumption that the fluorophore density should remain constant during the translation of the sample.If the signal changes and bleaching is ignored, a change in the measured intensity during sample translation provides information about the inhomogeneity profile.

[0033] None of the widely used shading correction methods [1] - [5] employ explicit modeling of the signal change during translation of the sample from which the field inhomogeneity profile is extracted. The extraction of shading information by explicitly modeling intensity changes within highly overlapping tiles is unique to the microscope and method according to the invention. In particular, the microscope and method according to the invention can operate with as few as three tiles at a minimum overlap of 50%.

[0034] The microscope and, in particular, the control unit can be configured to perform at least one of the method according to the invention for determining a field inhomogeneity in a field of view of a microscope and the microscopy method according to the invention.

[0035] In preferred embodiments, the methods according to the invention further include the use of the microscope according to the invention.

[0036] A key advantage of the invention is that not only the components of the illumination inhomogeneity caused by vignetting, but also those caused by dust spots, field stops and edges can be recorded with a high degree of accuracy.

[0037] Another important advantage of the invention is that any sample can be used to determine the illumination inhomogeneity. Unlike other methods, no specific test sample is therefore necessary.

[0038] In contrast to the method described, for example, in [3], the method described in the present application does not require any a priori assumptions about the shape and smoothness of the shading profile.

[0039] Unlike the background and shading correction algorithm described in [5], which requires a priori assumptions about smoothness, the microscope and the method according to the invention use an explicit physical model of the change in signal intensity during lateral translation of the sample against an inhomogeneous illumination profile.

[0040] A significant advantage of the microscopy method of the invention is that the determination of illumination inhomogeneity can be carried out with a high degree of accuracy on a light-insensitive sample, either before or after an examination of specific samples, for example, light-sensitive biological samples. This is generally possible because any fluctuations in a light source are usually negligible.

[0041] In general, the invention requires that at least three relative lateral positions be set in the setting step and that measurement data for at least two different relative lateral positions be recorded for each of the points in the subset of points. In a preferred embodiment of the invention, the set relative lateral positions are located on a two-dimensional regular grid. The grid can be, for example, rectangular or triangular.

[0042] The requirement that measurement data for at least two different relative lateral positions be recorded for each point in the subset of points can preferably be achieved in one embodiment of the invention by further configuring the control unit such that, in the two-dimensional grid of the set relative lateral positions during the setup step, the overlap between adjacent tiles in a first coordinate direction is at least 50%. In a further preferred embodiment of the invention, the control unit is further configured to ensure that, in the two-dimensional grid of the set relative lateral positions during the setup step, the overlap between adjacent tiles in a second coordinate direction, which may be perpendicular to the first coordinate direction, is at least 5% and preferably at least 10%.

[0043] In another preferred embodiment, the points in the subset of points are evenly distributed across the field of view. For example, the points in the subset of points can lie on a regular grid. The grid of the subset of points can be rectangular or triangular.

[0044] In a situation where field inhomogeneity needs to be determined with a high degree of accuracy, the acquisition step preferably includes acquiring measurement data for each of the points in the field of view.

[0045] In a situation where field inhomogeneity needs to be determined quickly, the evaluation step can include binning measurement data from a large number of points, especially a large number of pixels.

[0046] In a situation where field inhomogeneity needs to be determined quickly and / or with only a moderate degree of accuracy, it would suffice to evaluate only the measurement data of a subset of points. However, if field inhomogeneity is to be determined with a high degree of accuracy, the evaluation step preferably includes evaluating the measurement data of each individual point in the field of view.

[0047] In another preferred embodiment, the control unit is configured to perform the evaluation step as an iterative solution of a double-blind estimation problem based on an initial estimate of the field inhomogeneity and an initial estimate of the microscopic sample information.

[0048] More specifically, the control unit can be configured to perform at least some of the following steps during the evaluation step: • Calculating an estimate of the microscopic sample information based on the measurement data and using an initial estimate of the field inhomogeneity and • Iterative calculation of updated field inhomogeneity estimates based on measurement data and using the most recent estimate of microscopic sample information, as well as updated estimates of microscopic sample information based on measurement data and using the most recent estimate of field inhomogeneity, • Evaluating a degree of accuracy to which the most recent estimate of field inhomogeneity and the most recent estimate of microscopic sample information reproduce the measurement data, • Repeat the step of iteratively calculating updated estimates of field inhomogeneity and updated estimates of microscopic sample information until the measurement data are replaced by the most recent estimate of field inhomogeneity and the most recent estimate of microscopic sample information.

[0049] Sample information can be reproduced to a specific degree of accuracy.

[0050] Additionally or alternatively, the control unit can also be configured to perform at least some of the following steps during the evaluation step: • Calculating an initial estimate of the microscopic sample information based on the measurement data and using an initial estimate of the field inhomogeneity; • Calculating an initial updated estimate of field inhomogeneity based on the measurement data and using the initial estimate of the microscopic sample information; • Calculating an initial updated estimate of the microscopic sample information based on the measurement data and using the initial updated estimate of the field inhomogeneity; • Calculating an (n+1)th updated estimate of field inhomogeneity based on the measurement data and using the nth updated estimate of the microscopic sample information; and • Calculate an (n+1)th updated estimate of the microscopic sample information based on the measurement data and using the (n+1)th updated estimate of the field inhomogeneity.

[0051] The initial estimate of field inhomogeneity may, for example, be a flat profile.

[0052] In a further embodiment, the control unit can additionally or alternatively be configured to perform the evaluation step as minimizing a mathematical distance between the measurement data and a combination of the field inhomogeneity and the microscopic sample information, wherein the mathematical distance is based on any mathematical norm.

[0053] In a further embodiment, the control unit can additionally or alternatively be configured to perform the following steps in the evaluation step: accepting a new estimate of the field inhomogeneity as the new updated estimate of the field inhomogeneity and a new estimate of the microscopic sample information as the new updated estimate of the microscopic sample information.if the value of a first scalar cost function for the new estimate of field inhomogeneity and the new estimate of microscopic sample information is smaller than the value of the first scalar cost function for the most recent updated estimate of field inhomogeneity and the most recent updated estimate of microscopic sample information, and if the value of a second scalar cost function for the new estimate of field inhomogeneity and the new estimate of microscopic sample information is smaller than the value of the second scalar cost function for the most recent updated estimate of field inhomogeneity and the most recent updated estimate of microscopic sample information, where the first scalar cost function and the second scalar cost function each contain a mathematical distance between the measurement data and a combination of field inhomogeneity with microscopic sample information,where the first scalar cost function contains the mathematical norm of the field inhomogeneity and the second scalar cost function L2 contains the mathematical norm of the microscopic sample information.

[0054] The combination of field inhomogeneity with microscopic sample information can, for example, be a product of field inhomogeneity with microscopic sample information.

[0055] The mathematical norm can be one of the following: Lp norm, L2 norm (Euclidean norm), Manhattan norm (L1 norm), infinity norm (L∞ norm), Shannon entropy norm, Jensen-Shannon divergence norm, Renyi entropy norm, Tsallis entropy norm.

[0056] In a particularly preferred embodiment, the first cost function L1 and the second cost function L2 are each given by L1=12∑x∑m[I(x)S(x−m)−C(x|m)]p+μ12∑xI(x)p L2=12∑x∑m[I(x+m)S(x)−C(x+m|m)]p+μ22∑xS(x)p where C(x|m) the measurement data are I(x) the field inhomogeneity is S(x) the microscopic sample information are µ1 and µ2 are non-zero scalars. m is a two-dimensional vector n of the x1, x2 plane. x is a two-dimensional vector n of the x1, x2 plane. p is a positive integer.

[0057] In a preferred embodiment, p = 2, i.e., the norm used is the Euclidean norm. This minimization algorithm thus minimizes the squared distances between the combination of the field inhomogeneity with the microscopic sample information, e.g., the product of the field inhomogeneity with the microscopic sample information, and the measurement data.

[0058] This variant of the method according to the invention for determining a field inhomogeneity in a field of view of a microscope or the underlying algorithm can be called the Alternating Least Squares Shading Correction algorithm or ALS-SC algorithm.

[0059] For this embodiment, the (n+1)th updated estimate of the field inhomogeneity I can be obtained. n+1 (x) calculate as follows: In+1(x)=∑mSn(x−m)C(x|m)∑mSn(x−m)2+μ1 and the (n+1)th updated estimate of the microscopic sample information S can be obtained n+1 (x) calculate as follows: Sn+1(x)=∑mIn(x+m)C(x+m|m)∑mIn(x+m)2+μ2 where C(x|m) the measurement information is I n+1 (x) (n+1)-th updated estimate of field inhomogeneity I(x) is I n (x) the nth updated estimate of the field inhomogeneity I(x) is S n(x) the nth updated estimate of the microscopic sample information S(x) is S n+1 (x)) the (n+1)th updated estimate of the microscopic sample information S(x) is µ1 and µ2 are non-zero scalars. m is a two-dimensional vector in the x1, x2 plane. x is a two-dimensional vector in the x1, x2 plane p is a positive integer.

[0060] Further features and advantages of the invention are described below with reference to the accompanying drawings. In the drawings, the following applies: Fig. Figure 1 shows a schematic representation of a microscope according to the invention; Fig. Figure 2 shows a schematic representation of a field inhomogeneity of a detection beam path of a microscope; Fig. Figure 3 shows a schematic representation of a composite image of measurement data, consisting of a large number of image tiles, each showing the field inhomogeneity and, in some cases, also the proportions caused by a sample; Fig. Figure 4 shows a schematic representation of six overlapping image tiles; Fig. Figure 5 shows a schematic representation of the coordinates used; Fig. Figure 6 shows a schematic representation of two overlapping image tiles; Fig. Figure 7 shows a schematic representation of three overlapping image tiles; and Fig. Figure 8 shows another schematic representation of three overlapping image tiles.

[0061] An embodiment of a microscope 100 according to the invention is described with reference to Fig. Sections 1 to 7 are described. Identical and equivalent components are generally designated with the same reference numerals.

[0062] The microscope 100 initially comprises a light source 10, e.g., a laser, for providing excitation light 12, and an illumination beam path for guiding the excitation light 12 to a sample chamber 1. In the example shown, the illumination beam path includes a tube lens 20, a main beam splitter 23, and a microscope objective 40. The tube lens 20 generates an intermediate image plane 18, i.e., a plane that is optically conjugate to a plane 11 in a sample 2 in the sample chamber 1. In the illumination beam path, the excitation light 12 passes through the intermediate image plane 18 and via the tube lens 20 to the main beam splitter 23, where it is reflected towards the microscope objective 40. The excitation light 12 then passes through a rear focal plane 42 of the microscope objective 40 and is subsequently directed to the sample chamber 1. Sample 2 can be a biological sample and be prepared with fluorophores that can be excited by the excitation light 12.The wavelength and intensity of the excitation light 12 can be suitably selected with regard to the sample 2 and the fluorophores used. The light source can comprise a variety of different lasers. The wavelength and / or intensity can be adjustable.

[0063] Furthermore, the microscope 100 comprises a detector 50 for detecting emission light 16 emitted by a sample 2 in the sample chamber 1 and a detection beam path with a microscope objective 40 for guiding the emission light 16 to the detector 50. In the example shown, the microscope 100 is a wide-field microscope and the detector 50 is a camera, i.e., a field of view 30 (see Fig. 2) The detection beam path is imaged onto a sensor plane 51 of the camera 50. The sensor plane 51 is optically conjugate to plane 11 in sample chamber 1.

[0064] The emission light 16 emitted by sample 2 is typically redshifted fluorescence light emitted by the fluorophores in sample 2. The main beam splitter 23 is configured to transmit the redshifted emission light 16 and to reflect the emission light 12, thus preventing large portions of the emission light 12, which is backscattered from sample chamber 1, from reaching the camera 50.

[0065] In the detection beam path, the emission light 16 emitted by the sample 2 is captured by the microscope objective 40, passes through the main beam splitter 23 and is then imaged by a tube lens 22 into the sensor plane 51 of the camera 50.

[0066] As is generally known in engineering, an excitation filter and / or an emission filter can be present in the excitation beam path, e.g., between the tube lens 20 and the main beam splitter 23, and / or in the detection beam path, e.g., between the tube lens 22 and the main beam splitter 23. There can also be a multitude of different main beam splitters 23, each tuned to individual fluorophores, which can be inserted into the beam path.

[0067] Furthermore, the microscope 100 comprises a mechanical drive 44 for setting a relative lateral position x1, x2 between the sample 2 and the microscope objective 40 with respect to an optical axis 41 of the microscope objective 40, and a control unit 90, e.g., a PC, configured to control the mechanical drive 44 and to acquire and evaluate measurement data from the detector 50. In the example shown, the optical axis 41 of the microscope objective 40 runs in the direction of the x3 axis. The mechanical drive 44 can, for example, be part of a motorized sample slide and, in the example shown, serves to set a specific position of the sample 2, i.e., specific x1, x2 coordinates of the sample with respect to the optical axis 41. A right-handed and orthogonal coordinate system x1, x2, x3 is shown below the mechanical drive 44.In the example shown, the microscope 100 further includes an axial drive 46, which serves to set a specific axial distance, i.e. a distance in the x3 direction between the sample and the microscope objective 40.

[0068] According to the invention, the control unit 90 is configured to perform an adjustment step, wherein the mechanical drive is sequentially adjusted to at least three different relative lateral positions m, where m is a two-dimensional vector (m1, m2) in the x1,x2 plane. Each specific relative lateral position m corresponds to a different section or tile of the sample 2. The measurement data acquired by the camera 50 are designated C(x|m), where x is a two-dimensional vector that designates the detector coordinates in a suitable reference frame. For example, the origin is located at the center of a camera chip. Fig. Figure 5 shows an exemplary vector (m1, m2) in the x1, x2 plane for a tile 67 corresponding to the field of view 30 and located in a specific relative lateral position within an image 60 as defined by (m1, m2), as well as the corresponding coordinates x1, x2 within the tile 67.

[0069] According to the invention, the control unit 90 is further configured to perform a detection step, wherein, in each of the different relative lateral positions m, measurement data C(x|m) = C(x1,x2|m) are acquired by the detector 50 for at least a subset of points in the sample 2 within the field of view 30 of the detection beam path. For each point of the subset, measurement data are acquired for at least two different lateral positions m of the mechanical drive 44. The latter can be implemented, for example, in an embodiment in which the set relative lateral positions m are located on a two-dimensional regular grid configured such that the overlap between adjacent tiles in a first coordinate direction x1 is at least 50%.

[0070] This is in Fig. 6 and Fig. 7 shown schematically. Fig. Figure 6 shows a first tile 71 and a second tile 72 of the same size, which overlap with the first tile 71 in the first coordinate direction x1. The overlap a is more than 50% of the width of tiles 71 and 72 in the first coordinate direction x1. Fig. Figure 7 shows tiles 73, 74, and 75, each of the same size. Exactly half of tile 73 overlaps tile 74, and the other half of tile 74 overlaps tile 75; that is, the overlap a in the first coordinate direction x1 is exactly 50% in each case. The two-dimensional grid of the set relative lateral positions m can be configured such that an overlap b between adjacent tiles in a second coordinate direction x2, which is particularly perpendicular to the first coordinate direction x1, is at least 5% and preferably at least 10%. This is shown in Fig. 4 shown schematically. Fig. Figure 4 shows six identical tiles 61, ... 66, each overlapping by 50% (arrow a) with its respective neighbors in the first coordinate direction x1 and by 10% (arrows b) with its respective neighbors in the second coordinate direction x2. The points in the subset of points can be evenly distributed across the field of view, e.g., on a rectangular grid.

[0071] Fig. Figure 2 schematically shows the field of view 30 as seen by camera 50, without any contribution from a sample. This can be considered the image of a completely homogeneous sample. The image shows a field inhomogeneity I(x) = I(x1, x2), which in the example shown essentially consists of vignetting 19, i.e., reduced intensity or shading, in the corner regions. A dust particle 17 is also shown, which may be located, for example, in the plane 51 of camera 50. While the vignetting has a smooth gradient, the dust particle 17 has a comparatively sharp contour. By definition, the field inhomogeneity I(x) encompasses the profile of the detected radiation without any contribution from sample 2. The vignetting 19 in the corners is caused by the optical properties of the light source 10, the illumination beam path, the detection beam path, and the camera. In the example shown, the dust particle 17 is located in camera 50.Therefore, the structure of the field inhomogeneity I(x) does not depend on the specifically set relative lateral position m and is displayed as such in each of the recorded tiles.

[0072] This will be demonstrated using Fig. 3 explained in more detail. Fig. Figure 3 schematically shows a composite image 80, which consists of 80 individual tiles, each representing the measured data C(x|m) = C(x1,x2|m) acquired from detector 50. The set relative lateral positions m are located on a two-dimensional regular grid, namely a rectangular grid. As can be seen, each individual tile has the same field inhomogeneity I(x). In addition, at least some of the tiles show contributions S(x) = S(x1, x2) of a sample. It should be noted that Fig. 3 serves only to illustrate the fact that the field inhomogeneity I(x) is present in each of the tiles. Fig. Image 3 shows no overlapping tiles. Image 80 can be visualized as showing, for example, only half of the total measured tiles.

[0073] In reality, an image can consist of several hundred image tiles fused together to form a whole-slide image of a sample. Remarkably, the image exhibits in Fig. 3. Shading artifacts appear across each tile, resulting in a checkerboard-like artifact in the fused whole-slide image. The present invention describes a novel, non-parametric technique for removing field inhomogeneities, which allows for both an estimation of the in Fig. 2 shading profile shown as well as a corrected microphotograph of the sample, i.e. microscopic sample information, is generated.

[0074] In this context, the control unit 90 according to the invention is further configured to perform an evaluation step in which the following steps are carried out based on the measurement data C(x|m) in the acquisition step: • Extract, from the measurement data C(x|m), the field inhomogeneity I(x) in the field of view under the assumption that the field inhomogeneity I(x) is independent of the respective set relative lateral position m, and • Extract, from the measurement data C(x|m), the microscopic sample information S(x) that represents a part of the measurement data C(x|m) caused by sample 2.

[0075] The control unit 90 of the microscope 100 can be configured, in particular, to carry out the method according to the invention. The methods according to the invention can include the use of the microscope 100.

[0076] Removing field inhomogeneity artifacts from microscopy data can be viewed as an inverse problem. If multiple scattering of light in thick samples is neglected, it is possible to model the measurement data, e.g., a camera signal C(x|m), as the product of an illumination profile, i.e., the field inhomogeneity I(x), and a sample profile, i.e., the microscopic sample information S(x), as follows: C(x|m)=I(x)S(x−m)

[0077] Here, x is again a two-dimensional vector that denotes the detector coordinates in a suitable reference frame. Typically, the origin is located at the center of a camera chip, and m is the two-dimensional translation vector that describes the lateral translation experienced by sample 2 due to translation by the mechanical drive 44. The aim is to estimate the field inhomogeneity I(x) and the microscopic sample information S(x) from a sequence of measurement data, e.g., a sequence of camera images for a set of variable translation vectors m.

[0078] The control unit 90 can be configured to perform the evaluation step as an iterative solution to a double-blind estimation problem based on an initial estimate of the field inhomogeneity I(x) and a first estimate of the microscopic sample information S(x). More precisely, the control unit 90 can further be configured to perform the evaluation step as minimizing a mathematical distance between the measurement data C(x|m) and a combination of the field inhomogeneity I(x) and the microscopic sample information S(x), where the mathematical distance is based on an arbitrary mathematical norm.

[0079] More precisely, the control unit 90 can be configured to perform at least some of the following steps during the evaluation step: • Calculating an estimate of the microscopic sample information S(x) based on the measurement data C(x|m) and using an initial estimate of the field inhomogeneity I0(x) and • iterative calculation ▪ updated estimates of field inhomogeneity I n+1 (x) based on the measurement data C(x|m) and using the most recent estimate of the microscopic sample information S n (x) and ▪ updated estimates of microscopic sample information S n+1 (x) based on the measurement data C(x|m) and using the most recent estimate of field inhomogeneity I n+1 (x), • Evaluating a degree of accuracy to which the most recent estimate of field inhomogeneity I relates n+1 (x) and the most recent estimate of the microscopic sample information S n+1 (x) reproduce the measurement data C(x|m), and • Repeat the step of iteratively calculating updated estimates of field inhomogeneity I n+1 (x) and updated estimates of microscopic sample information S n+1 (x) until the measurement data C(x|m) is replaced by the most recent estimate of the field inhomogeneity I n+1 (x) and the most recent estimate of the microscopic sample information S n+1 (x) can be reproduced to a specific degree of accuracy.

[0080] More precisely, the control unit 90 can also be configured to perform some or all of the following steps during the evaluation step: • Calculating an initial estimate of the microscopic sample information S0(x) based on the measurement data C(x|m) and using an initial estimate of the field inhomogeneity I0(x); • Calculating a first updated estimate of the field inhomogeneity I1(x) based on the measurement data C(x|m) and using the initial estimate of the microscopic sample information S0(x); • Calculating a first updated estimate of the microscopic sample information S1(x) based on the measurement data C(x|m) and using the first updated estimate of the field inhomogeneity I1(x); • Calculating an (n+1)th updated estimate of field inhomogeneity I n+1 (x) based on the measurement data C(x|m) and using the nth updated estimate of the microscopic sample information S n (x); and • Calculating an (n+1)th updated estimate of the microscopic sample information S n+1 (x) based on the measurement data C(x|m) and using the (n+1)th updated estimate of the field inhomogeneity I n+1 (x).

[0081] The initial estimate of the field inhomogeneity I0(x) can be, for example, a flat profile.

[0082] More precisely, the control unit 90 can further be configured to determine the estimates of the field inhomogeneity I(x) and the microscopic sample information S(x) based on the minimization of scalar cost functions L1, L2. In this respect, the control unit 90 can also be configured to perform the following steps in the evaluation step: Taking a new estimate of the field inhomogeneity as the new updated estimate of the field inhomogeneity I. n+1 (x) and a new estimate of the microscopic sample information as a new updated estimate of the microscopic sample information S n+1(x) if the value of a first scalar cost function L1 for re-estimating the field inhomogeneity I(x) and the re-estimation of the microscopic sample information S(x) is less than the value of the first scalar cost function L1 for the most recent updated estimate of the field inhomogeneity I n (x) and the latest updated estimate of microscopic sample information S n (x) is and if the value of a second scalar cost function L2 for the new estimate of field inhomogeneity and the new estimate of microscopic sample information is less than the value of the second scalar cost function L2 for the most recent updated estimate of field inhomogeneity I n (x) and the latest updated estimate of microscopic sample information S n (x) is

[0083] The first scalar cost function L1 and the second scalar cost function L2 each contain a mathematical distance between the measurement data C(x|m) and a combination, in particular a product, of the field inhomogeneity I(x) with the microscopic sample information S(x). The first scalar cost function L1 contains the mathematical norm of the field inhomogeneity I(x), and the second scalar cost function L2 contains the mathematical norm of the microscopic sample information S(x).

[0084] The mathematical measure can, in principle, be based on any mathematical norm. Preferably, an Lp norm is used. In this case, the following first cost function L1 and second cost function L2 can be used: L1=12∑x∑m[I(x)S(x−m)−C(x|m)]p+μ12∑xI(x)p L2=12∑x∑m[I(x+m)S(x)−C(x+m|m)]p+μ22∑xS(x)p where the summations over x are taken over all points in the respective tile, the summations over m are taken over specific relative lateral positions m, and where p is a positive integer that defines the norm and µ1 and µ2 are non-zero scalars.

[0085] Preferably, p = 2, i.e., the norm used is the Euclidean norm. Then, an alternating least squares problem minimizing the cost functions L1 and L2 with respect to I(x) and S(x) must be solved. The (n+1)th updated estimate of the field inhomogeneity can then be computed as follows: In+1(x)=∑mSn(x−m)C(x|m)∑mSn(x−m)2+μ1 and the (n+1)th updated estimate of the microscopic sample information can be calculated as follows: Sn+1(x)=∑mIn(x+m)C(x+m|m)∑mIn(x+m)2+μ2 where µ1 and µ2 are non-zero scalars to prevent division by zero.

[0086] The following sections explain the minimum number of tiles required, overlap considerations, unmeasured points, and further algorithmic extensions.

[0087] An overlap parameter δ can be defined as δ = 1 - s / t, where s is the size of a scan step and t is the lateral size of a tile.

[0088] With reference to Fig. Section 6 considers a 2 × 1 scan with an overlap parameter δ in the interval 0 to 1. In this configuration, the number of known data points is the number of scan positions multiplied by the number of measured pixels. A single tile consists of a total of N pixels. Then there are K = 2N known data points in a 2 × 1 scan. However, the number of unknowns U is always greater than the number of known data points for a 2 × 1 scan, since the shading (N unknowns) and the sample (N pixels for the left tile plus (1-δ)N pixels for the right edge, where δ is the overlap parameter) must be estimated. Therefore, U > K holds only in the case of a 2 × 1 scan.

[0089] Following the same logic as above, an n × 1 scan consists of K = n × N known data, in contrast to U=2N+(n−1)(1−δ)N

[0090] The requirement that the number of known data points is at least as large as the number of unknowns, namely K ≥ U, yields the general condition (n−2) / (n−1)≥1−δ(*) where δ denotes the overlap and n is the number of scan positions. The inequality (*) above must be satisfied so that in the present inverse problem there are more data than unknowns.

[0091] Consider the specific case n = 3, for which the above condition is satisfied with equality at an overlap of 50%. This situation is in Fig. 7 shown.

[0092] In summary, it can be stated that 1) a scan with n = 2 scan points and no further a priori knowledge is theoretically impossible; 2) the condition (*) must be met for a general scan with n tiles; 3) the smallest number of scan points n = 3; 4) the minimum required overlap is 50%, i.e. δ = 0.5.

[0093] The argument presented above can easily be extended to two dimensions. Constraint counting again shows that the number of scan points n ≥ 3 must be present. In the special case of n = 3, an example would be an equilateral triangular scan grid.

[0094] This is schematically shown in Fig. Figure 8 shows three identical and partially overlapping tiles 76, 77, 78. Tiles 76, 77, 78 are shifted relative to each other by the displacement vectors c and form an equilateral triangle. For the situation in Fig. 8. It can be shown that the ratio between scan step c and tile size t must satisfy the following. c / t≤1(1+√3 / 2)

[0095] In practice, the observed data are contaminated by noise. It is possible to improve the signal-to-noise ratio by recording more scan positions (n ​​> 3) with higher overlap (δ > 0.5). Furthermore, the sample itself may not be densely populated with fluorophores. This introduces the possibility of unmeasured points in the illumination profile. In other words, the density of structures on the sample can be considered the sampling density of the shading, and vice versa. Without structure in the illumination, every sample measurement would look the same, except for a shift (the brightness of the sample would not change with lateral translation). Conversely, without structure in the sample, nothing can be learned about the illumination or the lateral translation. A perfectly fluorescent, homogeneous film would look the same with lateral translation.Similarly, a perfectly flat piece of glass would remain transparent during lateral translation in a brightfield transmission system. Although not everything is lost in the previous example, and a blank and perfectly flat piece of glass (e.g., a blank coverslip without dust or scratches) would allow a so-called flat-field measurement (as used in [0] for shading correction), this technique cannot be applied to fluorescence microscopes, which, by design, block the light incident on the sample after reflection into the detection path. Another challenging example for fluorescence microscopes would be the case where a single fluorescent bead would not be sufficient to scan a large-field illumination profile if only a few images are acquired. In this case, there are two measures to counteract unmeasured spots: 1) Increase the number of scan points and the overlap so that the illumination profile is adequately scanned from the sample, or 2) Regression or interpolation of unmeasured points from measured points.

[0096] The method for determining field inhomogeneity in the field of view of a microscope is fundamentally also a method for microscopy, since a sample is required and, moreover, microscopic sample information is determined for this sample. However, the determined field inhomogeneity can be used for the microscopy method according to the invention. That is, first, a field inhomogeneity I(x) in a microscope's field of view is determined in a determination step using the method according to the invention for determining field inhomogeneity and using a first sample. Subsequently, in a microscopy step, the steps of the method according to the invention for determining field inhomogeneity are carried out for a second sample, which differs from the first sample, and the field inhomogeneity I(x) determined in the determination step is used as an initial estimate for the field inhomogeneity I(x).

[0097] The present invention presents a method for detecting and removing field inhomogeneities from acquired microscopy data of various types, e.g., wide-field fluorescence, transmission bright-field, and confocal laser scanning microscopy. In addition to correcting inhomogeneities in the image, the detection of field inhomogeneities can also be used to monitor their changes over time. Based on such changes, instrument alignment adjustments can be initiated. In this method, a sample is laterally displaced, and spatially overlapping data are recorded. Field inhomogeneities can be computationally detected and removed by means of the spatial overlap of the acquired tiles.

[0098] The method according to the invention can be applied to a variety of imaging systems to remove vignetting and field inhomogeneities. Such systems include wide-field systems (e.g., the Axio Scan, Observer, DaVinci, Elyra, Apotome, AxioZoom, AxioVert systems from Carl Zeiss), scanning systems (e.g., the LSM, Light Sheet systems from Carl Zeiss), and light field systems.

[0099] In contrast to parametric shading correction methods, the method according to the invention includes estimating, for example, dust particles on the detectors, which are removed along with vignetting effects of the illumination profile. The final image is free of vignetting artifacts and obstructions caused, for example, by dust particles on the detectors.

[0100] It should be noted that the method according to the invention and in particular the ALS-SC algorithm described here works excellently for many different microscope modalities, e.g. fluorescence widefield data or transmission brightfield data.

[0101] In a test of the method according to the invention for transmission brightfield data, the illumination was spatially partially coherent. In particular, the numerical aperture of the illumination defined by the condenser was exactly the same as the numerical aperture of the lens. Thus, the method according to the invention, and in particular the ALS-SC algorithm, can be applied to spatially partially coherent illumination settings.

[0102] The method according to the invention was further tested on measurement data from a laser scanning microscope (LSM). In this context, it is emphasized that with the method according to the invention, it is not possible to confuse shading with effects caused, for example, by tilting the sample, which would lead to signal attenuation due to optical sectioning. The reason for this is that during lateral translation of blurred areas of the sample, the dark areas must move with the sample, since the sample remains blurred during purely lateral translation. However, if the dark areas on the sample remain constant during sample translation, the method according to the invention, and in particular the ALS-SC algorithm in LSM systems, can uniquely distinguish shading from sections. List of reference symbols 1 rehearsal room 2 Sample 10 Light source, e.g. laser 11 Sample level 12 Excitation light 16 emission light 17 speck of dust, e.g. B. on detector 50 18 Intermediate image plane, optically conjugate to the sample plane 11 19 Vignetting 20 Tube lens in the excitation beam path 22 Tube lens in the detection beam path 23 main beam splitters 30 Field of view of the detection beam path 40 microscope objective 41 optical axis of microscope objective 40 42 rear image plane, pupil plane 44 mechanical drive for lateral manipulations 46 mechanical drive for axial manipulations 50 Detector, e.g. camera 51 Detection plane, optically conjugate to the sample plane 11 60 merged images comprising 6 image tiles 61-66 Image tiles 67 image tiles 71-78 Image tiles 80 merged images comprising 80 image tiles 81 Unit Triangular Grid 90 Control unit, e.g. PC 100 microscopes according to the invention an overlap in the x1 direction b Overlap in the x2 direction c Scan step size C(x|m) measurement data = measurement information C(x1, x2|M) Measurement data I(x) Field inhomogeneity I(x1, x2) Field inhomogeneity I0(x) initial estimate of field inhomogeneity I(x) I1(x) first updated estimate of field inhomogeneity I(x) I n (x) nth updated estimate of field inhomogeneity I(x) I n+1 (x) (n+1)th updated estimate of field inhomogeneity I(x) L1 first scalar cost function L2 second scalar cost function m two-dimensional vector in x1, x2-plane m1 Component of m in the x1 direction m2 component of m in the x2 direction p positive integer S(x) microscopic sample information S(x1, x2) microscopic sample information S0(x) initial estimate of microscopic sample information S(x) S1(x) first updated estimate of microscopic sample information S(x) S n (x) nth updated estimate of microscopic sample information S(x) S n+1 (x)) (n+1)th updated estimate of microscopic sample information S(x) t lateral size of the tile x is a two-dimensional vector in the x1, x2 plane x1, x2, x3 right-handed orthogonal coordinate system x1 component of x in the x1 direction x2 component of x in the x2 direction x3 direction of the optical axis µ1 is a scalar different from zero µ2 of non-zero scalar References [0] US8253789B2 [1] Young, Ian T. „Shading correction: compensation for illumination and sensor inhomogeneities.“ Current Protocols in Cytometry 14, Nr. 1 (2000): 2-11. [2] Preibisch, Stephan, Stephan Saalfeld und Pavel Tomancak. „Globally optimal stitching of tiled 3D microscopic image acquisitions.“ Bioinformatics 25.11 (2009): 1463-1465. [3] Legesse, Fisseha Bekele, Olga Chernavskaia, Sandro Heuke, Thomas Bocklitz, Tobias Meyer, Jürgen Popp und Rainer Heintzmann. „Seamless stitching of tile scan microscope images.“ Journal of microscopy 258, Nr. 3 (2015): 223-232. [4] Smith, Kevin, Yunpeng Li, Filippo Piccinini, Gabor Csucs, Csaba Balazs, Alessandro Bevilacqua und Peter Horvath. „CIDRE: an illumination-correction method for optical microscopy.“ Nature methods 12, Nr. 5 (2015): 404-406. [5] Peng, Tingying, Kurt Thorn, Timm Schroeder, Lichao Wang, Fabian J. Theis, Carsten Marr and Nassir Navab. “A BaSiC tool for background and shading correction of optical microscopy images.” Nature Communications 8, No. 1 (2017): 14836. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 8253789B2

[0102] Cited non-patent literature

[0000] Young, Ian T. “Shading correction: compensation for illumination and sensor inhomogeneities.” Current Protocols in Cytometry 14, No. 1 (2000): 2-11

[0102] Preibisch, Stephan, Stephan Saalfeld und Pavel Tomancak. „Globally optimal stitching of tiled 3D microscopic image acquisitions.“ Bioinformatics 25.11 (2009): 1463-1465

[0102] Legesse, Fisseha Bekele, Olga Chernavskaia, Sandro Heuke, Thomas Bocklitz, Tobias Meyer, Jürgen Popp und Rainer Heintzmann. „Seamless stitching of tile scan microscope images.“ Journal of microscopy 258, Nr. 3 (2015): 223-232

[0102] Smith, Kevin, Yunpeng Li, Filippo Piccinini, Gabor Csucs, Csaba Balazs, Alessandro Bevilacqua und Peter Horvath. „CIDRE: an illumination-correction method for optical microscopy.“ Nature methods 12, Nr. 5 (2015): 404-406

[0102] Peng, Tingying, Kurt Thorn, Timm Schroeder, Lichao Wang, Fabian J. Theis, Carsten Marr und Nassir Navab. „A BaSiC tool for background and shading correction of optical microscopy images.“ Nature Communications 8, Nr. 1 (2017): 14836

[0102]

Claims

[1] microscope comprising a light source (10) to provide excitation light (12), a lighting beam path for guiding the excitation light (12) to a sample chamber (1), a detector (50) for detecting emission light (16) emitted by a sample (2) in the sample chamber (1), a detection beam path with a microscope objective (40) for guiding the emission light (16) to the detector (50), a mechanical drive (44) for setting a relative lateral position (x1, x2) between the sample (2) and the microscope objective (40) with respect to an optical axis (41) of the microscope objective (40), a control unit (90) for controlling the mechanical drive (44) and is configured for recording and evaluating measurement data from the detector (50), characterized by , that the control unit (90) is configured to perform the following steps: an adjustment step wherein the mechanical drive (44) is successively adjusted to at least three different relative lateral positions (m), a detection step, wherein in each of the different relative lateral positions (m) measurement data (C(x|m)) are acquired from the detector (50) for at least a subset of points in the sample (2) in a field of view (30) of the detection beam path, wherein for each of the points of the subset measurement data are acquired for at least two different lateral positions (m) of the mechanical drive (44), and an evaluation step, wherein, based on measurement data (C(x|m)) acquired in the acquisition step, the following steps are performed: • Extract, from the measurement data (C(x|m)), a field inhomogeneity (I(x)) in the field of view under the assumption that the field inhomogeneity (I(x)) is independent of the respective set relative lateral position (m), and • Extracting, from the measurement data (C(x|m)), a microscopic sample information (S(x)) that represents a part of the measurement data (C(x|m)) caused by the sample (2). [2] Microscope according to claim 1, characterized by , that the set relative lateral positions (m) are located on a two-dimensional regular grid. [3] Microscope according to claim 2, characterized by that the grid is rectangular or triangular. [4] Microscope according to claim 2 or 3, characterized by, that the control unit (90) is further configured to configure the two-dimensional grid of the set relative lateral positions (m) such that an overlap (a) between adjacent tiles (61, 62) in a first coordinate direction (x1) is at least 50%. [5] Microscope according to any one of claims 2 to 4, characterized by , that the control unit (90) is further configured to configure the two-dimensional grid of the set relative lateral positions (m) such that an overlap (b) between adjacent tiles (61, 64) in a second coordinate direction (x2), which is in particular perpendicular to the first coordinate direction (x1), is at least 5% and preferably at least 10%. [6] Microscope according to any one of claims 1 to 5, characterized by , that the points in the subset of points are evenly distributed across the field of view. [7] Microscope according to claim 6, characterized by, that the points in the subset of points lie on a regular grid. [8] Microscope according to claim 7, characterized by that the grid of the subset of points is rectangular or triangular. [9] Microscope according to any one of claims 1 to 8, characterized by , that the acquisition step includes the acquisition of measurement data (C(x|m)) for each of the points in the field of view. [10] Microscope according to any one of claims 1 to 9, characterized by , that the evaluation step includes a binning of the measurement data (C(x|m)) of a large number of points, in particular a large number of pixels. [11] Microscope according to any one of claims 1 to 10, characterized by , that the evaluation step includes evaluating the measurement data (C(x|m)) of each of the points in the field of view. [12] Microscope according to any one of claims 1 to 11, characterized by, that the control unit (90) is configured to perform the evaluation step as an iterative solution of a double-blind estimation problem based on an initial estimate of the field inhomogeneity (I(x)) and an initial estimate of the microscopic sample information (S(x)). [13] Microscope according to any one of claims 1 to 12, characterized by , that the control unit (90) is configured to perform at least some of the following steps in the evaluation step: • Calculating an estimate of the microscopic sample information (S(x)) based on the measurement data (C(x|m)) and using an initial estimate of the field inhomogeneity (I0(x)) and • iterative calculation ▪ updated estimates of field inhomogeneity (I n+1 (x)) based on the measurement data (C(x|m)) and using the most recent estimate of the microscopic sample information (S n (x)) and ▪ updated estimates of microscopic sample information (S n+1 (x)) based on the measurement data (C(x|m)) and using the most recent estimate of field inhomogeneity (I) n+1 (x)), • Evaluating a degree of accuracy to which the most recent estimate of field inhomogeneity (I) relates n+1 (x)) and the most recent estimate of the microscopic sample information (S n+1 (x)) reproduce the measurement data (C(x|m)), • Repeat the step of iteratively calculating updated estimates of field inhomogeneity (I n+1 (x)) and updated estimates of the microscopic sample information (S n+1 (x)), until the measurement data (C(xlm)) are replaced by the most recent estimate of the field inhomogeneity (I n+1 (x)) and the most recent estimate of the microscopic sample information (S n+1 (x)) can be reproduced to a certain degree of accuracy. [14] Microscope according to any one of claims 1 to 13, characterized by , that the control unit (90) is further configured to perform at least some of the following steps in the evaluation step: • Calculating an initial estimate of the microscopic sample information (S0(x)) based on the measurement data (C(x|m)) and using an initial estimate of the field inhomogeneity (I0(x)); • Calculating a first updated estimate of field inhomogeneity (I1(x)) based on the measurement data (C(x|m)) and using the initial estimate of the microscopic sample information (S0(x)); • Calculating a first updated estimate of the microscopic sample information (S1(x)) based on the measurement data (C(x|m)) and using the first updated estimate of the field inhomogeneity (I1(x)); • Calculating an (n+1)th updated estimate of the field inhomogeneity (I n+1(x)) based on the measurement data (C(xlm)) and using the nth updated estimate of the microscopic sample information (S n (x)); and • Calculating an (n+1)th updated estimate of the microscopic sample information (S n+1 (x)) based on the measurement data (C(xlm)) and using the (n+1)th updated estimate of field inhomogeneity (I n+1 (x)). [15] Microscope according to claim 13 or 14, characterized by , that the initial estimate of the field inhomogeneity (I0(x)) is a flat profile. [16] Microscope according to any one of claims 1 to 15, characterized by, that the control unit (90) is further configured to perform the evaluation step as minimizing a mathematical distance between the measurement data (C(x|m)) and a combination of the field inhomogeneity (I(x)) and the microscopic sample information (S(x)), wherein the mathematical distance is based on any mathematical norm. [17] Microscope according to claim 16, characterized by , that The control unit (90) is further configured to perform the following steps in the evaluation step: Taking a new estimate of field inhomogeneity as the new updated estimate of field inhomogeneity (I n+1 (x)) and a new estimate of the microscopic sample information as a new updated estimate of the microscopic sample information (S n+1 (x) • if the value of a first scalar cost function (L1) for the new estimate of field inhomogeneity (I(x)) and the new estimate of microscopic sample information (S(x)) is smaller than the value of the first scalar cost function (L1) for the most recent updated estimate of field inhomogeneity (I n (x)) and the most recent updated estimate of the microscopic sample information (S n (x)) and • if the value of a second scalar cost function (L2) for the new estimate of field inhomogeneity and the new estimate of microscopic sample information is smaller than the value of the second scalar cost function (L2) for the most recent updated estimate of field inhomogeneity (I n (x)) and the most recent updated estimate of the microscopic sample information (S n (x)), • where the first scalar cost function (L1) and the second scalar cost function (L2) each contain a mathematical distance between the measurement data (C(x|m)) and a combination of the field inhomogeneity (I(x)) with the microscopic sample information (S(x)) and • where the first scalar cost function (L1) contains the norm of the field inhomogeneity (I(x)) and the second scalar cost function (L2) contains the mathematical norm of the microscopic sample information (S(x)). [18] Microscope according to claim 16 or 17, characterized by , that the combination of field inhomogeneity (I(x)) with microscopic sample information (S(x)) is a product of field inhomogeneity (I(x)) and microscopic sample information (S(x)). [19] Microscope according to any one of claims 16 to 18, characterized by , that that the mathematical norms are one of: Lp-Norm L2 norm (Euclidean norm), Manhattan standard (L1 standard), Infinite norm (L∞ norm, Shannon entropy norm, Jensen-Shannon divergence norm, Renyi entropy norm, Tsallis entropy norm. [20] Microscope according to any one of claims 17 to 19, characterized by , that the first cost function (L1) and the second cost function (L2) are each given by L1=12∑x∑m[I(x)S(x−m)−C(x|m)]p+μ12∑xI(x)p L2=12∑x∑m[I(x+m)S(x)−C(x+m|m)]p+μ22∑xS(x)p where C(x|m) the measurement data are I(x) the field inhomogeneity is S(x) the microscopic sample information are µ1 and µ2 scalars not equal to zero m is a two-dimensional vector n of the x1, x2 plane. x is a two-dimensional vector n of the x1, x2 plane. p is a positive integer. [21] Microscope according to one of claims 19 or 20, characterized by , that p = 2. [22] Microscope according to one of claims 20 or 21, characterized by , that The (n+1)th updated estimate of field inhomogeneity is calculated as follows: In+1(x)=∑mSn(x−m)C(x|m)∑mSn(x−m)2+μ1 and that the (n+1)th updated estimate of the microscopic sample information is calculated as follows: Sn+1(x)=∑mIn(x+m)C(x+m|m)∑mIn(x+m)2+μ2 where C(x|m) the measurement data are I n+1 (x) (n+1)-th updated estimate of field inhomogeneity I(x) is I n (x) the nth updated estimate of the field inhomogeneity I(x) is S n (x) the nth updated estimate of the microscopic sample information S(x) is S n+1 (x)) the (n+1)th updated estimate of the microscopic sample information S(x) is µ1 and µ2 are non-zero scalars. m is a two-dimensional vector in the x1, x2 plane. x is a two-dimensional vector n of the x1, x2 plane. p is a positive integer. [23] Microscope according to any one of claims 1 to 22, characterized by , that the detector includes a two-dimensional spatially resolving photodetector (50), e.g. a camera, a one-dimensional spatially resolving detector, e.g. a linear detector array, or a single photodetector, e.g. a point-shaped photodetector. [24] Microscope according to claim 23, characterized by , that the detector (50) comprises at least one of the following: CCD element, CMOS element, SPAD element, PMT. [25] Microscope according to any one of claims 1 to 24, that is at least one of the following: a wide-field microscope a scanning microscope, a confocal microscope, a light field microscope a light-sheet microscope a TIRF microscope, a SIM microscope. [26] Microscope according to any one of claims 1 to 25, characterized by that the control unit (90) is configured to perform the method according to one of claims 28 or 29. [27] Method for determining a field inhomogeneity in a field of view of a microscope, comprising the following steps: Illuminating a sample (1) through an illumination beam path of the microscope (100) with excitation light (12), Guiding emission light (16) emitted from the sample (2) through a detection beam path comprising a microscope objective (40) to a detector (50), Detecting the emission light (16) with the detector (50), Setting specific lateral positions (x1, x2) between the sample (2) and the microscope objective (40) with respect to an optical axis (41) of the microscope objective (40) by means of a mechanical drive (44), characterized by , an adjustment step wherein the mechanical drive (44) is successively adjusted to at least three different relative lateral positions (m), a detection step, wherein in each of the different relative lateral positions (m) measurement data (C(x|m)) are acquired from the detector (50) for at least a subset of points in the sample (2) in a field of view (30) of the detection beam path, wherein for each of the points in the subset measurement data are acquired for at least two different lateral positions (m) of the mechanical drive (44), and an evaluation step, wherein, based on measurement data (C(x|m)) acquired in the acquisition step, the following steps are performed: Extracting, from the measurement data (C(x|m)), a field inhomogeneity (I(x)) in the field of view (30) under the assumption that the field inhomogeneity (I(x)) is independent of the respective set relative lateral position (m), and Extracting, from the measurement data (C(x|m)), a microscopic sample information (S(x)) that represents a part of the measurement data (C(x|m)), which are caused by sample (2). [28] Microscopy procedures comprise the following steps: a determination step wherein a field inhomogeneity (I(x)) in a field of view of a microscope is determined using the method according to claim 27 using a sample, a microscopy step comprising performing the steps of the method according to claim 27 for a sample that differs from the sample used in the determination step, wherein the field inhomogeneity (I(x)) in the determination step is used as an initial estimate for the field inhomogeneity (I(x)). [29] Method according to one of claims 27 or 28, further comprising the use of the microscope according to one of claims 1 to 26.

Images