METHOD FOR PROCESSING TILED IMAGES AND OPTICAL DEVICE

Abstract

The invention relates to a method for processing tiled images, in which a plurality of overlapping tiled images C are processed using an optical device. i (x, y) of a sample is recorded. The method is characterized by the fact that a reaction for C i (x,y) and a model function, in particular for a shadowing function S i (x, y) are specified so that correction factors are determined at support points in an overlap area of ​​adjacent tile images and the shading function S i (x,y) and unshaded sample images B i (x,y) can each be determined at least approximately. The invention also relates to an optical device.

Description

[0001] The invention relates, in a first aspect, to a method for processing tiled images according to the preamble of claim 1. In a further aspect, the invention relates to an optical device according to the preamble of claim 22.

[0002] In a generic method for processing tiled images, a plurality of overlapping tiled images C are processed using an optical device. i (x, y) of a sample recorded.

[0003] An optical device of this type comprises at least the following components: a detector for detecting emission radiation emitted by a sample in a sample region, a detection beam path with imaging optics for directing the emission radiation onto the detector, a mechanical drive for adjusting a relative lateral position between the sample and the imaging optics with respect to an optical axis of the imaging optics, and a control unit configured at least for controlling the mechanical drive and for evaluating measurement data from the detector. The control unit is also configured to control the optical device for recording a plurality of overlapping tile images. i (x, y) to target a sample.

[0004] In modern microscopy, large areas of a sample are typically imaged at high resolution by acquiring images at various positions using a digital camera or another type of detector, such as a line scanner system. The positions of the individual images are often chosen to form a regular grid, with the field of view (FOV) of adjacent images overlapping by a certain amount, for example, 10% of the FOV width or height. The acquired images are usually referred to as tiles, and the entire set of tiles is called a tiled image or mosaic image. In this description, the terms tile and tiled image are used synonymously. Such tiled images offer a large field of view and high resolution but are subject to two types of artifacts that often complicate further analysis.

[0005] The first type of artifact is caused by inaccuracies in the mechanical devices used to position the sample relative to the microscope objective, leading to a mismatch of the imaged sample structures visible at the tile boundaries. This is typically corrected by so-called stitching algorithms: The imaged sample structures in the overlapping area are examined to detect positional deviations between adjacent tiles, and appropriate horizontal (x-direction) and vertical (y-direction) shifts are calculated for the slightly displaced tiles, ensuring that the imaged sample structures at the tile boundaries align as closely as possible. The stitching process is also referred to as the structural alignment of adjacent tiles.

[0006] The second type of artifact is caused by brightness differences between adjacent tiles and tile edges. Such brightness differences, which can also be referred to as luminance or luminance differences, can have various causes: vignetting, which can be caused by the design of the imaging light path, i.e., the illumination beam path and / or the detection beam path, by inhomogeneous illumination, or by a spatially and / or temporally dependent sensitivity of the detector.The sample itself can also be the source of such artifacts: In fluorescence imaging, bleaching can occur in overlapping areas; in transmitted light or fluorescence imaging, differing transmission / emission properties of the sample holder or embedding medium can have an effect; a tiled image with highly reflective structures often appears brighter at the tile edge than the edge of the adjacent tile if the sample has no reflective structures in that area. Most of these artifacts exhibit a slow spatial change. Other artifacts with spatially abrupt changes within a tile can be caused, for example, by dust in the light path, defects in the detector or image sensor, or dust directly on or at the detector or image sensor. All these undesirable effects on image luminance taken together are often referred to as shading artifacts.

[0007] Image correction algorithms for fixing or correcting these shading artifacts are often referred to as de-shading algorithms or background shading correction algorithms [Wu] and can be broadly divided into prospective and retrospective methods.

[0008] In prospective methods, shading reference images are acquired from calibration samples that exhibit identical or similar artifacts to the subsequently acquired samples, but contain no other structures [Young, De Silva]. Using such shading reference images, the luminance artifacts can be compensated for by a simple multiplicative or additive correction scheme. While such algorithms can handle spatially slow and spatially abrupt luminance changes and can also be applied to non-overlapping tile images, they have some disadvantages: It can be difficult or sometimes even impossible to reproduce the shading effects that appear in the real samples using a calibration pattern. Furthermore, such methods do not work well if the shading artifacts depend on the tile content itself.To overcome the limitations of using calibration samples, a new method has recently been described which derives a shadow correction image from the sample image itself, but requires a special acquisition scheme with large tile overlap [Loetgering].

[0009] Retrospective methods [Legesse, Smith, Peng] attempt to compensate for shading artifacts by using the acquired tiling image data itself. Some methods apply the luminance correction only to the overlap region [Legesse]. Such an approach leads to unsatisfactory results for mosaic images with small overlaps and when the shading effect extends beyond the overlap region.

[0010] Some methods of this type attempt to separate shading artifacts from sample structures by averaging the image data of all tiles, using low-pass filters, or requiring user input to separate background and sample regions [Smith, Peng]. Other methods improve or combine this scheme by automatically detecting object and background regions and by averaging only the background regions [Legesse]. All these approaches that use averaging show good results only if the following conditions are met: The tiled image must contain a sufficient number of tiles. Furthermore, sufficient sample homogeneity is required, and it must be possible to separate background and sample areas.If only a small number of tiles are available for averaging, or if the sample contains pronounced edges or areas with significantly different luminance, the sample structure distorts the shading estimate.

[0011] Some methods [Wu] model the shadow background by fitting 2D area functions via a least-squares fit of the background pixel intensities or the averaged intensities of the background regions. With this approach as well, the tiled image must contain a sufficient number of background regions suitable for the least squares fit.

[0012] The described retrospective methods are based on the unsatisfactory situation that the quality of the calculated shading correction data depends heavily on the sample properties.

[0013] One object of the present invention can be considered to be to provide a method for processing tiled images and an optical device in which shadowing artifacts can be reduced with comparatively little computational effort and high-quality mosaic images can be achieved.

[0014] This problem is solved by the method with the features of claim 1 and by the optical device with the features of claim 22.

[0015] The method of the type described above is further developed according to the invention in that, for determining unshaded sample images B i (x, y) the following procedure steps are carried out: a) Specifying an approach C i (x,y) = S i (x, y)B i (x, y)+T i (x, y) for the measured tile images C i (x, y), wherein B i (x, y) an unshaded sample image of a tile i, S to be determined i(x, y) a shading function for tile i, T i (x, y) is an additive shadowing term for tile i and i is a running index of the individual tiles; b) Specifying a model function for the shadowing function S i (x, y) and a model term for the additive shadowing term T i (x, y), c) Determining correction factors at a plurality of support points in an overlap area of ​​adjacent tile images using tile images C i (x, y), d) at least approximately determining the shadowing function S i (x, y) and the shadowing term T i (x, y) by adjusting the model function specified in step b) to the correction factors determined in step c), e) approximate determination of an unshaded sample image B i (x, y) using the shadowing function S determined at least approximately in step d).i (x,y) and the at least approximately determined shadowing term T i (x, y).

[0016] The optical device of the type described above is further developed according to the invention in that the control unit is configured to determine unshaded sample images B i (x, y) to carry out the process steps a) to e) specified above for the method according to the invention.

[0017] Advantageous embodiments of the inventive method for processing tiled images and the inventive optical device are described below, particularly in connection with the dependent claims and the figures.

[0018] The optical device can be any type of optical device capable of capturing images of a sample. For example, the optical device can be designed as an aerial camera. This aerial camera can, for instance, be located in a satellite to capture images of the Earth or space.

[0019] In particularly preferred embodiments, the optical device is designed as a microscope and comprises a radiation source for providing illumination radiation and an illumination beam path for directing the illumination radiation into the sample area, wherein the imaging optics comprise microscope optics. The microscope can, in principle, be any type of microscope in which a sample is illuminated. For example, the microscope can be an electron microscope or an ion microscope. In particularly preferred embodiments, the microscope is a light microscope and the microscope optics comprise at least one microscope objective.The microscope can, for example, perform the function of at least one of the following microscopes: digital microscope, fluorescence microscope, light microscope, transmitted light microscope, reflected light microscope, wide-field microscope, scanning microscope, confocal microscope, light-field microscope, light-sheet microscope, TIRF microscope, SIM microscope, electron microscope, ion microscope.

[0020] Accordingly, in preferred variants of the inventive method for processing tiled images, a microscope is used as the optical device.

[0021] The radiation source can be any source of radiation capable of delivering the desired type of radiation with the desired wavelength(s) and at a suitable intensity. For example, the radiation source can be an electron or ion source.

[0022] In preferred embodiments, the radiation source is a light source. In principle, the light source can be any light source capable of supplying illumination light with a desired wavelength or wavelengths and a suitable intensity. For example, the light source can be a laser or an LED module. The excitation light can be coherent, at least partially coherent, or incoherent. The illumination light is electromagnetic radiation, particularly in the visible spectral range and adjacent regions. The illumination light can also be referred to as excitation light; these two terms are largely used synonymously in this description.

[0023] Radiation emitted by the sample under investigation as a result of irradiation by the illumination radiation is called emission radiation.

[0024] For example, light emitted by the sample under investigation as a result of irradiation by the illumination or excitation light is called emission light and reaches the detector, e.g., a camera, via the detection beam path. For the light to be classified as emission light, it is only necessary that the light is emitted by the illuminated sample or at least originates from the illuminated sample. Typically, the emission light can be fluorescence light, which the sample, especially any dye molecules present, emits as a result of irradiation by the excitation light. The emission light can also be reflected, transmitted, or scattered illumination light. The only requirement for the contrast-generating principle is that the sample emits light as a result of irradiation by the excitation light.The sample can also be referred to as a specimen.

[0025] The term illumination beam path refers to all components, particularly optical ones, that guide and modify the beam, e.g., lenses, mirrors, prisms, gratings, filters, apertures, beam splitters, with and via which the excitation radiation, especially the excitation light, is guided from a radiation source, particularly a light source, to the sample under investigation. The illumination beam path may include an illumination objective. The illumination objective and the microscope objective may each be of a known type. In principle, the illumination objective and the microscope objective can also be separate objectives. In preferred embodiments, however, the illumination objective and the microscope objective are one and the same objective.

[0026] The term sample space refers to the spatial area in which a sample to be examined can be arranged. In typical embodiments, a sample can be placed or secured, for example by means of a sample holder or sample frame, on an xy-stage that is adjustable laterally with respect to an optical axis. A z-drive may be provided to change the distance between the sample stage and the illumination objective or between the sample and the microscope objective.

[0027] The sample can be any type of sample in principle. The methods and the optical device designed as a microscope according to the invention are particularly suitable for the examination of biological samples.

[0028] The light emitted by the sample under investigation as a result of irradiation with the excitation light is called emission radiation, and in particular emission light, and travels via the detection beam path to the detector, e.g., a camera. The term detection beam path encompasses all beam-guiding and beam-modifying components, especially optical ones, such as lenses, mirrors, prisms, gratings, filters, apertures, and beam splitters, by which and through which the emission radiation is guided from the sample under investigation to the camera. Advantageously, a sensor plane of the detector can be arranged in a plane that is optically conjugate to a focal plane of the microscope objective.

[0029] The field of view of the detection beam path refers to the lateral spatial area in a sample plane from which emission radiation, especially emission light, can be collected and directed to the detector. The size of the field of view is generally determined by the optical parameters, e.g., the aperture and magnification, of the microscope objective and other components in the detection beam path, e.g., a tube lens.

[0030] The type of detector used for detecting the emission radiation generally depends on the type of optical device and, in particular, the type of microscope. In embodiments of the invention, the detector can comprise a two-dimensional spatially resolved photodetector, e.g., a camera, a one-dimensional spatially resolved detector, e.g., a linear detector array, or a single photodetector, e.g., a point-like photodetector. Specifically, the detector can comprise at least one of the following elements or components: CCD element, CMOS element, SPAD element, PMT.

[0031] The mechanical drive for adjusting the relative lateral position between the sample and the microscope objective can be configured to move the sample relative to an optical axis, such as that of the microscope objective, and / or to move the microscope objective and thus its optical axis relative to the sample. The mechanical drive can be mounted on a microscope stand. The microscope stand can be either upright or inverted.

[0032] The term control unit encompasses all hardware and software components that interact with the components of the optical device according to the invention to achieve the intended functionality of the latter. In particular, the control unit may include a computing unit, for example a PC, and a camera control unit capable of reading measurement signals. Detector measurement data refers to the measurement data generated by the detector when irradiated with emission light.

[0033] The term optical axis refers to the optical axis of the imaging optics, in particular the microscope objective, in the detection beam path.

[0034] Relative lateral position refers to the lateral position of the sample relative to the optical axis. If, as is usually the case, a coordinate system is chosen such that the optical axis coincides with the z-axis, the lateral coordinate directions are the x- and y-directions. The lateral coordinate directions also specifically include those coordinate directions in which the tiles and the tiled image extend.

[0035] Overlapping tile images mean that the individual tiles do not butt up at their edges, but rather overlap, particularly in both lateral coordinate directions. An overlap area of ​​adjacent tile images refers to the area of ​​the lateral coordinates in which the adjacent tiles overlap.

[0036] The term unshaded sample images B i(x, y) denote the sample images of the individual tiles that would theoretically be obtained if the shadowing artifacts described above were not present.

[0037] The shading function S i (x, y) and the additive shadowing term T i (x, y) for a specific tile i are scalar functions of the lateral coordinates. The support points are points in the plane spanned by the tiles and defined by the lateral coordinates. Support points refer to positions in a plane of the viewing field. The correction factors are scalar values.

[0038] The present invention has first recognized that the shadowing artifacts described above can generally be modeled by mathematically relatively simple types of model functions and model terms.

[0039] Furthermore, a basic idea of ​​the invention described here can be considered to be the determination of correction factors from the measurement data, which enable a straightforward adjustment of the model function and the additive model term.

[0040] A significant advantage of the invention is that shadowing artifacts, which at least change slowly in space, can be corrected very well and also with low computational effort, i.e., quickly.

[0041] The method according to the invention can be carried out with the optical device according to the invention. The microscopy method according to the invention can be carried out with an optical device according to the invention designed as a microscope. The optical device according to the invention, and in particular its control unit, can advantageously be configured to carry out the variants of the method according to the invention described herein.

[0042] In principle, it is possible that the additive shadowing term T can also be used in the method according to the invention. i (x, y) is adjusted. In practice, however, it has been shown that it is sufficient to adjust the shading function S. i (x, y) to be determined at least approximately. This means that in advantageous variants of the method according to the invention, the model term for the additive shadowing term T can be used. i (x, y) can be chosen as zero.

[0043] The model function can be defined by a plurality of parameters.

[0044] In a comparatively simple preferred variant of the method according to the invention, the model function is a parabolic function in the x- and / or y-coordinate.

[0045] In a more complex variant, where the shadowing functions can be fitted more precisely, the model function is a Zernike polynomial. For example, the model function could be a third-degree Zernike polynomial. Generally, it is advantageous to choose a function from an orthonormal system of functions as the model function. Functions for which an orthonormal basis exists are well-suited because the least-squares approximation can then be determined using simple scalar products. Examples of orthogonal systems of functions include Zernike polynomials, two-dimensional Hermite polynomials, two-dimensional Laguerre polynomials, and Jacobi polynomials. However, non-orthogonal polynomials can also be advantageous due to their efficient evaluation possibilities resulting from their polynomial nature.For example, multivariate Bernstein polynomials can also be used because they allow a so-called shape-preserving approximation, preferably on simplices. Polynomials that are in principle defined on non-rectangular domains can also be used advantageously if the domain of definition encompasses the relevant pixel area.

[0046] The model function is preferably chosen such that its value at the origin of the lateral coordinates, i.e., at position (0,0), is 1. This means that it is assumed there is no vignetting or other impairment of the propagated emission light on the optical axis and / or that the correction at the origin is normalized to 1, and that the corrections are therefore made relative to the center, where no correction is applied.

[0047] To implement the method according to the invention, it is basically only necessary that the support points at which the correction factors are calculated lie within overlap areas of the tile images.

[0048] Preferably, the correction factors are calculated for support points located at the edges of overlapping areas of the tile images. In particular, the correction factors can be advantageously calculated for support points lying on the coordinate axes. However, this is not strictly necessary. The support points can also be located within the overlapping area.

[0049] For more complex model functions, it may be preferable to calculate the correction factors for a large number of support points on the edge of a tile. These support points can, for example, be located at equal intervals along the edge of the tiles.

[0050] It is also advantageous to calculate a correction factor for at least one support point in each overlap area. It is also possible to calculate correction factors for support points located in areas of the tiles, and especially on the edge of the tile, where the tile in question overlaps with a diagonally adjacent tile.

[0051] For tiles at the edge of the mosaic image, where there are no overlapping areas with adjacent tiles, correction factors derived from support points at opposite edges can be used for the edges of the tiles in question without adjacent tiles. Alternatively or additionally, for the edges of tiles at the edge of the mosaic image, where there are no overlapping areas with adjacent tiles, mean values ​​of correction factors from tiles that are completely surrounded by other tiles can be used.

[0052] Using the model function and assuming that the measured tile images of neighboring tiles are structurally adapted to each other, values ​​of the sample image B can be determined. i (x, y) are calculated at support points on the coordinate axes and at the boundaries of the overlap areas. The correction factors can then be expressed as quotients of the values ​​of the tile images C. i (x, y) and the values ​​of sample image B i (x, y) are calculated at support points on the coordinate axes and at the boundaries of the overlap regions. This alternative is particularly suitable if, as described above, a parabolic approach is chosen for the model function.

[0053] Additionally or alternatively, it is also possible that the correction factors are quotients of values ​​of the tile images measured from neighboring tiles.

[0054] To obtain even more information, it can be advantageous to insert additional support points in central areas of the tile images, setting the function value of the model function at these additional support points to a suitable value, for example, 1. These additional support points can be beneficial when a model function with a very large number of parameters needs to be adjusted.

[0055] The adjustment of the shadowing functions and / or the sample images, i.e., the actual "fitting" to the determined correction factors, can generally be carried out using known methods. For example, it is possible to approximate at least one of the shadowing functions and / or the sample images using the correction factors with a least-squares fit.

[0056] In a particularly preferred embodiment of the method according to the invention, the approximate determination of the shading function S is carried out in step d). i (x, y) and in step e) the approximate determination of the unshaded sample image B i (x,y) each iteratively performed by determining updated correction factors using an (n - 1)th approximation of the shadowing function and / or an (n - 1)th approximation of the unshaded sample image, determining an nth approximation of the shadowing function by adjusting the correction factors to the respective given model function, and determining an nth approximation of the unshaded sample image using the nth approximation of the shadowing function.

[0057] In this process, a number of free parameters for fitting at least one of the shadowing functions and / or the sample images can be higher in later iteration steps than at the beginning of the iteration.

[0058] To obtain information about the progress of the iteration, a distance between the nth approximation of the sample images can also be determined for a sample image of tile i. Bin(x,y) and the n - 1-th approximation of the sample images Bin−1(x,y) The iteration can be determined. Iteration can then continue if the distance is still greater than a predefined threshold, and it can be terminated if the determined distance is less than the predefined threshold. In principle, different mathematical norms can be used as the distance measure. For example, a Euclidean distance can be defined.

[0059] To reduce the computational effort and obtain a good approximation of the shadowing function more quickly, it may also be useful to first perform the iterative procedure for a model function for the shadowing function with a first number of free parameters and then for a more complex model function with a second number of free parameters that is larger than the first number.

[0060] Once an unshaded sample image of sufficient accuracy has been obtained for each of the tiles, the resulting unshaded sample images can be assembled into a mosaic image.

[0061] In principle, it is possible and also preferred that the shading function is calculated at least approximately individually for each tile image.

[0062] In some situations, it may be useful to calculate an average of the approximate shading functions determined individually for the tile images. Consistency tests can then be performed using this average. For example, it can be checked how far an individual approximate shading function deviates from the calculated average. An approximate shading function can then be replaced by the average if it differs from the average by more than a predefined threshold.

[0063] To reduce noise and compensate for outliers of excessively bright or dark pixels, it is preferable to evaluate the values ​​of the tiled images for groups of several adjacent pixels, rather than pixel by pixel, when calculating the shading function. For example, a mean or median value can be calculated for the pixels in these groups and used for further analysis. Pixels that are too bright or too dark are also referred to as hot or cold pixels, respectively. Sometimes, outliers are not only hot / cold pixels but also originate from the sample size. If the sampling rate is high, considering only immediately adjacent pixels may not be sufficient. In such cases, mean and / or median values ​​of pixels from a neighborhood can be used preferentially.Several pixels from a localized region around a central pixel can be used to obtain a representative value of the tiling pattern at the central pixel by calculating the mean and / or median. The groups of pixels can each be localized, for example, in rectangular or square areas. The edge length of the rectangular or square areas can be, for example, between 2% and 5%, and preferably 3%, of the overlap between adjacent tiles.

[0064] Further advantages and features of the present invention are explained below in connection with the figures. These show: Fig. 1: schematic representation of an embodiment of an optical device according to the invention; Fig. 2: Diagrams illustrating a first embodiment of the method according to the invention; Fig. 3: a schematic representation of overlapping tiles to illustrate the first embodiment of the method according to the invention; Fig. 4: a diagram illustrating a second embodiment of the method according to the invention and Fig. 5: a further schematic representation of overlapping tiles to illustrate the second embodiment of the method according to the invention.

[0065] Identical and equivalent components are generally marked with the same reference symbols in the figures.

[0066] An embodiment of an optical device according to the invention, designed as a microscope 100, is described with reference to Fig. 1 described.

[0067] The microscope 100 initially comprises a light source 10, e.g., a laser, to provide illumination 12 and an illumination beam path to guide the illumination 12 to a sample chamber 1. In the example shown, the illumination beam path includes a tube lens 20, a principal beam splitter 23, and a microscope objective 40. The tube lens 20 generates an intermediate image plane 18, i.e., a plane that is optically conjugate to a plane 11 in a sample 2 in the sample chamber 1. In the illumination beam path, the excitation light 12 passes through the intermediate image plane 18 and via the tube lens 20 to the principal beam splitter 23, where it is reflected toward the microscope objective 40. The excitation light 12 then passes through a rear focal plane 42 of the microscope objective 40 and is subsequently directed by the microscope objective 40 into the sample chamber 1.Sample 2 can be a biological sample prepared with fluorophores that can be excited by the excitation light 12. The wavelength and intensity of the excitation light 12 can be selected appropriately with regard to sample 2 and the fluorophores used. The light source 10 can consist of a variety of different lasers. The wavelength and / or intensity can be adjustable.

[0068] Furthermore, the microscope 100 comprises a detector 50 for detecting emission light 16 emitted by a sample 2 in sample chamber 1, and a detection beam path with a microscope objective 40 for guiding the emission light 16 to the detector 50. In the example shown, the microscope 100 is a wide-field microscope and the detector 50 is a camera, i.e., a field of view 30 (FOV) of the detection beam path is imaged onto a sensor plane 51 of the camera 50. The sensor plane 51 is optically conjugate to the plane 11 in sample chamber 1.

[0069] The emission light 16 emitted by sample 2 can typically be redshifted fluorescence light emitted by the fluorophores in sample 2. The main beam splitter 23 is configured to transmit the redshifted emission light 16 and reflect the excitation light 12, thus preventing large portions of the excitation light 12 backscattered from sample chamber 1 from reaching the camera 50.

[0070] In the detection beam path, the emission light 16 emitted by the sample 2 is captured by the microscope objective 40, passes through the main beam splitter 23 and is then imaged by a tube lens 22 into the sensor plane 51 of the camera 50.

[0071] As is known in the prior art, an excitation filter can be located in the excitation beam path, e.g., between the tube lens 20 and the main beam splitter 23, and / or an emission filter can be located in the detection beam path, e.g., between the tube lens 22 and the main beam splitter 23. Several different main beam splitters 23 are also possible, each tuned to individual fluorophores and connected in the beam path.

[0072] Furthermore, the microscope 100 comprises a mechanical drive 44, particularly an automated one, for setting a relative lateral position x, y between the sample 2 and the microscope objective 40 with respect to an optical axis 41 of the microscope objective 40, and a control unit 90, e.g., a PC, configured to control the mechanical drive 44 and to acquire and evaluate measurement data from the detector 50. In the example shown, the optical axis 41 of the microscope objective 40 extends in the direction of the z-axis. The mechanical drive 44 can, for example, be part of a motorized sample stage and, in the example shown, serves to set a predetermined position of the sample 2, i.e., predetermined x, y coordinates of the sample 2 with respect to the optical axis 41. A right-handed and orthogonal coordinate system x, y, z is shown below the mechanical drive 44.In the example shown, the microscope 100 further includes an axial drive 46, which serves to set a predetermined axial distance, i.e. a distance in the z-direction, between the sample 2 and the microscope objective 40.

[0073] The illustrated embodiment of the microscope according to the invention is also configured to carry out the method according to the invention. For this purpose, the control unit 90 is configured according to the invention to determine unshaded sample images B. i (x, y) to perform the following procedural steps: a) Specifying an approach (S103) C i (x, y) = S i (x, y)B i (x, y)+T i (x, y) for the measured tile images C i (x, y), wherein B i (x,y) an unshaded sample image of a tile i, S to be determined i (x, y) a shading function for tile i and T i (x,y) is an additive shadowing term for tile i; b) Specifying (S104; S204) a model function (P(x, y), F i (x,y); G i (x, y)) for the shadowing function S i (x,y) and a model term for the additive shadowing term T i (x, y), c) Determining correction factors (f k ; g k ; S105, S205) at a plurality of support points ((x i , y i ), (x j ,y j )) in an overlap area of ​​adjacent tile images (C i (x, y), C j (x, y)) using the tile images C i (x, y), d) at least approximately determining (S108; S207) the shadowing function S i (x,y) and the shadowing term T i (x,y) by adjusting the model function (P(x, y); F) specified in step b). i (x, y); G i (x,y)) to the correction factors determined in step c) (f k ; g k ), e) approximate determination (S109; S208) of an unshaded sample image B i (x,y) using the shadowing function S determined at least approximately in step d). i (x, y) and the at least approximately determined shadowing term T i (x,y).

[0074] A first embodiment of the method according to the invention is described with reference to the process steps S101 to S118 reproduced below and with reference to the Fig. 2 and Fig. 3 explained.

[0075] Fig. Figure 2 shows diagrams for a first tile i=0 and a second tile i=1. The coordinate systems are shown as solid lines for the first tile i=0 and as dashed lines for the second tile i=1. The diagrams a in Fig. Figure 2 above shows the intensities of the desired sample images B0(x, y) and B1(x, y) as a function of the x-coordinate. It should be noted that the x-coordinate refers to the center of the respective tile. Diagrams b in Fig. The diagrams in the middle show the shadowing functions S0(x,y) and S1(x,y) to be determined. Diagrams c in Fig. Figure 2 below shows the data C0(x, y), C1(x, y) measured for the first tile i=0 and the second tile i=1 respectively.

[0076] Fig. Figure 3 shows five rectangular tiles of equal size, each with a width b and a height h, arranged in a rectangular grid. Areas where the central tile overlaps with an adjacent tile are shown with checkered hatching. Areas where the central tile overlaps with two adjacent tiles are shown with diagonally lined hatching. Fig. The directions of the x-coordinate and the y-coordinate are also shown. The center of the central tile lies at the origin of the coordinate system. The horizontal line g runs along the x-direction and divides the central tile horizontally in the middle. The vertical line v in Fig. Line 3 runs along the y-direction and divides the central tile vertically in the middle. Support points where correction factors are calculated are shown schematically in Fig. 3 is indicated by thick black dots. This will be described in detail below. Procedure #1 S101 Collecting measurement data with a microscope S102 Matching the measurement data of adjacent tiles (“stitching”) Result C i (x,y), i = 0,.., N - 1 S103 Approach C i (x, y) = B i (x, y)S i (x, y) S104 Approach S i (x,y) ≈ P i (x,y) = 1 + P1x + P2x 2 + p3y + p4y 2 S105 Determining correction factors according fk=1(xk,yk)=Ck(xk,yk) / BL(xk,yk), fk=2(xk,yk)=Ck(xk,yk) / BR(xk,yk), and so forth where B L and B R These can be calculated according to equations 3 and 4 (see explanation below). In total, 8 correction factors can be determined in this way for the remaining tiles overlapping in the x and y directions. k for a tile not located at the edge (see Fig. 3). S106 Approach for the shadowing function: Si(x,y)≈Fi(x,y)=1+aix+bix2+ciy+diy2 S107 Setting an index n of the approximation of the shadowing function F i (x, y) and the approximation of the desired sample image B i (x, y) on n = 1 S108 Determining a first approximation of the shadowing function Fi1(x,y) by fitting, for example using the method of least squares, of F i(x,y) to the correction factors f determined in step S105 k ; Result: Fi1(x,y) S109 Determining a first approximation of the desired sample image Bi1(x,y) by inserting the shading function approximately determined in S108 Fi1(x,y) as S i (x,y) into the approach of S103: Bi1(x,y)=Ci(x,y) / Fi1(x,y) S110 Increasing n by 1 S111 Determining updated correction factors f k as in step S105 using Bin−1(x,y) for C i (x,y) and Fin−1(x,y) for S i (x,y) S112 Determining the nth approximation of the shadowing function Fin(x,y) by adjusting (“fitting”), for example using the least squares method, a curve of the type specified in S106 to the updated correction factors f determined in S111 k ; Result: Fin(x,y) S113 Determining the nth approximation of the sample image B i (x, y) by substituting Fin(x,y) from S112 in Bin(x,y)=Bin−1(x,y) / Fin(x,y) S114 Determining the, for example Euclidean, distance d between Bin(x,y) and Bin−1(x,y) S115 if d is greater than the threshold S to be set: Continue at S110 S116 if d is smaller than the specified threshold S: S117 Perform steps S103 to S116 for all tiles i S118 Assembling the respective current approximations for the B i (x, y) to B(x', y'): Result: B(x', y')

[0077] First, using a microscope, for example the microscope according to the invention, 100 of the Fig. 1, in step S101 measurement data, namely the image data of the detector 50 for the individual tiles i with i = 0,..,N - 1, were collected.

[0078] Then, in step S102, the measurement data in the respective overlap areas of the individual tiles are structurally aligned. This corresponds to the process of "stitching". The resulting data sets are referred to as measured tile images C. i (x, y) denotes the coordinate x, where x represents a discrete integer position x of a pixel within a tiled image in the x-direction, and y represents a discrete integer position y of a pixel within a tiled image in the y-direction. For C i The following approach is then written for (x, y): Ci(x,y)=Bi(x,y)Si(x,y)+Ti(x,y).

[0079] It contains B i (x, y) the brightness generated by the sample at the tile position (x, y) without shadowing effects, S i(x, y) is the shadowing function, i.e., a multiplicative shadowing effect at the tile position (x, y) caused, for example, by vignetting and / or contamination on the detector and T. i (x,y) is the additive shadowing term, i.e., an additive shadowing effect at the tile position (x, y).

[0080] The invention recognizes that, as a rule, all shading effects are caused by a multiplicative shading effect, i.e., by the shading function S. i (x, y) can be represented. The approach thus simplifies (step S103) to Ci(x,y)=Bi(x,y)Si(x,y)

[0081] The image composed of the tiles is called a mosaic image. Without loss of generality, one can assume that the mosaic image is composed of N overlapping tiles of equal size, arranged on a regular grid. The index i of the tiles thus ranges, for example, from 0 to N-1. The i-th tile is shifted relative to the global origin by a relative displacement vector m. i = (α i m x , β i m y ) shifted with α i ,β i ∈ ℤ0, m x < w and m y < h, where w is the width and h is the height of the tiles.

[0082] The overlap of the tiles in the horizontal direction, i.e. the x-direction, is therefore d x = w - m x and the overlap of the tiles in the vertical direction, i.e. the y-direction, is d y = h - m yTypically, the tiles can have an overlap of 10%. In this case, m x = 0.9w and m y = 0.9h. A global position (x', y') in the mosaic image can therefore be written as (x',y')=(x+αimx,y+βimy)

[0083] The brightness of mosaic image C measured by the detector i (x', y') can be written as Ci(x',y')=C(x+αimx,y+βimy)

[0084] The shading function S i (x,y) is fundamentally different for each individual tile. Because the essential contributions to S i (x,y) are each given by the optical characteristics of the illumination and detection beam paths, S i (x, y) are usually similar for the individual tiles.

[0085] A typical situation is described below with reference to Fig. 2 explained. As from Fig. As can be seen from 2, the following four equations (I) to (IV) can be formulated. C0(w2−dx,y)=B0(w2−dx,y)S0(w2−dx,y) C1(−w2,y)=B1(−w2,y)S1(−w2,y) C0(w2,y)=B0(w2,y)S0(w2,y) C1(−w2+dx,y)=B1(−w2+dx,y)S1(−w2+dx,y)

[0086] In each of equations (I) to (IV), the values ​​of the quantities C on the left-hand side are known by measurement with the detector for tiles i=0 and i=1. The values ​​of the quantities on the right-hand side of equations (I) to (IV) are unknown and therefore sought.

[0087] From the further condition that adjacent tile images are structurally adapted to each other (“stitched”) as described above, the following further conditions (V) and (VI) result: B0(w2−dx,y)=B1(−w2,y):=BL B0(w2,y)=B1(−w2,−dxy):=BR

[0088] Equations (V) and (VI) reduce the number of unknown parameters B and S from eight to six unknown parameters.

[0089] In this embodiment of the method according to the invention, the following parabolic approach is used for the shadowing function S(x, y) according to step S104: S(x,y) ≈ P(x,y) = 1 + p1x + p2x 2 p3y + p4y 2At the center of the image (x, y) = (0,0) P(x,y) = P(0,0) = 1. The approach therefore assumes that the brightness value at the center of each tile is undisturbed. This is generally a good assumption. Furthermore, the same shading function P(x,y) is initially assumed for tiles i=0 and i=1. However, the algorithm can and generally will produce different approximate shading functions for tile i and tile j because the least-squares fit, which will be explained in detail later, incorporates different correction factors for the left and right image overlaps for tiles i and j.

[0090] If we first set y = 0 in this model, the two parameters p1 and p2 remain and therefore need to be determined. Using this model, the number of unknown parameters for equations (I), (II), (III), (IV) is reduced from six to four, resulting in the following system of equations for the four unknown parameters. p1,p2,B0(w2−dx,0):=BL and B0(w2,y):=BR can be solved. C0(w2−dx,0)=BLP(w2−dx,0) C1(−w2,0)=BLP1(−w2,0) C0(w2,0)=BRP0(w2,0) C1(−w2+dx,0)=BRP(−w2+dx,0)

[0091] With the sizes introduced for the sake of clarity I1:=C0(w2−dx,0) I2:=C1(−w2,0) I3:=C0(w2,0) I4:=C1(−w2+dx,0) The result for B is obtained L on the left edge of the tile and B R on the right edge of the tile: BL=8I2I3dx3−2I2I4wdx2−12I2I3wdx2−2I1I3wdx2w34dx(dx−w)(2I3dx−I4w−I3w)+ +I2I4w2dx+6I2I3w2dx+I1I3w2dx+I1I4w3−I2I3w34dx(dx−w)(2I3dx−I4w−I3w) and BR=8I2I3dx3−2I2I4wdx2−12I2I3wdx2−2I1I3wdx2w34dx(dx−w)(2I2dx−I2w−I1w)+ +I2I4w2dx+6I2I3w2dx+I1I3w2dx+I1I4w3−I2I3w34dx(dx−w)(2I2dx−I2w−I1w)

[0092] Similarly, B L and B R for x = 0 and thus in the vertical direction. With these results, correction factors can now be calculated for support points on the coordinate axes and at the respective boundaries of the overlap areas. These support points are in the Fig. 3 are highlighted by thick black dots. The correction factors for these points can be calculated as already described in step 105: fk=1(xk,yk)=Ck(xk,yk) / BL(xk,yk) fk=2(xk,yk)=Ck(xk,yk) / BR(xk,yk) where k is a running index of the support points (see location in Fig. 3) It is important to note that these results apply regardless of whether the image shows the background or the sample at the specified positions. However, one limitation is that none of these positions should contain overexposed or black pixels. For the horizontal line g, which runs through the center of the tile (i.e., at y = 0), and for the vertical line h, which runs through the center of the tile (i.e., at x = 0), this can be done for all pairs of adjacent tiles, resulting in a total of 8 corresponding correction factors.

[0093] In step S106, the shading function S to be determined is then specified. i (x, y) the parabolic approach S i (x, y) ≈ F i (x,y) = 1 + a i x + b i x 2 + c i y + d i y 2 chosen.

[0094] The eight correction factors at each of the eight positions are then used in step S108 to make a first approximation using a least-squares fit. Fi1(x,y) the shading function F i to determine (x, y). Each of the in Fig. Each of the three marked positions yields a correction factor. Correction factors are also obtained for the adjacent tile at the same positions, but these are used to determine the approximation of the shadowing function of the adjacent tile. Thus, for the four parameters a, b, c, d of the parabolic approach for the i-th tile, there are eight input values ​​that are used for the least-squares fit.

[0095] With Fi1(x,y) In step S109, a first approximation B is then made. i (x, y) for the desired sample image Bi1(x,y) calculated as follows: Bi1(x,y)=Ci(x,y) / Fi1(x,y)

[0096] Although a very simple model function is used here, it typically compensates for or reduces most of the slowly varying shadowing artifacts. The index i indicates that the function F i (x, y) is fundamentally different for each tile i. The computational effort is comparatively low compared to a pixel-wise iterative optimization of cost functions, such as those used by [Lötgering]. It is therefore relatively easy to implement the shading correction function F. i (x, y) are calculated and used individually for each tile. In many cases, the functions F calculated for the different tiles are i (x,y) but be comparatively similar. It can therefore be useful to use an average function 〈F i (x,y)〉 from the set of functions F ito calculate (x, y). Furthermore, consistency checks can be performed to determine whether the result of a single F i (x, y) for example, differs particularly strongly from the other F i (x, y) or from the averaged function 〈F i (x, y)〉 deviates. In particular, overexposed or very dark pixels can be taken into account.

[0097] Calculating the approximation of the shadowing function F i (x, y) for tiles at the left, right, top, or bottom edge of the mosaic image is made more difficult by the lack of neighboring tiles. Because the functions F i Since (x,y) are usually similar, the calculation for these tiles can be assisted by using either values ​​mirrored from opposite tile boundaries or averaged values ​​of tiles completely surrounded by other tiles for the correction factors.

[0098] From a mathematical point of view, the functions F i (x, y) are calculated from the brightness values ​​of the respective individual pixels. However, it must be taken into account that the brightness value of an individual pixel is affected by noise, and that the tiles must also be structurally aligned correctly, i.e., properly stitched. This means that overlapping luminance or brightness values ​​of the pixels must originate from exactly the same sample area. These circumstances can complicate the calculations. The influence of both problems, namely noise and proper stitching, can be mitigated by using spatially averaged brightness values ​​for C. i (x, y) for calculating the approximation of the shadowing function F i(x,y) can be reduced, for example, by calculating averaged brightness values ​​within small areas. This averaging process also makes it possible to identify and exclude overexposed or underexposed pixels, or at least to reduce their effect. Alternatively, a median of the respective measured values ​​can be calculated for the pixels in the small areas or surroundings.

[0099] It is important that the correction method described here ideally describes the shadowing artifacts for the theoretical case where all tiles have exactly the same parabolic curve P(x,y), thus making it possible to obtain the desired undisturbed brightness values ​​of sample B. i (x,y) to calculate directly.

[0100] Because the shading in real cases does not have an exactly parabolic shape, it is useful to approximate the shading function F i(x, y) and thus the sample image B i (x,y) is determined iteratively. This is explained with reference to procedure steps S110 to S116.

[0101] First, in step S110, the index n is incremented by 1. Then, in step S111, updated correction factors are determined. This is done in principle as described above, with the proviso that instead of C i (x, y) the last determined approximation, i.e. the (n - 1)th approximation of the sample image Bin−1(x,y) and instead of C i (x, y) the last determined approximation, i.e. the (n - 1)th approximation of the shadowing function Fin−1(x,y) for S i (x, y) is used.

[0102] Using the updated correction factors f determined in step S111 k In step S112, an nth approximation of the shadowing function is then performed. Fin(x,y) This is determined by fitting a curve of the type specified in step S106. This nth approximation of the shadowing function Fin(x,y) can then be used in step S113 to obtain an nth approximation of the sample image B. i (x, y) can be calculated as follows: Bin(x,y)=Bin−1(x,y) / Fin(x,y). For numerical stabilization and / or if the denominator in this equation becomes zero for a specific pixel, divisions of the form a / b can be performed in a regularized form as: a / b = a*b / (b 2 +e) with a small regularization parameter e.

[0103] To check the progress of the iteration, a distance d can then be measured in step S114, using, for example, a Euclidean metric, between Bin(x,y) and Bin−1(x,y) The calculation is performed. If a check reveals that the distance d is still larger than a predefined threshold S (step S115), the iteration continues at step S110. However, if the check reveals that the distance d is smaller than the predefined threshold S (step S116), the iteration is terminated. The interim result for tile i is then the current approximation of the sample image B. i (x, y) are approximated with an accuracy set by the defined threshold S. Following step S117, steps S103 to S116 described here are then performed for all tiles i, and in step S118 the respective current approximations for the B are then calculated. i (x, y) combined to form B(x', y').

[0104] The above explanations make it clear that even more complex functions can, in principle, be used to approximate B. i(x, y) can be used if more sampling points are used to determine the brightness values ​​in the overlapping areas, and thus more correction factors are considered. Each additional correction factor allows for two further equations with a new unknown value for B. i Formulating (x, y) at the relevant support point allows for an additional parameter of the model function. However, the resulting equations can become complex for model functions with more free parameters.

[0105] A second embodiment of the method according to the invention is described below, which allows for the creation of more complex model functions G. i to use (x, y), which requires only a few iterations and where the background of a tiled image does not need to be separated from sample regions. The model function G iThe shading function (x, y) can be defined, for example, by Zernike polynomials. For third-degree Zernike polynomials, for instance, 10 free parameters would need to be fitted.

[0106] The second embodiment of the method according to the invention is described with reference to the process steps S201 to S217 reproduced below and with reference to the Fig. 4 and Fig. 5 explained. The diagram in Fig. 4 corresponds to diagram c from Fig. 2, i.e., it shows the data C0(x, y), C1(x, y) measured for a first tile i=0 and a second tile i=1 respectively.

[0107] Fig. Figure 5 shows nine rectangular tiles of equal size. The size and arrangement of these tiles correspond to those of the tiles from [reference missing]. Fig. 3. The hatching in the overlap areas is the same as in Fig. 3. Also in Fig. Five points are support points where correction factors are calculated, indicated by thick black dots. This is described in detail below. Procedure #2 S201 Collecting measurement data with a microscope S202 Matching the measurement data of adjacent tiles (“stitching”) Result C i (x,y),i = 1,..,N S203 Approach C i (x, y) = B i (x, y)S i (x, y) S204 Approach S i (x, y) ≈ G i (x, y) with G i (x, y) Zernike polynomial S205 Determination of correction factors gk=Ci(xi,yi) / Cj(xj,yj) for a specific tile i for points at the edges of the overlap areas ( Fig. 5), where j denotes the adjacent tiles of tile i S206 Setting the index n of the approximation of a shadowing function Gin(x,y) and the approximation of the sought-after sample image B i (x, y) on n = 1 S207 Determining a first approximation of the shadowing function Gi1(x,y) by fitting, for example using the method of least squares, of G i (x, y) to the correction factors g determined in S205 k ; Result: Gi1(x,y) S208 Determining a first approximation of the sought sample image B i (x, y) by substituting the shadowing function G determined approximately in S207 i (x, y) as S i (x,y) into the approach of S204: Bi1(x,y)=Ci(x,y) / Gi1(x,y) S209 Increasing n by 1 S210 Determining updated correction factors g k as in S205 using Bin−1(x,y) for C i (x, y) S211 Determining the nth approximation of the shadowing function Gin(x,y) by fitting, for example using the least squares method, a curve of the type specified in step S104 to the updated correction factors g determined in S211 k ; Result: Gin(x,y) S212 Determining the nth approximation of the sample image Bin(x,y) by inserting Gin(x,y) from S211 in Bin(x,y)=Bin−1(x,y) / Gin(x,y) S213 Determining the, for example Euclidean, distance d between Bin(x,y) and Bin−1(x,y): S214 if d is greater than the threshold to be set S: Continue at S209 S215 if d is smaller than the specified threshold S: S216 Perform steps S203 to S215 for all tiles S217 Assembling the respective current approximations for the B i (x, y) to B(x',y'): Result: B(x',y')

[0108] Steps S201 to S203 are identical to steps S101 to S103 of the first method variant described above. In step S204, the shading function S i A Zernike polynomial, for example of degree 3, is chosen as the model function for (x,y). It is advantageous to select a suitable domain for the Zernike polynomial and / or to normalize the correction values ​​appropriately in order to describe a rectangular shadowing function, corresponding to the measured image, as the values ​​of a Zernike polynomial. For example, the radial term components of the Zernike polynomial can be normalized to the length of the image diagonal.

[0109] In step S205, correction factors g are then applied. k = C i (x i ,y i ) / C j (x j ,y j ) determined for points at the edges of the overlapping areas. These points are in Fig. 5. The central tile is schematically represented by thick black dots. The calculation of these correction factors is illustrated using the diagram in [reference to diagram]. Fig. 4 explained. For the left tile i = 0, a correction factor g can be applied at position (w / 2, y). 0R calculated at the right edge of the tile as g0R=C1(−w / 2+dx,y) / C0(w / 2,y)

[0110] This correction factor is designed so that by multiplying it by the brightness value of the pixel at the boundary position, its brightness value is corrected to match the brightness value of the overlapping tile at the same position. This is indicated by an arrow in Fig. 4 indicated.

[0111] Accordingly, a correction factor g can be applied to the right tile i = 1. 1L calculated at the left edge of the tile as g1L=C0(w / 2−dx,y) / C1(−w / 2,y)

[0112] The effect of the correction factors g 1L and g0R is in Fig. 4 each illustrated by a vertical arrow. The support points of the correction factors g 0R and g 1L are also in Fig. 5 shown. It is important that g 0R and g 1L Examples of correction factors g k The indices 0R and 1L are intended to indicate the respective locations of the support points.

[0113] Such correction factors g k These correction factors can then be calculated for many y-positions along the vertical boundaries of the overlapping tiles. Similarly, corresponding values ​​can also be calculated for the horizontal boundaries. The support points for which these correction factors for the central tile are calculated are in Fig. 5 represented by black dots. As with method variant #1, it is useful for the robustness of the calculation to use spatially averaged brightness or luminance values, i.e., averaged over several pixels.

[0114] It is also possible to add correction factors to the positions of diagonally oriented tiles. These overlap areas are in Fig. Figure 5 is shown with diagonal hatching. Furthermore, it is possible to add additional correction factors that are not located at the tile edges. For example, one or more correction factors with a value of 1.0 can be added in or near the center of the tile. This means that the function values ​​of the model function at these locations should tend towards these additional correction factors as iteration progresses.

[0115] If the number of calculated correction factors is greater than the number of free parameters of the model function to be adjusted, the model function can be determined via a least-squares fit. The model function can then be used to determine correction factors for all pixels within the tile. By applying these correction factors to the brightness values ​​C i (x i ,y i The remaining brightness deviations can be reduced at the tile boundaries. This has the advantage that such an adjustment of the brightness values ​​occurs naturally across the entire tile area. Therefore, it is possible to achieve good correction of shading artifacts even in mosaic images with small overlaps.

[0116] In step S207, a first approximation of the shading function can then be made. Gi1(x,y) determined by adjusting (“fitting”) the model function G specified in step S204 i (x, y), for example using the method of least squares, to the determined correction factors g k .

[0117] Substituting the shadowing function G, which is approximately determined in S207 i (x, y) as S i Substituting (x,y) into the approach of step S204 then provides a first approximation of the desired object function. Bi1(x,y):Bi1(x,y)=Ci(x,y) / Gi1(x,y).

[0118] The iteration of the procedure in the following steps S209 to S212, the verification of the progress of the iteration by determining a distance d between Bin(x,y) and Bin−1(x,y) In steps S213 and S214, the termination of the process with sufficient accuracy in step S215, the determination of the sample images for all tiles in step S216 and the assembly of the mosaic image from the individual tile images in step S217 then take place analogously to the steps S110 to S118 of the first embodiment of the inventive method described above.

[0119] Because the model function is typically a slowly varying function and its free parameters are calculated using a least-squares fit, the result will not completely eliminate the luminance mismatch. However, this process can be iteratively applied to the resulting sample image as described to reduce the luminance mismatch with each iteration. The process usually converges after just a few iterations, for example, three or four.

[0120] It is also possible to first iterate this procedure for a model function for the shadowing function with L1 free parameters and then for a more complex model function with more free parameters L2 > L1.

[0121] After calculating the corrected mosaic images according to procedure variant #1 and / or according to procedure variant #2, mean values ​​〈F i (x, y)〉, 〈G i (x, y)〉 of the determined approximate shadowing functions G i (x, y), F i (x, y) are calculated, which can be used as a reference for future acquisitions. This offers the advantage that this type of reference data for the shadowing functions is calculated from a typical sample acquisition and not from an artificial calibration sample. In particular, this allows correction even for single images without having to acquire an image of several or all tiles.

[0122] The present invention proposes a novel, robust retrospective method for correcting shadowing artifacts in tiled images. In particular, two method variants for compensating spatially slowly changing shadowing artifacts in tiled images are described in detail. Both employ image processing methods that utilize only the overlapping tile regions and do not require pre-calibration or the acquisition of shadow reference images. The new method is independent of any distinction between background and foreground regions. Reference symbol list 2 Sample 10. Radiation source, light source, e.g. laser or LED source 10 Microscope stand 12 Illumination radiation, illumination light, excitation light 16 emission radiation or emission light emitted from sample 2 into sample area 1 40 microscope objective 41 optical axis of the microscope objective 40 44 mechanical drive 50 Detector 90 Control unit 100 optical devices according to the invention, microscope (0,0) Origin of the xy-coordinate system a i Parameters of the parabolic model function F i (x, y) b i Parameters of the parabolic model function F i (x, y) B i (x, y) Brightness generated by the sample at the tile position (x, y) without shadowing effects, unshaded sample image (n - 1)th approximation of B i (x, y) nth approximation of B i (x, y) B i (x', y') = B(x + a i m x , y + b i m y Mosaic image B L Value of sample image B i (x,y) on a left edge of the tile i B R Value of sample image Bi (x,y) on a right edge of the tile i B i (x', y') Brightness generated by probe at global position (x', y') without shadowing effects c i Parameters of the parabolic model function F i (x, y) C i (x,y) brightness measured by the detector at the tile position (x, y) in the i-th tile C i (x', y') = C(x + a i m x ,y + b i m y ) measured brightness of the mosaic image d distance between Bin(x,y) and Bin−1(x,y) d i Parameters of the parabolic model function F i (x, y) d x = w - m x Tile overlap in the x-direction d y = w - m y Tile overlap in the y-direction f k Correction factors F i(x,y) Model function for shadowing function, parabolic function (n - 1)th approximation of the shadowing function F i (x, y) nth approximation of the shadowing function F i (x, y) 〈F i (x, y)〉 Mean of the approximately determined shadowing functions F i (x, y) g horizontal direction at y=0 within a tile G k Correction factors G i (x,y) Model function for shadowing function, for example Zernike polynomial (n - 1)th approximation of the shadowing function G i (x, y) nth approximation of the shadowing function G i (x,y) <G i (x, y)〉 Mean of the approximately determined shadowing functions G i (x,y) h Height of a tile image i Index of tiles j Index of tiles k Index of support points m i = (αi m x , β i m y ) Shift vector for i-th tile in the mosaic image m x Length of lattice vector in the x-direction m y Length of lattice vector in the y-direction p1 Parameter of the parabolic function P(x, y) p2 Parameter of the parabolic function P(x, y) p3 Parameters of the parabolic function P(x, y) p4 Parameters of the parabolic function P(x, y) P(x,y) Model function for shadowing function, parabolic function S threshold for distance from Bin−1(x,y) and Bin(x,y) S i (x, y) multiplicative shadowing function, multiplicative shadowing effect at the tile position (x, y) caused e.g. by vignetting T i (x, y) and / or contamination on the detector, shadow correction function additive shadow term v vertical direction at x=0 within a tile w width of a tile image x first lateral coordinate in the sample plane x discrete integer position of a pixel within a tiled image in the x-direction x' first global lateral coordinate (x', y') = (x + a i m x , y + b i m y ) global position (x', y') in mosaic image (x i , y i )| k Support points (x j , y j )| k Support points y second lateral coordinate in the sample plane y discrete integer position of a pixel within a tile image in the y-direction y' second global lateral coordinate α i ∈ ℤ0 β i ∈ ℤ0 δ Distance of a certain approximately determined shadowing function F i (x, y) or G i(x, y) to a mean value of the approximately determined shadowing functions 〈F i (x, y)〉 or 〈G i (x, y)〉 References [Wu] Q. Wu, FA Merchant, KR Castleman, 'Microscope Image Processing', Cap. 12.5.3 - 12.5.6, Academic Press, Elsevier, ISBN 978-0-12-372578-3 [Young] Young, lan T. “Shading correction: compensation for illumination and sensor inhomogeneities.” Current Protocols in Cytometry 14, no. 1 (2000): 2-11. [De Silva] V. De Silva, V. Chesnokov, D. Larkin, “A Novel Adaptive Shading Correction Algorithm for Camera Systems,” 2016 Society for Imaging Science and Technology, DOI: 10.2352 / ISSN.2470-1173.2016.18.DPMI-249 [Preibisch] Preibisch, Stephan, Stephan Saalfeld, and Pavel Tomancak. “Globally optimal stitching of tiled 3D microscopic image acquisitions.” Bioinformatics 25.11 (2009): 1463-1465. [Artists] Actors, Fisseha Bekele, Olga Chernavskaia, Sandro Heuke, Thomas Bocklitz, Tobias Meyer, Jürgen Popp, and Rainer Heintzmann. "Seamless stitching of tile scan microscope images." Journal of microscopy 258, no. 3 (2015): 223–2 [Smith] Smith, Kevin, Yunpeng Li, Philip Piccinini, Gabor Csucs, Csaba Balazs, Alessandro Bevilacqua, and Peter Horvath. “CIDRE: an illumination-correction method for optical microscopy.” Nature methods 12, no. 5 (2015): 404–4 [Peng] Peng, Tingying, Kurt Thorn, Timm Schroeder, Lichao Wang, Fabian J. Theis, Carsten Marr, and Nassir Navab. „A BaSiC tool for background and shading correction of optical microscopy images.“ Nature communications 8, no. 1 ( 2017 ): 14836 . [Loetgering] Deutsche Patent Notice 10 2024 124 248.5 ZITATE ENTHALTEN IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2024 124 248.5

[0122] Cited non-patent literature

[0000] Wu] Q. Wu, FA Merchant, KR Castleman, 'Microscope Image Processing', Cap. 12.5.3 - 12.5.6, Academic Press, Elsevier, ISBN 978-0-12-372578-3

[0122] Young] Young, Lan T. “Shading correction: compensation for illumination and sensor inhomogeneities.” Current Protocols in Cytometry 14, no. 1 (2000): 2-11

[0122] De Silva] V. De Silva, V. Chesnokov, D. Larkin, “A Novel Adaptive Shading Correction Algorithm for Camera Systems,” 2016 Society for Imaging Science and Technology, DOI: 10.2352 / ISSN.2470-1173. Preibisch] Preibisch, Stephan, Stephan Saalfeld, and Pavel Tomancak. “Globally optimal stitching of tiled 3D microscopic image acquisitions.” Bioinformatics 25.11(2009):1463–1465

[0122] Legesse] Legesse, Fisseha Bekele, Olga Chernavskaia, Sandro Heuke, Thomas Bocklitz, Tobias Meyer, Jürgen Popp, and Rainer Heintzmann. "Seamless stitching of tile scan microscope images." Journal of microscopy 258, no. 3 (2015): 223–2 Smith] Smith, Kevin, Yunpeng Li, Philip Piccinini, Gabor Csucs, Csaba Balazs, Alessandro Bevilacqua, and Peter Horvath. “CIDRE: an illumination-correction method for optical microscopy.” Nature methods 12, no. 5 (2015): 404–4 Peng] Peng, Tingying, Kurt Thorn, Timm Schroeder, Lichao Wang, Fabian J. Theis, Carsten Marr, and Nassir Navab. „A BaSiC tool for background and shading correction of optical microscopy images.“ Nature communications 8, no. 1 (2017): 14836

[0122]

Claims

[1] Method for processing tiled images, wherein a plurality of overlapping tiled images C are processed by an optical device (100). i (x,y) of a sample (2) is recorded (S101), characterized by , that to determine unshaded sample images B i (x, y) the following procedure steps are carried out: a) Specifying an approach (S103) C i (x,y) = S i (x, y)B i (x, y)+T i (x,y) for the measured tile images C i (x,y), wherein B i (x, y) an unshaded sample image of a tile i to be determined, S i (x, y) a shading function for tile i, T i (x, y) an additive shadowing term for tile i and and i is a running index of the tiles; b) Specifying (S104; S204) a model function (P(x, y), F i (x,y); G i(x,y)) for the shadowing function S i (x,y) and a model term for the additive shadowing term T i (x, y), c) Determining correction factors (f k ; g k ; S105; S205) at a plurality of support points ((x i ,y i )| k , (x j ,y j )| k ) in an overlap area of ​​adjacent tile images (C i (x,y), C j (x,y)) using the tile images C i (x, y), d) at least approximately determining (S108; S207) the shadowing function S i (x,y) and the shadowing term T i (x,y) by adjusting the model function (P(x, y), F) specified in step b). i (x,y); G i (x,y)) to the correction factors determined in step c) (f k ; g k ), e) approximate determination (S109; S208) of an unshaded sample image B i(x,y) using the shadowing function S determined at least approximately in step d). i (x, y) and the at least approximately determined shadowing term T i (x, y). [2] Method according to claim 1, characterized by , that the model function (P i (x, y), F i (x, y)) is a parabolic function in the x- and / or y-coordinate. [3] Method according to claim 1 or 2, characterized by , that the model function 〈G i (x, y)) has at least one of the following polynomials or is realized by at least one of the following polynomials: Zernike polynomial, two-dimensional Hermite polynomial, two-dimensional Laguerre polynomial, Jacobi polynomial multivariate Bernstein polynomial. [4] Method according to any one of claims 1 to 3, characterized by , that the correction factors (f k ; g k ) for support points ((x i, y i )| k , (x j ,y j )| k ) are calculated, which are located at the edges of overlapping areas of the tile images C i (x,y) lie. [5] Method according to any one of claims 1 to 4, characterized by , that the correction factors (f k ; g k ) for support points ((x i , y i )| k , (x j ,y j )| k ) are calculated, which lie on the coordinate axes (x, y). [6] Method according to any one of claims 1 to 4, characterized by , that the correction factors (f k ; g k ) for a large number of support points ((x i , y i )| k , (x j , y j )| k ) are calculated at points on the edge of a tile (i, j). [7] Method according to any one of claims 1 to 6, characterized by , that for tiles (i) at the edge of a mosaic image (B i(x', y')), where there are no overlapping areas with adjacent tiles, correction factors obtained for support points at opposite edges (f k ; g k ) for the edges of the tiles in question (i) where there are no adjacent tiles. [8] Method according to any one of claims 1 to 7, characterized by , that for the edges of tiles (i) at the edge of a mosaic image (B i (x', y')), where there are no overlapping areas with adjacent tiles, mean values ​​of correction factors (f k ; g k ) of tiles (i) that are completely surrounded by other tiles (i). [9] Method according to any one of claims 1 to 8, characterized by , that using the model function (P(x, y), F i (x, y); G i (x, y)) and assuming that the measured tile images C i(x, y) of adjacent tiles (i, j) are structurally adapted to each other, values ​​(B L , B R ) of sample image B i (x, y) are calculated at support points on the coordinate axes and at the boundaries of the overlap areas. [10] Method according to claim 9, characterized by , that the correction factors (f k ) Quotients of values ​​of the tile images C i (x, y) and the values ​​(B L , B R ) of sample image B i (x, y) are at support points on the coordinate axes and at the boundaries of the overlap areas. [11] Method according to any one of claims 1 to 10, characterized by , that the correction factors (g k ) Quotients of values ​​of the tile images (C i (x,y),C j (x,y)) of adjacent tiles (i,j). [12] Method according to any one of claims 1 to 11, characterized by , that in central areas of the tile images C i(x, y) additional support points are inserted, where the function value of the model function (P(x, y), F i (x,y); G i (x, y)) is set to a suitable value, for example to 1, at these additional support points. [13] Method according to any one of claims 1 to 12, characterized by , that in step d) the approximate determination of the shadowing function S i (x,y) and in step e) the approximate determination of the sample images B i (x,y) is performed iteratively (S110-S113; S209-S212) by determining (S111; S210) updated correction factors (f k ; g k ) using one (n - 1)th approximation of the shadowing function in each case (Fin−1(x,y);Gin−1(x,y)) and / or an (n - 1)th approximation of the sample images (Bin−1(x,y)), Determine (S112; S211) an nth approximation of the shadowing function (Fin(x,y);Gin(x,y)) by adjusting the correction factors (f k ; g k ) to the respective given model function 〈F i (x, y); G i (x, y)) and Determine (S113; S212) an nth approximation of the unshaded sample images (Bin(x,y)) using the nth approximation of the shadowing function (Fin(x,y);Gin(x,y)). [14] Method according to claim 13, characterized by , that a distance (d) between Bin(x,y) and Bin−1(x,y) is determined that the iteration continues if the distance (d) is greater than a predetermined threshold (S), and that the iteration is terminated when the distance (d) is smaller than the specified threshold (S). [15] Method according to one of claims 13 or 14, characterized by, that the iterative procedure is first carried out for a model function for the shadowing function with a first number of free parameters and subsequently for a more complex model function with a second number of free parameters that is larger than the first number. [16] Method according to any one of claims 1 to 15, characterized by , that the resulting sample images B i (x, y) to form a mosaic image (B i (x',y')) can be composed. [17] Method according to any one of claims 1 to 16, characterized by , that the shading function S i (x, y) is calculated at least approximately individually for each tile (i). [18] Method according to any one of claims 1 to 17, characterized by , that it is checked how far a single one of the approximately determined shadowing functions (F) i (x, y); G i (x, y)) of a mean (〈F i (x,y)〉; 〈G i(x, y)〉) deviates from the approximately determined shadowing functions, and that an approximately determined shadowing function (F i (x,y); G i (x, y)) by the mean (〈F i (x,y)〉; 〈G i (x, y)〉) is replaced when it differs from the mean (〈F i (x,y)〉; 〈G i (x, y)〉) differs by more than a threshold (δ) to be determined. [19] Method according to any one of claims 1 to 18, characterized by , that for the approximate calculation of the shadowing function 〈F i (x, y); G i (x,y)) are evaluated as groups of several adjacent pixels. [20] Method according to claim 19, characterized by , that for each group of several neighboring pixels, a mean or median of the data measured by the pixels is calculated. [21] Method according to any one of claims 1 to 20, characterized by, that the optical device is a microscope (100). [22] Optical device with a detector (50) for detecting emission radiation (16) emitted from a sample (2) in a sample area (1), with a detection beam path with an imaging optic (40) for guiding the emission radiation (16) onto the detector (50), with a mechanical drive (44) for setting a relative lateral position between the sample (2) and the imaging optics (40) with respect to an optical axis (41) of the imaging optics (40), with a control unit (90) configured to control the mechanical drive (44) and to evaluate measurement data from the detector (50), wherein the control unit (90) is configured to control the optical device (100) for recording (S101) a plurality of overlapping tile images C i (x, y) of the sample (2) to target, characterized by , that the control unit (90) is configured to determine unshaded sample images B i (x,y) to perform the following procedural steps: a) Specifying an approach (S103) C i (x,y) = S i (x, y)B i (x, y)+T i (x,y) for the measured tile images C i (x,y), wherein B i (x, y) an unshaded sample image of a tile i, S to be determined i (x,y) a shading function for tile i and T i (x, y) is an additive shadowing term for tile i and i is a running index of the tiles; b) Specifying (S104; S204) a model function (P(x,y), F i (x,y); G i (x,y)) for the shadowing function S i (x,y) and a model term (0) for the additive shadowing term T i (x, y), c) Determining correction factors (f k ; g k; S105, S205) at a plurality of support points ((x i , y i )| k , (x j , y j )| k ) in an overlap area of ​​adjacent tile images (C i (x,y), C j (x,y)) using the tile images C i (x, y), d) at least approximately determining (S108; S207) the shadowing function S i (x,y) and the shadowing term T i (x,y) by adjusting the model function (P(x,y), F) specified in step b) i (x,y); G i (x,y)) to the correction factors determined in step c) (f k ; g k ), e) approximate determination (S109; S208) of the unshaded sample images B i (x, y) using the shadowing function S determined at least approximately in step d). i (x,y) and the at least approximately determined shadowing term T i (x, y). [23] Optical device according to claim 22, characterized by , that the control unit (90) is configured to carry out the method according to any one of claims 2 to 21. [24] Optical device according to one of claims 22 or 23, designed as an aerial camera. [25] Optical device according to one of claims 22 or 23, which is designed as a microscope (100) and has a radiation source (10) for providing illumination radiation (12) and an illumination beam path for directing the illumination radiation (12) into the sample area (1), wherein the imaging optics (40) has a microscope optic. [26] Optical device according to claim 25, characterized by , that the microscope optics have at least one microscope objective (40). [27] Microscope according to one of claims 25 or 26, which implements at least one of the following microscopes Electron microscope, Ion microscope, Digital microscope Fluorescence microscope, Light microscope transmitted light microscope reflected light microscope Wide-field microscope scanning microscope confocal microscope, Light field microscope Light-sheet microscope, TIRF microscope, SIM microscope.

Images