Method and optical apparatus for processing tiled image

Abstract

The invention relates to a method and an optical device for processing tiled images, wherein a plurality of overlapping tile images of a sample are captured using the optical device. The method is characterized in that a model function is specified for a specified relation, in particular for a shadow function, correction factors at support positions in the overlapping region of adjacent tile images are determined, and a shadow function and a shadow-free sample image are at least approximately determined in each case.

Description

Methods and optical devices for processing tiled images Technical Field

[0001] In a first aspect, the present invention relates to a method for processing a tiled image as described in the preamble of claim 1. In another aspect, the present invention relates to an optical device as described in the preamble of claim 22. Background Technology

[0002] In categorized methods for processing tiled images, multiple overlapping patch images of the sample are captured using optical equipment. .

[0003] The optical device of this category comprises at least the following components: a detector for detecting emitted radiation from a sample within a sample region; a detection optical path having imaging optics for guiding the emitted radiation to the detector; a mechanical actuator for adjusting the relative lateral positioning of the sample and the imaging optics about the optical axis of the imaging optics; and a control unit configured to at least drive the mechanical actuator and evaluate measurement data from the detector. The control unit is also configured to drive the optical device to capture multiple overlapping patch images of the sample. .

[0004] In modern microscopy, large areas of a sample are typically imaged at high resolution by taking images at different locations using a digital camera or other type of detector (e.g., a line scan system). The locations of individual images are usually chosen such that they form a regular grid, and the fields of view (FOV) of adjacent individual images overlap by a specific amount, such as 10% of the FOV width or 10% of the FOV height. These captured images are usually called tiles, and the collection of entire tiles is called a tiled image or mosaic image. In this specification, the terms "tile" and "tile image" are used synonymously. Such tile images provide a wide field of view and high resolution, but two types of artifacts exist that often make further analysis difficult.

[0005] Type I artifacts are caused by inaccuracies in the mechanical devices used to position the sample relative to the microscope objective, resulting in visible mismatches in the imaged sample structure at patch boundaries. This is typically corrected using a so-called "stitching" algorithm: the imaged sample structure is studied in overlapping areas to identify positioning deviations between adjacent patches, and appropriate offsets are calculated along the horizontal (x-direction) and vertical (y-direction) directions for slightly offset patches, thus ensuring the imaged sample structure matches as closely as possible at patch boundaries. The "stitching" process will be referred to below as structural matching of adjacent patches.

[0006] The second type of artifact is caused by brightness differences between adjacent patches and patch edges. This brightness difference, also known as optical density difference or illuminance difference, can have several causes: halos, which may be due to the design of the imaging optical path (i.e., the illumination path and / or probe path), non-uniform illumination, or the location- and / or time-varying sensitivity of the detector. The sample itself can also be a source of this artifact: in fluorescence imaging, fading may occur in overlapping areas; in transmitted light or fluorescence imaging, different transmission / emission characteristics of the sample holder or embedding medium can have an effect; patches with strongly reflective structures tend to appear brighter at patch edges than at the edges of adjacent patches (when the sample does not have reflective structures in that area). These artifacts mostly exhibit slow spatial variations. Other artifacts with abrupt spatial variations within a patch may be caused by dust in the optical path, defects in the detector or image sensor, or dust directly on or at the detector or image sensor. All these undesirable effects on the overall image illumination are generally referred to as shadow artifacts.

[0007] Image correction algorithms used to eliminate or correct these shadow artifacts are often called deshading algorithms or background shadow correction algorithms [Wu], and can be broadly classified into prospective measures and retrospective measures.

[0008] In prospective approaches, shadow reference images from calibration samples are detected. These calibration samples exhibit the same or similar artifacts as samples detected later, but do not contain additional structures [Young, De Silva]. With this shadow reference image, density artifacts can be compensated for using simple multiplicative or additive correction schemes. While these algorithms can handle both spatially slow and spatially abrupt illuminance changes and can be applied to non-overlapping tile images, they have some drawbacks: it can be difficult, or sometimes even impossible, to reproduce the shadow effects shown in the real sample through the calibration pattern. Furthermore, this approach does not work well when shadow artifacts are related to the tile content itself. To overcome the limitations when using calibration samples, a new approach has recently been proposed: deriving the shadow correction image from the sample image itself. However, this approach requires a special detection scheme with large tile overlap [Loetgering].

[0009] Retrospective measures [Legesse, Smith, Peng] attempt to compensate for shadow artifacts by using the detected tile image data itself. Some measures apply illumination correction [Legesse] only to overlapping areas. In mosaic images with minimal overlap, and when shadow effects extend beyond the overlapping areas, this approach yields undesirable results.

[0010] Some approaches attempt to separate shadow artifacts from sample structure by averaging the image data of all patches, using a low-pass filter, or requiring user input to separate background and sample areas [Smith, Peng]. Other approaches improve or integrate the scheme by automatically identifying object and background areas and by averaging only the background area [Legesse]. All these methods using averaging are only effective when the following conditions are met: the patch image must contain a sufficient number of patches; the sample must be sufficiently homogeneous; and it must be possible to separate the background and sample areas. If only a small number of patches are available for averaging, or if the sample contains prominent edges or areas with significant density differences, the sample structure will distort the shadow estimation.

[0011] Some methods [Wu] model shadowed backgrounds by matching 2D surface functions using least-squares matching of background pixel intensities or the average intensity of background regions. In this approach, the tile image must also contain a sufficient number of background regions suitable for least-squares matching.

[0012] The retrospective measures described are based on an unsatisfactory situation, namely, the quality of the calculated shading correction data is highly dependent on the sample characteristics. Summary of the Invention

[0013] The objective of this invention is to describe a method for processing tiled images and an optical device in which shadow artifacts can be reduced with relatively low computational cost and high-quality mosaic images can be obtained.

[0014] This task is accomplished by a method having the features of claim 1 and an optical device having the features of claim 22.

[0015] According to the present invention, the method of the above type is improved in the following way: to obtain a shadowless sample image Perform the following steps: a) For the measured patch image Pre-given relation ,in, It is a tile The unshaded sample images that are yet to be obtained. It is for tiles The shadow function, It is for tiles The additive shaded items, and a) Consecutive indices of each tile; b) Pre-defined shading function. Model functions and shaded terms for additive properties Model items; c) using tile images d) Determine correction coefficients at multiple support locations in the overlapping region of adjacent tile images; d) Determine the shadow function at least approximately by matching the model function pre-given in step b) with the correction coefficients determined in step c). and shaded items e) By using the shadow function that was at least approximately determined in step d). and at least approximately determined shaded terms To approximately determine the shadowless sample image. .

[0016] According to the present invention, the optical device of the above type is improved by configuring the control unit to perform steps a) to e) provided by the method according to the present invention to obtain a shadowless sample image. .

[0017] Advantageous embodiments of the method for processing tiled images according to the invention and the optical device according to the invention will be described below, particularly in conjunction with the dependent claims and drawings.

[0018] The optical device can be any type of optical device capable of capturing images of a sample. For example, the optical device can be constructed as an aerial camera. This aerial camera can, for example, be deployed on a satellite to capture images of the Earth or space.

[0019] In a particularly preferred embodiment, the optical device is configured as a microscope and has a radiation source for providing illumination radiation and an illumination optical path for directing the illumination radiation to a sample region, wherein the imaging optics have microscopic optics. The microscope can, in principle, be any type of microscope that applies illumination radiation to a sample. For example, the microscope can be an electron microscope or an ion microscope. In a particularly preferred embodiment, the microscope is an optical microscope, and the microscopic optics have at least one microscope objective. The microscope can, for example, perform the functions of at least one of the following microscopes: digital microscope, fluorescence microscope, optical microscope, transmitted light microscope, reflected light microscope, wide-field microscope, scanning microscope, confocal microscope, light field microscope, light sheet microscope, TIRF microscope, SIM microscope, electron microscope, and ion microscope.

[0020] Accordingly, in a preferred variant of the method for processing tiled images according to the present invention, a microscope is used as the optical device.

[0021] A radiation source can be any source capable of providing a desired type of radiation with one or more desired wavelengths and appropriate intensity. For example, a radiation source can be an electron source or an ion source.

[0022] In a preferred embodiment, the radiation source is a light source. In principle, the light source can be any light source capable of providing illumination light with one or more desired wavelengths and appropriate intensity. For example, the light source can be a laser or an LED module. The excitation light can be coherent, at least partially coherent, or incoherent. Illumination light is electromagnetic radiation, particularly electromagnetic radiation in and near the visible spectrum. Illumination light can also be referred to as excitation light; in this specification, the two terms are used largely synonymously.

[0023] The radiation emitted from the sample under study due to illumination is called emitted radiation.

[0024] For example, light emitted from a sample under study by illumination or excitation light is called emitted light, and it reaches a detector (e.g., a camera) via a detection optical path. Since this light can be called emitted light, it is only required that the light is emitted from or at least originates from the illuminated sample. Typically, emitted light can be fluorescence emitted or emitted by the sample, especially pigment molecules present there, after being illuminated by excitation light. Emitted light can also be illumination light that has been reflected, transmitted, or scattered. The only requirement for the principle of contrast is that the sample emits emitted light after being illuminated by excitation light. The sample can also be referred to as a slice.

[0025] The term "illumination path" refers to all components, especially optical components, that guide and modify the radiation beam, such as lenses, mirrors, prisms, gratings, filters, apertures, and beam splitters, which guide excitation radiation (especially excitation light) from a radiation source (especially a light source) to the sample under study. The illumination path may include an illumination objective. Both the illumination objective and the microscope objective can be microscope objectives of known types. In principle, the illumination objective and the microscope objective can also be separate objectives. However, in a preferred embodiment, the illumination objective and the microscope objective are a single, identical objective.

[0026] The term "sample space" refers to the area in the space where the sample to be studied can be placed. In a typical implementation, the sample can be placed or secured on an xy-stage, for example, by means of a sample holder or sample frame, which is laterally adjustable about the optical axis. A z-drive may be present to change the distance between the sample stage and the illumination objective or between the sample and the microscope objective.

[0027] In principle, the sample can be of any type. The method of this invention and the optical device configured as a microscope are particularly suitable for the study of biological samples.

[0028] The light emitted from the sample under study due to illumination by excitation light is called emitted radiation, and more specifically, emitted light, and travels through the probe optical path to the detector (e.g., a camera). The term "probe optical path" refers to all the components, especially optical components, that guide and modify the beam, such as lenses, mirrors, prisms, gratings, filters, apertures, and beam splitters, using and via these components to guide the emitted radiation from the sample under study to the camera. Suitably, the sensor plane of the detector can be arranged in a plane optically conjugate to the focal plane of the microscope objective.

[0029] The field of view of the probe optical path refers to the area in the sample plane that can collect emitted radiation (especially emitted light) and redirect it to the detector in the lateral direction. The size of the field of view is usually determined by the optical parameters (such as aperture and magnification) of the microscope objective and other components in the probe optical path (such as the tube lens).

[0030] The type of detector used to detect emitted radiation is generally related to the type of optical equipment, and especially to the type of microscope. In embodiments of the invention, the detector may include a two-dimensional spatially resolved photodetector (e.g., a camera), a one-dimensional spatially resolved detector (e.g., a linear detector array), or a single photodetector (e.g., a point photodetector). Specifically, the detector may include at least one of the following elements or components: a CCD element, a CMOS element, a SPAD element, or a PMT.

[0031] A mechanical actuator for adjusting the relative lateral positioning between the sample and the microscope objective can be configured to move the sample relative to the optical axis (e.g., the microscope objective), and / or configured to move the microscope objective, and thus its optical axis, relative to the sample. The mechanical actuator can be secured to a microscope support. The microscope support can be either upright or inverted.

[0032] The term "control unit" refers to all hardware and software components that work in conjunction with the components of the optical device according to the invention to perform the conventional functions of those components. In particular, the control unit may include a computing device (e.g., a PC) and a camera control unit capable of reading measurement signals. The detector's measurement data is the measurement data generated when the detector is illuminated by emitted light.

[0033] The term "optical axis" refers to the optical axis of an imaging optics (especially a microscope objective) in the probe optical path.

[0034] Relative lateral positioning refers to the lateral positioning of the sample relative to the optical axis. If, by convention, the coordinate system is chosen so that the optical axis coincides with the z-axis, then the lateral coordinate directions are the x and y directions. The lateral coordinate directions specifically refer to the directions in which the patch and tiled image extend.

[0035] Overlapping tile images mean that the tiles are not connected by their edges, but rather overlap, specifically along two horizontal coordinate directions. The overlapping area of ​​adjacent tile images refers to the region along these horizontal coordinates where adjacent tiles overlap.

[0036] The term "shading-free sample image" "This refers to the sample image that can be theoretically obtained when the aforementioned shadow artifacts do not exist for each tile.

[0037] For a certain tile shadow function And additive shaded items It is a scalar function of the horizontal coordinate. The support position is a point in the tile plane unfolded through the horizontal coordinate. The support position refers to its location within the field of view plane. The correction factor is a scalar value.

[0038] This invention first recognizes that the aforementioned shadow artifacts can generally be modeled using mathematically relatively simple model functions and model terms.

[0039] Furthermore, the basic concept of the invention as described herein is that correction coefficients are obtained from measurement data, which enable effortless matching of model functions and addition model terms.

[0040] A significant advantage of this invention is that shadow artifacts, at least those that change slowly in space, can be corrected well and with little computation (i.e., quickly).

[0041] The method according to the invention can be performed using the optical device according to the invention. The method for microscopy according to the invention can be performed using the optical device according to the invention configured as a microscope. The optical device according to the invention, and in particular its control unit, can be advantageously configured to perform variations of the method according to the invention described herein.

[0042] While it is feasible in principle to also include additive shaded terms in the method according to the invention, Perform a matching. However, in practice, it has also been shown that the shadow function can be determined at least approximately. That is sufficient. This means that, in advantageous variations of the method according to the invention, zero can be chosen as the shaded term for additiveness. The model items.

[0043] The model function can be specified by multiple parameters.

[0044] In a relatively simple preferred variant of the method according to the invention, the model function is a parabolic function in the x-coordinate and / or y-coordinate.

[0045] In more complex variations that can more accurately match the shadow function, the model function is a Zernike polynomial. For example, the model function could be a third-order Zernike polynomial. Generally, choosing a function with an orthogonal normalized basis is advantageous for the model function. Functions providing an orthogonal normalized basis are well-suited because least-squares approximation can then be determined via a simple scalar product. Examples of orthogonal normalized basis systems are: Zernike polynomials, two-dimensional Hermite polynomials, two-dimensional Laguerre polynomials, and Jacobi polynomials. However, non-orthogonal polynomials are also advantageous due to their efficient evaluation capabilities arising from their polynomial representation. For example, multivariate Bernstein polynomials can also be used because they enable so-called form-preserving approximations, preferably on the simplex. If the bounded domain covers the relevant pixel region, polynomials that are in principle bound to a non-rectangular domain can also be advantageously used.

[0046] With particular preference, the model function is chosen such that the function value at the origin of the horizontal coordinate (i.e., at location (0,0)) is 1. This means that, assuming there are no halos or other adverse effects on the propagated light along the optical axis, and / or the correction at the origin is normalized to 1, and therefore all corrections are performed relative to the center where no correction occurs.

[0047] In order to achieve the method according to the present invention, in principle all that is required is that the support position for calculating the correction coefficient is located within the overlapping area of ​​the tile image.

[0048] Preferably, correction coefficients are calculated for support positions located at the edges of overlapping regions of the tile image. In particular, it is advantageous to calculate correction coefficients for support positions located on coordinate axes. However, this is not absolutely necessary. Support positions can also be located inside the overlapping region.

[0049] For more complex model functions, it is preferable to calculate correction coefficients for multiple support locations at points on the edge of the tile. For example, these support locations can be positioned at equal distances along the edge of the tile.

[0050] Furthermore, it is advantageous to calculate a correction factor for at least one support location in each overlapping region. It is also possible to calculate correction factors for support locations located in regions of a tile, and particularly at the edges of a tile where the relevant tile overlaps with a diagonally adjacent tile.

[0051] For tiles at edges in a mosaic image where there is no overlapping area with adjacent tiles, correction coefficients obtained for the support positions at the respective opposite edges can be applied to the edges of the relevant tiles where there are no adjacent tiles. Alternatively or supplementarily, for edges of tiles at the edges of a mosaic image where there is no overlapping area with adjacent tiles, the average of the correction coefficients of tiles completely surrounded by other tiles can be used.

[0052] Sample images can be calculated by using a model function and assuming that the measured patch images of adjacent patches are structurally matched. The values ​​at the support locations on the coordinate axes and at the boundaries of the overlapping regions. Then, the correction factor can be calculated as the patch image at the support locations on the coordinate axes and at the boundaries of the overlapping regions. Values ​​and sample images The quotient of the values. As mentioned above, this alternative is particularly worth considering when choosing a parabolic relation for a model function.

[0053] Alternatively, the correction factor may be the quotient of the values ​​of the tile images measured from adjacent tiles.

[0054] To obtain more information, it may be advantageous to add additional support locations in the central region of the tile image, where the model function is set to appropriate values, such as 1, at these additional support locations. These additional support locations can be beneficial when matching model functions with a large number of parameters is required.

[0055] The matching of the shadow function and / or the sample image, i.e., the actual "fitting" of the known correction coefficients, can in principle be performed using known methods. For example, it is feasible to approximate at least one of the shadow function and / or the sample image by using correction coefficients achieved through least-squares matching.

[0056] In a particularly preferred variant of the method according to the invention, the shadow function is adjusted in step d). The approximate determination and the image of the shadowless sample in step e). The approximate determination is performed iteratively by using the shadow function respectively in the following manner: The first approximation and / or shadowless sample image The updated correction coefficients are determined by approximating the model function with each correction coefficient against a pre-given model function. The second approximation, and by using the shadow function's first... The first approximation is used to determine the first shadowless sample image. Second approximation.

[0057] Here, the number of free parameters used to match (“fit”) at least one of the shadow functions and / or sample images will be higher in subsequent iterations than at the beginning of the iteration.

[0058] To draw conclusions about the progress of the iteration, we can also consider the tiles. The sample image determines the first sample image Second approximation The first image of the sample Second approximation The distance between them is determined. If the distance is still greater than the specified threshold, iteration can continue; if the known distance is less than the specified threshold, iteration can terminate. In principle, different mathematical norms can be used as distance metrics. For example, Euclidean distance can be used as the distance.

[0059] To reduce computation and obtain a better approximation of the shadow function more quickly, it is also appropriate to first perform the iterative method on the model function for the shadow function with a first number of free parameters, and then perform the iterative method on a more complex model function with a second number of free parameters that is larger than the first number.

[0060] Finally, once each of the tiles has obtained a sufficiently accurate, shadowless sample image, these final shadowless sample images can be combined into a mosaic image.

[0061] It is feasible and preferred in principle to compute the shadow function individually, at least approximately, for each tile image.

[0062] In some cases, it may be appropriate to calculate the average of the shading functions that are approximately individually known for each tile image on these tiles. Then, a consistency test is performed using this average. For example, the degree of deviation between each of the approximately known shading functions and the known average can be examined. If the difference between a certain approximately known shading function and the average exceeds a specified threshold, the average can be used instead of the approximately known shading function.

[0063] To reduce noise and balance outliers from overly bright or dark pixels, it is preferable that the calculation of the shading function not evaluates the value of the patch image pixel by pixel, but rather evaluates it for groups of multiple neighboring pixels. For example, the mean or median can be calculated for these groups of pixels and used for further evaluation. Overly bright or dark pixels are also referred to as hot or cold pixels. Sometimes, outliers are provided not only by hot / cold pixels but also by samples. When the sampling rate is high, it may be insufficient to consider only directly adjacent pixels. In this case, it is preferable to consider using the mean and / or median from neighboring pixels. Here, multiple pixels from a region whose location is defined around the center pixel can be utilized to obtain a representative value of the patch image at the center pixel by forming the mean and / or median. Groups of pixels can, for example, be located in rectangular or square regions. The side length of the rectangular or square region can, for example, be 2% to 5% of the overlap of adjacent patches, and preferably 3%. Attached Figure Description

[0064] Further advantages and features of the present invention will be explained below in conjunction with the accompanying drawings. Wherein:

[0065] Figure 1: A schematic diagram showing an embodiment of an optical device according to the present invention;

[0066] Figure 2: A simplified diagram illustrating a first embodiment of the method according to the present invention;

[0067] Figure 3: A schematic diagram illustrating overlapping blocks for explaining a first embodiment of the method according to the present invention;

[0068] Figure 4: A simplified diagram illustrating a second embodiment of the method according to the present invention; and

[0069] Figure 5: Another schematic diagram showing overlapping blocks used to explain a second embodiment of the method according to the present invention.

[0070] Components that are identical or have the same function are usually given the same reference numerals in the accompanying drawings. Detailed Implementation

[0071] Referring to FIG1, an embodiment of an optical device according to the present invention is illustrated, which is configured as a microscope 100.

[0072] Microscope 100 first includes a light source 10 (e.g., a laser) for providing illumination light 12 and an illumination optical path for guiding the illumination light 12 into sample space 1. In the example shown, the illumination optical path includes a tube lens 20, a main beam splitter 23, and a microscope objective 40. The tube lens 20 produces an intermediate image plane 18, which is optically conjugate to a plane 11 within the sample 2 in sample space 1. In the illumination optical path, the excitation light 12 passes through the intermediate image plane 18 and reaches the main beam splitter 23 via the tube lens 20, where it is reflected toward the microscope objective 40. The excitation light 12 then passes through the back focal plane 42 of the microscope objective 40 and is subsequently guided by the microscope objective 40 into sample space 1. Sample 2 can be a biological sample and can be prepared using a photoluminescent material that can be excited by the excitation light 12. The wavelength and intensity of the excitation light 12 can be appropriately selected based on sample 2 and the fluorescent agent used. The light source 10 can be composed of a variety of different lasers. The wavelength and / or intensity can be adjustable.

[0073] Furthermore, the microscope 100 also includes a detector 50 for detecting emitted light 16 emitted from the sample 2 in the sample space 1, and a detection optical path having a microscope objective 40 for guiding the emitted light 16 to the detector 50. In the example shown, the microscope 100 is a wide-field microscope, and the detector 50 is a camera, meaning that the field of view 30 (FOV) of the detection optical path is imaged onto the sensor plane 51 of the camera 50. The sensor plane 51 is optically conjugate to the plane 11 in the sample space 1.

[0074] The emitted light 16 from sample 2 can typically be redshifted fluorescence emitted by fluorophores in sample 2. The main beam splitter 23 is configured to transmit the redshifted emitted light 16 and reflect the excitation light 12, thereby preventing most of the excitation light 12 scattered back from sample space 1 from reaching the direction of camera 50.

[0075] In the detection optical path, the emitted light 16 emitted from sample 2 is captured by microscope objective lens 40, passes through main beam splitter 23, and is then imaged onto sensor plane 51 of camera 50 by lens 22.

[0076] It is known in the prior art that an excitation filter may be present in the excitation optical path (e.g., between the lens 20 and the main beam splitter 23), and / or an exit filter may be present in the probe optical path (e.g., between the lens 22 and the main beam splitter 23). Multiple different main beam splitters 23 are also possible, coordinated with individual fluorophores, and can be connected to the optical path.

[0077] Furthermore, the microscope 100 also includes, in particular, an automated mechanical actuator 44 for adjusting the relative lateral positioning (x, y) between the sample 2 and the microscope objective 40 about the optical axis 41 of the microscope objective 40, and includes a control unit 90, such as a PC, configured to control the mechanical actuator 44 and to detect and evaluate measurement data from the detector 50. In the example shown, the optical axis 41 of the microscope objective 40 extends along the z-axis. The mechanical actuator 44 may be, for example, part of a motor-driven sample stage, and in the example shown, is used to adjust the pre-given positioning of the sample 2, i.e., the pre-given x, y coordinates of the sample 2 relative to the optical axis 41. A right-handed orthogonal coordinate system (x, y, z) is shown below the mechanical actuator 44. In the example shown, the microscope 100 also includes an axial actuator 46 for adjusting a pre-given axial distance, i.e., the distance between the sample 2 and the microscope objective 40 along the z-direction.

[0078] The illustrated embodiment of the microscope according to the invention is also configured to perform the method according to the invention. For this purpose, according to the invention, the control unit 90 is configured to perform the following method steps for obtaining a shadow-free sample image. a) The measured patch image Predefined relation (S103) ,in, It is a tile The unshaded sample images that are yet to be obtained. It is for tiles The shadow function, and It is for tiles a) Additive shaded terms; b) Pre-given (S104; S204) for the shaded function Model function ( ) and shaded items used for additive properties Model items; c) using tile images Determine the adjacent image tiles ( Multiple support locations in the overlapping area of ​​) Correction factor at () ;S105, S205); d) By using the model function pre-given in step b) ; ) and the correction factor determined in step c) Perform matching to at least approximately determine (S108; S207) the shadow function. and shaded items e) By using the shadow function that was at least approximately determined in step d). and at least approximately determined shaded terms The shadowless sample image is approximately determined (S109; S208). .

[0079] The first embodiment of the method according to the present invention will be explained with reference to the following reproduced steps S101 to S118 and with reference to FIG2 and FIG3.

[0080] Figure 2 shows the first block. =0 and the second tile A simplified diagram of =1. (For the first tile) The coordinate system with =0 is represented by solid lines, while for the second tile... The coordinate system with 1 is represented by dashed lines. The simplified diagram a at the top of Figure 2 shows the image of the sample being searched. , The intensity of the shadow is related to the x-coordinate. It is important to note that the x-coordinates are referenced to the center of each respective patch. The simplified diagram b in the middle of Figure 2 illustrates the shadow functions to be determined. , The simplified diagram c at the bottom of Figure 2 shows the diagrams for the first block respectively. =0 and the second tile =1 data measured , .

[0081] Figure 3 shows five rectangular tiles of equal size, each with a width of b and a height of h, arranged in a rectangular grid. The area where the central tile overlaps with an adjacent tile is shown in grid shading. The area where the central tile overlaps with two adjacent tiles is shown in diagonal shading. The x and y coordinates are also indicated in Figure 3. The center point of the central tile is located at the origin of the coordinate system. The horizontal line g extends along the x-direction, horizontally centering the central tile. The vertical line v in Figure 3 extends along the y-direction, vertically centering the central tile. The support locations where the correction coefficients are calculated are schematically indicated in Figure 3 with thick black dots. This will be described in detail below.

[0082] Method #1

[0083] S101 uses a microscope to collect measurement data.

[0084] S102 matches ("stitching") the measurement data of adjacent tiles; Result:

[0085] S103 Setting Relationships =

[0086] S104 Setting Relationships

[0087] S105 The correction coefficient is determined according to the following equation, namely Equation 1: Equation 2: And so on, and These can be calculated using Equations 3 and 4 (see below). In summary, this method allows for the calculation of eight correction coefficients f for the remaining tiles that overlap along the x and y directions for tiles not located at the edges. k (See Figure 3).

[0088] S106 sets the relation for the shadow function.

[0089] S107 Approximation of the shadow function Approximation of the searched sample image index Set as

[0090] S108 By, for example, using the least squares method... With the correction coefficient obtained in step S105 A matching ("fitting") is performed to determine the first approximation of the shadow function. ;result:

[0091] S109 uses the shadow function that will be approximately determined in S108. As Substituting this into the relation S103, we can determine the first approximation of the searched sample image. :

[0092] S110 will Improve 1

[0093] S111 by using replace as well as replace Determine the updated correction coefficient as in step S105.

[0094] S112 corrects the curve of the pre-given type in S106 to the updated value obtained in S111 by means of, for example, the least squares method. Perform matching ("fitting") to determine the first shading function. Second approximation ;result:

[0095] S113 via S112 Substitution To determine the first sample image Second approximation .

[0096] S114 Confirmed and The distance d between them, for example, the Euclidean distance:

[0097] S115 When d is greater than the threshold S to be specified: Continue executing S110

[0098] S116 When d is less than the specified threshold S:

[0099] S117 for all tiles Execute steps S103 to S116

[0100] S118 will target these The current approximate combinations are Final result:

[0101] First, in step S101, measurement data is collected using a microscope (e.g., the microscope 100 according to the invention in FIG1), i.e., the detector 50 for each patch. ,in Image data.

[0102] Then, in step S102, the measurement data of each tile in its respective overlapping area are structurally matched with each other. This is equivalent to a "stitching" process. The dataset thus obtained is called the measured tile image. In this context, the coordinate x represents the discrete integer location of a pixel within the tile image along the x-direction. And the coordinate y represents the discrete integer location of the pixel along the y-direction within the tile image. Therefore, targeting The following relation can be written: .

[0103] Among them, Locating tiles without shadow effects The brightness generated by the sample; It's a shadow function, meaning it's used for locating tiles. The multiplicative shadowing effect produced, for example, by halos and / or stains on the detector; and It is an additive shaded item, that is, located in the tile. The additive shadowing effect produced at that location.

[0104] This invention recognizes that, generally, all shadowing effects can be derived through multiplicative shadowing effects (i.e., shadowing functions). This can be represented by ). Therefore, the simplified relation (step S103) is as follows: .

[0105] An image composed of these tiles is called a mosaic image. Without limiting its generality, it can be assumed that a mosaic image consists of N overlapping tiles of the same size, arranged on a regular grid. Therefore, the tile indexes... For example, from 0 to N-1. Each tile is offset by a relative offset vector relative to the global coordinate origin. ,in , and ,in, The width of the tile, and The height of the tile.

[0106] Therefore, the overlap of the tiles along the horizontal direction (i.e., the x-direction) is: The overlap of the tiles along the vertical direction (i.e., the y-direction) is... Typically, tiles can have 10% overlap. In this case, and Global localization in mosaic images Therefore, it can be written as

[0107] Mosaic image measured by the detector The brightness can be written as

[0108] Shadow function In principle, each individual tile is different. However, due to... The main contributions are given by the optical characterization of the illumination and detection optical paths, respectively. The individual tiles are usually similar.

[0109] The typical case will be explained below with reference to Figure 2. As can be seen from Figure 2, the following four equations (I) to (IV) can be listed. (I) (II) (III) (IV)

[0110] In each of these equations (I) to (IV), on the left side... The parameter values ​​are obtained by using the detector to analyze the patch. =0 and =1 is known after measurement. The values ​​of the parameters on the right side of equations (I) to (IV) are unknown, and therefore need to be solved.

[0111] Based on the additional condition that adjacent image tiles are structurally matched (“stitched”) as described above, the following additional conditions (V) and (VI) are derived: (V) (VI)

[0112] Using equations (V) and (VI), the eight unknown parameters are... and The number of parameters has been reduced to six unknown parameters.

[0113] In this embodiment of the method according to the invention, according to step S104, the shadow function is... Use the following parabolic equation: At the center of the image Place, This formula assumes that the brightness value at the center of each tile is undisturbed. Under normal circumstances, this is a reasonable assumption. Furthermore, this is primarily for the tile itself. =0 and =1 Assuming they have the same shadow function P Nevertheless, and under normal circumstances, the algorithm will still produce tiles. and tiles Different approximate shading functions are provided, as explained in detail below for tiles. and In the least squares fitting, different correction coefficients were used for the overlap of the left and right images.

[0114] If we first set in this model So, the two parameters and It remains free and therefore needs to be determined. By using this model, the number of unknown parameters for equations (I), (II), (III), and (IV) is reduced from six to four, allowing us to solve the following system of equations to obtain the four unknown parameters: , , and .

[0115] To illustrate this more clearly, the following parameters are introduced: The left tile edge was obtained and the edge of the right-side tile Result: Equation 3: Equation 4:

[0116] Similarly, it can be calculated that Then it is along the vertical direction. and Using these results, correction factors can now be calculated for the support locations on the coordinate axes and at their respective boundaries in the overlapping regions. These support locations are highlighted with bold black dots in Figure 3. For these points, the correction factors can be calculated as described in step 105: in, These are continuous indices of the support locations (see locations in Figure 3). Importantly, these results apply regardless of whether the image at a given location represents the background or the sample. However, a limitation is that these locations should not contain overexposed or black pixels. For a horizontal line g passing through the center of the patch (i.e., at...) (time) and the vertical line h passing through the center of the tile (i.e., at) (At any time), this operation can be performed on the pairing of all adjacent tiles to obtain the correction coefficients corresponding to a total of 8 support positions.

[0117] In step S106, the shadow function to be determined is... Choose the parabolic equation .

[0118] Then, in step S108, a total of eight correction coefficients at eight locations are used to determine the shadow function via least-squares matching. The first approximation Each location marked in Figure 3 provides a correction coefficient. Correction coefficients are also obtained for adjacent tiles at the same location, but these coefficients are used to determine the approximation of the shading function for adjacent tiles. Therefore, the four parameters a, b, c, and d of the parabolic relation for the i-th tile have eight input values, which are used for least-squares fitting.

[0119] Then, in step S109, using For the sample images being searched The first approximation is calculated as follows: :

[0120] Although a very simple model function is used here, it can typically compensate for or reduce most of the slowly changing shadow artifacts. Index This indicates that for each tile ,function They are fundamentally different. The computational cost is relatively small compared to, for example, the pixel-by-pixel iterative optimization cost function used in [Lötgering]. Therefore, it is relatively easy to compute and use the shading correction function separately for each tile. However, in many cases, the functions calculated for different tiles... Relatively similar. Therefore, it is meaningful that all functions... Calculate the averaged function In addition, consistency checks can be performed to specify individual Whether the result is, for example, with other or averaged function There are particularly noticeable deviations. In this regard, overexposed or very dark pixels should be considered in particular.

[0121] Shadow function for tiles at the left, right, top, and bottom edges of a mosaic image The approximate calculation becomes difficult due to the lack of adjacent tiles. However, due to the function... These tiles are generally similar, so the calculation of these tiles can be supported by either using the value mirrored from the boundary of the opposite tile or the average value of the tiles that are completely surrounded by other tiles for the correction coefficients.

[0122] From a mathematical perspective, a function It can be calculated based on the brightness value of the corresponding individual pixel. However, it is important to note that the brightness value of an individual pixel is affected by noise, and the patches need to be structurally matched, i.e., correctly aligned ("stitched"). This means that the illuminance or brightness values ​​of overlapping pixels must accurately come from the same sample area. These factors can make the calculation difficult. By using methods targeting... The shading function is calculated using the spatial average brightness value. An approximation can reduce the impact of these two problems (i.e., noise and proper stitching), for example, by calculating the average brightness value within a small area. This averaging process also enables the identification and exclusion of overexposed or underexposed pixels, or at least reduces their impact. Alternatively, the median of the corresponding measurements can also be calculated from pixels in these small areas or the surrounding environment.

[0123] Importantly, for the theoretical case, the shadows of all tiles have exactly the same parabolic trajectory. The correction method described here ideally describes shadow artifacts and thus enables the direct calculation of the interference-free brightness value of the searched sample. .

[0124] Since shadows do not follow a precise parabolic trajectory in reality, it is appropriate to determine an approximation of the shadow function through iteration. To determine the sample image This will be explained with reference to method steps S110 to S116.

[0125] First, in step S110, the index is... Increase by 1. Then, in step S111, determine the updated correction coefficients. In principle, this is the same as above, but with the condition that the most recently determined approximation (i.e., the first one of the sample images) is used. Second approximation )replace And using the most recently determined approximation (i.e., the first shaded function) Second approximation )replace .

[0126] By using the updated correction coefficients obtained in step S111 Subsequently, in step S112, the shadow function is determined by matching ("fitting") the curve of the pre-given type in step S106. Second approximation Then, in step S113, the first... The first approximation is performed so that the sample image can be calculated as follows: Second approximation : To achieve numerical stabilization, and / or when the denominator in the equation is zero for a particular pixel, division of the form a / b can be implemented in a regularized manner: , where e is a small regularization parameter.

[0127] To check the progress of the iteration, it can then be calculated in step S114, for example, by using Euclidean metrics. and The distance d between them. If the result shows that the distance d is still greater than the specified threshold S (step S115), the iteration continues in step S110. However, if the check result shows that the distance d is less than the specified threshold S (step S116), the iteration is interrupted. As an intermediate result, then for the tiles... The current approximation of the existing sample image It has the accuracy adjusted by the specified threshold S. According to step S117, then all blocks are then... Perform steps S103 to S116 as described herein, and then in step S118, [the following steps are performed]. Each of the current approximate combinations .

[0128] It is clear from the above statements that, in principle, more complex functions can be used to approximate the brightness values ​​in the sampling overlap region when considering more support positions and thus more correction coefficients. Each additional correction factor allows for adjustments at the relevant support locations. Establishing two additional equations with new unknown values ​​will allow us to account for additional parameters of the model function. However, in model functions with more free parameters, the resulting equations may become complex.

[0129] The following describes a second embodiment of the method according to the invention, which allows for the use of more complex model functions. This model function requires very few iterations and does not require separating the background of the patch image from the sample region. For example, the model function used for the shadow function... It can be constrained by Zernike polynomials. For a third-order Zernike polynomial, it is necessary to match, for example, 10 free parameters.

[0130] The second embodiment of the method according to the invention is explained with reference to steps S201 to S217, which are reproduced herein, and with reference to Figures 4 and 5. The simplified diagram in Figure 4 corresponds to simplified diagram c in Figure 2, showing the first block. =0 and the second tile =1 Data measured respectively , .

[0131] Figure 5 shows nine rectangular blocks of the same size. The size and arrangement of these blocks correspond to those of the blocks in Figure 3. The shading in the overlapping areas is the same as in Figure 3. In Figure 5, the support locations where the correction factors are calculated are also indicated by thick black dots. This will be described in detail below.

[0132] Method #2

[0133] S201 uses a microscope to collect measurement data.

[0134] S202 matches (“stitches”) the measurement data of adjacent tiles; results

[0135] S203 Setting Relationships

[0136] S204 Setting Relationships ,in, Zernike polynomials

[0137] S205 Targeting a specific tile For the points at the edges of the overlapping region (Figure 5), determine the correction coefficients according to Equation 5: in, Representing a block Adjacent tiles

[0138] S206 Approximation of the shadow function Approximation of the searched sample image index Set as

[0139] S207 By (e.g., using least squares method) With the correction factor known in S205 A matching ("fitting") process is performed to determine the first approximation of the shadow function. ;result:

[0140] S208 uses the shadow function approximately determined in S207. As Substituting into the relation in S204, we can determine the first approximation of the searched sample image. :

[0141] S209 will Improve 1

[0142] S210 by using replace Determine the updated correction factor as in S205.

[0143] S211 compares the curve of the pre-given type in step S104 with the updated correction coefficients obtained in step S211 (e.g., using the least squares method). Perform matching ("concatenation") to determine the first shading function. Second approximation ;result:

[0144] S212 passes through S211 Substitution To determine the first sample image Second approximation .

[0145] S213 Confirmed and The distance d between them (e.g., Euclidean distance):

[0146] S214 If d is greater than the threshold S to be specified: continue with S209

[0147] S215 If d is less than the specified threshold S:

[0148] S216 Perform steps S203 to S215 on all blocks.

[0149] S217 will Each of the current approximate combinations Final result:

[0150] Steps S201 to S203 are exactly the same as steps S101 to S103 of the first method variant described above. In step S204, the shadow function is... Choose, for example, a third-order Zernike polynomial as the model function. Appropriately, the limiting range of the Zernike polynomial and / or the correction values ​​are appropriately normalized so that a rectangular shadow function, corresponding to the measured image, can be described as the value of the Zernike polynomial. For example, the radial component of the Zernike polynomial can be normalized to the length of the image diagonal.

[0151] Then, in step S205, correction coefficients are determined for points at the edges of the overlapping regions. In Figure 5, these points are schematically represented by thick black dots for the central patch. The calculation of these correction factors is illustrated by way of example with the simplified diagram in Figure 4. For the left patch... =0, in positioning Correction factor at the edge of the right-hand tile. It can be calculated as

[0152] This correction factor is designed to adjust the brightness value of a pixel at the boundary location by multiplying it by the brightness value of the pixel at the same location, making it comparable to the brightness value of an overlapping patch at the same location. This is indicated by the arrow in Figure 4.

[0153] Accordingly, for the right-hand panel =1, the correction factor at the edge of the tile on the left. It can be calculated as

[0154]

[0155] Figure 4 shows the correction coefficients using vertical arrows. and The effect. Furthermore, Figure 5 shows the correction coefficient. and The support position. Importantly, and It refers to the correction factor. Example. Index. and This should be used here as an indication of the corresponding location of each support position.

[0156] Then, this correction factor can be calculated for many y-positions along the vertical boundaries of overlapping tiles. Similarly, the corresponding values ​​can also be calculated for the horizontal boundaries. In Figure 5, black dots indicate the support positions used to calculate these correction coefficients for the central tile. Similar to method variant #1, to improve the robustness of the calculation, spatially averaged (i.e., averaged over multiple pixels) brightness or light density values ​​are used.

[0157] Alternatively, correction coefficients can be added at the locations of diagonally opposite tiles. These overlapping areas are indicated by diagonal shading in Figure 5. Furthermore, additional correction coefficients can be added not at the tile edges. For example, multiple correction coefficients with a value of 1.0 can be added at or near the tile center. This means that as iterations proceed, the model function values ​​at these locations should tend towards these additional correction coefficients.

[0158] If the number of calculated correction coefficients exceeds the number of free parameters that the model function needs to match, the model function can be determined via least-squares matching. This model function can then be used to determine the correction coefficients for all pixels within the tile. These correction coefficients are then applied to the brightness values. This can reduce residual brightness deviations at tile boundaries. The advantage of this is that it allows for natural matching of brightness values ​​across the entire tile area. Therefore, it is feasible to achieve good correction of shadow artifacts even in mosaic images with minimal overlap.

[0159] Subsequently, in step S207, the model function pre-given in step S204 can be transformed (for example, using the least squares method). With the known correction coefficient A matching ("fitting") is performed to determine the first approximation of the shadow function. .

[0160] The shadow function approximately determined in step S207 As Substituting the relational expression into step S204, a first approximation of the searched object function is then provided. : .

[0161] The method is iterated in subsequent steps S209 to S212, and determined in steps S213 and S214. and The distance d between them is used to check the iteration progress. In step S215, the method is terminated when the accuracy is sufficient. In step S216, sample images of all patches are determined, and in step S217, the individual patch images are combined into a mosaic image. These are similar to steps S110 to S118 of the method according to the first embodiment of the present invention.

[0162] Because the model function is typically a slowly changing function, and its free parameters are calculated via least-squares matching, the result does not completely eliminate luminance mismatch. However, this process can be iteratively applied to the final sample image as described above to reduce luminance mismatch with each iteration. Typically, the process converges after a few iterations (e.g., 3 or 4).

[0163] Alternatively, the method can first iterate over the model function with the shadow function having L1 free parameters, and then iterate over the more complex model function with more free parameters L2 > L1.

[0164] After calculating the modified mosaic image based on method variant #1 and / or method variant #2, the approximate shader function can be calculated. , average , These average values ​​can be used as a reference in future shooting. The advantage of this is that this reference data for the shading function is calculated based on typical sample testing rather than on artificially calibrated samples. In particular, corrections can therefore be achieved even with several single images, without needing to capture images from multiple or all patches.

[0165] This invention presents a novel and robust retrospective method for correcting shadow artifacts in tiled images. Specifically, two methods for compensating for spatially slowly varying shadow artifacts in tiled images are described in detail. The image processing methods used here utilize only overlapping tile regions and do not require pre-calibration or capturing shadow reference images. This new method does not rely on the distinction between background and foreground regions.

[0166] List of reference numerals

[0167] 2 samples

[0168] 10. Radiation sources or light sources, such as lasers or LED light sources.

[0169] 10. Microscope stand

[0170] 12. Illumination radiation, illumination light, excitation light

[0171] 16. Emitted radiation or emitted light from sample 2 in sample region 1

[0172] 40 Microscope Objectives

[0173] 41. Optical axis of microscope objective lens: 40°

[0174] 44 Mechanical drive

[0175] 50 detectors

[0176] 90 Control Unit

[0177] 100 Optical devices and microscopes according to the present invention

[0178] The origin of the (0,0) xy coordinate system

[0179] Parabolic model function parameters

[0180] Parabolic model function parameters

[0181] Positioning of samples on blocks without shadow effects The generated brightness, shadow-free sample image

[0182] The Second approximation

[0183] The Second approximation

[0184] Mosaic image

[0185] Sample image In the block Value at the left edge

[0186] Sample image In the block Value at the right edge

[0187] Global positioning of the sample without shadowing effects The brightness generated at the location

[0188] Parabolic model function parameters

[0189] The detector locates the i-th patch. Brightness measured at

[0190] Measured brightness of mosaic image

[0191] d and Distance between

[0192] Parabolic model function parameters

[0193] Patch overlap along the x-direction

[0194] Overlap of tiles along the y-direction

[0195] Correction coefficient

[0196] The model function of the shadow function, the parabolic function

[0197] Shadow function The Second approximation

[0198] Shadow function The Second approximation

[0199] Approximately known shadow function average

[0200] g is the horizontal direction within the tile when y=0.

[0201] Correction coefficient

[0202] G_i The model function of the shadow function, such as the Zernike polynomial.

[0203] Shadow function The Second approximation

[0204] Shadow function The Second approximation

[0205] Approximately determined shadow function average

[0206] Height of the tile image

[0207] Tile Index

[0208] Tile Index

[0209] Index of support position

[0210] The offset vector of the i-th patch in the mosaic image

[0211] Length of the grid vector along the x-direction

[0212] Length of the grid vector along the y-direction

[0213] parabolic function P parameters

[0214] parabolic function P parameters

[0215] parabolic function P parameters

[0216] parabolic function P parameters

[0217] Model function and parabolic function used for shadowing functions

[0218] S and Threshold of distance between

[0219] Multiplicative shading function, tile positioning For example, multiplicative shadow effects caused by vignetting and / or dirt on the detector, shadow correction functions.

[0220] Additive shaded items

[0221] v is in the vertical direction within the tile when x=0.

[0222] Width of the tile image

[0223] First lateral coordinate in the sample plane

[0224] The discrete integer positioning of pixels within the tile image along the x-direction.

[0225] First global horizontal coordinate

[0226] Global localization in mosaic images

[0227] ( Support position

[0228] Support position

[0229] The second lateral coordinate in the sample plane

[0230] The discrete integer positioning of pixels along the y-direction within the tile image.

[0231] Second global horizontal coordinate

[0232]

[0233]

[0234] A certain approximate shadow function or The average value of the approximate shaded function < >or Distance between

[0235] References

[0236] [Wu] Q. Wu, FA Merchant, K. R. Castleman, 'Microscope Image Processing', Cap. 12.5.3 – 12.5.6, Academic Press, Elsevier, ISBN 978-0-12-372578-3

[0237] [Young] Young, Ian T. "Shading correction: compensation for illumination and sensor inhomogeneities." Current Protocols in Cytometry 14, no. 1 (2000): 2-11.

[0238] [DeSilva] V. De Silva, V. Chesnokov, D. Larkin, "A Novel Adaptive Shading Correction Algorithm for Camera Systems", 2016 Society for Imaging Science and Technology, DOI: 10.2352 / ISSN.2470-1173.2016.18.DPMI-249

[0239] [Preibisch] Preibisch, Stephan, Stephan Saalfeld, and Pavel Tomancak. "Globally optimal stitching of tiled 3D microscopic image acquisitions." Bioinformatics 25.11 (2009): 1463-1465.

[0240] [Legesse] Legesse, Fisseha Bekele, Olga Chernavskaia, SandroHeuke, Thomas Bocklitz, Tobias Meyer, Popp, and RainerHeintzmann. "Seamless stitching of tile scan microscope images." Journal of microscopy 258, no. 3 (2015): 223-232. [Legesse, Fisseha Bekele, Olga Chernavskaia, Sandro Heuke, Thomas Bocklitz, Tobias Meyer, Popp and Rainer Heintzmann, “Seamless stitching of tiled scanning microscope images,” *Journal of Microscopy*, Vol. 258, No. 3 (2015): 223-232.

[0241] [Smith] Smith, Kevin, Yunpeng Li, Filippo Piccinini, Gabor Csucs, Csaba Balazs, Alessandro Bevilacqua, and Peter Horvath. "CIDRE: an illumination-correction method for optical microscopy." Nature Methods 12, no. 5 (2015): 404-406.

[0242] [Peng] Peng, Tingying, Kurt Thorn, Timm Schroeder, Lichao Wang, Fabian J. Theis, Carsten Marr, and Nassir Navab. "A BaSiC tool for background and shading correction of optical microscopy images." Nature Communications 8, no. 1 (2017): 14836.

[0243] [Loetgering] German Patent Application 10 2024 124 248.5

Claims

1. A method for processing tiled images, wherein, Multiple overlapping patch images of sample (2) were captured using optical equipment (100). The feature is that, in order to obtain a shadowless sample image Perform the following steps: a) For the measured patch image Predefined relation (S103) ,in, It is a tile The unshaded sample images that are yet to be obtained. It is for tiles The shadow function, It is for tiles The additive shaded items; and a) is a continuous index of the tiles; b) pre-given (S104; S204) for the shading function. Model function ( ) and the shaded item used for the additive nature Model items; c) using the tile image Determine the adjacent image tiles ( Multiple support locations in the overlapping area of ​​) Correction factor at () ;S105, S205); d) By using the model function pre-given in step b) ； ) and the correction factor determined in step c) Perform matching to at least approximately determine (S108; S207) the shadow function. and the shaded item e) By using the shadow function that was at least approximately determined in step d). and at least approximately determined shaded terms The shadowless sample image is approximately determined (S109; S208). 。 2. The method according to claim 1, characterized in that, The model function ( () is a parabolic function in the x-coordinate and / or y-coordinate.

3. The method according to claim 1 or 2, characterized in that, The model function ( It has at least one of the following polynomials or is realized by at least one of the following polynomials: Zernike polynomial, two-dimensional Hermitian polynomial, two-dimensional Laguerre polynomial, Jacobi polynomial, multivariate Bernstein polynomial.

4. The method according to any one of claims 1 to 3, characterized in that, For the image of the tile Support positions at the edges of overlapping areas ( ) Calculate the correction coefficient ( )。 5. The method according to any one of claims 1 to 4, characterized in that, For the support positions located on the coordinate axes (x, y) ) Calculate the correction factor ( ； ）。 6. The method according to any one of claims 1 to 4, characterized in that, For tiles ( Multiple support locations at points on the edge (( ) Calculate the correction factor ( ； ）。 7. The method according to any one of claims 1 to 6, characterized in that, For mosaic images ( There are no tiles at the edges of areas that overlap with adjacent tiles. The correction factor obtained for the support positions at the respective opposite edges ( ； ) for related tiles ( () has no adjacent tile edges.

8. The method according to any one of claims 1 to 7, characterized in that, For mosaic images ( (Patches at the edge) The edges of the non-overlapping areas with adjacent tiles are used, and the edges are completely covered by other tiles. The tile surrounded by () Correction factor () ； The average value of ).

9. The method according to any one of claims 1 to 8, characterized in that, By using the model function ( ) and assuming adjacent tiles ( The measured patch image The sample image is calculated assuming that the structures are already structurally matched. The value at the support position on the coordinate axis and at the boundary of the overlapping area ( ）。 10. The method according to claim 9, characterized in that, The correction coefficient ( The image is a tile located at a support position on the coordinate axis and at the boundary of the overlapping area. The value of the sample image The value ( ) of the commerce.

11. The method according to any one of claims 1 to 10, characterized in that, The correction coefficient ( ) are adjacent tiles ( ) tile image ( 、 The quotient of the value of ).

12. The method according to any one of claims 1 to 11, characterized in that, In the tile image Add additional support locations in the central region, wherein the model function ( Set the function values ​​at these additional support locations to appropriate values, such as 1.

13. The method according to any one of claims 1 to 12, characterized in that, In step d), the shadow function The approximate determination and the sample image in step e) The approximate determination is performed iteratively (S110~S113; S209~S212) respectively by means of the following: that is, by using the shadow function respectively. Sub-approximation ( ) and / or the first of the sample images Sub-approximation ( To determine the updated correction coefficients (S111; S210) ), by the correction coefficient ( ； ) and the pre-given model functions ( Matching is performed to determine the first shaded function of (S112; S211). Sub-approximation ( And by using the first shadow function Sub-approximation ( ), to determine the first shadowless sample image of (S113; S212) Sub-approximation ( ）。 14. The method according to claim 13, characterized in that, Sure and The iteration continues when the distance (d) between the two points is greater than the threshold (S) to be specified; the iteration terminates when the distance (d) is less than the threshold (S).

15. The method according to any one of claims 13 or 14, characterized in that, The iterative method is first performed on a model function for the shadow function with a first number of free parameters, and then on a more complex model function with a second number of free parameters that is larger than the first number.

16. The method according to any one of claims 1 to 15, characterized in that, Final sample image Combined into a mosaic image ( ）。 17. The method according to any one of claims 1 to 16, characterized in that, For each tile ( The shadow function is calculated at least approximately individually. 。 18. The method according to any one of claims 1 to 17, characterized in that, Examine each of the approximate shaded functions ( ； ) and the average value of the approximate shaded function ( ； The degree of deviation, and if a certain approximate shaded function ( ； ) and the average value ( ； The difference exceeds the threshold to be specified. ), then use the average value ( ； ) replace the approximately known shadow function.

19. The method according to any one of claims 1 to 18, characterized in that, In order to approximate the calculation of the shadow function ( ； ), and evaluate groups of multiple adjacent pixels respectively.

20. The method according to claim 19, characterized in that, For each group of multiple adjacent pixels, the average or median of the data measured from each pixel is generated.

21. The method according to any one of claims 1 to 20, characterized in that, The optical device is a microscope (100).

22. An optical device comprising: a detector (50) for detecting emitted radiation (16) emitted from a sample (2) in a sample region (1); a detection optical path having an imaging optics (40) for guiding the emitted radiation (16) onto the detector (50); a mechanical actuator (44) for adjusting the relative lateral positioning of the sample (2) and the imaging optics (40) about the optical axis (41) of the imaging optics (40); and a control unit (90) configured to drive the mechanical actuator (44) and evaluate measurement data from the detector (50), wherein, The control unit (90) is configured to drive the optical device (100) to capture (S101) multiple overlapping patch images of the sample (2). The control unit (90) is characterized in that it is configured to perform the following method steps to obtain a shadowless sample image. a) The measured patch image Predefined relation (S103) ,in, It is a tile The unshaded sample images that are yet to be obtained. It is for tiles The shadow function, and It is for tiles The additive shaded items; and a) is a continuous index of the tiles; b) pre-given (S104; S204) for the shading function. Model function ( ) and the shadowed item for the additive nature Model items; c) using the tile image Determine the adjacent image tiles ( Multiple support locations in the overlapping area of ​​) Correction factor at () ;S105, S205); d) By using the model function pre-given in step b) ； ) and the correction factor determined in step c) Perform matching to at least approximately determine (S108; S207) the shadow function. and the shaded item e) By using the shadow function that was at least approximately determined in step d). and at least approximately determined shaded terms The shadowless sample image is approximately determined (S109; S208). 。 23. The optical device according to claim 22, characterized in that, The control unit (90) is configured to perform the method of any one of claims 2 to 21.

24. The optical device according to any one of claims 22 or 23, wherein the optical device is configured as an aerial camera.

25. The optical device according to any one of claims 22 or 23, wherein the optical device is configured as a microscope (100) and has a radiation source (10) for providing illumination radiation (12) and an illumination optical path for guiding the illumination radiation (12) into a sample region (1), wherein, The imaging optics (40) has microscopic optics.

26. The optical device according to claim 25, characterized in that, The microscopic optical device includes at least one microscope objective (40).

27. The microscope according to any one of claims 25 or 26, comprising at least one of the following microscopes: electron microscope, ion microscope, digital microscope, fluorescence microscope, optical microscope, transmitted light microscope, reflected light microscope, wide field microscope, scanning microscope, confocal microscope, light field microscope, light sheet microscope, TIRF microscope, and SIM microscope.

Images