Method and apparatus for producing a corrected image

Abstract

The invention relates to a method for producing a corrected image by using an image capturing device and a device for carrying out the method. The method comprises a first operating mode in which a plurality of image pieces of an object present in a sample chamber are each captured as a plurality of image pixels. Control commands are generated by a control unit such that each image piece is captured so that it partially overlaps at least one other captured image piece in an image piece overlap area of a defined minimum size. Each of the captured image pieces and / or a composite image is combined in a pixel-by-pixel manner by a brightness correction image by calculation. In a second operating mode of the method, the brightness correction image is produced by capturing a plurality of correction image pieces of the object as a plurality of image pixels, respectively, in which control commands are generated by the control unit such that the object is moved relative to the path of the detection beam and, in the process, the size of the correction image overlap area is greater than the minimum size of the image piece overlap area.

Description

Technical Field

[0001] This invention relates to a method for generating a corrected image according to the preamble of the independent claim. Furthermore, this invention relates to an apparatus for carrying out the method. Background Technology

[0002] Reducing system-related imaging aberrations is crucial in many imaging methods, particularly in microscopy. For example, such aberrations can arise from the influence of the optical elements used in the corresponding beam path, as well as from the technical limitations of the detector used. Furthermore, sample holders, such as coverslips, slides, microtiter plates, or the bottom of petri dishes, can cause imaging aberrations not directly caused by the object being imaged (the sample).

[0003] Correction of brightness differences (shadows; shadow correction) in captured images is particularly important. This is especially significant if multiple image patches are captured and stitched together to form a complete image. In this process, individual image patches overlap each other by a specific area (overlap area; overlap), allowing the image patches to be stitched together with positional accuracy based on the information from the multiple captured images contained within. If the image patches have different brightness levels, the viewer will perceive it as an error.

[0004] Several methods for brightness correction have been disclosed in the prior art. In principle, these methods can be subdivided into two categories: i) reference-based methods and ii) purely computational methods.

[0005] Reference-based methods rely on a reference image (also known as a brightness-corrected image, hereinafter referred to as the corrected image), such as a reference image of a uniform sample, which should result in a uniform signal. Imaging aberrations that will occur anyway are contained in the reference image, and therefore the captured image of the sample can be corrected by subtracting the reference image from it. Many procedures for generating reference images are known (e.g., DE 10 2014 112 002 A1).

[0006] Instead of purely computational methods using reference images, captured images of the sample, which include imaging aberrations, are employed. The BaSiC tool (published by Peng et al. in 2017) provides an example of such a method.

[0007] The following section aims to emphasize reference-based correction methods.

[0008] For example, so-called camera-based brightness correction can be achieved as a global correction. In this process, a sample holder, such as a slide, is positioned in the beam path such that a region without a sample is captured. The optical conditions correspond to those during sample capture. For instance, the selected region contains the same coverslip and embedding medium as the sample. The captured image of this region can then be used as a reference image. This process is specific to currently used objective lenses and is particularly useful for conventional image capture under transmitted light.

[0009] For example, the correction just described can be performed on each individual channel in multicolor experiments, and in particular, its application can be found in the correction of reflected light fluorescence recording.

[0010] Modifications to the described correction method can be applied in a variety of ways. For example, multiple image slices of the sample-free region of a slide can be captured and averaged, for example, to eliminate sample contamination present in the image or to compensate for variance between image slices. This process can be implemented in a channel-specific manner.

[0011] It is also possible to capture multiple image slices of a sample and determine a reference image from them. In this case, it is assumed that, for example, the image structure is removed within the range that forms the mean, and only information about the brightness difference is retained. This process is particularly applicable to complex or intricate fluorescence images and requires many image slices, typically more than 200. This requires a slide with the desired fluorophore and the sample, the size of which allows for the capture of a sufficient number of image slices. In this case, channel-specific corrections can also be performed.

[0012] In another variation, the beam is focused on the sample, and then the slide is removed from the beam path. At least one image of the "free space" is captured and used as a reference image.

[0013] A particular challenge arises when calibrating fluorescence images in which the sample to be imaged is labeled with molecules (fluorophores) that can excite fluorescence. In this case, the inherent structure of the sample in the reference image should be prevented from being reflected in the reference image. Furthermore, bleaching should be avoided during the generation of the reference image, if possible, as this can lead to discrepancies in optical conditions between the creation of the reference image and sample capture. Summary of the Invention

[0014] The present invention aims to propose alternative options for generating reference images, thereby reducing the disadvantages of the prior art.

[0015] This objective is achieved by using an image capture device to generate a corrected image. Furthermore, the invention includes an apparatus for implementing this method. Advantageous development examples are also described below.

[0016] A suitable apparatus for implementing the method according to the invention includes a detection beam path with a detector unit and a control unit for generating control commands. In a first operating mode of the method, multiple image patches of an object present in a sample chamber are captured. In this case, each image patch is captured as multiple image pixels. Thus, each image patch consists of multiple image pixels (picture elements). Furthermore, control commands are generated by the control unit to capture each image patch such that it partially overlaps with at least one other captured image patch in an image patch overlap region (overlap) having a defined minimum size. A composite image or a whole image is generated by stitching the image patches together. Here, image data of the multiple captured image pixels in the overlap region is used to position the image patches in the correct location and to arrange the image patches relative to each other in the correct orientation. The captured image patches and / or composite images are corrected by a brightness correction image, wherein each image pixel of the image patch and / or composite image is combined by calculation with image data of the corresponding pixel from the brightness correction image.

[0017] The method according to the invention is characterized in that, in the second operating mode, a reference image is generated as a brightness correction image by capturing multiple images of an object as multiple image pixels in various cases, wherein a control command is generated by a control unit to move the object relative to the detection beam path, and in the process, the size of the overlapping region of the correction image is greater than the minimum size of the overlapping region of the image patch.

[0018] Relative motion between the object and the detection beam path can be achieved through an adjustable sample stage and / or by pivoting or displacing the detection beam path. Advantageously, relative motion can be achieved via a motor.

[0019] This invention utilizes a deviation from the minimum size of the overlapping region of the image patch, at least for the purpose of creating a brightness-corrected image. In this way, a significantly greater number of different image patches of the sample can be captured compared to maintaining the minimum size.

[0020] The minimum size of the commonly used overlapping region is 10% of the area of ​​the image patch in one image direction. Therefore, an advantageous improvement to the method lies in the control commands used to control a drive unit, through which the object can be moved, and the size of the corrected image overlapping region in one image direction is selected to be greater than 10% of the area of ​​the image in that image direction. The corrected image overlapping region can be, for example, 20%, 30%, 40%, 50%, 60%, 70%, 80%, or 90% in one image direction.

[0021] To obtain a sufficiently high-quality luminance-corrected image for practical use, currently available luminance-corrected images can be analyzed and evaluated against predefined quality criteria. If the quality criteria are not met, the currently available luminance-corrected image can be combined with other images of the object through calculation. Optionally, this step can be repeated until the quality criteria are met. The variance of the luminance-corrected image is one possible quality criterion. The goal is to produce a low-variance luminance-corrected image, where a uniform distribution of luminance intensity values ​​is thus present.

[0022] In this context, it is advantageous to select the size of the overlapping region of the corrected image based on the evaluation results of the quality criteria in each case. In this way, other brightness-corrected images that are likely to meet the quality criteria can be generated.

[0023] In principle, this invention can be used in any microscopy method that utilizes contrast. Therefore, a sample holder can be selected as the object, for example, to reduce system-related imaging aberrations. This method can be particularly advantageously used in fluorescence microscopy. In this case, a sample containing molecules (e.g., fluorophores) emitting radiation to be captured and detected is selected as the object.

[0024] The relative displacement between image patches can be generated incrementally, for example, by appropriately controlling the drive of the sample stage. In a further improvement to the method, the image used to generate the brightness-corrected image can also be captured by time-limited illumination of the object as it moves continuously with the object and / or the sample stage. Time-limited illumination can be achieved within the range of strobe illumination.

[0025] Alternatively, to capture an image, the object is moved a specific path length to a position specified by a control command and stopped there. At that current position, additional images are captured to generate a brightness-corrected image.

[0026] An apparatus for implementing the method according to the invention is used to capture images of objects present on an adjustable sample stage in a sample chamber, and includes a detection beam path, a detector unit, and a control unit for generating control commands. The control unit is configured to generate control commands. The control commands can be used to execute a first operating mode and a second operating mode.

[0027] In the first operating mode, the detection unit and the sample stage are controlled by control commands to capture multiple image patches of an object. After an image patch has been captured, the object is moved relative to the optical axis of the detection beam path by a defined path length, which is less than the range of the captured image patches in the direction of movement, such that each image patch partially overlaps with at least one other captured image patch in a defined minimum-size image patch overlap region.

[0028] In the second operating mode, the sample stage is controlled by control commands to move the object relative to the optical axis of the detection beam path by a path length that is less than the path length defined in the first operating mode after the image patch has been captured, so that the minimum size of the overlapping area of ​​the corrected image is greater than the minimum size of the overlapping area of ​​the image patch.

[0029] In other embodiments, there is strobe lighting, which is also controllable by a control unit.

[0030] Therefore, this method can also be used to generate corrected images. In this case, multiple image patches of the object are captured, each image patch being captured as multiple image pixels, and each image patch overlapping with at least one other captured image patch in an image patch overlap region of a defined minimum size. A composite image is generated by stitching the image patches, wherein image data of the multiple captured image pixels in the overlap region is used to arrange the patch images in the correct position and to arrange the patch images relative to each other in the correct orientation. Each captured image patch and / or composite image is corrected by a brightness correction image, wherein each image pixel of the image patch and / or composite image is combined by calculating image data with the corresponding pixel from the brightness correction image. This method is characterized by generating the brightness correction image by capturing multiple images of the object as multiple image pixels in each case, wherein the minimum size of the correction image overlap region is selected to be larger than the minimum size of the image patch overlap region.

[0031] This invention can potentially be used in all contrast-based microscopy methods, with particular suitability for fluorescence microscopy and bright-field microscopy, especially in transmitted light bright fields. Attached Figure Description

[0032] The invention will now be explained in more detail with reference to the accompanying drawings. In the drawings:

[0033] Figure 1 A schematic example of generating a brightness-corrected image according to the prior art is shown;

[0034] Figure 2 A schematic representation of an improvement to the method according to the invention is shown; and

[0035] Figure 3 A schematic representation of an exemplary embodiment of the device according to the present invention is shown. Detailed Implementation

[0036] Existing methods for generating luminance-corrected images KB (also simply called corrected images KB) are known in the art. Figure 1 The diagram is shown schematically at a medium height. Multiple (corrected) image patches TB1, TB2 to TBn are captured and processed by applying alternatively selected mathematical methods (characterized by a feature sequence: ∑).

[0037] (e.g., averaging) to combine the individual pixels of the image patch, and this is used to create the corrected image KB. It is evident that, compared to, for example, the first image patch TB1, the corrected image KB has a smaller variance in luminance intensity values ​​and is presented more uniformly.

[0038] Figure 2 An improvement to the method according to the invention is illustrated. The composite image rB, which has been stitched together from multiple image patches (not shown) and illustrates sample 2, is shown for illustrative purposes only. The first to nth image patches TB1 to TBn are shown by way of example, overlapping each other in a range in the horizontal image direction in the case shown. The correction image overlap region Ov (overlap) that occurs during this process is shown by way of example. The correction image overlap region Ov is chosen to be larger than that of sample 2 when it is imaged to produce the resulting image rB, in particular 10% larger. The correction image KB is generated mathematically from the partially overlapping image patches TB1 to TBn. In this case, the corresponding variance of each correction image patch TB1 to TBn has, for example, a value ranging from 18 to 80, while the variance of the generated correction image KB is, for example, only 3.

[0039] Figure 3 An exemplary embodiment of an apparatus for carrying out the method according to the invention is schematically illustrated. A sample 2 may be arranged on a controllable and adjustable sample stage 1. The sample is illuminated by a light source 3. Detection radiation emitted from the sample 2 (e.g., fluorescence radiation and / or reflected components of illumination radiation) is captured by an objective lens 4 and imaged along the optical axis (indicated by discontinuous lines) onto a spatially resolved detector 5, such as a CCD, CMOS, or sCMOS chip, a SPAD (single-photon avalanche diode) array, or an array of multiple PMTs (photomultiplier tubes). The image data captured by the detector 5 is sent to an evaluation unit 8 and there, through calculation, combined to form a synthetic image rB or a corrected image KB.

[0040] In an alternative embodiment of the device, illumination can be achieved using transmitted light, that is, for example in a bright field, by a properly positioned light source 3 (shown by discontinuous lines in an exemplary and optional manner).

[0041] In addition, there is a control unit 6, which may be, for example, a computer or an FPGA, and is configured to generate control commands. Generating control commands enables the execution of either of the aforementioned operating modes. In each case, the control unit 6 is connected to the motor-driven motion drive unit 7, detector 5, and light source 3 of the sample stage 1 in a manner suitable for data transmission.

[0042] The evaluation unit 8 may optionally be connected to the control unit 6, for example, to facilitate the adaptation of control commands generated there based on the captured image data or the degree of satisfaction of selected quality criteria.

[0043] List of reference numerals

[0044] 1 Sample Stage

[0045] 2. Samples, objects

[0046] 3. Light source

[0047] 4. Objective lens

[0048] 5 detectors

[0049] 6 Control Unit

[0050] 7 drive units

[0051] 8 Evaluation Units

[0052] TB1,...,TBn (first image slice,..., nth image slice)

[0053] Ov (corrected image) overlapping area

[0054] rB composite image / overall image

[0055] KB Brightness Correction Image

Claims

1. A method for generating a corrected image using an image capture device, wherein, The device has a detection beam path with a detector (5) and a control unit (6) for generating control commands. The method includes the following steps: In the first operating mode: - Capture multiple image slices of the object (2) present in the sample chamber, wherein: Each image patch is captured as multiple image pixels, and ○ Control commands are generated by the control unit (6) to capture each of the image pieces such that they partially overlap with at least one other captured image piece in an image piece overlap area of ​​a defined minimum size; - A composite image (rB) is generated by stitching the image pieces together, wherein image data of multiple captured image pixels in the overlapping region is used to arrange the image pieces in the correct position and to arrange the image pieces relative to each other in the correct orientation; - Each of the captured image patches and / or the composite image (rB) is corrected by a brightness-corrected image (KB), wherein each image pixel of the image patch and / or the composite image (rB) is combined with image data from the corresponding pixel in the brightness-corrected image (KB) by calculation; Its features are, In the second operating mode, - The brightness correction image (KB) is generated by capturing multiple image patches (Tb1, ..., TBn) of the object (2) as multiple image pixels in various cases, wherein the control unit (6) generates control commands to move the object (2) relative to the detection beam path, and in the process, the size of the overlapping area of ​​the correction image is greater than the minimum size of the overlapping area (Ov) of the image patches.

2. The method according to claim 1, characterized in that, The control command is used to control the drive unit (7), through which the object (2) can be moved, and the size of the corrected image overlap region in one image direction is selected to be greater than 10% of the range of the image patch (Tb1, ..., TBn) in the image direction.

3. The method according to claim 1 or 2, characterized in that, The currently available luminance-corrected image (KB) is analyzed and evaluated according to predefined quality criteria, wherein if the quality criteria are not met, the currently available luminance-corrected image (KB) is combined with other image patches (TBn+1) by calculation.

4. The method according to claim 3, characterized in that, The size of the overlapping region of the corrected image is selected in each case based on the evaluation results of the quality criteria.

5. The method according to claim 1 or 2, characterized in that, The sample holder was selected as the object (2).

6. The method according to claim 1 or 2, characterized in that, A sample (2) containing molecules that emit radiation to be captured and detected is selected as the object (2).

7. The method according to claim 1 or 2, characterized in that, The object (2) is illuminated by transmitted light.

8. The method according to claim 1 or 2, characterized in that, Image patches (Tb1, ..., TBn) are captured by time-limited illumination of the object (2) to generate the brightness-corrected image (KB).

9. The method according to claim 1 or 2, characterized in that, The object (2) used to capture image patches (Tb1, ..., TBn) is moved along a path length to a position specified by a control command and stopped there, at which other image patches (Tb1, ..., TBn) are captured to generate the brightness-corrected image (KB).

10. An apparatus for capturing an image of an object (2) present on an adjustable sample stage (1) in a sample chamber, comprising a detection beam path, a detector (6) and a control unit (6) for generating control commands. in, The control unit (6) is configured to generate control commands, and a first operating mode and a second operating mode are executable by the control commands. In the first operating mode The detector (5) and the sample stage (1) are controlled by the control command to capture multiple image patches of the object (2), wherein, after the image patches (Tb1, ..., TBn) have been captured, the object (2) moves relative to the optical axis of the detection beam path by a defined path length, the defined path length being less than the range of the captured image patches (Tb1, ..., TBn) in the moving direction, such that each of the image patches (Tb1, ..., TBn) partially overlaps with at least one other captured image patch (Tb1, ..., TBn) in a defined minimum-sized corrected image overlap region (Ov); and, In the second operating mode, The sample stage (1) is controlled by the control command to move the object (2) relative to the optical axis of the detection beam path by a path length after the image patch (Tb1, …, TBn) has been captured. The path length is less than the defined path length of the first operating mode, such that the minimum size of the corrected image overlap region (Ov) is greater than the minimum size of the image patch overlap region.

Images